Avoid empty lines with spaces to be transformed to empty string

[ritwizsinha]

• closes BUG: read_html returns empty list #59147 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[Aloqeely]

Thanks for the PR! I'm not sure if this fixes the problem in the linked issue. Can you write a test that asserts the result of read_html on '<table><tr><td> </td></tr></table>' is not an empty list?

[Aloqeely]

Is this necessary? If it's left as the default then what will the result be?

[ritwizsinha]

Leaving it as default which is true would give the same behaviour as the bug, because if this is true, python_parser calls _remove_empty_lines which gets rid of lines with only spaces.

[Aloqeely]

So running read_html without passing skip_blank_lines=False will produce an empty list, but the doc states that no empty list should be returned, so the bug isn't really fixed.

[ritwizsinha]

I see, so removing the new argument to parse_html, I can instead pass the skip_blank_lines to be False by default to _parse function to always get space only lines as a row instead of avoiding them

[Aloqeely]

How does this relate to fixing the bug? Generally to introduce new parameters to a function you'd have to open an enhancement issue.

[ritwizsinha]

This argument is needed because this function might need to specify the option whether they want to consider columns with just spaces as a valid column or not. If this is true, any column which is blank would be skipped as before. Hereis the place where this is checked to remove whitespaced lines

[ritwizsinha]

@Aloqeely addressed your comments and removed the new named argument

[Aloqeely]

Are there any implications of passing skip_blank_lines=False as the default now? I'm sure that would break some existing code.
To be quite frank I'm not very familiar with the read_html code, ping @mroeschke

[ritwizsinha]

Are there any implications of passing skip_blank_lines=False as the default now? I'm sure that would break some existing code. To be quite frank I'm not very familiar with the read_html code, ping @mroeschke

The difference now would be that, every line with only spaces would be included as a new row in the DataFrame

[mroeschke]

I am not too familiar with the read_html code either, but given the issue I lean toward @Aloqeely opinion that this PR might cause more of a behavior change as opposed to a "bug fix". And reading the original issue it seems like the docs were clear enough given this edge case.

I would lean toward more updating this docs as opposed to changing the code.

[ritwizsinha]

I understand, accordingly I will close this pr, so do you suggest @Aloqeely and @mroeschke to update the docs to say that data elements with only spaces will not be interpreted as rows and all spaces in the html elements would be stripped as a sidenote?