Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add str_length built-in check for PySpark backend #1709

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

marrov
Copy link

@marrov marrov commented Jun 25, 2024

Adds a builtin-check for str_length for the pyspark.sql backend, solving these issues:

Copy link

codecov bot commented Jun 25, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 9 lines in your changes missing coverage. Please review.

Project coverage is 93.46%. Comparing base (812b2a8) to head (b002e70).
Report is 120 commits behind head on main.

Files Patch % Lines
pandera/backends/pyspark/builtin_checks.py 40.00% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1709      +/-   ##
==========================================
- Coverage   94.28%   93.46%   -0.82%     
==========================================
  Files          91      117      +26     
  Lines        7013     8856    +1843     
==========================================
+ Hits         6612     8277    +1665     
- Misses        401      579     +178     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cosmicBboy
Copy link
Collaborator

thanks @marrov ! please run pre-commit to fix linter errors: https://pandera.readthedocs.io/en/stable/CONTRIBUTING.html#set-up-pre-commit

@marrov
Copy link
Author

marrov commented Jun 27, 2024

thanks @marrov ! please run pre-commit to fix linter errors: https://pandera.readthedocs.io/en/stable/CONTRIBUTING.html#set-up-pre-commit

Yes, sorry for that I added it too hastily. Should be fixed now @cosmicBboy.

@cosmicBboy
Copy link
Collaborator

thanks @marrov ! sorry to extend this a little longer, but this check needs unit tests, which can live here: https://github.com/unionai-oss/pandera/blob/main/tests/pyspark/test_pyspark_check.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants