Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support datetime arguments for column range functions #142

Merged
merged 7 commits into from
Jan 29, 2025

Conversation

ghanse
Copy link
Contributor

@ghanse ghanse commented Jan 28, 2025

Changes

Added support for both datetime.date and datetime.datetime arguments to the following column functions:

  • not_less_than
  • not_greater_than
  • is_in_range
  • is_not_in_range

I used type parameter lists for the limit parameter argument of each function. I created the limit as a pyspark.sql.Column using lit(...) which handles both datetime.date and datetime.datetime arguments.

Linked issues

Resolves #136

Tests

Integration tests for each column function have been updated to check the condition for int, datetime.date, and datetime.datetime arguments against IntegerType, DateType, and TimestampType columns, respectively.

  • manually tested
  • added unit tests
  • added integration tests

@ghanse ghanse added the enhancement New feature or request label Jan 28, 2025
@ghanse ghanse requested a review from a team as a code owner January 28, 2025 19:53
@ghanse ghanse requested review from tombonfert and removed request for a team January 28, 2025 19:53
Copy link

github-actions bot commented Jan 28, 2025

✅ 107/107 passed, 1 skipped, 27m10s total

Running from acceptance #265

Copy link
Contributor

@mwojtyczka mwojtyczka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mwojtyczka mwojtyczka merged commit daf37e4 into main Jan 29, 2025
8 checks passed
@mwojtyczka mwojtyczka deleted the feature_datetime_range_checks branch January 29, 2025 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE]: Add support for date and timestamp to range type of checks
2 participants