Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Improve type validation for bare types #8997

Merged
merged 7 commits into from
Mar 11, 2025

Conversation

mdrazak2001
Copy link
Contributor

@mdrazak2001 mdrazak2001 commented Mar 6, 2025

Related Issues

Proposed Changes:

  • Treat bare types (e.g., List, Dict) as generic types with Any arguments during type compatibility checks.

How did you test it?

  • added unit tests

Notes for the reviewer

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added unit tests and updated the docstrings
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@mdrazak2001 mdrazak2001 requested a review from a team as a code owner March 6, 2025 18:00
@mdrazak2001 mdrazak2001 requested review from sjrl and removed request for a team March 6, 2025 18:00
@mdrazak2001 mdrazak2001 requested a review from a team as a code owner March 6, 2025 18:02
@mdrazak2001 mdrazak2001 requested review from dfokina and removed request for a team March 6, 2025 18:02
@coveralls
Copy link
Collaborator

coveralls commented Mar 6, 2025

Pull Request Test Coverage Report for Build 13782703256

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 25 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.02%) to 89.94%

Files with Coverage Reduction New Missed Lines %
core/type_utils.py 1 95.83%
utils/type_serialization.py 10 86.08%
document_stores/in_memory/document_store.py 14 95.6%
Totals Coverage Status
Change from base Build 13702551434: -0.02%
Covered Lines: 9691
Relevant Lines: 10775

💛 - Coveralls

@sjrl
Copy link
Contributor

sjrl commented Mar 7, 2025

@mdrazak2001 thanks for your work on this! Could we also expand the test suite a bit to include

Nested checks:

@pytest.mark.parametrize(
    "sender_type,receiver_type",
    [
        # Nested Lists
        pytest.param(List[List[int]], List[List], id="nested-list-to-partially-bare-list"),
        pytest.param(List[List[int]], List, id="nested-list-to-bare-list"),
        # Nested Dicts
        pytest.param(Dict[str, Dict[str, int]], Dict[str, Dict], id="nested-dict-to-partially-bare-dict"),
        pytest.param(Dict[str, Dict[str, int]], Dict, id="nested-dict-to-bare-dict"),
        # Nested Sets
        pytest.param(Set[Set[int]], Set[Set], id="nested-set-to-partially-bare-set"),
        pytest.param(Set[Set[int]], Set, id="nested-set-to-bare-set"),
        # Nested Tuples
        pytest.param(Tuple[Tuple[int, str], bool], Tuple[Tuple, bool], id="nested-tuple-to-partially-bare-tuple"),
        pytest.param(Tuple[Tuple[int, str], bool], Tuple, id="nested-tuple-to-bare-tuple"),
        # Mixed nesting
        pytest.param(List[Dict[str, int]], List[Dict], id="list-of-dict-to-list-of-bare-dict"),
        pytest.param(Dict[str, List[int]], Dict[str, List], id="dict-with-list-to-dict-with-bare-list"),
        pytest.param(Set[Tuple[int, str]], Set[Tuple], id="set-of-tuple-to-set-of-bare-tuple"),
        pytest.param(Tuple[List[int], Dict[str, bool]], Tuple[List, Dict], id="tuple-of-containers-to-tuple-of-bare-containers"),
        # Triple nesting
        pytest.param(List[List[List[int]]], List[List[List]], id="triple-nested-list-to-partially-bare"),
        pytest.param(List[List[List[int]]], List[List], id="triple-nested-list-to-double-bare"),
        pytest.param(List[List[List[int]]], List, id="triple-nested-list-to-completely-bare"),
    ],
)
def test_nested_container_compatibility(sender_type, receiver_type):
    assert _types_are_compatible(sender_type, receiver_type)
    # Bare container types should not be compatible with their typed counterparts
    assert not _types_are_compatible(receiver_type, sender_type)

@mdrazak2001
Copy link
Contributor Author

@sjrl: I merged the following if conditional block into the final return statement because the function had too many returns, which was causing it to fail to commit.

if len(sender_args) > len(receiver_args):
        return False


return all(_strict_types_are_compatible(*args) for args in zip(sender_args, receiver_args))

def _check_callable_compatibility(sender_args, receiver_args):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work on this, but for now let's not worry about the callable compatibility and do this in a separate PR so we can focus on just the other bare types. Could you open a new issue/PR with these changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, @sjrl , I will include callable compatibility in a separate PR. However, I noticed a minor limitation in the current implementation for bare types: even Callable bare types require special handling via _check_callable_compatibility due to their unique structure. For example, the test case pytest.param(Callable[[Any], Any], Callable) fails with the current implementation (without _check_callable_compatibility).

Would it be acceptable to exclude Callable bare types from this PR and instead address all cases involving them in the separate PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Would it be acceptable to exclude Callable bare types from this PR and instead address all cases involving them in the separate PR?

Yes that sounds good to me.

pytest.param(List[int], List, id="list-of-primitive-to-bare-list"),
pytest.param(Dict[str, int], Dict, id="dict-of-primitive-to-bare-dict"),
pytest.param(Set[float], Set, id="set-of-primitive-to-bare-set"),
pytest.param(Tuple[int, str], Tuple, id="tuple-of-primitive-to-bare-tuple"),
pytest.param((List[int]), list, id="list-of-primitive-to-list-object"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one needs to be put into a separate test for now we expect it to not work.

@mdrazak2001 mdrazak2001 requested a review from sjrl March 10, 2025 17:53
Copy link
Contributor

@sjrl sjrl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work on this!

@sjrl sjrl self-assigned this Mar 11, 2025
@sjrl sjrl merged commit 7291134 into deepset-ai:main Mar 11, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve type validation to handle bare typing objects
3 participants