Skip to content

Improve pytest.warns() docs to clarify difference with catch_warnings() #9002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nicoddemus opened this issue Aug 13, 2021 · 6 comments
Closed
Labels
plugin: warnings related to the warnings builtin plugin type: docs documentation improvement, missing or needing clarification
Milestone

Comments

@nicoddemus
Copy link
Member

We should add some stdlib links to the relevant docs:

  • If you want to get a list of warnings, but no error if no warnings were emitted, use warnings.catch_warnings()
  • To say "any warning here is unexpected / an error", use with catch_warnings(): simplefilter("error"); ...

I didn't intend to break anything, but nor did I consider that people might be using pytest.warns() like warnings.catch_warnings(). My apologies for the inconvenience!

Originally posted by @Zac-HD in #8677 (comment)

@nicoddemus nicoddemus added plugin: warnings related to the warnings builtin plugin type: docs documentation improvement, missing or needing clarification labels Aug 13, 2021
@Zac-HD Zac-HD added this to the 7.0 milestone Dec 7, 2021
@Zac-HD
Copy link
Member

Zac-HD commented Dec 7, 2021

Per #9386 we should also expand the deprecation message to suggest use of stdlib catch_warnings() for cases where you either allow or expect no warnings to be emitted.

@The-Compiler
Copy link
Member

I wonder if we should instead provide some kind of pytest convenience API for this case, though? I imagine something like pytest.warns(optional=True) or perhaps pytest.warns(raising=False). This would make it a bit easier for people to migrate and probably also make our deprecation guide simpler.

@WarrenWeckesser
Copy link

FYI: We need to replace a few occurrences of pytest.warns(None) in the SciPy tests (see scipy/scipy#15186), and we observe the same issue as noted in #9386: pytest.warns() in 7.0.0rc1 does not work the same as pytest.warns(None) in 6.2.5, so the message in the deprecation warning is a bit misleading.

WarrenWeckesser added a commit to WarrenWeckesser/scipy that referenced this issue Dec 9, 2021
…ngs`.

The use of `pytest.warns(None)` is deprecated in pytest 7.0.0.
The alternative suggested by the deprecation warning, `pytest.warns()`,
does not match the old behavior.  In fact, `pytest.warns()` is intended
specifically to raise an error if the code in the context does not raise
the given warning, and in pytest 7.0.0rc1, the default warning used by
`pytest.warns()` is the generic `Warning` class.

This issue is discussed in the following pytest issues:

* pytest-dev/pytest#9002
* pytest-dev/pytest#9386

There, the recommended alternative is to use `warnings.catch_warnings`.
That is the change made here.

Closes scipygh-15186
@bashtage
Copy link

bashtage commented Dec 10, 2021

It would be very helpful if the best way to handle runtime-dependent warnings is documented. In a nutshell, we have parameterized tests that warn under some parameterizations and do not warn under others. We have been using something like

expected_warn = FutureWarning if param == "n" else None
with pytest.warns(expected_warn):
    ...

This worked in the sense that it would ensure that FutureWarning was raised when it should even if it silenced any warning otherwise, and so was not strict.

What is the best way to handle a situation like this? I had thought of

expected_warn = [FutureWarning] if param == "n" else []
with pytest.warns(*expected_warn):
    ...

but it seems a bit unnatural, and I'm not even sure this would work given this thread and others.

@nicoddemus
Copy link
Member Author

Please see #9404.

tylerjereddy pushed a commit to tylerjereddy/scipy that referenced this issue Dec 19, 2021
…ngs`.

The use of `pytest.warns(None)` is deprecated in pytest 7.0.0.
The alternative suggested by the deprecation warning, `pytest.warns()`,
does not match the old behavior.  In fact, `pytest.warns()` is intended
specifically to raise an error if the code in the context does not raise
the given warning, and in pytest 7.0.0rc1, the default warning used by
`pytest.warns()` is the generic `Warning` class.

This issue is discussed in the following pytest issues:

* pytest-dev/pytest#9002
* pytest-dev/pytest#9386

There, the recommended alternative is to use `warnings.catch_warnings`.
That is the change made here.

Closes scipygh-15186
@nicoddemus
Copy link
Member Author

Fixed by #9495

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: warnings related to the warnings builtin plugin type: docs documentation improvement, missing or needing clarification
Projects
None yet
Development

No branches or pull requests

5 participants