-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Surprising test run order when test function reused #5163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I believe tests are attempted to be executed in definition order. The |
I figured that might be the case. Would we want to change this to run in the expected order*, or is the current behaviour desirable? *This is definitely not going to be a trivial change, and may well be more effort than it’s worth, but I’m willing to have a poke around if it’s worth it |
I'm not sure it's possible to support this for python<3.6 since it's not known where the assignment is from just the imported module (though maybe we could mark up the ast with an import hook? hmmmm) but once we're py36+ I think we can stop sorting and let the definition order handle it (due to ordered mapping types by default) Just curious -- is this breaking something for you? |
didn't mean to close, oops |
I have some incremental tests (end-to-end UI tests) where I wanted to reuse a class and just change one of the methods, so yes, but it isn’t a big deal. Might be something to revisit if/when we drop 2.7 and 3.5 support, but sounds like a no-go for now. I’m happy to close this until then. |
Agreed. Closing in favor of #5196 then. |
issue
When a class is composed form another class, the tests do not run in the order they are defined.
e.g:
I would expect the last 2 tests to be executed in the opposite order (i.e. TestBar::test_foo then TestBar::test_bar) since this is the order they appear in the file. I'm not sure this is a bug per se, but it's something I'd like to see documented if it's decided it should stay this way.
venv
MacOS 10.14.4 (Mojave)
The text was updated successfully, but these errors were encountered: