gh-103053: Skip test_freeze_simple_script() on PGO build #109591

vstinner · 2023-09-20T00:35:51Z

Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags.

test.pythoninfo now gets also get_build_info() of
test.libregrtests.utils.

Issue: test_tools: test_freeze_simple_script() times out after 15 min: --with-lto --enable-optimizations (s390x LTO buildbots) #103053

Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils.

vstinner · 2023-09-20T00:39:07Z

The previous attempt PR #103170 has two issues:

It skipped the test depending on the CPU architecture: only on s390x.
It checks for support.PGO, but this is the --pgo command line option used by PROFILE_TASK: to build Python (PROFILE_TASK runs the test suite with --pgo), not to run tests. When the buildbot runs the test suite, it doesn't pass --pgo.

I prefer to now make a skip dependent on the CPU architecture.

vstinner · 2023-09-20T00:40:43Z

If later we see that LTO also makes test_freeze_simple_script() too slow, we can also skip it if LTO is used.

miss-islington · 2023-09-20T01:58:38Z

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

miss-islington · 2023-09-20T01:58:40Z

Sorry, @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 81cd1bd713624c3d26b647f3d28f2fd905887a0d 3.12

miss-islington · 2023-09-20T01:58:43Z

Sorry, @vstinner, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 81cd1bd713624c3d26b647f3d28f2fd905887a0d 3.11

bedevere-bot · 2023-09-20T02:31:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 3.x has failed when building commit 81cd1bd.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/185/builds/5048) and take a look at the build logs.
Check if the failure is related to this commit (81cd1bd) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/185/builds/5048

Failed tests:

test.test_asyncio.test_subprocess

Failed subtests:

test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: [('pi[29 chars]t'), 'pipe_connection_lost', ('pipe_data_recei[57 chars]ted'] != [('pi[29 chars]t'), ('pipe_data_received', 2, b'stderr'), 'pi[57 chars]ted']

bedevere-bot · 2023-09-20T03:57:24Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO + PGO 3.x has failed when building commit 81cd1bd.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/43/builds/4256) and take a look at the build logs.
Check if the failure is related to this commit (81cd1bd) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/43/builds/4256

Failed tests:

test.test_asyncio.test_subprocess

Failed subtests:

test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 788, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le.lto-pgo/build/Lib/test/test_asyncio/test_subprocess.py", line 780, in main
    self.assertEqual(events, [
AssertionError: Lists differ: ['process_exited', ('pipe_data_received', 1, b'stdout')] != [('pipe_data_received', 1, b'stdout'), ('p[95 chars]ted']

bedevere-bot · 2023-09-20T04:22:17Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 LTO 3.x has failed when building commit 81cd1bd.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/503/builds/3932) and take a look at the build logs.
Check if the failure is related to this commit (81cd1bd) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/503/builds/3932

Failed tests:

test.test_multiprocessing_spawn.test_manager

Summary of the results of the build (if available):

==

Click to see traceback logs

Note: checking out '81cd1bd713624c3d26b647f3d28f2fd905887a0d'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 81cd1bd... gh-103053: Skip test_freeze_simple_script() on PGO build (#109591)
Switched to and reset branch 'main'

./Include/internal/pycore_semaphore.h:54:1: warning: type of ‘_PySemaphore_Wakeup’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wakeup(_PySemaphore *sema);
 ^
Python/parking_lot.c:197:1: note: ‘_PySemaphore_Wakeup’ was previously declared here
 _PySemaphore_Wakeup(_PySemaphore *sema)
 ^
Python/parking_lot.c:197:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:57:18: warning: type of ‘_PySemaphore_Init’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Init(_PySemaphore *sema);
                  ^
Python/parking_lot.c:53:1: note: ‘_PySemaphore_Init’ was previously declared here
 _PySemaphore_Init(_PySemaphore *sema)
 ^
Python/parking_lot.c:53:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:49:1: warning: type of ‘_PySemaphore_Wait’ does not match original declaration [-Wlto-type-mismatch]
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout_ns, int detach);
 ^
Python/parking_lot.c:178:1: note: ‘_PySemaphore_Wait’ was previously declared here
 _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout, int detach)
 ^
Python/parking_lot.c:178:1: note: code may be misoptimized unless -fno-strict-aliasing is used
./Include/internal/pycore_semaphore.h:58:18: warning: type of ‘_PySemaphore_Destroy’ does not match original declaration [-Wlto-type-mismatch]
 PyAPI_FUNC(void) _PySemaphore_Destroy(_PySemaphore *sema);
                  ^
Python/parking_lot.c:81:1: note: ‘_PySemaphore_Destroy’ was previously declared here
 _PySemaphore_Destroy(_PySemaphore *sema)
 ^
Python/parking_lot.c:81:1: note: code may be misoptimized unless -fno-strict-aliasing is used

make: *** [Makefile:2032: buildbottest] Error 5

…n#109591) Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils. (cherry picked from commit 81cd1bd)

bedevere-app · 2023-09-20T15:18:52Z

GH-109614 is a backport of this pull request to the 3.12 branch.

bedevere-app · 2023-09-20T15:22:46Z

GH-109616 is a backport of this pull request to the 3.11 branch.

) (#109616) gh-103053: Skip test_freeze_simple_script() on PGO build (#109591) Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils. (cherry picked from commit 81cd1bd)

…n#109591) Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils.

) (#109614) gh-103053: Skip test_freeze_simple_script() on PGO build (#109591) Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils. (cherry picked from commit 81cd1bd)

…n#109591) Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils.

bedevere-app bot added the awaiting review label Sep 20, 2023

bedevere-app bot mentioned this pull request Sep 20, 2023

test_tools: test_freeze_simple_script() times out after 15 min: --with-lto --enable-optimizations (s390x LTO buildbots) #103053

Closed

vstinner force-pushed the skip_freeze_pgo branch from a4c39c9 to 4d93f9f Compare September 20, 2023 00:36

vstinner added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 20, 2023

vstinner mentioned this pull request Sep 20, 2023

gh-103053: Skip the whole test_freeze module in tests on s390x with PGO #103170

Closed

vstinner enabled auto-merge (squash) September 20, 2023 01:40

vstinner merged commit 81cd1bd into python:main Sep 20, 2023

vstinner deleted the skip_freeze_pgo branch September 20, 2023 01:58

bedevere-app bot removed the awaiting review label Sep 20, 2023

miss-islington assigned vstinner Sep 20, 2023

bedevere-app bot removed the needs backport to 3.12 only security fixes label Sep 20, 2023

bedevere-app bot removed the needs backport to 3.11 only security fixes label Sep 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-103053: Skip test_freeze_simple_script() on PGO build #109591

gh-103053: Skip test_freeze_simple_script() on PGO build #109591

Uh oh!

vstinner commented Sep 20, 2023 •

edited by bedevere-app bot

Loading

Uh oh!

vstinner commented Sep 20, 2023

Uh oh!

vstinner commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

bedevere-bot commented Sep 20, 2023

Uh oh!

bedevere-bot commented Sep 20, 2023

Uh oh!

bedevere-bot commented Sep 20, 2023

Uh oh!

bedevere-app bot commented Sep 20, 2023

Uh oh!

bedevere-app bot commented Sep 20, 2023

Uh oh!

Uh oh!

Uh oh!

gh-103053: Skip test_freeze_simple_script() on PGO build #109591

gh-103053: Skip test_freeze_simple_script() on PGO build #109591

Uh oh!

Conversation

vstinner commented Sep 20, 2023 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Sep 20, 2023

Uh oh!

vstinner commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

miss-islington commented Sep 20, 2023

Uh oh!

bedevere-bot commented Sep 20, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 20, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Sep 20, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-app bot commented Sep 20, 2023

Uh oh!

bedevere-app bot commented Sep 20, 2023

Uh oh!

Uh oh!

vstinner commented Sep 20, 2023 •

edited by bedevere-app bot

Loading