Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

poetry run fails with a keyerror for poetry 2.0.0 #9961

Closed
bjoernricks opened this issue Jan 6, 2025 · 8 comments · Fixed by python-poetry/poetry-core#805
Closed

poetry run fails with a keyerror for poetry 2.0.0 #9961

bjoernricks opened this issue Jan 6, 2025 · 8 comments · Fixed by python-poetry/poetry-core#805
Labels
kind/bug Something isn't working as expected status/confirmed Issue is reproduced and confirmed

Comments

@bjoernricks
Copy link

Description

With the latest poetry release 2.0.0 calling a script via poetry run always fails with a KeyError.

  KeyError

  'format'

  at ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/core/masonry/utils/module.py:79 in __init__
       75│             self._package_includes.append(
       76│                 PackageInclude(
       77│                     self._path,
       78│                     package["include"],
    →  79│                     formats=package["format"],
       80│                     source=package.get("from"),
       81│                     target=package.get("to"),
       82│                 )
       83│             )

Steps to reproduce.

  • Install poetry via pipx
  • Create a temporary directory
  • Change into the temporary directory
  • Copy the following pyproject.toml into that new directory
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "tmp"
version = "0.1.0"
description = ""
authors = ["Foo Bar <[email protected]>"]
readme = "README.md"

packages = [{ include = "somepackage" }]

[tool.poetry.scripts]
sometool = "somepackage:main"
  • Create a someproject/__init__.py file in that temporary directory with the following content
def main():
    print("Hello from somepackage")
  • Call poetry run sometool

Workarounds

None

Poetry Installation Method

pipx

Operating System

Ubuntu 24.04

Poetry Version

Poetry (version 2.0.0)

Poetry Configuration

cache-dir = "/home/bricks/.cache/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/bricks/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false


### Python Sysconfig

_No response_

### Example pyproject.toml

```TOML
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "tmp"
version = "0.1.0"
description = ""
authors = ["Foo Bar <[email protected]>"]
readme = "README.md"

packages = [{ include = "somepackage" }]

[tool.poetry.scripts]
sometool = "somepackage:main"

Poetry Runtime Logs

poetry run -vvv sometool

  KeyError

  'format'

  at ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/core/masonry/utils/module.py:79 in __init__
       75│             self._package_includes.append(
       76│                 PackageInclude(
       77│                     self._path,
       78│                     package["include"],
    →  79│                     formats=package["format"],
       80│                     source=package.get("from"),
       81│                     target=package.get("to"),
       82│                 )
@bjoernricks bjoernricks added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Jan 6, 2025
@finswimmer
Copy link
Member

finswimmer commented Jan 6, 2025

Thanks for reporting this @bjoernricks. I can confirm this issue.

This was introduced in poetry-core with python-poetry/poetry-core#773. @radoering can we simply change https://github.com/python-poetry/poetry-core/pull/773/files#diff-62e4eae7349a9699c84076b1a65f0371d26b188e396dcee122356110960adeffR78 to package.get("format")? I mean it works, but I'm not sure if we kill something else. (Tests are still running fine).

Edit: Hm, I think we also need to reintroduce this lines:

formats = package.get("format")
if formats and not isinstance(formats, list):
    formats = [formats]

@finswimmer finswimmer added status/confirmed Issue is reproduced and confirmed and removed status/triage This issue needs to be triaged labels Jan 6, 2025
honzajavorek added a commit to juniorguru/chick that referenced this issue Jan 6, 2025
honzajavorek added a commit to juniorguru/plucker that referenced this issue Jan 6, 2025
DinisCruz added a commit to owasp-sbot/OSBot-Utils that referenced this issue Jan 6, 2025
This has been long time coming (for example for performance reasons), but was triggered by this poetry update that crashed the CI pipeline:

 - https://github.com/owasp-sbot/OSBot-Utils/actions/runs/12632720785/job/35198284338
 - python-poetry/poetry#9961
 - python-poetry/poetry#9957
 - python-poetry/poetry-core#803
simonsmh added a commit to simonsmh/telegram-bili-feed-helper that referenced this issue Jan 6, 2025
simonsmh added a commit to simonsmh/telegram-bili-feed-helper that referenced this issue Jan 6, 2025
@anuragnatoo
Copy link

anuragnatoo commented Jan 6, 2025

Hi @bjoernricks, even I faced this issue, for now a quick workaround would be adding format in the packages
packages = [{ include = "somepackage", format = ["sdist"] }]

Also it's mentioned in the poetry documentation here that
If no format is specified, include defaults to only sdist.
image

I think this handling needs to be added where if format is not specified it should default to sdist

github-merge-queue bot pushed a commit to Significant-Gravitas/AutoGPT that referenced this issue Jan 6, 2025
- Resolves #9196

Make all changes necessary to make everything work with Poetry v2.0.0.

## Changes
- Removed `--no-update` flag from `poetry lock` command in codebase
- Removed extra path arguments from `poetry -C [path] run [command]`
occurrences
- Regenerated all lock files in hierarchical order
- Added workaround for Poetry bug where `packages.[i].format` is now
suddenly required

Additionally:
- Fixed up .dockerignore
  - Fixes .venv being erroneously copied over from local
  - Fixes build context bloat (300MB -> 2.5MB)
- Fixed warnings about entrypoint script not being installed in docker
builds

### Relevant (breaking) changes in v2.0.0
- `--no-update` flag no longer exists for `poetry lock` as it has become
default behavior
- The `-C` option now actually changes the directory, so any path
arguments in `poetry run` commands can/must be removed
- Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have
to be regenerated to avoid false-positive lock file updates and checks
on future PRs
- **BUG:** when specifying `poetry.tool.packages`, `format` is required
now (python-poetry/poetry#9961)

Full release notes and change log:
https://python-poetry.org/blog/announcing-poetry-2.0.0
github-merge-queue bot pushed a commit to Significant-Gravitas/AutoGPT that referenced this issue Jan 6, 2025
- Resolves #9196

Make all changes necessary to make everything work with Poetry v2.0.0.

## Changes
- Removed `--no-update` flag from `poetry lock` command in codebase
- Removed extra path arguments from `poetry -C [path] run [command]`
occurrences
- Regenerated all lock files in hierarchical order
- Added workaround for Poetry bug where `packages.[i].format` is now
suddenly required

Additionally:
- Fixed up .dockerignore
  - Fixes .venv being erroneously copied over from local
  - Fixes build context bloat (300MB -> 2.5MB)
- Fixed warnings about entrypoint script not being installed in docker
builds

### Relevant (breaking) changes in v2.0.0
- `--no-update` flag no longer exists for `poetry lock` as it has become
default behavior
- The `-C` option now actually changes the directory, so any path
arguments in `poetry run` commands can/must be removed
- Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have
to be regenerated to avoid false-positive lock file updates and checks
on future PRs
- **BUG:** when specifying `poetry.tool.packages`, `format` is required
now
  - python-poetry/poetry#9961

Full release notes and change log:
https://python-poetry.org/blog/announcing-poetry-2.0.0
github-merge-queue bot pushed a commit to Significant-Gravitas/AutoGPT that referenced this issue Jan 6, 2025
- Resolves #9196

Make all changes necessary to make everything work with Poetry v2.0.0.

## Changes
- Removed `--no-update` flag from `poetry lock` command in codebase
- Removed extra path arguments from `poetry -C [path] run [command]`
occurrences
- Regenerated all lock files in hierarchical order
- Added workaround for Poetry bug where `packages.[i].format` is now
suddenly required

Additionally:
- Fixed up .dockerignore
  - Fixes .venv being erroneously copied over from local
  - Fixes build context bloat (300MB -> 2.5MB)
- Fixed warnings about entrypoint script not being installed in docker
builds

### Relevant (breaking) changes in v2.0.0
- `--no-update` flag no longer exists for `poetry lock` as it has become
default behavior
- The `-C` option now actually changes the directory, so any path
arguments in `poetry run` commands can/must be removed
- Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have
to be regenerated to avoid false-positive lock file updates and checks
on future PRs
- **BUG:** when specifying `poetry.tool.packages`, `format` is required
now
  - python-poetry/poetry#9961

Full release notes and change log:
https://python-poetry.org/blog/announcing-poetry-2.0.0
github-merge-queue bot pushed a commit to Significant-Gravitas/AutoGPT that referenced this issue Jan 6, 2025
- Resolves #9196

Make all changes necessary to make everything work with Poetry v2.0.0.

## Changes
- Removed `--no-update` flag from `poetry lock` command in codebase
- Removed extra path arguments from `poetry -C [path] run [command]`
occurrences
- Regenerated all lock files in hierarchical order
- Added workaround for Poetry bug where `packages.[i].format` is now
suddenly required

Additionally:
- Fixed up .dockerignore
  - Fixes .venv being erroneously copied over from local
  - Fixes build context bloat (300MB -> 2.5MB)
- Fixed warnings about entrypoint script not being installed in docker
builds

### Relevant (breaking) changes in v2.0.0
- `--no-update` flag no longer exists for `poetry lock` as it has become
default behavior
- The `-C` option now actually changes the directory, so any path
arguments in `poetry run` commands can/must be removed
- Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have
to be regenerated to avoid false-positive lock file updates and checks
on future PRs
- **BUG:** when specifying `poetry.tool.packages`, `format` is required
now
  - python-poetry/poetry#9961

Full release notes and change log:
https://python-poetry.org/blog/announcing-poetry-2.0.0
Pwuts added a commit to Significant-Gravitas/AutoGPT that referenced this issue Jan 6, 2025
Make all changes necessary to make everything work with Poetry v2.0.0.

- Resolves #9196

## Changes
- Removed `--no-update` flag from `poetry lock` command in codebase
- Removed extra path arguments from `poetry -C [path] run [command]`
occurrences
- Regenerated all lock files in hierarchical order
- Added workaround for Poetry bug where `packages.[i].format` is now
suddenly required

Additionally:
- Fixed up .dockerignore
  - Fixes .venv being erroneously copied over from local
  - Fixes build context bloat (300MB -> 2.5MB)
- Fixed warnings about entrypoint script not being installed in docker
builds

### Relevant (breaking) changes in v2.0.0
- `--no-update` flag no longer exists for `poetry lock` as it has become
default behavior
- The `-C` option now actually changes the directory, so any path
arguments in `poetry run` commands can/must be removed
- Poetry v2.0.0 uses the new v2.1 lock file spec, so all lock files have
to be regenerated to avoid false-positive lock file updates and checks
on future PRs
- **BUG:** when specifying `poetry.tool.packages`, `format` is required
now
  - python-poetry/poetry#9961

Full Poetry v2.0.0 release notes and change log:
https://python-poetry.org/blog/announcing-poetry-2.0.0
honzajavorek added a commit to juniorguru/eggtray that referenced this issue Jan 7, 2025
@aliceinwire
Copy link

aliceinwire commented Jan 8, 2025

I don't know too much about poetry but I think it would be nice to trying to minimize problems on stable versions.
One option other than add more checks on pytest would be to do a dev/pre release on pypi repository before moving it to a stable release on major version changes,
so that the community can try it before going directly to stable
https://pypi.org/project/poetry/#history

https://peps.python.org/pep-0440/#developmental-releases

@aliceinwire
Copy link

Hi @bjoernricks, even I faced this issue, for now a quick workaround would be adding format in the packages packages = [{ include = "somepackage", format = ["sdist"] }]

Also it's mentioned in the poetry documentation here that If no format is specified, include defaults to only sdist. image

I think this handling needs to be added where if format is not specified it should default to sdist

I think defaulting to sdist is true only for 2.0.0
on 1.8.5 doing poetry build without any format option gives

$ poetry build
Building kci-dev (0.1.1)
  - Building sdist
  - Built kci_dev-0.1.1.tar.gz
  - Building wheel
  - Built kci_dev-0.1.1-py3-none-any.whl

with 2.0.0 with sdist only format gives

$ poetry build
Building kci-dev (0.1.1)
  - Building sdist
  - Built kci_dev-0.1.1.tar.gz
  - Building wheel

No file/folder found for package kci-dev

aliceinwire added a commit to aliceinwire/kci-dev that referenced this issue Jan 8, 2025
resolves kernelci#79

Added sdist and wheel as format on pyproject.toml as a temporary fix
for poetry format issue

bug: python-poetry/poetry#9961
Signed-off-by: Arisu Tachibana <[email protected]>
aliceinwire added a commit to aliceinwire/kci-dev that referenced this issue Jan 8, 2025
resolves kernelci#79

Added sdist and wheel as format on pyproject.toml as a temporary fix
for poetry format issue

bug: python-poetry/poetry#9961
Signed-off-by: Arisu Tachibana <[email protected]>
@bjoernricks
Copy link
Author

I suppose for packages include the default is [wheel, sdist].

@Atokulus
Copy link

Atokulus commented Jan 8, 2025

Dear all

I have the same issue and was quite perplex when greeted with a red 'format' message (there might seem to potential in improving the masonry logging in case of errors, as the exception stack was not printed on Windows). I searched quite some time in my own code before running poetry run --verbose my_script 🤦)

I have now added the appropriate key format to all my packages:

[tool.poetry]
packages = [
    { include = "package_a" },
    { include = "package_b" },
]

should now be:

[tool.poetry]
packages = [
    { include = "package_a", format = [
        "sdist",
        "wheel",
    ] },
    { include = "package_b", format = [
        "sdist",
        "wheel",
    ] },
]

Prior to the migration to Poetry 2.0, my package used to build to wheel by default in Poetry 1.8 and I depended upon this (maybe others as well). To not break compatibility I guess falling back to ["sdist", "wheel"] would be more appropriate than just ["sdist"].

What is/was the motivation behind the change to only build sdist? The current Poetry 2.0 release still uses sdist and wheel for the default package if not specified (see https://github.com/python-poetry/poetry-core/blame/5e5ae745d2f236f0758915b86f20d5638dad2d37/src/poetry/core/masonry/utils/module.py#L71)

Best regards
Markus

@flipbit03
Copy link

Faced the same issue here, which only happens if my project name = is different from the include = XX name, and the workaround is to simply specify format = sdist in your include packages, even though sdist is the default, from the documentation.

Copy link

github-actions bot commented Feb 9, 2025

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected status/confirmed Issue is reproduced and confirmed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants