Adopt compose-specification for compose file support in docker stack command

[ndeloof]

https://compose-spec.io/ is now the place to define Compose file format and evolution. Changes like #2517 should not happen within this repo anymore without a sibling issue/PR on compose-spec.

http://github.com/compose-spec/compose-go was contributed as a git extraction for the types and loader packages from docker/cli. To avoid code to diverge, docker/cli should adopt compose-go ASAP

last but not least, using compose-go would make docker stack command compliant with the compose specification, and allow to mix docker compose config command with docker stack deploy (which seem to be a common pattern)

[smidge84]

Is there a way we can bump this issue?

It's been open for over a year and a half with no comments, but there's a steadily increasing list of related issues which I think shows desire for this issue to be resolved.

Is there a view of the Docker Cli roadmap and what work is in-scope?

[ndeloof]

compose-go is not just a drop-in replacement for legacy compose support in docker/cli, so this change will require some extra testing for backward compatibility.
#3257 will provide a temporary workaround

[Radiergummi]

So... different Docker components aren't compatible anymore, nobody thought of marking this as a breaking change, lots of users report this, and... nothing happens in more than two years?

I don't even know how it'd be possible to use Swarm secrets without versioning them, which in turn isn't possible without variables in the compose file, which in turn isn't possible with stack deploy, which in turn requires the config command to return a preprocessed version of the compose file, which in turn requires it to include a version or stack deploy to follow the spec.

As someone actually paying for Docker and using Swarm in production, I cannot wrap my head around why this happens, or why I actually continue to bother with this.
This is not targeted at anyone personally of course, but it's incredibly frustrating to hunt down problems in issues, leaving the impression of beta-grade software that is sold as production ready...

[ndeloof]

I don't even know how it'd be possible to use Swarm secrets without versioning them, which in turn isn't possible without variables in the compose file ...

A potential fix is for Swarm maintainers to introduce support for variables in the legacy compose file parser

[Radiergummi]

A potential fix is for Swarm maintainers to introduce support for variables in the legacy compose file parser

Please no. I don't trust them to not introduce a heap of new bugs and inconsistencies while doing so, making stuff even worse.

Would that include interpolation in curly braces? Trimming rules inherited from bash? Default values? What if the variable is defined, but an empty string? What if it is used in an illegal location? What if it breaks Yaml parsing?

I'd rather they just finally implement the compose spec proper and get this over with. But I guess that won't happen.

[dazinator]

I see in the compose spec, it's possible to create a secret from an env variable:

https://docs.docker.com/compose/compose-file/09-secrets/#example-2

I eagerly tried this with docker stack deploy but was greeted with: secrets.accesskey Additional property environment is not allowed

Please oh pretty please would some kind soul add support for the compose spec? This would resolve #4406

[slavnycoder]

Please fix it. It's difficult to use swarm. Need to use sed as workaround for converting published from string to int. Or add ability to use .env file to format docker-compose.yml file

[mbbyn]

btw, instead of sed we rely on yq, which is an additional dependency, but grants superpowers when dealing with YAML.

[Radiergummi]

A potential fix is for Swarm maintainers to introduce support for variables in the legacy compose file parser

@ndeloof forgive me for bugging you with this repeatedly, but I'd like to ask one more question: Who are those Swarm maintainers? The only people interacting with Swarm mode issues are people associated with the Docker org. Who are the people that actually maintain this project?

[kinghuang]

AFAIK, there's effectively no real Swarm maintainers these days. Mirantis provided support for two years after the enterprise spin off in 2019.

[Leopere]

well theres still very much a need for it and a good quantity of uses for it.

[ndeloof]

#4863

[melutovich]

@dperny Perhaps https://github.com/aaraney/deployx could be used to replace the existing docker swarm deploy ? [If this issue is in the wrong place, where should it be?]

[Radiergummi]

@dperny Perhaps https://github.com/aaraney/deployx could be used to replace the existing docker swarm deploy ? [If this issue is in the wrong place, where should it be?]

I had no idea that existed. Moving to a plugin would be very in line with the rest of Docker CLIENT IMHO. Thanks for sharing!