Docker compose config omitting the compose file version element

[smidge84]

Description

When using Docker compose config to interpolate service definitions into a single flat stack file, the output omits the "version" field which specifies the compose file version. The is required for deployment, either via Docker compose of Docker swarm, resulting in the following error message:

"unsupported Compose file version: 1.0"

This is because, without the "version" field in the compose file, Docker default back to compose file version 1.0

Steps to reproduce the issue:

1. Install Docker Desktop for Mac (version 4.4.2), with Docker compose V2.2.3
2. Ensure that "Use Docker Compose V2" is enabled in the General settings
3. Interpolate a stack file using the docker compose config command
4. Deploy the interpolated stack file

Describe the results you received:

Outcome received:

• "unsupported Compose file version: 1.0"

Describe the results you expected:

Expected outcome:

• The stack deploys as expected

Additional information you deem important (e.g. issue happens only occasionally):

This issue happens consistently every time.

Output of docker compose version:

Docker Compose version v2.2.3

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.7.1)
  compose: Docker Compose (Docker Inc., v2.2.3)
  scan: Docker Scan (Docker Inc., v0.16.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 217
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: active
  NodeID: 28rbg4r10zzhtsnn5myfv8vzy
  Is Manager: true
  ClusterID: 8zpel2girwhhqt3zgoskc610l
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.65.3
  Manager Addresses:
   192.168.65.3:2377
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 6.789GiB
 Name: docker-desktop
 ID: BKDG:IFZT:G4WM:4NQN:2PVA:QRKX:MZUO:OGFN:YZMV:OFJX:P276:XEDJ
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 58
  Goroutines: 160
  System Time: 2022-01-31T14:16:09.1871698Z
  EventsListeners: 4
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

macOS Monterey 12.2

[Kangaroux]

Interpolate a stack file using the docker compose config command

I'm a bit confused, what's your goal? If you need the compose file contents you can use cat docker-compose.yml

[ndeloof]

Docker Compose adopted the Compose Specification, which makes the version element obsolete and irrelevant.

[robertbartel]

This is likely a duplicate of #8950, and it relates back to this issue.

[smidge84]

Thanks @ndeloof and @robertbartel.
Sorry for the duplication of the issue, I did try searching for tickets before I posted, maybe I just didn't use the right combination of keywords to find the previous issue which you liked to #8950.

Robert, you got it exactly right. We use docker-compose config to interpolate all our environment variables before we then deploy the interpolated stack file into a Docker Swarm.
I feel this is a very common practice used around the world.
However, if Docker Compose has moved on, can someone please highlight the proposed way of working now.

So basically, there's been a divergence in the code bases here.
From what I'm aware, support for Docker Swarm hasn't been deprecated to the changes to Docker Compose should not have removed something which another component requires.

We're using Docker Compose file format version 3.x (3.4 specifically, but we could probably upgrade that not), which is supported by Docker Engines 17.09.0+ (according to this compatibility matrix).

@ndeloof I don't get what you are referring to about this "Compose Specification"? Please can you link to more information.
It looks clear that whilst Docker Compose might have moved on to this "specification" the Docker CLI has not.

In the mean time, I'll downgrade back to Docker Compose v1.26.2, which is the last release for the "Compose Specification" was adopted in v1.27.0 (source).

[ndeloof]

@smidge84 https://compose-spec.io/

docker stack deploy indeed didn't adopted the spec, and is still stuck with legacy versioned schema.
But I wonder about your use-case, why don't you just pass your compose files to docker stack deploy and ask compose to pre-process those?

[smidge84]

Thanks @ndeloof.

We like to interpolate all our compose files into a single "flat/interpolated" file for the following reasons:

1. This means we have a single immutable artefact which will always be the same not matter when you deploy it. This is extremely useful when you need to troubleshoot a historical stack as it ensures you have the same version as the customer experiencing the issue.
2. In our use case, our deployment environment is an air-gapped system, which means we need to package everything up and transport it to the system manually. This means that all the Environment Variables which toggle the configuration of the stack are not present in the environment where the stack is being deployed, so it is essential that we interpolate all the environment variables from our build/release environment before transporting the release artefact to the target system.

Does that help you understand the context a bit more? I hope I've managed to explain this is a clear way.

Also, thanks for the immediate reply btw.

[smidge84]

@ndeloof

Why have you Closed this issue? We haven't finished discussing yet.
Please re-open the issue.

[ndeloof]

closed as a duplicate for docker/cli#2527

[smidge84]

Thanks @ndeloof, I understand now, and thanks for the link to the open issue on the Docker Cli project.
I'll be watching that a lot to see if it gets resolved.

For now, I'll downgrade to Docker Compose v1.26.2 until the open issue is resolved.

[smidge84]

@thaJeztah Has just commented in issue #2475 that the following Pull Request might resolve the problem we're seeing:

• cli/compose: add schema 3.10, with optional version field (default to "latest") #3257