History in base image is reset when registry uses manifest V2.1 such as Quay

[chanseokoh]

While investgating #1627, I noticed that history of all the layers in the base image is reset if the registry is returning manifest V.21. Check "auto-generated by Jib" below:

8dec7460fca9        44 minutes ago      jib-maven-plugin:1.1.1-SNAPSHOT   4.57kB              extra files
<missing>           44 minutes ago      jib-maven-plugin:1.1.1-SNAPSHOT   1.37kB              classes
<missing>           44 minutes ago      jib-maven-plugin:1.1.1-SNAPSHOT   5B                  resources
<missing>           44 minutes ago      jib-maven-plugin:1.1.1-SNAPSHOT   2.8MB               dependencies
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        658B                auto-generated by Jib
<missing>           44 minutes ago                                        11.4kB              auto-generated by Jib
<missing>           44 minutes ago                                        658B                auto-generated by Jib
<missing>           44 minutes ago                                        249MB               auto-generated by Jib
<missing>           44 minutes ago                                        639B                auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        0B                  auto-generated by Jib
<missing>           44 minutes ago                                        1.65kB              auto-generated by Jib
<missing>           44 minutes ago                                        203MB               auto-generated by Jib

This is because we are not calling addHistory() in JsonToImageTranslato.rtoImage(V21ManifestTemplate). Low priority because it happens only when the registry is using the old schema and the history is basically informative comments.

[chanseokoh]

See #1641, which is closely related. History is low priority though.

[chanseokoh]

Note that #1644 doesn't fix this at least for the example in #1641. The container config JSON in the history.v1Compatibility field in the V2.1 manifest doesn't have history entries.

For example, with the following Dockerfile,

FROM quay.io/tremolosecurity/myvirtualdirectory-base:latest
LABEL something=new

$ docker history test gives

IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
0d6135882251        About a minute ago   /bin/sh -c #(nop)  LABEL something=new          0B                  
d39cd3e7f332        12 days ago          /bin/sh -c #(nop)  LABEL io.openshift.build.…   0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  ENV OPENSHIFT_BUILD_NAME=…   0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  CMD ["usage"]                0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  EXPOSE 10636/tcp             0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  EXPOSE 10389/tcp             0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  USER [431]                   0B                  
<missing>           12 days ago          /bin/sh -c chown -R myvd:myvd     /etc/myvd …   658B                
<missing>           12 days ago          /bin/sh -c #(nop) COPY file:23008a3686e7f955…   11.4kB              
<missing>           12 days ago          /bin/sh -c #(nop) ADD file:6dbb519f74a1f2e19…   658B                
<missing>           12 days ago          /bin/sh -c yum -y install curl java-1.8.0-op…   249MB               
<missing>           12 days ago          /bin/sh -c #(nop) ADD file:f7382511150bd66d1…   639B                
<missing>           12 days ago          /bin/sh -c #(nop)  LABEL name=MyVirtualDirec…   0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  ENV BUILDER_VERSION=1.0 J…   0B                  
<missing>           12 days ago          /bin/sh -c #(nop)  MAINTAINER Tremolo Securi…   0B                  
<missing>           3 months ago                                                         1.65kB              
<missing>           3 months ago                                                         203MB               Imported from -

The V2.1 manifest in this case is

   "history": [
      {  
         "v1Compatibility": "{\"architecture\":\"amd64\",\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"config\":{\"Hostname\":\"generic-buildconfig-3-build\",\"Domainname\":\"\",\"User\":\"431\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"ExposedPorts\":{\"10389/tcp\":{},\"10636/tcp\":{}},\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"container=oci\",\"BUILDER_VERSION=1.0\",\"JDK_VERSION=1.8.0\",\"OPENSHIFT_BUILD_NAME=generic-buildconfig-3\",\"OPENSHIFT_BUILD_NAMESPACE=ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6\",\"OPENSHIFT_BUILD_SOURCE=https://github.com/TremoloSecurity/myvd-jib-builder.git\",\"OPENSHIFT_BUILD_REFERENCE=master\",\"OPENSHIFT_BUILD_COMMIT=fdc3ce38fe184d8e07c659a7011fadece4a18a47\"],\"Cmd\":[\"usage\"],\"ArgsEscaped\":true,\"Image\":\"sha256:1fc51e093597b53b875903ac0296644c1aa204231af981f4a6a7ac2905a2f38e\",\"Volumes\":null,\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":[],\"Labels\":{\"architecture\":\"x86_64\",\"authoritative-source-url\":\"registry.access.redhat.com\",\"build-date\":\"2019-01-22T20:02:12.681170\",\"com.redhat.build-host\":\"cpt-0012.osbs.prod.upshift.rdu2.redhat.com\",\"com.redhat.component\":\"ubi7-container\",\"description\":\"MyVirtualDirectory is an LDAP virtual directory and LDAP proxy\",\"distribution-scope\":\"public\",\"io.k8s.description\":\"Builder for MyVD\",\"io.k8s.display-name\":\"MyVirtualDirectory\",\"io.openshift.build.commit.author\":\"Marc Boorshtein \\\\[email protected]\\\\u003e\",\"io.openshift.build.commit.date\":\"Thu Apr 11 05:32:18 2019 -0700\",\"io.openshift.build.commit.id\":\"fdc3ce38fe184d8e07c659a7011fadece4a18a47\",\"io.openshift.build.commit.message\":\"Merge pull request #4 from mlbiam/master\",\"io.openshift.build.commit.ref\":\"master\",\"io.openshift.build.name\":\"generic-buildconfig-3\",\"io.openshift.build.namespace\":\"ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6\",\"io.openshift.build.source-location\":\"https://github.com/TremoloSecurity/myvd-jib-builder.git\",\"io.openshift.expose-services\":\"10389:10636\",\"io.openshift.tags\":\"ldap,virtual directory,identity management\",\"name\":\"MyVirtualDirectory Base\",\"release\":\"20190411\",\"run\":\"do not run this image, use it as a base for building a custom MyVirtualDirectory build\",\"summary\":\"MyVirtualDirectory base image\",\"url\":\"https://www.tremolosecurity.com/myvirtualdirectory/\",\"vcs-ref\":\"8ff3e2f260c10576fabc5c6daca1884ea06b2c5b\",\"vcs-type\":\"git\",\"vendor\":\"Tremolo Security, Inc.\",\"version\":\"1.0.0\"}},\"container\":\"a2004bd2e13fe832b03bd8154504b95465289f2c257d59775169e093dbf49c64\",\"container_config\":{\"Hostname\":\"generic-buildconfig-3-build\",\"Domainname\":\"\",\"User\":\"431\",\"AttachStdin\":false,\"AttachStdout\":false,\"AttachStderr\":false,\"ExposedPorts\":{\"10389/tcp\":{},\"10636/tcp\":{}},\"Tty\":false,\"OpenStdin\":false,\"StdinOnce\":false,\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\",\"container=oci\",\"BUILDER_VERSION=1.0\",\"JDK_VERSION=1.8.0\",\"OPENSHIFT_BUILD_NAME=generic-buildconfig-3\",\"OPENSHIFT_BUILD_NAMESPACE=ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6\",\"OPENSHIFT_BUILD_SOURCE=https://github.com/TremoloSecurity/myvd-jib-builder.git\",\"OPENSHIFT_BUILD_REFERENCE=master\",\"OPENSHIFT_BUILD_COMMIT=fdc3ce38fe184d8e07c659a7011fadece4a18a47\"],\"Cmd\":[\"/bin/sh\",\"-c\",\"#(nop) \",\"LABEL io.openshift.build.commit.author=Marc Boorshtein \\\\[email protected]\\\\u003e io.openshift.build.commit.date=Thu Apr 11 05:32:18 2019 -0700 io.openshift.build.commit.id=fdc3ce38fe184d8e07c659a7011fadece4a18a47 io.openshift.build.commit.message=Merge pull request #4 from mlbiam/master io.openshift.build.commit.ref=master io.openshift.build.name=generic-buildconfig-3 io.openshift.build.namespace=ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6 io.openshift.build.source-location=https://github.com/TremoloSecurity/myvd-jib-builder.git\"],\"ArgsEscaped\":true,\"Image\":\"sha256:1fc51e093597b53b875903ac0296644c1aa204231af981f4a6a7ac2905a2f38e\",\"Volumes\":null,\"WorkingDir\":\"\",\"Entrypoint\":null,\"OnBuild\":[],\"Labels\":{\"architecture\":\"x86_64\",\"authoritative-source-url\":\"registry.access.redhat.com\",\"build-date\":\"2019-01-22T20:02:12.681170\",\"com.redhat.build-host\":\"cpt-0012.osbs.prod.upshift.rdu2.redhat.com\",\"com.redhat.component\":\"ubi7-container\",\"description\":\"MyVirtualDirectory is an LDAP virtual directory and LDAP proxy\",\"distribution-scope\":\"public\",\"io.k8s.description\":\"Builder for MyVD\",\"io.k8s.display-name\":\"MyVirtualDirectory\",\"io.openshift.build.commit.author\":\"Marc Boorshtein \\\\[email protected]\\\\u003e\",\"io.openshift.build.commit.date\":\"Thu Apr 11 05:32:18 2019 -0700\",\"io.openshift.build.commit.id\":\"fdc3ce38fe184d8e07c659a7011fadece4a18a47\",\"io.openshift.build.commit.message\":\"Merge pull request #4 from mlbiam/master\",\"io.openshift.build.commit.ref\":\"master\",\"io.openshift.build.name\":\"generic-buildconfig-3\",\"io.openshift.build.namespace\":\"ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6\",\"io.openshift.build.source-location\":\"https://github.com/TremoloSecurity/myvd-jib-builder.git\",\"io.openshift.expose-services\":\"10389:10636\",\"io.openshift.tags\":\"ldap,virtual directory,identity management\",\"name\":\"MyVirtualDirectory Base\",\"release\":\"20190411\",\"run\":\"do not run this image, use it as a base for building a custom MyVirtualDirectory build\",\"summary\":\"MyVirtualDirectory base image\",\"url\":\"https://www.tremolosecurity.com/myvirtualdirectory/\",\"vcs-ref\":\"8ff3e2f260c10576fabc5c6daca1884ea06b2c5b\",\"vcs-type\":\"git\",\"vendor\":\"Tremolo Security, Inc.\",\"version\":\"1.0.0\"}},\"created\":\"2019-04-11T12:35:48.142248384Z\",\"docker_version\":\"1.13.1\",\"id\":\"30160f6a8d8dd32bd5097a6e8d368bf034db84d58366b50686cf889f2455ba50\",\"os\":\"linux\",\"parent\":\"6edf4d31e956e3d1fafabb6af2c682c11ab13456a0f655b0e9fe4c8b0ec4fdf7\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"6edf4d31e956e3d1fafabb6af2c682c11ab13456a0f655b0e9fe4c8b0ec4fdf7\",\"parent\":\"3e1f6831f1b3505bda628bd16066010a274a90e9a1c83c557410442752cca254\",\"created\":\"2019-04-11T12:35:48.045330146Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  ENV OPENSHIFT_BUILD_NAME=generic-buildconfig-3 OPENSHIFT_BUILD_NAMESPACE=ospid-5b00d266-75c8-4d31-99cc-de52007ae4a6 OPENSHIFT_BUILD_SOURCE=https://github.com/TremoloSecurity/myvd-jib-builder.git OPENSHIFT_BUILD_REFERENCE=master OPENSHIFT_BUILD_COMMIT=fdc3ce38fe184d8e07c659a7011fadece4a18a47\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"3e1f6831f1b3505bda628bd16066010a274a90e9a1c83c557410442752cca254\",\"parent\":\"1c31c1b96dcee6cee9f87efe1453dc1a0137a61d512ea664e8d4881d3bd4725f\",\"created\":\"2019-04-11T12:35:47.949845255Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  CMD [\\\"usage\\\"]\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {
         "v1Compatibility": "{\"id\":\"1c31c1b96dcee6cee9f87efe1453dc1a0137a61d512ea664e8d4881d3bd4725f\",\"parent\":\"c89f7f5e3969e987381bbf55dba9bc9cd624cb6add4a5076185601f6f177b5bf\",\"created\":\"2019-04-11T12:35:47.857045079Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  EXPOSE 10636/tcp\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {
         "v1Compatibility": "{\"id\":\"c89f7f5e3969e987381bbf55dba9bc9cd624cb6add4a5076185601f6f177b5bf\",\"parent\":\"f04c9f1e07293cb5cc7e14a2eaa28b66018f70b817b9eb0987cad1873cb6c631\",\"created\":\"2019-04-11T12:35:47.756437475Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  EXPOSE 10389/tcp\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"f04c9f1e07293cb5cc7e14a2eaa28b66018f70b817b9eb0987cad1873cb6c631\",\"parent\":\"0c070fea72b0c56f6f039332818849a681f6244a240a2b58c5698ca0d9faa9ad\",\"created\":\"2019-04-11T12:35:47.668449288Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  USER [431]\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"0c070fea72b0c56f6f039332818849a681f6244a240a2b58c5698ca0d9faa9ad\",\"parent\":\"8bfa1bfaaf5b1455654d09c725faf5422828a34e1324233c2e51a1d892263f91\",\"created\":\"2019-04-11T12:35:47.567387787Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c chown -R myvd:myvd     /etc/myvd     /etc/myvd-local     /usr/local/myvd   \\u0026\\u0026 chmod +x /usr/local/myvd/bin/*\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"}"
      },
      {  
         "v1Compatibility": "{\"id\":\"8bfa1bfaaf5b1455654d09c725faf5422828a34e1324233c2e51a1d892263f91\",\"parent\":\"5b7c55814a6e8b29b6ef044c7e451213c2d46f8e5d8debe99fc825484cc05977\",\"created\":\"2019-04-11T12:35:47.250471194Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop) COPY file:23008a3686e7f955c7bce68081c843c43e1bb4afa230c123e56fef8e19873642 in /licenses \"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"}"
      },
      {  
         "v1Compatibility": "{\"id\":\"5b7c55814a6e8b29b6ef044c7e451213c2d46f8e5d8debe99fc825484cc05977\",\"parent\":\"2742adc6f805f707c49680755bf78a4a66e50f2ce2dd80f82cb0b2aa4f6e266c\",\"created\":\"2019-04-11T12:35:47.117212679Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop) ADD file:6dbb519f74a1f2e1963ffb96d1ea611d4b447d0f1bf38bf6ebf83aa69c40d3e0 in /usr/local/myvd/bin/run_myvd.sh \"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"}"
      },
      {  
         "v1Compatibility": "{\"id\":\"2742adc6f805f707c49680755bf78a4a66e50f2ce2dd80f82cb0b2aa4f6e266c\",\"parent\":\"c8d81ccdc09f4781678912f5d699f8b374d28319c2ab28cf48baf129202a86dc\",\"created\":\"2019-04-11T12:35:46.730580097Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c yum -y install curl java-1.8.0-openjdk-headless.x86_64 wget unzip;yum -y upgrade;yum clean all \\u0026\\u0026     mkdir -p /etc/myvd \\u0026\\u0026     mkdir -p /etc/myvd-local \\u0026\\u0026     mkdir -p /usr/local/myvd \\u0026\\u0026     groupadd -r myvd -g 433 \\u0026\\u0026     useradd -u 431 -r -g myvd -d /usr/local/myvd -s /sbin/nologin -c \\\"MyVD Docker image user\\\" myvd \\u0026\\u0026     mkdir -p /usr/local/myvd/work \\u0026\\u0026     mkdir -p /usr/local/myvd/bin \\u0026\\u0026     mkdir /licenses \\u0026\\u0026     rm -rf /var/cache/yum\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"}"
      },
      {  
         "v1Compatibility": "{\"id\":\"c8d81ccdc09f4781678912f5d699f8b374d28319c2ab28cf48baf129202a86dc\",\"parent\":\"58e84dbc348cc03152ed033916f5f5e51b39e9ed365d9b194b58bee2e68ad097\",\"created\":\"2019-04-11T12:33:47.769129794Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop) ADD file:f7382511150bd66d166eff578eab6c440a5765e0304fbedb0051eed2eb7235c4 in /tmp/help.md \"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"}"
      },
      {
         "v1Compatibility": "{\"id\":\"58e84dbc348cc03152ed033916f5f5e51b39e9ed365d9b194b58bee2e68ad097\",\"parent\":\"46b7dd86c7e84043b90598524c03a61e0307c25d43be2543bd1648e917a69784\",\"created\":\"2019-04-11T12:33:47.635121367Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  LABEL name=MyVirtualDirectory Base vendor=Tremolo Security, Inc. version=1.0.0 release=20190411 url=https://www.tremolosecurity.com/myvirtualdirectory/ summary=MyVirtualDirectory base image description=MyVirtualDirectory is an LDAP virtual directory and LDAP proxy run=do not run this image, use it as a base for building a custom MyVirtualDirectory build io.k8s.description=Builder for MyVD io.k8s.display-name=MyVirtualDirectory io.openshift.expose-services=10389:10636 io.openshift.tags=ldap,virtual directory,identity management\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"46b7dd86c7e84043b90598524c03a61e0307c25d43be2543bd1648e917a69784\",\"parent\":\"13a1ea149358c5760741587e94df4f723443189da1701e874fd70bfce90a5c5e\",\"created\":\"2019-04-11T12:33:47.539141914Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  ENV BUILDER_VERSION=1.0 JDK_VERSION=1.8.0\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"13a1ea149358c5760741587e94df4f723443189da1701e874fd70bfce90a5c5e\",\"parent\":\"b42180df0bcdc82a9e6fda99043eedb4cef2d9ec6b416ea4f86507cd1d2d3211\",\"created\":\"2019-04-11T12:33:47.450141111Z\",\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  MAINTAINER Tremolo Security, Inc. - Docker \\[email protected]\\u003e\"]},\"author\":\"Tremolo Security, Inc. - Docker \\[email protected]\\u003e\",\"throwaway\":true}"
      },
      {  
         "v1Compatibility": "{\"id\":\"b42180df0bcdc82a9e6fda99043eedb4cef2d9ec6b416ea4f86507cd1d2d3211\",\"parent\":\"dd6280311988ffd70b32c4d5e1128f46bd267294f18dff0c937c3ab61870b4cd\",\"created\":\"2019-01-22T20:02:54.295067Z\",\"container_config\":{\"Cmd\":[\"\"]}}"
      },
      {
         "v1Compatibility": "{\"id\":\"dd6280311988ffd70b32c4d5e1128f46bd267294f18dff0c937c3ab61870b4cd\",\"comment\":\"Imported from -\",\"created\":\"2019-01-22T20:02:49.244835069Z\",\"container_config\":{\"Cmd\":[\"\"]}}"
      }
   ]

So reconstructing proper history from V2.1 requires quite a lot of work.

[loosebazooka]

So what does this exactly involve?
Updating the container config 2.1 object to read history?
Passing that history around?

[chanseokoh]

To clarify, there's only one version of container config template being used when consuming both V2.1 and V2.2 manifests. By its definition, the container config template defines a config.History array.

For V2.1, the template is matched against the first history.v1Compatibility value. Looks like this v1Compatibility value is almost complete in that it has all the important entries except for the config.History array. So, other values like config.Labels, config.Cmd, config.ExposedPorts, etc. are there, except config.History.

And seems like to restructure the proper history, we need to go through all history.v1Compatbility values (bottom to top) and extract container_config.Cmd. An example value is

         "v1Compatibility": "{\"id\":..., ... ,\"container_config\":{\"Cmd\":[\"/bin/sh -c #(nop)  LABEL name=MyVirtualDirectory Base vendor=Tremolo Security, Inc. version=1.0.0 ...\"]}, \"author\":...}"

The container_config.Cmd values seem like the history values appearing in CREATED BY.

[chanseokoh]

Will not work on V2.1 manifest.