Reduce images size #1636

Aschen · 2020-05-24T18:27:58Z

Reduce image size

~~⚠️ Depends on #1635~~

Reduce Kuzzle images size:

kuzzleio/kuzzle:2: from 563 Mo to 165 Mo
kuzzleio/plugin-dev:2: from 968 Mo to 410 Mo

This PR also includes 2 new tags:

kuzzleio/kuzzle:2-alpine: 140 Mo
kuzzleio/kuzzle:2-scratch: 94 Mo

I followed the tricks explained here: https://medium.com/trendyol-tech/how-we-reduce-node-docker-image-size-in-3-steps-ff2762b51d5a

Also I made few other changes for the production image:

remove all object and build files (.o, .h)
add only relevant files (lib, config, plugins, etc)
remove .git
remove useless npm dependencies
minify JS
strip re2 and node binaries

The scratch image contains only Node.js (with libstdc++), musl and Kuzzle source code.

Scripts

All docker related scripts are in docker/scripts/:

entrypoint.sh: the entrypoint
clean-node.sh: remove every unnecessary file from a directory. If NODE_ENV=production then the JS and JSON files will be minified
run-dev.sh: used by the core-dev image to run a core-developer stack
run-plugin.sh: used by plugin-dev to run an application composed of plugins (like we do now)
run-prod.sh: used by the kuzzle images to run kuzzle in production

How should this be manually tested?

Build the images and play the functional tests

Other changes

Elasticsearch initialization is now waited with the bin/wait-elasticsearch script instead of curl
Functionals tests are run in parallel in 2 travis jobs

Aschen · 2020-05-24T18:29:32Z

I really struggled with the .dockerignore file, it does not seem to work if anyone know why

alexandrebouthinon

Update kuzzle directory location in Docker image is a breaking change. Otherwise very nice job 👍

alexandrebouthinon · 2020-05-28T11:04:48Z

.ci/privatebuild/Dockerfile

@@ -10,8 +10,8 @@ ARG SSH_KEY
 ENV GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no"
 ENV NODE_ENV=production

-WORKDIR /var/app
-COPY . /var/app
+WORKDIR /app


This is a breaking change

benoitvidis · 2020-05-28T11:09:35Z

bin/wait-elasticsearch

+  }
+};
+
+run();


Suggested change

run();

if (require.main === module) {

run();

}

Else, will be called on require (i.e. from kuzzle-start)

benoitvidis · 2020-05-28T11:12:44Z

docker/images/kuzzle.alpine/Dockerfile

+  && ln -s /app/plugins/available/kuzzle-plugin-logger /app/plugins/enabled/kuzzle-plugin-logger \
+  # Install plugins dependencies
+  && for plugin in plugins/enabled/*; do cd "$plugin"; npm install --production; cd /app; done \
+  && for plugin in plugins/available/*; do cd "$plugin"; npm install --production; cd /app; done \


Suggested change

&& for plugin in plugins/available/*; do cd "$plugin"; npm install --production; cd /app; done \

&& for plugin in plugins/*/*; do npm install --production --prefix "$plugin"; done \

benoitvidis · 2020-05-28T11:15:09Z

docker/scripts/everest-clean-node.sh

+
+# Minify JS & JSON
+# I wasn't able to find a portable way to do string comparison with if
+( echo $NODE_ENV | grep production ) || exit 0


Suggested change

( echo $NODE_ENV | grep production ) || exit 0

[ "${NODE_ENV}" = "production" ] || exit 0

This is posix

alexandrebouthinon

I used Shellcheck on scripts, there it is some of the easiest warning/error to correct 🙂

alexandrebouthinon · 2020-05-28T11:52:16Z

docker/build-docker-images.sh

+  promise_run docker_build 'kuzzle' $TRAVIS_BRANCH 'kuzzle'
+    promise_then docker_push 'kuzzle' $TRAVIS_BRANCH
+
+  promise_run docker_build 'kuzzle' $TRAVIS_BRANCH-alpine 'kuzzle.alpine'
+    promise_then docker_push 'kuzzle' $TRAVIS_BRANCH-alpine


Suggested change

promise_run docker_build 'kuzzle' $TRAVIS_BRANCH 'kuzzle'

promise_then docker_push 'kuzzle' $TRAVIS_BRANCH

promise_run docker_build 'kuzzle' $TRAVIS_BRANCH-alpine 'kuzzle.alpine'

promise_then docker_push 'kuzzle' $TRAVIS_BRANCH-alpine

promise_run docker_build 'kuzzle' "$TRAVIS_BRANCH" 'kuzzle'

promise_then docker_push 'kuzzle' "$TRAVIS_BRANCH"

promise_run docker_build 'kuzzle' "$TRAVIS_BRANCH-alpine" 'kuzzle.alpine'

promise_then docker_push 'kuzzle' "$TRAVIS_BRANCH-alpine"

alexandrebouthinon · 2020-05-28T11:52:34Z

docker/build-docker-images.sh

+  promise_run docker_build 'plugin-dev' $TRAVIS_BRANCH 'plugin-dev'
+    promise_then docker_push 'plugin-dev' $TRAVIS_BRANCH


Suggested change

promise_run docker_build 'plugin-dev' $TRAVIS_BRANCH 'plugin-dev'

promise_then docker_push 'plugin-dev' $TRAVIS_BRANCH

promise_run docker_build 'plugin-dev' "$TRAVIS_BRANCH" 'plugin-dev'

promise_then docker_push 'plugin-dev' "$TRAVIS_BRANCH"

alexandrebouthinon · 2020-05-28T11:54:17Z

docker/build-docker-images.sh

+    promise_then docker_push 'kuzzle' $TRAVIS_BRANCH
+
+  promise_run docker_build 'kuzzle' $TRAVIS_BRANCH-alpine 'kuzzle.alpine'
+    promise_then docker_push 'kuzzle' $TRAVIS_BRANCH-alpine

 elif [[ "$TRAVIS_BRANCH" == "master" ]] || [[ "$TRAVIS_BRANCH" == *"-stable" ]]; then
  RELEASE_TAG=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[\",]//g' | tr -d '[[:space:]]')


Suggested change

RELEASE_TAG=$(cat package.json | grep version | head -1 | awk -F: '{ print $2 }' | sed 's/[\",]//g' | tr -d '[[:space:]]')

RELEASE_TAG=$(grep version package.json | head -1 | awk -F: '{ print $2 }' | sed 's/[\",]//g' | tr -d '[:space:]')

docker/build-docker-images.sh

alexandrebouthinon · 2020-05-28T12:00:25Z

docker/scripts/everest-clean-node.sh

@@ -0,0 +1,22 @@
+#!/bin/sh


Suggested change

#!/bin/sh

#!/usr/bin/env sh

Aschen · 2020-06-02T04:55:31Z

I will wait for Kaaf before finishing this PR

Aschen · 2020-06-19T07:12:08Z

@benoitvidis About the compression, the decompression is very fast (~4-5s) and it allows to have only 19 mo transiting through the network instead of 94. It could be very useful for K8S environment where the pods have to download the new image before running for example.
Also keeping a lot of images on a server takes a lot of storage. You can clean it but it's not always the case and you also may want to keep previous versions to rollback quickly.

Also the way I copy Kuzzle still allows developers to add plugins in the plugins/* directories

I will leave the kuzzleio/kuzzle, kuzzleio/plugin-dev and kuzzleio/kuzzle:*-alpine without compression and I will create a new kuzzleio/kuzzle:*-everest image with the compression for those who want extra small images, what do you think?

Image sizes:

kuzzleio/kuzzle      plugin-dev      97624376beb0    53 seconds ago  385MB
kuzzleio/kuzzle      kuzzle          be537f211ae9    37 minutes ago  165MB
kuzzleio/kuzzle      kuzzle:alpine   d826c568d69a    10 minutes ago  140MB
kuzzleio/kuzzle      kuzzle:everest  67ef01c1d47e    9 minutes ago   19.5MB

codecov · 2020-06-23T05:11:32Z

Codecov Report

Merging #1636 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1636   +/-   ##
=======================================
  Coverage   93.52%   93.52%           
=======================================
  Files         113      113           
  Lines        7197     7197           
=======================================
  Hits         6731     6731           
  Misses        466      466

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c81e218...1ea882f. Read the comment docs.

alexandrebouthinon · 2020-06-23T09:04:06Z

.ci/scripts/install-plugins.sh

+set -e
+set -x


Suggested change

set -e

set -x

set -ex

alexandrebouthinon · 2020-06-23T09:05:32Z

.ci/scripts/run-test-arm.sh

@@ -1,8 +1,7 @@
 #!/bin/sh

 set -e
-
-elastic_host=${kuzzle_services__storageEngine__client__node:-http://elasticsearch:9200}
+set -x


set arguments could be combined

alexandrebouthinon · 2020-06-23T09:05:49Z

.ci/scripts/run-test.sh

@@ -1,57 +1,21 @@
 #!/bin/sh

 set -e
-
-elastic_host=${kuzzle_services__storageEngine__client__node:-http://elasticsearch:9200}
+set -x


alexandrebouthinon · 2020-06-23T09:09:41Z

docker/scripts/run-dev.sh

+
+set -e
+
+if [ ! -z "$WITHOUT_KUZZLE" ]; then


Suggested change

if [ ! -z "$WITHOUT_KUZZLE" ]; then

if [ -n "$WITHOUT_KUZZLE" ]; then

docker/scripts/run-dev.sh

alexandrebouthinon · 2020-06-23T09:11:01Z

docker/scripts/everest-extract.sh

@@ -0,0 +1,34 @@
+#!/bin/sh
+


Missing set -e?

Co-authored-by: Alexandre Bouthinon <[email protected]>

…zzle into reduce-production-image-size

benoitvidis · 2020-06-23T14:00:58Z

@Aschen , I am still not sure about everest image.

docker layers are already compressed. May be worth checking but my guess is there is gain to extra compress the layer for download time.
because of the mergefs system, you will get 2 layers on the disk: the one with the compressed tar + the diff once is is extracted, which is also counter productive considering the goal. The docker image ls here needs to be compared vs the real size of the corresponding layers in /var/lib/docker/overlay/merged|diff
the rollback issues you are mentioning should not have much impact in most cases, as only the application layer should be updated.

These 2 points + the already discussed UX issues make me really dubious about this image.

Aschen · 2020-06-24T04:35:14Z

@benoitvidis I replaced the image by a scratch image without the compression (size 94 Mo)

Docker compress layers with Gzip which is not as efficient as Lzma, especially for binary files:

The difference between compressed application layers is not that huge (13Mo gz vs 9.7 Mo lzma) so I agree that for subsequent download of the same image the gain won't be huge:

I still think the compression is useful in a K8S environment because the pods has to download the image whenever they are spawned so it allows to save bandwidth, which is quite expensive in most of the cloud providers.

But I understand your point of view and we need those smaller image so LGTM

…zzle into reduce-production-image-size

sonarqubecloud · 2020-06-25T07:00:12Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

No Coverage information
No Duplication information

Aschen added 2 commits May 24, 2020 11:56

Use nodemon instead of PM2

5c4bab5

Reduce production image size

a3daaeb

Aschen added the changelog:enhancements label May 24, 2020

Aschen self-assigned this May 24, 2020

Aschen added 2 commits May 24, 2020 20:37

Nit

dfff4d9

Update scripts

c184da6

Aschen changed the title ~~Reduce production image size~~ Reduce images size May 25, 2020

Aschen marked this pull request as draft May 25, 2020 07:10

Aschen mentioned this pull request May 25, 2020

Use the new smaller image kuzzleio/kuzzle-plugin-graphql#2

Closed

Aschen added 6 commits May 25, 2020 14:05

Fix plugin-dev

104092b

Save 4mo

6ca77b1

Move images

93bc34e

asynchronous extract

c76aafa

Fix images

cd5a6b1

Nit

ee0bb17

Aschen marked this pull request as ready for review May 27, 2020 11:47

Aschen added 6 commits May 27, 2020 13:48

nit

e423165

Reduce size

437bdca

wip

c429fb4

Plugin-dev ok

b04ce12

merge ok

9e0d229

Add wait-elasticsearch

f87ed79

alexandrebouthinon requested changes May 28, 2020

View reviewed changes

benoitvidis reviewed May 28, 2020

View reviewed changes

alexandrebouthinon reviewed May 28, 2020

View reviewed changes

Aschen marked this pull request as draft June 2, 2020 05:00

kuzzleio/alpine ok

6862c52

Aschen added 4 commits June 19, 2020 09:51

all images ok

ff1ae5e

Add README

bb3818f

Fix tests

19f3314

trigger ci

383f8b5

Aschen marked this pull request as ready for review June 19, 2020 09:21

Aschen and others added 6 commits June 19, 2020 11:39

fix test

271413d

fix sh

0df3a7b

use node for startup scripts

119bb7d

Bump node 12.18.1

70e5fa8

Merge branch 'master' into reduce-production-image-size

41d1cf5

Fix dependencies install in tests

17660d7

alexandrebouthinon approved these changes Jun 23, 2020

View reviewed changes

Aschen and others added 5 commits June 23, 2020 11:45

Update docker/scripts/run-dev.sh

e331000

Co-authored-by: Alexandre Bouthinon <[email protected]>

Update scripts

f14f4c4

Merge branch 'master' into reduce-production-image-size

b33d5b9

Update scripts

bd5a9c2

Merge branch 'reduce-production-image-size' of github.com:kuzzleio/ku…

8b219d4

…zzle into reduce-production-image-size

Aschen added 4 commits June 24, 2020 06:35

replace everest by scratch

3764998

Merge branch 'reduce-production-image-size' of github.com:kuzzleio/ku…

5904564

…zzle into reduce-production-image-size

fix scratch image

2525a86

factorize steps

1ea882f

benoitvidis approved these changes Jun 26, 2020

View reviewed changes

Aschen merged commit c48fa76 into master Jun 26, 2020

Aschen deleted the reduce-production-image-size branch June 26, 2020 09:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reduce images size #1636

Reduce images size #1636

Aschen commented May 24, 2020 •

edited

Loading

Aschen commented May 24, 2020 •

edited

Loading

alexandrebouthinon left a comment

alexandrebouthinon May 28, 2020

benoitvidis May 28, 2020

benoitvidis May 28, 2020

benoitvidis May 28, 2020

alexandrebouthinon left a comment

alexandrebouthinon May 28, 2020

alexandrebouthinon May 28, 2020

alexandrebouthinon May 28, 2020

alexandrebouthinon May 28, 2020

Aschen commented Jun 2, 2020

Aschen commented Jun 19, 2020 •

edited

Loading

codecov bot commented Jun 23, 2020 •

edited

Loading

alexandrebouthinon Jun 23, 2020

alexandrebouthinon Jun 23, 2020

alexandrebouthinon Jun 23, 2020

alexandrebouthinon Jun 23, 2020

alexandrebouthinon Jun 23, 2020

benoitvidis commented Jun 23, 2020

Aschen commented Jun 24, 2020 •

edited

Loading

sonarqubecloud bot commented Jun 25, 2020

+                }
+              };
+              run();

-run();
+if (require.main === module) {
+  run();
+}

	&& for plugin in plugins/available/*; do cd "$plugin"; npm install --production; cd /app; done \
	&& for plugin in plugins//; do npm install --production --prefix "$plugin"; done \

	( echo $NODE_ENV \| grep production ) \|\| exit 0
	[ "${NODE_ENV}" = "production" ] \|\| exit 0

		promise_run docker_build 'plugin-dev' $TRAVIS_BRANCH 'plugin-dev'
		promise_then docker_push 'plugin-dev' $TRAVIS_BRANCH

	RELEASE_TAG=$(cat package.json \| grep version \| head -1 \| awk -F: '{ print $2 }' \| sed 's/[\",]//g' \| tr -d '[[:space:]]')
	RELEASE_TAG=$(grep version package.json \| head -1 \| awk -F: '{ print $2 }' \| sed 's/[\",]//g' \| tr -d '[:space:]')

	if [ ! -z "$WITHOUT_KUZZLE" ]; then
	if [ -n "$WITHOUT_KUZZLE" ]; then

Reduce images size #1636

Reduce images size #1636

Conversation

Aschen commented May 24, 2020 • edited Loading

Reduce image size

Scripts

How should this be manually tested?

Other changes

Aschen commented May 24, 2020 • edited Loading

alexandrebouthinon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexandrebouthinon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Aschen commented Jun 2, 2020

Aschen commented Jun 19, 2020 • edited Loading

codecov bot commented Jun 23, 2020 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benoitvidis commented Jun 23, 2020

Aschen commented Jun 24, 2020 • edited Loading

sonarqubecloud bot commented Jun 25, 2020

Aschen commented May 24, 2020 •

edited

Loading

Aschen commented May 24, 2020 •

edited

Loading

Aschen commented Jun 19, 2020 •

edited

Loading

codecov bot commented Jun 23, 2020 •

edited

Loading

Aschen commented Jun 24, 2020 •

edited

Loading