Merge pull request #129 from LandRegistry/develop

2.0

Author: sichapman

.gitignore

@@ -1,4 +1,3 @@
-.vagrant/
 .dev-env-context
 .update-check-context
 .quick-reload

README.md

@@ -11,22 +11,13 @@ It provides several hooks for applications to take advantage of, including:
 
 ### Prerequisites
 
-#### Recommended setup
-
 * **Docker and Docker Compose**. Exactly what toolset you use depends on your OS and personal preferences, but recommended are:
   * [Docker For Mac](https://docs.docker.com/docker-for-mac/)
   * [Docker for Windows 10](https://docs.docker.com/docker-for-windows/) (See [the wiki](https://github.com/LandRegistry/common-dev-env/wiki/Windows-setup) for more information on getting a working Windows environment set up)
   * [Docker CE for Linux](https://docs.docker.com/install/linux/docker-ce/ubuntu/)
 * **Git**
 * **Ruby 2.5+**
 
-#### Alternative setup
-
-* **Vagrant** (v2+)
-* **VirtualBox** (v6+)
-
-A Vagrantfile is provided so `vagrant up` will result in a virtual machine containing all the prerequisites from the Recommended setup above. From there, just `vagrant ssh` in and use the dev-env as you would natively.
-
 ### Git/SSH
 
 You must ensure the shell you are starting the dev-env from can access all the necessary Git repositories - namely the config repo and the application repos it specifies. If they are to be accessed via SSH, [this wiki page](https://github.com/LandRegistry/common-dev-env/wiki/Git---SSH-setup) has some proven techniques for generating keys and making them available to the shell.
@@ -115,25 +106,21 @@ This file specifies which commodities the dev-env should create and launch for t
 
 The list of allowable commodity values is:
 
-1. postgres
-2. postgres-9.6
-3. postgres-13
-4. db2_devc (**Warning:** source image deprecated by IBM; use db2_community instead)
-5. db2_community
-6. elasticsearch
-7. elasticsearch5
-8. nginx
-9. rabbitmq
-10. redis
-11. swagger
-12. wiremock
-13. squid
-14. auth
-15. cadence
-16. cadence-web
-17. activemq
-18. ibmmq
-19. localstack
+1. postgres-13
+2. db2_community
+4. elasticsearch5
+5. nginx
+6. rabbitmq
+7.  redis
+8.  swagger
+9.  wiremock
+10. squid
+11. auth
+12. cadence
+13. cadence-web
+14. activemq
+15. ibmmq
+16. localstack
 
 * The file may optionally also indicate that one or more services are resource intensive ("expensive") when starting up. The dev env will start those containers seperately - 3 at a time - and wait until each are declared healthy (or crash and get restarted 10 times) before starting any more. This requires a healthcheck command specified here or in the Dockerfile/docker-compose-fragment (in which case just use 'docker' in this file).
   * If one of these expensive services prefers another one to be considered "healthy" before a startup attempt is made (such as a database, to ensure immediate connectivity and no expensive restarts) then the dependent service can be specified here, with a healthcheck command following the same rules as above.
@@ -156,35 +143,31 @@ If you want to spatially enable your database see the following example:
 
 [Example - Spatial](snippets/spatial_postgres-init-fragment.sql)
 
-The default Postgres port 5432 will be available for connections from other containers and on the host. Postgres 9.6 will be exposed on port 5433, Postgres 13 on 5434.
+The default Postgres port 5432 will be available for connections from other containers. Port 5434 is exposed for external connections from the host.
 
 **`/manage.py`**
 
-This is a standard Alembic management file - if it exists, then a database migration will be run on every `up` or `reload`. This can be disabled by setting the key `perform_alembic_migration` to `false` in `configuration.yml` - for example if the file does not actually relate to Alembic, or you do your own migration during app startup.
-
-##### DB2
+This is a standard Alembic management file - if it exists, then a database migration will be run on every `up` or `reload`. This functionality can be enabled by setting the key `perform_alembic_migration` to `true` in `configuration.yml`. It is recommended however that you do your own migration during app startup.
 
-`db2_community` (DB2 Community Edition 11.5) is recommended over `db2_devc` (DB2 Developer C 11.0)
+##### DB2 Community
 
-Note that DB2 Developer C is exposed on the host ports 50001/55001 and DB2 Community on 50002/55002 to avoid port clashes.
+Note that DB2 Community is exposed on 30002/35002 to avoid port clashes.
 
-**`/fragments/db2-devc-init-fragment.sql`**
 **`/fragments/db2-community-init-fragment.sql`**
 
 This file contains any one-off SQL to run in DB2 - at the minimum it will normally be creating a database.
 
 [Example](snippets/db2-community-init-fragment.sql)
 
-##### ElasticSearch
+##### ElasticSearch 5
 
-**`/fragments/elasticsearch-fragment.sh`**
 **`/fragments/elasticsearch5-fragment.sh`**
 
 This file is a shell script that contains curl commands to do any setup the app needs in elasticsearch - creating indexes etc. It will be passed a single argument, the host and port, which can be accessed in the script using `$1`.
 
-The ports 9200/9300 and 9202/9302 are exposed on the host for Elasticsearch versions 2 and 5 respectively.
+The ports 9300 and 9302 are exposed on the host.
 
-[Example](snippets/elasticsearch-fragment.sh)
+[Example](snippets/elasticsearch5-fragment.sh)
 
 ##### Nginx
 

Vagrantfile

@@ -29,7 +29,6 @@
 root_loc = __dir__
 
 # Define the DEV_ENV_CONTEXT_FILE file name to store the users app_grouping choice
-# As vagrant up can be run from any subdirectory, we must make sure it is stored alongside the Vagrantfile
 DEV_ENV_CONTEXT_FILE = root_loc + '/.dev-env-context'
 
 # Where we clone the dev env configuration repo into
@@ -76,7 +75,7 @@
 
 # Does a version check and self-update if required
 if options['self_update']
-  this_version = '1.21.2'
+  this_version = '2.0.0'
   puts colorize_lightblue("This is a universal dev env (version #{this_version})")
   # Skip version check if not on master (prevents infinite loops if you're in a branch that isn't up to date with the
   # latest release code yet)
@@ -117,7 +116,7 @@
 # Then use that config to clone/update apps, create commodities and custom provision lists
 # and download supporting files
 if options['prepare_config']
-  # Check if a DEV_ENV_CONTEXT_FILE exists, to prevent prompting for dev-env configuration choice on each vagrant up
+  # Check if a DEV_ENV_CONTEXT_FILE exists, to prevent prompting for dev-env configuration choice on each up
   if File.exist?(DEV_ENV_CONTEXT_FILE)
     puts ''
     puts colorize_green("This dev env has been provisioned to run for the repo: #{File.read(DEV_ENV_CONTEXT_FILE)}")
@@ -197,16 +196,13 @@
   puts colorize_lightblue('Building images...')
   # v2 already builds in parallel
   if ENV['DC_VERSION'] == '2'
-    if run_command("#{ENV['DC_CMD']} build --memory 384m " + (options['nopull'] ? '' : '--pull')) != 0
-      puts colorize_yellow('Build command failed. Trying without a memory limit')
-      if run_command("#{ENV['DC_CMD']} build " + (options['nopull'] ? '' : '--pull')) != 0
-        puts colorize_red('Something went wrong when building your app images. Check the output above.')
-        exit
-      end
+    if run_command("#{ENV['DC_CMD']} build " + (options['nopull'] ? '' : '--pull')) != 0
+      puts colorize_red('Something went wrong when building your app images. Check the output above.')
+      exit
     end
   else
-    if run_command("#{ENV['DC_CMD']} build --memory 384m --parallel " + (options['nopull'] ? '' : '--pull')) != 0
-      puts colorize_yellow('Build command failed. Trying without --parallel and no memory limit')
+    if run_command("#{ENV['DC_CMD']} build --parallel " + (options['nopull'] ? '' : '--pull')) != 0
+      puts colorize_yellow('Build command failed. Trying without --parallel')
       # Might not be running a version of compose that supports --parallel, try one more time
       if run_command("#{ENV['DC_CMD']} build " + (options['nopull'] ? '' : '--pull')) != 0
         puts colorize_red('Something went wrong when building your app images. Check the output above.')

logic.rb

@@ -42,19 +42,17 @@ then
     echo -e "\e[36mBeginning UP\e[0m"
     ruby logic.rb --check-for-update --prepare-config --update-apps --prepare-compose "${subcommands}" &&
     source scripts/docker_prepare.sh &&
-    ruby logic.rb --build-images --provision-commodities --start-apps "${subcommands}" &&
-    source scripts/after-ups.sh &&
-    source scripts/add-aliases.sh
+    source scripts/add-aliases.sh &&
+    ruby logic.rb --build-images --provision-commodities --start-apps "${subcommands}"
 fi
 
 if [ "$command" = "quickup" ]
 then
     echo -e "\e[36mBeginning UP (Quick mode)\e[0m"
     ruby logic.rb --check-for-update --prepare-compose &&
     source scripts/docker_prepare.sh &&
-    ruby logic.rb --start-apps &&
-    source scripts/after-ups.sh &&
-    source scripts/add-aliases.sh
+    source scripts/add-aliases.sh &&
+    ruby logic.rb --start-apps
 fi
 
 if [ "$command" = "halt" ]
@@ -75,9 +73,8 @@ then
     source scripts/docker_prepare.sh &&
     ruby logic.rb --stop-apps --prepare-config --update-apps --prepare-compose "${subcommands}" &&
     source scripts/docker_prepare.sh &&
-    ruby logic.rb --build-images --provision-commodities --start-apps "${subcommands}" &&
-    source scripts/after-ups.sh &&
-    source scripts/add-aliases.sh
+    source scripts/add-aliases.sh &&
+    ruby logic.rb --build-images --provision-commodities --start-apps "${subcommands}"
 fi
 
 if [ "$command" = "quickreload" ]
@@ -87,9 +84,8 @@ then
     source scripts/docker_prepare.sh &&
     ruby logic.rb --stop-apps --prepare-compose &&
     source scripts/docker_prepare.sh &&
-    ruby logic.rb --start-apps &&
-    source scripts/after-ups.sh &&
-    source scripts/add-aliases.sh
+    source scripts/add-aliases.sh &&
+    ruby logic.rb --start-apps
 fi
 
 if [ "$command" = "destroy" ]

run.sh

@@ -15,11 +15,7 @@ alias livelogs="docker attach --no-stdin --sig-proxy=false"
 alias ex="$DC_CMD exec"
 alias status="$DC_CMD ps"
 alias run="$DC_CMD run --rm"
-alias psql="$DC_CMD exec postgres psql -h postgres -U root -d"
-alias db2="$DC_CMD exec --user db2inst1 db2 bash -c '~/sqllib/bin/db2'"
-alias psql96="$DC_CMD exec postgres-96 psql -h postgres-96 -U root -d"
 alias psql13="$DC_CMD exec postgres-13 psql -h postgres-13 -U root -d"
-alias db2c="$DC_CMD exec --user db2inst1 db2_devc bash -c '~/sqllib/bin/db2'"
 alias db2co="$DC_CMD exec --user db2inst1 db2_community bash -c '~/sqllib/bin/db2'"
 alias gitlist="bash $DEV_ENV_ROOT_DIR/scripts/git_list.sh"
 alias gitpull="bash $DEV_ENV_ROOT_DIR/scripts/git_pull.sh"
@@ -215,8 +211,8 @@ function devenv-help(){
                                                            if you add -r it will output reports to the test-output folder
                                                            if you add -f it will automatically fix issues where possible
                                                            (flags can be combined)
-    psql[96] <name of database>                      -     run psql in the postgres/postgres-96 container
-    db2[c][co]                                       -     run db2 command line in the db2/db2_devc/db2_community container
+    psql13 <name of database>                        -     run psql in the postgres-13 container
+    db2co                                            -     run db2 command line in the db2_community container
     manage <name of container> <command>             -     run manage.py commands in a container
     alembic <name of container> <command>            -     run an alembic db command in a container, with the appropriate environment variables preset
     add-to-docker-compose