Release GN v4.4.0 #107

juanluisrp · 2023-07-11T11:59:41Z

Main changes

Run on Java 11

GeoNetwork 4.4.x use Java 11 (See geonetwork/core-geonetwork#7186)

Improved configuration

Documentation explains how to configure various aspects of the application (eg. root context, database, elasticsearch index, security, ...) using environment variables.

Changes and new options:

Allow to set the application web context path using the new WEBAPP_CONTEXT_PATH (defaults to /geonetwork)
Move the exploded war to /opt/geonetwork
Add a new GN_CONFIG_PROPERTIES for passing addional configuration options to GN in the form of Java properties, for example GN_CONFIG_PROPERTIES=-Dldap.base.provider.url=ldap://ldap:389
ES_XYZ variables are removed. Use JVM arguments instead.
Add Jetty configuration to support large metadata records editing

Move from Nginx to Traefik

Moving to traefik was mainly motivated for easier setup of the multiple instances configuration (see below).
Traefik dashboard is available on http://localhost:8000/
It defines a router to GeoNetwork with a redirect when requesting "/" on http://geonetwork.localhost/ and then load balance to the instances available.

Add health check

Add service health check and start services in order (eg. avoid to have error on OGC API Records when the database is empty).

Multiple instances configuration (experimental)

The composition allows to start multiple GeoNetwork nodes. eg. to start 2 more instances :

docker-compose --profile scaled up --scale geonetwork-replica=2 -d

and traefik will register them as servers for the route

Related works

* Allow to set the application web context path using the new `WEBAPP_CONTEXT_PATH` (defaults to `/geonetwork`) * Move the exploded war to `/opt/geonetwork` * Add a new `GN_CONFIG_PROPERTIES` for passing addional configuration options to GN in the form of Java properties, for example `GN_CONFIG_PROPERTIES=-Dldap.base.provider.url=ldap://ldap:389` * `ES_HOST` is no longer mandatory and now defaults to `localhost`

* Java 11 * Fix Config properties * Use OGC API records 4.2.5 for now * Fix OGC API records fail to start because of empty GN database * Move config to JVM args * Add Jetty http forward module for X-Forwarded header.

4.4.0/nginx/default.conf.template

…ances.

This is a parameter to customize depending on size of metadata records (eg. many languages or many contacts)

Main idea was to more easily set up load balancing with sticky session. Co-authored-by: Joachim Nielandt <[email protected]>

Main idea is to avoid errors on some case (eg. OGC API records error starting on an empty database) and start services in order. Co-authored-by: Joachim Nielandt <[email protected]>

…nx config.

edevosc2c · 2023-09-14T08:20:29Z

It should be noted that there is a sticky cookie here for forcing the user to use the same geonetwork across his visit: https://github.com/geonetwork/docker-geonetwork/pull/107/files#diff-bed7ab158ecf2f50be93c45dd9ae77da44d0689a155d95771d091515fb6d1ba7R59

If we have 2 geonetwork instances, the requests won't be evenly sent to all the geonetwork instances, in case we have 50% of users stuck to 1 geonetwork sending 80% of the overall requests, then one geonetwork will be overloaded while the other one will be underutilized.

I guess it's inevitable because geonetwork is a stateful app, but that's something the users should be aware of it.

Co-authored-by: Joachim Nielandt <[email protected]>

when starting multiple instances using the same data dir - XSD are copied to this folder on startup and copy may clash)

joachimnielandt · 2023-09-22T08:20:29Z

I have encountered a configuration issue that I can reproduce based on this 4.4.0 setup. The swagger docs for ogc-records-service are not accessible through Geonetwork:

geonetwork: http://localhost:8080/geonetwork/srv/eng/catalog.search#/home
ogc records service: http://localhost:8080/geonetwork/api
ogc records service swagger api
- link on page is http://localhost:8080/geonetwork/openapi
- the service itself listens for /openapi and generates a link for baseurl + /openapi, whereas geonetwork wants to send traffic for the service to /api

I've tried configuring the context path for ogc-records-service such that it matches /api, and the location for the swagger api itself to match:

server.servlet.context-path: /api
springfox.documentation.sawggerUi.baseUrl: /api/openapi

However, in that case, the desired path results in a 404: http://localhost:8081/api/openapi/swagger-ui/. Probably not the way to go then. Could you advise on how to get the swaggerui accessible?

joachimnielandt · 2023-09-26T09:07:10Z

4.4.0/README.md

@@ -195,7 +195,7 @@ To enable it use the `scaled` profile. In this mode:
 * any node can take a harvesting task manually triggered from the harvesting console
 * webserver is configured with sticky session (ie. a user stay on the same node) 

-To start new instances:
+First, start the main composition which will start all services (including the main node). Then start new instances with:


Do you need to start this up in multiple goes? I remember having all instances (regular + replicas) start up fine when triggering the command below.

4.4.0/docker-entrypoint.sh

fxprunayre · 2023-10-19T08:31:46Z

However, in that case, the desired path results in a 404: http://localhost:8081/api/openapi/swagger-ui/. Probably not the way to go then. Could you advise on how to get the swaggerui accessible?

Not sure @joachimnielandt . We have to fix it in OGC API Records side and it will not be fixed for 4.4.0.

fxprunayre · 2023-10-19T11:39:52Z

However, in that case, the desired path results in a 404: http://localhost:8081/api/openapi/swagger-ui/. Probably not the way to go then. Could you advise on how to get the swaggerui accessible?

Checking a bit more, it looks like
http://localhost:8080/geonetwork/api/openapi/swagger-ui/
is fine so the issue is when swagger redirect from /openapi to /openapi/swagger-ui/.

4.4.0/README.md

Co-authored-by: joachimnielandt <[email protected]>

juanluisrp · 2023-10-25T11:20:38Z

Created docker-library/official-images#15618 in official images repo.

juanluisrp mentioned this pull request Jul 11, 2023

Release GN v4.4.0 #103

Closed

juanluisrp marked this pull request as draft July 11, 2023 12:01

Update

c4e01cb

* Java 11 * Fix Config properties * Use OGC API records 4.2.5 for now * Fix OGC API records fail to start because of empty GN database * Move config to JVM args * Add Jetty http forward module for X-Forwarded header.

fxprunayre reviewed Jul 21, 2023

View reviewed changes

4.4.0/nginx/default.conf.template Outdated Show resolved Hide resolved

fxprunayre and others added 11 commits August 31, 2023 14:29

Move env to variable for future easier configuration of multiple inst…

71cad20

…ances.

Jetty / Increase max form keys

f21fd8c

This is a parameter to customize depending on size of metadata records (eg. many languages or many contacts)

Jetty config / Allows larger form.

9d1cb4f

Java options to be able to return metrics in Java 11.

8bf8eda

Clustering / Add instruction for testing and add a simple load balancer.

da562a5

Nginx / Add body size parameter and fix hard coded scheme.

4f6488d

Move to traefik.

8c37971

Main idea was to more easily set up load balancing with sticky session. Co-authored-by: Joachim Nielandt <[email protected]>

Add health check.

d64c0f1

Main idea is to avoid errors on some case (eg. OGC API records error starting on an empty database) and start services in order. Co-authored-by: Joachim Nielandt <[email protected]>

Readme update for traefik change.

df6a147

Monitoring / Load traefik log using filebeat. Removing Apache and Ngi…

fdf9372

…nx config.

Update multiple instances limitations.

724036d

fxprunayre mentioned this pull request Sep 13, 2023

Improvements to allow future deployment of multiple instances geonetwork/core-geonetwork#7337

Merged

fxprunayre and others added 2 commits September 18, 2023 18:31

Jetty / Update version and fix sending mail on java 11.

cd6af05

Co-authored-by: Joachim Nielandt <[email protected]>

Add timezone config and use separate schemapublication dir (avoid issue

ad067fd

when starting multiple instances using the same data dir - XSD are copied to this folder on startup and copy may clash)

fxprunayre added 3 commits September 25, 2023 10:23

Update README.md

12bee86

Update README.md

f768ddd

Update README.md

9f06cdd

joachimnielandt reviewed Sep 26, 2023

View reviewed changes

juanluisrp commented Sep 28, 2023

View reviewed changes

4.4.0/docker-entrypoint.sh Outdated Show resolved Hide resolved

Fix typo in variable expansion

b249c6e

fxprunayre marked this pull request as ready for review October 13, 2023 07:45

Update MD5 for 4.4.0

410cd70

When building image, set image name for docker compose to find it.

0b9ff2f

Add proxy variables example

b13cc6f

joachimnielandt reviewed Oct 20, 2023

View reviewed changes

4.4.0/README.md Outdated Show resolved Hide resolved

Update 4.4.0/README.md

09af312

Co-authored-by: joachimnielandt <[email protected]>

fxprunayre approved these changes Oct 24, 2023

View reviewed changes

Fix Markdown warnings

a2e70d7

juanluisrp added the new release label Oct 25, 2023

juanluisrp merged commit 317a9e7 into main Oct 25, 2023

f-necas mentioned this pull request Nov 16, 2023

Geonetwork 4.4 georchestra/geonetwork#261

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release GN v4.4.0 #107

Release GN v4.4.0 #107

juanluisrp commented Jul 11, 2023 •

edited by fxprunayre

Loading

edevosc2c commented Sep 14, 2023

joachimnielandt commented Sep 22, 2023 •

edited

Loading

joachimnielandt Sep 26, 2023

fxprunayre commented Oct 19, 2023

fxprunayre commented Oct 19, 2023

juanluisrp commented Oct 25, 2023

Release GN v4.4.0 #107

Release GN v4.4.0 #107

Conversation

juanluisrp commented Jul 11, 2023 • edited by fxprunayre Loading

Main changes

Run on Java 11

Improved configuration

Move from Nginx to Traefik

Add health check

Multiple instances configuration (experimental)

Related works

edevosc2c commented Sep 14, 2023

joachimnielandt commented Sep 22, 2023 • edited Loading

joachimnielandt Sep 26, 2023

Choose a reason for hiding this comment

fxprunayre commented Oct 19, 2023

fxprunayre commented Oct 19, 2023

juanluisrp commented Oct 25, 2023

juanluisrp commented Jul 11, 2023 •

edited by fxprunayre

Loading

joachimnielandt commented Sep 22, 2023 •

edited

Loading