improve jest/sonar integration.

[mshima]

Fixes #19433

Switch from archived jest-sonar-reporter to jest-sonar

---

Please make sure the below checklist is followed for Pull Requests.

• All continuous integration tests are green
• Tests are added where necessary
• The JDL part is updated if necessary
• jhipster-online is updated if necessary
• Documentation is added/updated where necessary
• Coding Rules & Commit Guidelines as per our CONTRIBUTING.md document are followed

When you are still working on the PR, consider converting it to Draft (below reviewers) and adding skip-ci label, you can still see CI build result at your branch.

[mshima]

I saw some warnings, will try to fix.

[mraible]

Testing this now...

[mraible]

Looks like there are some new issues:

No coverage report can be found with sonar.coverage.jacoco.xmlReportPaths='build/reports/jacoco/test/jacocoTestReport.xml'. 
Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
Reports path not found or is not a directory: /Users/mraible/Downloads/sonar/build/test-results/integrationTest

> Task :sonarqube FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sonarqube'.
> Error during parsing of generic test execution report '/Users/mraible/Downloads/sonar/build/test-results/jest/TESTS-results-sonar.xml'. Look at the SonarQube documentation to know the expected XML format.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 2m 39s
16 actionable tasks: 14 executed, 2 up-to-date
Execution time: 2 min. 40 s.

[mraible]

@mshima Any ideas on how to fix the errors I experienced above?

[mshima]

@mraible I will test, have you run gradle test npm run ci:backend:test?
Maybe sonar with gradle is broken at main already. I will test it later.

[mraible]

I did not use npm run ci:backend:test, I used what the generated README has:

./gradlew -Pprod clean check jacocoTestReport sonarqube

Our Sonar docs also use this command, albeit with an additional -Dsonar.host.url=http://localhost:9001 argument.

I just tried npm run ci:backend:test and it passes, but doesn't seem to send anything to Sonar. I tried the Gradle command above after running this command and get the following error:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sonarqube'.
> Error during parsing of generic test execution report '/Users/mraible/Downloads/21/build/test-results/jest/TESTS-results-sonar.xml'. Look at the SonarQube documentation to know the expected XML format.

[mshima]

Looks like jest-sonar is not compatible with <rootDir>.

[mraible]

I tried again by generating the app and running the Gradle command. Looks good!

BUILD SUCCESSFUL in 3m 55s
21 actionable tasks: 19 executed, 2 up-to-date
Execution time: 3 min. 56 s.

[mraible]

Related: I had to add the following to the Sonar image for it to work on M1.

platform: linux/amd64

Should I create a PR for this?

[mraible]

Hmmm, Docker shows it started, but it's unreachable, so adding linux/amd64 is probably a bad idea. Docker logs show:

2022.08.27 16:08:16 ERROR web[][o.s.s.m.ElasticSearchMetricTask] Failed to query ES status
org.sonar.server.es.ElasticsearchException: Fail to execute es request
        at org.sonar.server.es.EsClient.execute(EsClient.java:313)
        at org.sonar.server.es.EsClient.execute(EsClient.java:305)
        at org.sonar.server.es.EsClient.nodesStats(EsClient.java:216)
        at org.sonar.server.monitoring.ElasticSearchMetricTask.updateFileSystemMetrics(ElasticSearchMetricTask.java:78)
        at org.sonar.server.monitoring.ElasticSearchMetricTask.run(ElasticSearchMetricTask.java:51)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.SocketTimeoutException: 60,000 milliseconds timeout on connection http-outgoing-3 [ACTIVE]
        at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:903)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:299)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:287)
        at org.sonar.server.es.EsClient.lambda$nodesStats$29(EsClient.java:218)
        at org.sonar.server.es.EsClient.execute(EsClient.java:311)
        ... 10 common frames omitted
Caused by: java.net.SocketTimeoutException: 60,000 milliseconds timeout on connection http-outgoing-3 [ACTIVE]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92)
        at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39)
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:261)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:502)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:211)
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
        ... 1 common frames omitted

[mshima]

Sonar docker containter doesn’t support arm64 and fails bad with amd64 image at apple silicon.
There are third part containers, we could add a comment to use a third part container

[carminevassallo]

Hi @mraible, hi @mshima,

I just came across your PR. As Marcelo pointed out, the official SonarQube image available on Docker Hub does not support the execution on Apple M1 yet. However, I can give you some info on how to get the latest SonarQube running as a container on M1.

SonarQube uses a base image (currently alpine:3:15) that supports linux/arm64/v8 and all the required libraries are installed through a package manager (apk). Therefore, if you want to run SonarQube as a container on M1, you can simply check out your preferred SonarQube 9 edition from the official repository and build the corresponding Dockerfile locally. You should then be able to run the resulting image without problems on M1.

Please let me know if this comment solves your problems with SonarQube and how we can help further on this 😃

[mraible]

Hi @carminevassallo,

Thanks for providing this information. Do you know when your official image will support Apple M1?

Our users expect everything to work out-of-the-box and we'll have to provide this workaround for them (both in my JHipster Mini-Book and in our docs). We currently have a workaround for Keycloak documented, but it's no longer needed as of Keycloak v18. Do you have a similar list of steps we could use?

FWIW, here are our current versions of SonarQube: https://github.com/jhipster/generator-jhipster/blob/main/generators/generator-constants.js#L88-L89

And the Docker Compose file we distribute with new apps: https://github.com/jhipster/generator-jhipster/blob/main/generators/server/templates/src/main/docker/sonar.yml.ejs

[carminevassallo]

Hi @mraible,

We do not have a date for supporting Apple M1 yet, but this is already under discussion as it requires some adaptations in our docker release process. I’ll follow up on this when I know more.

In the meantime, I see that you are using the sonarqube 9.6.0 community edition. We can use a workaround similar to the Keycloak one. To build and run the latest sonarqube 9 community edition, I’d suggest these steps:

• Clone the docker-sonarqube repository: git clone [email protected]:SonarSource/docker-sonarqube.git
• Build the latest sonarqube 9 image (currently 9.6.1): docker build docker-sonarqube/9/community -t sonarqube-arm64-v8:9.6.1-community
• Generate a container that will be accessible on port 9000 docker run -d -p 9000:9000 sonarqube-arm64-v8:9.6.1-community

Depending on your docker setup, you should be able to access sonarqube at http://localhost:9000/. Please let me know if this workaround is useful in your case.

[mraible]

@carminevassallo Is there a way to checkout a particular version like we can with Keycloak?

git clone [email protected]:keycloak/keycloak-containers.git
cd keycloak-containers/server
git checkout 16.1.0

We're currently using sonarqube:9.6.0-community in JHipster 7.9.3. To have a proper workaround, we need to build that version locally to replace the one that JHipster downloads.

[mraible]

I think I figured it out:

git clone [email protected]:SonarSource/docker-sonarqube.git
cd docker-sonarqube/9/community
git checkout 9.6.0
docker build -t sonarqube:9.6.0-community .

Then, I ran the following in my JHipster app's directory:

docker-compose -f src/main/docker/sonar.yml up

Then, I ran ./gradlew -Pprod clean check jacocoTestReport sonarqube and I'm able to see the results.

[carminevassallo]

I think I figured it out:

git clone [email protected]:SonarSource/docker-sonarqube.git
cd docker-sonarqube/9/community
git checkout 9.6.0
docker build -t sonarqube:9.6.0-community .

Indeed, we have a tag for each release, so it's possible to do a checkout similar to keycloak. Anyways, 9.6.1 contains important bug fixes, you might want to move to that version soon :)

[mraible]

@carminevassallo I agree we should upgrade. I'll work on that.

Here's a PR to add the workaround to our docs. jhipster/jhipster.github.io#1228

[carminevassallo]

Hi @mraible,

I have good news for you on this topic. With the new SonarQube LTS launched last month, we do support Apple M1 in our docker image. Therefore, if you set your docker-compose file to use the sonarqube:9.9.0-community(or the latest) image, you can remove the workaround suggested in your docs.

[mraible]

@carminevassallo It looks like we might already be using the version you recommend: https://github.com/jhipster/generator-jhipster/blob/main/generators/server/templates/Dockerfile#L49

@DanielFran Can you confirm? If so, I can update our docs.

[DanielFran]

@carminevassallo It looks like we might already be using the version you recommend: https://github.com/jhipster/generator-jhipster/blob/main/generators/server/templates/Dockerfile#L49

@DanielFran Can you confirm? If so, I can update our docs.

@mraible Yes, I confirm!

[mraible]

I created a PR to remove this warning in our docs. jhipster/jhipster.github.io#1269