Skip to content

chore(druid): Add 33.0.0, Remove 30.0.0 #1110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
May 13, 2025
Merged
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ All notable changes to this project will be documented in this file.
- airflow: Add `2.10.5` ([#1108]).
- druid: check for correct permissions and ownerships in /stackable folder via
`check-permissions-ownership.sh` provided in stackable-base image ([#1039]).
- druid: Add `33.0.0` ([#1110]).
- hadoop: check for correct permissions and ownerships in /stackable folder via
`check-permissions-ownership.sh` provided in stackable-base image ([#1029]).
- hbase: check for correct permissions and ownerships in /stackable folder via
Expand Down Expand Up @@ -78,6 +79,7 @@ All notable changes to this project will be documented in this file.
- Remove ubi8-rust-builder image ([#1091]).
- opa: Remove `0.67.1` ([#1103]).
- opa: Remove legacy bundle-builder from container build ([#1103]).
- druid: Remove `30.0.0` ([#1110]).

[#1025]: https://github.com/stackabletech/docker-images/pull/1025
[#1027]: https://github.com/stackabletech/docker-images/pull/1027
Expand Down Expand Up @@ -107,6 +109,7 @@ All notable changes to this project will be documented in this file.
[#1103]: https://github.com/stackabletech/docker-images/pull/1103
[#1106]: https://github.com/stackabletech/docker-images/pull/1106
[#1108]: https://github.com/stackabletech/docker-images/pull/1108
[#1110]: https://github.com/stackabletech/docker-images/pull/1110
[#1116]: https://github.com/stackabletech/docker-images/pull/1116

## [25.3.0] - 2025-03-21
Expand Down
20 changes: 18 additions & 2 deletions druid/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,29 @@ COPY --chown=${STACKABLE_USER_UID}:0 druid/stackable/patches/${PRODUCT} /stackab
# with a "directory not empty" error on the first builder to finish, as other builders
# are still working in the cache directory.

# Let's have patchable as a dedicated step, as it fetches the druid sourcecode over the network,
# thus taking a bit (which is annoying while development)
RUN /stackable/patchable --images-repo-root=src checkout druid ${PRODUCT} > /tmp/DRUID_SOURCE_DIR

RUN --mount=type=cache,id=maven-${PRODUCT},uid=${STACKABLE_USER_UID},target=/stackable/.m2/repository \
--mount=type=cache,id=npm-${PRODUCT},uid=${STACKABLE_USER_UID},target=/stackable/.npm \
--mount=type=cache,id=cache-${PRODUCT},uid=${STACKABLE_USER_UID},target=/stackable/.cache \
<<EOF
cd "$(/stackable/patchable --images-repo-root=src checkout druid ${PRODUCT})"
cd "$(cat /tmp/DRUID_SOURCE_DIR)" || exit 1
rm /tmp/DRUID_SOURCE_DIR

mvn \
--batch-mode \
--no-transfer-progress \
clean install \
-Pdist,stackable-bundle-contrib-exts \
-Dmaven.test.skip `# Skip test compilation` \
-DskipTests `# Skip test execution` \
-Dcheckstyle.skip `# Skip checkstyle checks. We dont care if the code is properly formatted, it just wastes time` \
-Dmaven.javadoc.skip=true `# Dont generate javadoc` \
-Dmaven.gitcommitid.skip=true `# The gitcommitid plugin cannot work with git workspaces (ie: patchable)` \
$(if [[ ${PRODUCT} != 30.* ]]; then echo --projects '!quidem-ut'; fi) `# This is just a maven module for tests. https://github.com/apache/druid/pull/16867 added https://raw.githubusercontent.com/kgyrtkirk/datasets/repo/ as a Maven repository, which fails to pull for us (Failed to execute goal on project druid-quidem-ut: Could not resolve dependencies for project org.apache.druid:druid-quidem-ut:jar:33.0.0: com.github.kgyrtkirk.datasets:kttm-nested:jar:0.1 was not found in https://build-repo.stackable.tech/repository/maven-public/). By disabling the maven module we dont pull in this weird dependency...`

mvn --batch-mode --no-transfer-progress clean install -Pdist,stackable-bundle-contrib-exts -DskipTests -Dmaven.javadoc.skip=true
mv distribution/target/apache-druid-${PRODUCT}-bin/apache-druid-${PRODUCT} /stackable/
mv distribution/target/bom.json /stackable/apache-druid-${PRODUCT}/apache-druid-${PRODUCT}.cdx.json
rm -rf /stackable/apache-druid-${PRODUCT}-src
Expand Down

This file was deleted.

2 changes: 0 additions & 2 deletions druid/stackable/patches/30.0.0/patchable.toml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
From 0517d9fb940879ced639eb0d717ba4b5f67d9670 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Wed, 10 Jul 2024 17:07:13 +0200
Subject: Removes all traces of the druid ranger extension

---
distribution/pom.xml | 2 --
pom.xml | 1 -
2 files changed, 3 deletions(-)

diff --git a/distribution/pom.xml b/distribution/pom.xml
index 8ee50f9029..cb54001d20 100644
--- a/distribution/pom.xml
+++ b/distribution/pom.xml
@@ -458,8 +458,6 @@
<argument>org.apache.druid.extensions.contrib:druid-rabbit-indexing-service</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:grpc-query</argument>
- <argument>-c</argument>
- <argument>org.apache.druid.extensions.contrib:druid-ranger-security</argument>
</arguments>
</configuration>
</execution>
diff --git a/pom.xml b/pom.xml
index 1f7d6a69df..09daa8b2e9 100644
--- a/pom.xml
+++ b/pom.xml
@@ -258,7 +258,6 @@
<module>extensions-contrib/druid-deltalake-extensions</module>
<module>extensions-contrib/spectator-histogram</module>
<module>extensions-contrib/rabbit-stream-indexing-service</module>
- <module>extensions-contrib/druid-ranger-security</module>
<!-- distribution packaging -->
<module>distribution</module>
<!-- Revised integration tests -->
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 0cee640900bb8119ba6be12911578927aeb191ed Mon Sep 17 00:00:00 2001
From 7cd9440c7f484db79ca4a7b107fc0404dc9421a0 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Mon, 17 Feb 2025 16:42:34 +0100
Subject: Include Prometheus emitter in distribution
Expand All @@ -8,10 +8,10 @@ Subject: Include Prometheus emitter in distribution
1 file changed, 46 insertions(+)

diff --git a/distribution/pom.xml b/distribution/pom.xml
index e27329e96d..d5918710ef 100644
index cb54001d20..c8b7e13054 100644
--- a/distribution/pom.xml
+++ b/distribution/pom.xml
@@ -464,6 +464,52 @@
@@ -466,6 +466,52 @@
</plugins>
</build>
</profile>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 62f340d66dae20da8d7566f992b56223d29d4174 Mon Sep 17 00:00:00 2001
From 2259629e248db402a660b42d7eca75f17e164b7f Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Mon, 17 Feb 2025 16:42:34 +0100
Subject: Stop building unused extensions.
Expand All @@ -7,18 +7,16 @@ By default Druid builds all community extensions and then discards them
while assembling the final distribution. This patch removes unused
extensions from the build.
---
pom.xml | 32 +-------------------------------
1 file changed, 1 insertion(+), 31 deletions(-)
pom.xml | 31 -------------------------------
1 file changed, 31 deletions(-)

diff --git a/pom.xml b/pom.xml
index 3ab467e468..171a98f803 100644
index 09daa8b2e9..df781fac63 100644
--- a/pom.xml
+++ b/pom.xml
@@ -200,39 +200,9 @@
<module>extensions-core/google-extensions</module>
@@ -226,38 +226,7 @@
<module>extensions-core/druid-catalog</module>
<module>extensions-core/testing-tools</module>
+
<!-- Community extensions -->
- <module>extensions-contrib/compressed-bigdecimal</module>
- <module>extensions-contrib/influx-extensions</module>
Expand Down Expand Up @@ -47,11 +45,11 @@ index 3ab467e468..171a98f803 100644
- <module>extensions-contrib/aliyun-oss-extensions</module>
<module>extensions-contrib/prometheus-emitter</module>
- <module>extensions-contrib/opentelemetry-emitter</module>
- <module>extensions-contrib/kubernetes-overlord-extensions</module>
- <module>extensions-contrib/grpc-query</module>
- <module>extensions-contrib/druid-iceberg-extensions</module>
- <module>extensions-contrib/druid-deltalake-extensions</module>
- <module>extensions-contrib/spectator-histogram</module>
- <module>extensions-contrib/rabbit-stream-indexing-service</module>

<!-- distribution packaging -->
<module>distribution</module>
<!-- Revised integration tests -->
Loading