Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Bakefile Multi Arch builds #333

Closed
wants to merge 7 commits into from

Conversation

ryanolee
Copy link
Contributor

@ryanolee ryanolee commented Nov 16, 2024

Note

This is a request for comment, it is not a final implementation. I am happy to push it forwards to a completed state if it seems like it will be useful to the overall project structure / aims. Though given how substantial this change is to the overall build process can understand if the changes are too large for the stated aims of this PR.

Warning

It should be noted that bakefiles are still somewhat experimental https://docs.docker.com/build/bake/

First thanks all for the amazing project, really cool to try out this project!

What

This PR aims to address #290 #330 and #128 by migrating the build process from github actions into a docker bakefile for more centralized control over which docker files are built and to allow for multi-arch builds.

Why

In order to be able to build multiarch docker files so that ARM platforms can be tested with. Especially with ARM containers normally being cheaper to run on certain CaaS Platforms for similar amounts of compute. (Also I thought it would be funny to run dask on a raspberry Pi cluster🙊

How

This PR implements a new bakefile which aims to simplify the overall build flow in CI offloading much of the responsibility to docker itself.

It should be noted that:

  • Tagging as been replicated in the bakefile as it currently functions in CI
  • Base images are pulled directly from GIT to be built at the versions we require things to be built with

How to test

See passing CI Jobs here: https://github.com/ryanolee/dask-docker/actions
And relevant docker images here: https://github.com/ryanolee?tab=packages&repo_name=dask-docker

Try:

version: "3.1"

services:
  scheduler:
    image: ghcr.io/ryanolee/dask:2024.9.1-py3.12
    hostname: scheduler
    ports:
      - "8786:8786"
      - "8787:8787"
    command: ["dask-scheduler"]

  worker:
    image: ghcr.io/ryanolee/dask:2024.9.1-py3.12
    command: ["dask-worker", "tcp://scheduler:8786"]
    # For Docker swarm you can specify multiple workers, this is ignored by `docker-compose up`
    deploy:
      replicas: 4

  notebook:
    image: ghcr.io/ryanolee/dask-notebook:2024.9.1-py3.12
    ports:
      - "8888:8888"
    environment:
      - DASK_SCHEDULER_ADDRESS="tcp://scheduler:8786"

OR

Run make build-docker and import the images from your build context and reference them in a similar way.

Has been tested on 4 Node Raspberry Pi cluster using Docker swarm and works as expected for everything I have tried it with so for.

x86 Builds have been tested on WSL2, Ubuntu 24.04 and have worked as expected also.

@jacobtomlinson
Copy link
Member

I just merged #337 which adds multi arch support using the existing GitHub Actions based workflow. I appreciate there may be more benefits to doing things with bake files but it felt like a simpler approach. Thanks for the effort here but it looks like this draft stalled and I'm going to close this out.

@ryanolee
Copy link
Contributor Author

ryanolee commented Jan 8, 2025

No worries @jacobtomlinson ! Built really for my own needs given a requirement for multi-arch docker containers. Opened as an RFC to see if there was any interest in pushing it further! Great job getting arm64 builds into the mainline 🚀

@ryanolee
Copy link
Contributor Author

ryanolee commented Jan 8, 2025

As a quick follow up you might need to quickly add export G_SLICE=always-malloc to where you call mamba (See in this PR)

As QEMU can cause mamba to hang as per details in this comment dask/dask-kubernetes#919 (comment)

It seem current main is susceptible to this
https://github.com/dask/dask-docker/actions/runs/12659677038/job/35279267366

@ryanolee
Copy link
Contributor Author

ryanolee commented Jan 8, 2025

Opened #339 to resolve 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants