Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added to GitHub Workflow to run examples in CI #214

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

RobertJacobsonCDC
Copy link
Collaborator

Modified build-test.yaml to run this script in CI.

@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch 2 times, most recently from 5188766 to 4b2f797 Compare February 12, 2025 02:18
Modified `build-test.yaml` to run this script in CI.
@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch from 4b2f797 to d8ec460 Compare February 12, 2025 02:24
@RobertJacobsonCDC RobertJacobsonCDC marked this pull request as ready for review February 12, 2025 02:32
@RobertJacobsonCDC
Copy link
Collaborator Author

I'm not a fan of how this script works right now, because it exits when the first command fails to run. I'd rather it run every example and return an error code if any single example fails.

@RobertJacobsonCDC
Copy link
Collaborator Author

Here's an alternative approach that runs every example and exits with an error code if any single example fails.

#!/bin/bash

# Initialize a variable to track failures
has_error=0

# Function to run a command and log failures
run_command() {
    echo "Running: $1"
    $1
    if [ $? -ne 0 ]; then
        echo "Command failed: $1"
        has_error=1
    fi
}

# Run each command
run_command "cargo run --example basic-infection"
run_command "cargo run --example births-deaths"
# ... etc. ...
run_command "cargo run --example time-varying-infection ./examples/time-varying-infection/input.json"

# Exit with the appropriate code based on whether any command failed
if [ $has_error -eq 1 ]; then
    echo "One or more commands failed."
    exit 1
else
    echo "All commands executed successfully."
    exit 0
fi

@ekr-cfa
Copy link
Collaborator

ekr-cfa commented Feb 12, 2025

I agree that it's undesirable to have the whole job fail when a single example fails. In addition, I think we want to have individual failures visible in the GitHub interface. I think this pushes us towards each example being a separate job in GitHub actions using the GitHub actions "matrix" feature.

Here's a partial example:

#216

This is unfinished. There are a few areas for improvement here:

  1. As you can see, I'm using two jobs connected by an artifact so we only need to build once. However, for some reason (timestamps?) we're actually getting a rebuild. This needs some investigation.
  2. You need some way to give time-varying-infection an argument for the config file. I assume you need some list of the arguments for each example.
  3. It should be possible to auto-detect all the examples, e.g., run cargo --example and then use a script to feed this into the matrix. Of course that only works if they don't take arguments

At minimum we need to fix (1) and (2). However, I think this is a more promising angle than a script.

@ekr-cfa
Copy link
Collaborator

ekr-cfa commented Feb 12, 2025

This is what it looks like: https://github.com/CDCgov/ixa/actions/runs/13291542297

@RobertJacobsonCDC
Copy link
Collaborator Author

Thanks for this work! I'll add "WIP" to this PR and keep working on this.

I've been getting a list of examples with this bash one-liner:

EXAMPLES=$(cargo run --example 2>&1 | tail -n +3)

But you're right, this only works for examples without arguments.

@RobertJacobsonCDC RobertJacobsonCDC changed the title Added the script examples/run_examples.sh, which runs all examples. WIP: Added the script examples/run_examples.sh, which runs all examples. Feb 12, 2025
@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch from cb62f57 to db07707 Compare February 13, 2025 04:38
@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch 2 times, most recently from f981772 to 2bbc74b Compare February 13, 2025 04:58
@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch from 2bbc74b to 8242e2d Compare February 13, 2025 05:07
@RobertJacobsonCDC RobertJacobsonCDC self-assigned this Feb 13, 2025
@RobertJacobsonCDC RobertJacobsonCDC changed the title WIP: Added the script examples/run_examples.sh, which runs all examples. WIP: Added GitHub Workflow examples.yaml to run examples in CI Feb 13, 2025
…f the dependencies and dependency build artifacts.
@RobertJacobsonCDC RobertJacobsonCDC force-pushed the RobertJacobsonCDC_198_examples_in_ci branch from cf567c5 to fa413b6 Compare February 13, 2025 22:36
@RobertJacobsonCDC RobertJacobsonCDC changed the title WIP: Added GitHub Workflow examples.yaml to run examples in CI Added GitHub Workflow examples.yaml to run examples in CI Feb 13, 2025
@RobertJacobsonCDC RobertJacobsonCDC changed the title Added GitHub Workflow examples.yaml to run examples in CI Added to GitHub Workflow to run examples in CI Feb 13, 2025
@RobertJacobsonCDC
Copy link
Collaborator Author

Cached artifacts appear to be saved and restored correctly, but I cannot observe any difference in compile times.

  • Other examples across the web do not cache target/debug/.fingerprint nor do they update timestamps, so these steps may be unnecessary in a working solution.
  • It's cache, not cash.
  • Note that cargo says "compiling" event when it doesn't actually compile, so use --verbose.

@RobertJacobsonCDC RobertJacobsonCDC changed the title Added to GitHub Workflow to run examples in CI WIP: Added to GitHub Workflow to run examples in CI Feb 14, 2025
@k88hudson-cfa k88hudson-cfa requested review from k88hudson-cfa and removed request for ekr-cfa February 18, 2025 14:46
@RobertJacobsonCDC RobertJacobsonCDC changed the title WIP: Added to GitHub Workflow to run examples in CI Added to GitHub Workflow to run examples in CI Feb 18, 2025
Manually bumped version to invalidate the cache.
@RobertJacobsonCDC
Copy link
Collaborator Author

Examples are running. Caching of dependencies and their build artifacts has been bumped to the backlog as a separate task for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants