Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control build.sh and .gitignore #1526

Merged
merged 5 commits into from
Mar 25, 2024
Merged

Control build.sh and .gitignore #1526

merged 5 commits into from
Mar 25, 2024

Conversation

dmikusa
Copy link
Contributor

@dmikusa dmikusa commented Mar 16, 2024

Summary

This PR adds control of the build.sh and .gitignore files over to octo. When it runs, it will generate those files. The build.sh script comes from a template file which is static at the moment. This could be modified to be a go template or something more flexible if necessary in the future. The .gitignore file is generated from a list of strings. For now, this ends up being static, but we could add in logic to generate the list dynamically if needed in the future.

Use Cases

Automatically update the build.sh and .gitignore files, which are basically the same across all the repos. This helps make changes to the resources easier to apply.

Resolves #1100

The only caveat to this change is that we'll need to be very careful on the first run though. This change assumes that the build.sh file is the same for all managed buildpacks. I think that's true, but the first time we'll want to merge all the PRs manually and review carefully to make sure that we're not overwriting anything important.

Checklist

  • I have viewed, signed, and submitted the Contributor License Agreement.
  • I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
  • I have added an integration test, if necessary.
  • I have reviewed the styleguide for guidance on my code quality.
  • I'm happy with the commit history on this PR (I have rebased/squashed as needed).

This PR adds control of the build.sh and .gitignore files over to `octo`. When it runs, it will generate those files. The build.sh script comes from a template file which is static at the moment. This could be modified to be a go template or something more flexible if necessary in the future. The .gitignore file is generated from a list of strings. For now, this ends up being static, but we could add in logic to generate the list dynamically if needed in the future.

Signed-off-by: Daniel Mikusa <[email protected]>
@dmikusa dmikusa added type:enhancement A general enhancement semver:minor A change requiring a minor version bump labels Mar 16, 2024
@dmikusa dmikusa requested a review from a team as a code owner March 16, 2024 02:00
dmikusa added 3 commits March 15, 2024 22:05
This prevents the tool from adding the script to repos that don't need it. If a repo needs it, it will already have the file. For new repos, you'll need to touch the file first or copy it from another repo.

Signed-off-by: Daniel Mikusa <[email protected]>
Some buildpacks also need to build helpers, like JVM provider buildpacks. The helper needs to be configured in pipeline-descriptor.yml. This commit adds a new `helpers` section where you can define the helpers to be created. The helpers map has the key of the file into which the helper is compiled and the value of which is the module to compile.

For example:

```
helpers:
  "bin/helper": "github.com/paketo-buildpacks/libjvm/cmd/helper"
```

Will result in two lines being added to the build script:

```
GOOS="linux" go build -ldflags='-s -w' -o "bin/helper" "github.com/paketo-buildpacks/libjvm/cmd/helper"
GOOS="linux" GOARCH="arm64" go build -ldflags='-s -w' -o "bin/helper" "github.com/paketo-buildpacks/libjvm/cmd/helper":w
```

If you need to reference the module for the current buildpack, you can use the literal `$GOMOD` which is present when the build script runs. For example `"bin/helper": "$GOMOD/cmd/helper"` would build `cmd/helper` from the present module.

Signed-off-by: Daniel Mikusa <[email protected]>
@dmikusa
Copy link
Contributor Author

dmikusa commented Mar 17, 2024

Thanks, I hit some similar issues. I have a change to account for this but am doing some more testing with it. Will update soon.

Signed-off-by: Daniel Mikusa <[email protected]>
@dmikusa
Copy link
Contributor Author

dmikusa commented Mar 18, 2024

Ok, I think this is ready now.

What I added is a helpers: section in the pipeline-descriptor.yml file. That is an optional map that is used to define any additional binaries that should be compiled for this buildpack. The format is key=path and value=go-mod-build-path. The path is relative to the os/arch-specific folder in the generated buildpack file structure. The go-mod-build-path is the part you'd pass to go build to tell it what to build.

Ex: build cmd/helper from the current buildpack

helpers:
  "bin/helper": "$GOMOD/cmd/helper"

Ex: build cmd/helper from libjvm

helpers:
  "bin/helper": "github.com/paketo-buildpacks/libjvm/cmd/helper"

Then you end up with a structure like:

linux/amd64/bin:
total 21392
lrwxr-xr-x  1 dmikusa  staff        4 Mar 16 17:29 build@ -> main
lrwxr-xr-x  1 dmikusa  staff        4 Mar 16 17:29 detect@ -> main
-rwxr-xr-x  1 dmikusa  staff  4325528 Mar 16 17:29 helper*
-rwxr-xr-x  1 dmikusa  staff  6619288 Mar 16 17:29 main*

linux/arm64/bin:
total 21392
lrwxr-xr-x  1 dmikusa  staff        4 Mar 16 17:29 build@ -> main
lrwxr-xr-x  1 dmikusa  staff        4 Mar 16 17:29 detect@ -> main
-rwxr-xr-x  1 dmikusa  staff  4325528 Mar 16 17:29 helper*
-rwxr-xr-x  1 dmikusa  staff  6619288 Mar 16 17:29 main*

@dmikusa
Copy link
Contributor Author

dmikusa commented Mar 19, 2024

When we release this, we need to be careful. We can't auto-merge all the PRs because some of the buildpacks will need updates to pipeline-descriptor.yml (for helpers to be added).

Copy link
Member

@anthonydahanne anthonydahanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all good!

@dmikusa dmikusa merged commit fa9ceca into main Mar 25, 2024
5 checks passed
@dmikusa dmikusa deleted the update-build-ignore branch March 25, 2024 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver:minor A change requiring a minor version bump type:enhancement A general enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable pipeline builder to manage scripts/build.sh
2 participants