GODRIVER-3599 Add task and script to generate CycloneDX SBOM #2154
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| name: Generate SBOM | ||
| # Generates sbom.json using cyclonedx-gomod and opens a PR if it has changed. | ||
| # Triggered when go.mod changes on master, or manually. | ||
| # Internal documentation: go/sbom-scope | ||
| on: | ||
| workflow_dispatch: {} | ||
| push: | ||
| branches: | ||
| - master | ||
| paths: | ||
| - go.mod | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| jobs: | ||
| sbom: | ||
| name: Generate SBOM and Create PR | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: sbom-${{ github.ref }} | ||
| cancel-in-progress: false | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| - name: Set up Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version-file: go.mod | ||
| - name: Stash existing SBOM | ||
| run: | | ||
| # Strip nondeterministic fields before diffing | ||
| jq 'del(.metadata.timestamp, .version)' sbom.json > ${{ runner.temp }}/sbom.existing.json | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
| - name: Generate SBOM | ||
| run: bash etc/generate-sbom.sh | ||
| - name: Download CycloneDX CLI | ||
| run: | | ||
| curl -L -s -o ${{ runner.temp }}/cyclonedx \ | ||
| "https://github.com/CycloneDX/cyclonedx-cli/releases/download/v0.32.0/cyclonedx-linux-x64" | ||
| echo "454879e6a4a405c8a13bff49b8982adcb0596f3019b26b0811c66e4d7f0783e1 ${{ runner.temp }}/cyclonedx" | sha256sum --check | ||
| chmod +x ${{ runner.temp }}/cyclonedx | ||
| - name: Validate SBOM | ||
| run: ${{ runner.temp }}/cyclonedx validate --input-file sbom.json --fail-on-errors | ||
| - name: Check for SBOM changes | ||
| id: sbom_diff | ||
| run: | | ||
| jq 'del(.metadata.timestamp, .version)' sbom.json > ${{ runner.temp }}/sbom.generated.json | ||
| RESULT=$(diff --brief ${{ runner.temp }}/sbom.existing.json ${{ runner.temp }}/sbom.generated.json || true) | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
| echo "result=$RESULT" | tee -a $GITHUB_OUTPUT | ||
| - name: Open Pull Request if SBOM has changed | ||
| if: steps.sbom_diff.outputs.result | ||
| uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 | ||
| with: | ||
| add-paths: sbom.json | ||
| branch: auto-update-sbom-${{ github.ref_name }} | ||
| commit-message: "chore: Update SBOM after dependency changes" | ||
| delete-branch: true | ||
| labels: sbom | ||
| title: "Automation: Update SBOM [${{ github.ref_name }}]" | ||
| body: | | ||
| ## Automated SBOM Update | ||
|
|
||
| This PR was automatically generated because `go.mod` changed. | ||
|
|
||
| ### Triggered by | ||
| - Commit: ${{ github.sha }} | ||
| - Workflow run: [${{ github.run_id }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| #!/usr/bin/env bash | ||
|
Member
There was a problem hiding this comment. [blocking] CycloneDX notes that you can use as a library ( > v1.2.0). We should opt for this to avoid a CLI dependency in local dev and VM CI and convert etc/generate-sbom.sh to a go script at internal/cmd/generate-sbom/. Despite being more LoC, it's easier for the team to read, update, and bump. Here is a gist: https://gist.github.com/prestonvasquez/04a194f2f0aff1ba010754e5e9057eb5 Typically, we would defer this work to a followup ticket, but I'm concerned that we could easily miss something with a de facto re-write. Let me know if this would work better if we wrote the Go code and you reviewed for correctness. |
||
| set -e | ||
|
jasonhills-mongodb marked this conversation as resolved.
Outdated
|
||
|
|
||
| CHECK_CURRENCY="false" | ||
|
|
||
| # Options are: | ||
| # -c : check currency of staged sbom.json versus go.mod. | ||
| while getopts "c" opt; do | ||
| case $opt in | ||
| c) | ||
| CHECK_CURRENCY="true" | ||
| ;; | ||
| *) | ||
| echo "usage: $0 [-c]" >&2 | ||
| echo " -c : (optional) check currency of staged sbom.json versus go.mod." >&2 | ||
| exit 1 | ||
| ;; | ||
| esac | ||
| done | ||
| #shift $((OPTIND - 1)) | ||
|
|
||
| if ! $CHECK_CURRENCY; then | ||
|
jasonhills-mongodb marked this conversation as resolved.
Outdated
|
||
| # The cyclonedx-gomod 'mod' subcommand is used to generate a CycloneDX SBOM with GOWORK=off to exclude example/test code. | ||
|
|
||
| ## The pipe to jq is a temporary workaround until this issue is resolved: https://github.com/CycloneDX/cyclonedx-gomod/issues/662. | ||
| ## When resolved, bump version and replace with commented line below. | ||
| # GOWORK=off go run github.com/CycloneDX/cyclonedx-gomod/cmd/cyclonedx-gomod@[UPDATED VERSION] mod -type library -licenses -assert-licenses -output-version 1.5 -json -output sbom.json . | ||
| GOWORK=off go run github.com/CycloneDX/cyclonedx-gomod/cmd/cyclonedx-gomod@v1.10.0 mod -type library -licenses -assert-licenses -output-version 1.5 -json . | jq '.metadata.component.purl |= split("?")[0]' | jq '.components[].purl |= split("?")[0]' > sbom.json | ||
|
|
||
| # Derive the libmongocrypt version from the install script and inject it as an optional component. | ||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| LIBMONGOCRYPT_VERSION=$(grep 'LIBMONGOCRYPT_TAG=' "${SCRIPT_DIR}/install-libmongocrypt.sh" | head -1 | cut -d'"' -f2) | ||
| LIBMONGOCRYPT_PURL="pkg:github/mongodb/libmongocrypt@${LIBMONGOCRYPT_VERSION}" | ||
|
|
||
| jq --arg version "$LIBMONGOCRYPT_VERSION" --arg purl "$LIBMONGOCRYPT_PURL" ' | ||
| .metadata.component."bom-ref" as $main_ref | | ||
| .components += [{ | ||
| "type": "library", | ||
| "bom-ref": $purl, | ||
| "supplier": {"name": "MongoDB, Inc.", "url": ["https://mongodb.com"]}, | ||
| "author": "MongoDB, Inc.", | ||
| "group": "mongodb", | ||
| "name": "libmongocrypt", | ||
| "version": $version, | ||
| "description": "Required C library for Client Side and Queryable Encryption in MongoDB", | ||
| "scope": "optional", | ||
| "licenses": [{"license": {"id": "Apache-2.0"}}], | ||
| "copyright": "Copyright 2019-present MongoDB, Inc.", | ||
| "cpe": ("cpe:2.3:a:mongodb:libmongocrypt:" + $version + ":*:*:*:*:*:*:*"), | ||
| "purl": $purl, | ||
| "externalReferences": [{"url": "https://github.com/mongodb/libmongocrypt.git", "type": "distribution"}] | ||
| }] | | ||
| (.dependencies[] | select(.ref == $main_ref) | .dependsOn) += [$purl] | | ||
| .dependencies += [{"ref": $purl, "dependsOn": []}] | ||
| ' sbom.json > sbom.tmp.json && mv sbom.tmp.json sbom.json | ||
|
|
||
| elif [[ $(git diff --name-only --cached go.mod) && ! $(git diff --name-only --cached sbom.json) ]]; then | ||
| echo "'go.mod' has changed. 'sbom.json' must be re-generated (run 'task generate-sbom' or 'etc/generate-sbom.sh') and staged." && exit 1 | ||
|
jasonhills-mongodb marked this conversation as resolved.
Outdated
|
||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| #!/usr/bin/env bash | ||
|
Member
There was a problem hiding this comment. [blocking] Same with generate-sbom.sh, this could be a Go script that leverages the docker API. Here is a gist: https://gist.github.com/prestonvasquez/7d83b6bd919fbcb133bcbcc053e4a2ed |
||
|
|
||
| set -o errexit | ||
| set -o pipefail | ||
|
|
||
| : "${branch_name:?}" | ||
| : "${DOCKER_CONFIG:?}" | ||
| : "${AWS_ACCESS_KEY_ID:?}" | ||
| : "${AWS_SECRET_ACCESS_KEY:?}" | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
|
|
||
| command -v podman >/dev/null || { | ||
| echo "missing required program podman" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| command -v jq >/dev/null || { | ||
| echo "missing required program jq" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| silkbomb="901841024863.dkr.ecr.us-east-1.amazonaws.com/release-infrastructure/silkbomb:2.0" | ||
|
|
||
| podman pull "${silkbomb:?}" | ||
|
jasonhills-mongodb marked this conversation as resolved.
Member
There was a problem hiding this comment. [blocking] Is there a reason we use podman over docker? Docker is preferred by the team. If it needs to be agnostic, we should make this an env variable. E.g. |
||
|
|
||
| silkbomb_augment_flags=( | ||
| --repo mongodb/mongo-go-driver | ||
| --branch "${branch_name:?}" | ||
| --sbom-in /pwd/sbom.json | ||
| --sbom-out /pwd/augmented.sbom.json.new | ||
|
|
||
| # Any notable updates to the Augmented SBOM version should be done manually after careful inspection. | ||
| # Otherwise, it should be equal to the existing SBOM version. | ||
| --no-update-sbom-version | ||
| ) | ||
|
|
||
| podman run -it --rm -v "$(pwd):/pwd" --env 'AWS_ACCESS_KEY_ID' --env 'AWS_SECRET_ACCESS_KEY' \ | ||
|
jasonhills-mongodb marked this conversation as resolved.
Outdated
jasonhills-mongodb marked this conversation as resolved.
Outdated
|
||
| "${silkbomb:?}" augment "${silkbomb_augment_flags[@]:?}" | ||
|
|
||
| [[ -f ./augmented.sbom.json.new ]] || { | ||
| echo "failed to download Augmented SBOM" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| echo "Comparing Augmented SBOM..." | ||
|
|
||
| jq -S 'del(.metadata.timestamp)' ./augmented.sbom.json >|old.json | ||
| jq -S 'del(.metadata.timestamp)' ./augmented.sbom.json.new >|new.json | ||
|
|
||
| if ! diff -sty --left-column -W 200 old.json new.json >|diff.txt; then | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
| declare status | ||
| status='{"status":"failed", "type":"test", "should_continue":true, "desc":"detected significant changes in Augmented SBOM"}' | ||
| curl -sS -d "${status:?}" -H "Content-Type: application/json" -X POST localhost:2285/task_status || true | ||
| fi | ||
|
|
||
| cat diff.txt | ||
|
|
||
| echo "Comparing Augmented SBOM... done." | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.