GODRIVER-3599 Add task and script to generate CycloneDX SBOM #2154
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| name: Generate SBOM | ||
| # Generates sbom.json using cyclonedx-gomod and opens a PR if it has changed. | ||
| # Triggered when go.mod changes on master, or manually. | ||
| # Internal documentation: go/sbom-scope | ||
| on: | ||
| workflow_dispatch: {} | ||
| push: | ||
| branches: | ||
| - master | ||
| paths: | ||
| - go.mod | ||
| - go.sum | ||
| - etc/install-libmongocrypt.sh | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| jobs: | ||
| sbom: | ||
| name: Generate SBOM and Create PR | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: sbom-${{ github.ref }} | ||
| cancel-in-progress: false | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| - name: Set up Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version-file: go.mod | ||
| - name: Stash existing SBOM | ||
| run: | | ||
| # Strip nondeterministic fields before diffing | ||
| jq 'del(.metadata.timestamp, .version)' sbom.json > ${{ runner.temp }}/sbom.existing.json | ||
| - name: Generate SBOM | ||
| run: bash etc/generate-sbom.sh | ||
| - name: Download CycloneDX CLI | ||
| run: | | ||
| curl -fsSL -o ${{ runner.temp }}/cyclonedx \ | ||
| "https://github.com/CycloneDX/cyclonedx-cli/releases/download/v0.32.0/cyclonedx-linux-x64" | ||
| echo "454879e6a4a405c8a13bff49b8982adcb0596f3019b26b0811c66e4d7f0783e1 ${{ runner.temp }}/cyclonedx" | sha256sum --check | ||
| chmod +x ${{ runner.temp }}/cyclonedx | ||
| - name: Validate SBOM | ||
| run: ${{ runner.temp }}/cyclonedx validate --input-file sbom.json --fail-on-errors | ||
| - name: Check for SBOM changes | ||
| id: sbom_diff | ||
| run: | | ||
| jq 'del(.metadata.timestamp, .version)' sbom.json > ${{ runner.temp }}/sbom.generated.json | ||
| RESULT=$(diff --brief ${{ runner.temp }}/sbom.existing.json ${{ runner.temp }}/sbom.generated.json || true) | ||
| echo "result=$RESULT" | tee -a $GITHUB_OUTPUT | ||
| - name: Open Pull Request if SBOM has changed | ||
| if: steps.sbom_diff.outputs.result | ||
| uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1 | ||
| with: | ||
| add-paths: sbom.json | ||
| branch: auto-update-sbom-${{ github.ref_name }} | ||
| commit-message: "chore: Update SBOM after dependency changes" | ||
| delete-branch: true | ||
| labels: sbom | ||
| title: "Automation: Update SBOM [${{ github.ref_name }}]" | ||
| body: | | ||
| ## Automated SBOM Update | ||
|
|
||
| This PR was automatically generated because a change to `go.mod`, `go.sum`, or | ||
| `etc/install-libmongocrypt.sh` was merged into `${{ github.ref_name }}`. | ||
|
|
||
| `sbom.json` has been regenerated to reflect the current module dependency graph | ||
| and `libmongocrypt` version. The SBOM serial number is stable; the `.version` | ||
| field has been incremented to reflect this update. | ||
|
|
||
| Once merged, an Evergreen CI task will upload the updated SBOM to the internal | ||
| tracking system. | ||
|
|
||
| ### Triggered by | ||
| - Commit: ${{ github.sha }} | ||
| - Workflow run: [${{ github.run_id }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| #!/usr/bin/env bash | ||
|
Member
There was a problem hiding this comment. [blocking] CycloneDX notes that you can use as a library ( > v1.2.0). We should opt for this to avoid a CLI dependency in local dev and VM CI and convert etc/generate-sbom.sh to a go script at internal/cmd/generate-sbom/. Despite being more LoC, it's easier for the team to read, update, and bump. Here is a gist: https://gist.github.com/prestonvasquez/04a194f2f0aff1ba010754e5e9057eb5 Typically, we would defer this work to a followup ticket, but I'm concerned that we could easily miss something with a de facto re-write. Let me know if this would work better if we wrote the Go code and you reviewed for correctness. |
||
| set -eo pipefail | ||
|
|
||
| SERIAL_NUMBER="urn:uuid:b7adcdf8-bafc-43c5-a529-a73130697171" | ||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
|
|
||
| # Generate the base SBOM. -short-purls strips ?type=module query strings from PURLs, | ||
| # resolving https://github.com/CycloneDX/cyclonedx-gomod/issues/662. | ||
| GOWORK=off go run github.com/CycloneDX/cyclonedx-gomod/cmd/cyclonedx-gomod@v1.10.0 \ | ||
| mod -type library -licenses -assert-licenses -output-version 1.5 -json \ | ||
| -serial "$SERIAL_NUMBER" -std=true -short-purls=true . > sbom.tmp.json | ||
|
|
||
| # Inject libmongocrypt as an optional component. | ||
| LIBMONGOCRYPT_VERSION=$(grep 'LIBMONGOCRYPT_TAG=' "${SCRIPT_DIR}/install-libmongocrypt.sh" | head -1 | cut -d'"' -f2) | ||
| LIBMONGOCRYPT_PURL="pkg:github/mongodb/libmongocrypt@${LIBMONGOCRYPT_VERSION}" | ||
|
|
||
| jq --arg version "$LIBMONGOCRYPT_VERSION" --arg purl "$LIBMONGOCRYPT_PURL" ' | ||
| .metadata.component."bom-ref" as $main_ref | | ||
| .components += [{ | ||
| "type": "library", | ||
| "bom-ref": $purl, | ||
| "supplier": {"name": "MongoDB, Inc.", "url": ["https://mongodb.com"]}, | ||
| "author": "MongoDB, Inc.", | ||
| "group": "mongodb", | ||
| "name": "libmongocrypt", | ||
| "version": $version, | ||
| "description": "Required C library for Client Side and Queryable Encryption in MongoDB", | ||
| "scope": "optional", | ||
| "licenses": [{"license": {"id": "Apache-2.0"}}], | ||
| "copyright": "Copyright 2019-present MongoDB, Inc.", | ||
| "cpe": ("cpe:2.3:a:mongodb:libmongocrypt:" + $version + ":*:*:*:*:*:*:*"), | ||
| "purl": $purl, | ||
| "externalReferences": [{"url": "https://github.com/mongodb/libmongocrypt.git", "type": "distribution"}] | ||
| }] | | ||
| (.dependencies[] | select(.ref == $main_ref) | .dependsOn) += [$purl] | | ||
| .dependencies += [{"ref": $purl, "dependsOn": []}] | ||
| ' sbom.tmp.json > sbom.new.json | ||
| rm sbom.tmp.json | ||
|
|
||
| # Increment .version only when meaningful content has changed. | ||
| CURRENT_VERSION=$(jq -r '.version // 0' sbom.json 2>/dev/null || echo 0) | ||
| NEW_CONTENT=$(jq -S 'del(.version, .metadata.timestamp)' sbom.new.json) | ||
| OLD_CONTENT=$(jq -S 'del(.version, .metadata.timestamp)' sbom.json 2>/dev/null || echo '{}') | ||
|
|
||
| if [ "$NEW_CONTENT" = "$OLD_CONTENT" ]; then | ||
| NEW_VERSION=$CURRENT_VERSION | ||
| else | ||
| NEW_VERSION=$((CURRENT_VERSION + 1)) | ||
| fi | ||
|
|
||
| jq --argjson v "$NEW_VERSION" '.version = $v' sbom.new.json > sbom.json | ||
| rm sbom.new.json | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| #!/usr/bin/env bash | ||
|
Member
There was a problem hiding this comment. [blocking] Same with generate-sbom.sh, this could be a Go script that leverages the docker API. Here is a gist: https://gist.github.com/prestonvasquez/7d83b6bd919fbcb133bcbcc053e4a2ed |
||
|
|
||
| set -o errexit | ||
| set -o pipefail | ||
|
|
||
| : "${branch_name:?}" | ||
| : "${DOCKER_CONFIG:?}" | ||
| : "${AWS_ACCESS_KEY_ID:?}" | ||
| : "${AWS_SECRET_ACCESS_KEY:?}" | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
| : "${AWS_SESSION_TOKEN:?}" | ||
|
|
||
| command -v podman >/dev/null || { | ||
| echo "missing required program podman" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| command -v jq >/dev/null || { | ||
| echo "missing required program jq" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| silkbomb="901841024863.dkr.ecr.us-east-1.amazonaws.com/release-infrastructure/silkbomb:2.0" | ||
|
|
||
| podman pull "${silkbomb:?}" | ||
|
jasonhills-mongodb marked this conversation as resolved.
Member
There was a problem hiding this comment. [blocking] Is there a reason we use podman over docker? Docker is preferred by the team. If it needs to be agnostic, we should make this an env variable. E.g. |
||
|
|
||
| silkbomb_augment_flags=( | ||
| --repo mongodb/mongo-go-driver | ||
| --branch "${branch_name:?}" | ||
| --sbom-in /pwd/sbom.json | ||
| --sbom-out /pwd/augmented.sbom.json.new | ||
|
|
||
| # Any notable updates to the Augmented SBOM version should be done manually after careful inspection. | ||
| # Otherwise, it should be equal to the existing SBOM version. | ||
| --no-update-sbom-version | ||
| ) | ||
|
|
||
| podman run --rm -v "$(pwd):/pwd" \ | ||
| --env 'AWS_ACCESS_KEY_ID' --env 'AWS_SECRET_ACCESS_KEY' --env 'AWS_SESSION_TOKEN' \ | ||
| "${silkbomb:?}" augment "${silkbomb_augment_flags[@]:?}" | ||
|
|
||
| [[ -f ./augmented.sbom.json.new ]] || { | ||
| echo "failed to download Augmented SBOM" 1>&2 | ||
| exit 1 | ||
| } | ||
|
|
||
| echo "Comparing Augmented SBOM..." | ||
|
|
||
| jq -S 'del(.metadata.timestamp)' ./augmented.sbom.json >|old.json | ||
| jq -S 'del(.metadata.timestamp)' ./augmented.sbom.json.new >|new.json | ||
|
|
||
| if ! diff -sty --left-column -W 200 old.json new.json >|diff.txt; then | ||
|
jasonhills-mongodb marked this conversation as resolved.
|
||
| declare status | ||
| status='{"status":"failed", "type":"test", "should_continue":true, "desc":"detected significant changes in Augmented SBOM"}' | ||
| curl -sS -d "${status:?}" -H "Content-Type: application/json" -X POST localhost:2285/task_status || true | ||
| fi | ||
|
|
||
| cat diff.txt | ||
|
|
||
| echo "Comparing Augmented SBOM... done." | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.