From 1d7fab62b6c25fdef2de64200dcebe7cfd0e0e30 Mon Sep 17 00:00:00 2001 From: Kirk Rodrigues <2454684+kirkrodrigues@users.noreply.github.com> Date: Mon, 24 Aug 2026 18:28:47 -0400 Subject: [PATCH 1/2] docs: Document GitHub workflows and jobs missed in previous PRs. --- docs/src/dev-docs/tooling-gh-workflows.md | 105 ++++++++++++++++++---- 1 file changed, 89 insertions(+), 16 deletions(-) mode change 100644 => 100755 docs/src/dev-docs/tooling-gh-workflows.md diff --git a/docs/src/dev-docs/tooling-gh-workflows.md b/docs/src/dev-docs/tooling-gh-workflows.md old mode 100644 new mode 100755 index e619fd4ea0..216d0964ef --- a/docs/src/dev-docs/tooling-gh-workflows.md +++ b/docs/src/dev-docs/tooling-gh-workflows.md @@ -5,11 +5,17 @@ builds, testing, and linting. We briefly describe each workflow below. ## clp-artifact-build -This workflow is responsible for: +This workflow is responsible for the following: -1. building (Linux) container images containing CLP-core's dependencies, -2. building CLP-core and running its unit & integration tests, and -3. building a container image containing CLP's package components. +1. Building container images containing the dependencies necessary to build all CLP artifacts. +2. Building CLP-core, the CLP-core Python wheels, and the CLP package. +3. Building container images containing: + + * CLP-core; + * the CLP package; and + * the binaries necessary for the Spider worker. + +4. Running C++ linting checks, unit tests, and integration tests on the built artifacts. To minimize build times, the jobs in the workflow are organized in the directed acyclic graph (DAG) shown below. @@ -29,24 +35,48 @@ shown below. } }%% flowchart LR + %% CLP-artifact-dependency container build jobs filter-relevant-changes --> centos-stream-9-deps-image filter-relevant-changes --> manylinux_2_28-deps-image filter-relevant-changes --> musllinux_1_2-deps-image - filter-relevant-changes --> ubuntu-jammy-deps-image + filter-relevant-changes --> ubuntu-jammy-aarch64-deps-image + filter-relevant-changes --> ubuntu-jammy-x86_64-deps-image + manylinux_2_28-deps-image --> manylinux_2_28-deps-image-merge + musllinux_1_2-deps-image --> musllinux_1_2-deps-image-merge + + %% CLP-core build jobs filter-relevant-changes --> centos-stream-9-binaries - filter-relevant-changes --> manylinux_2_28-x86_64-binaries - filter-relevant-changes --> musllinux_1_2-x86_64-binaries - filter-relevant-changes --> ubuntu-jammy-binaries centos-stream-9-deps-image --> centos-stream-9-binaries - manylinux_2_28-deps-image --> manylinux_2_28-deps-image-merge + filter-relevant-changes --> manylinux_2_28-x86_64-binaries manylinux_2_28-deps-image --> manylinux_2_28-x86_64-binaries manylinux_2_28-deps-image-merge --> manylinux_2_28-x86_64-binaries - musllinux_1_2-deps-image --> musllinux_1_2-deps-image-merge + filter-relevant-changes --> musllinux_1_2-x86_64-binaries musllinux_1_2-deps-image --> musllinux_1_2-x86_64-binaries musllinux_1_2-deps-image-merge --> musllinux_1_2-x86_64-binaries - ubuntu-jammy-deps-image --> ubuntu-jammy-binaries - ubuntu-jammy-deps-image --> package-image + filter-relevant-changes --> ubuntu-jammy-binaries + ubuntu-jammy-x86_64-deps-image --> ubuntu-jammy-binaries + + %% CLP-core binaries container build jobs ubuntu-jammy-binaries --> ubuntu-jammy-binaries-image + + %% CLP-core Python-wheel build jobs + filter-relevant-changes --> manylinux_2_28-x86_64-python-wheels + manylinux_2_28-deps-image --> manylinux_2_28-x86_64-python-wheels + manylinux_2_28-deps-image-merge --> manylinux_2_28-x86_64-python-wheels + + %% CLP-package container build jobs + filter-relevant-changes --> package-image + ubuntu-jammy-aarch64-deps-image --> package-image + ubuntu-jammy-x86_64-deps-image --> package-image + package-image --> package-image-multiarch-manifest + + %% Spider-worker container build jobs + filter-relevant-changes --> spider-worker-image + ubuntu-jammy-x86_64-deps-image --> spider-worker-image + + %% Lint & test jobs + filter-relevant-changes --> ubuntu-jammy-lint + ubuntu-jammy-x86_64-deps-image --> ubuntu-jammy-lint ubuntu-jammy-binaries --> ubuntu-jammy-integration-tests-core ::: @@ -68,21 +98,28 @@ Arrows between jobs indicate a dependency. The jobs are as follows: arch-suffixed tag (e.g. `:main-amd64`). * `musllinux_1_2-deps-image-merge`: On push to `main`, merges the per-arch tags produced by `musllinux_1_2-deps-image` into a single multi-arch `:main` manifest. -* `ubuntu-jammy-deps-image`: Builds a container image containing the dependencies necessary to build - CLP-core in an Ubuntu Jammy x86 environment. +* `ubuntu-jammy-x86-deps-image` / `ubuntu-jammy-aarch64-deps-image`: Builds a container image + containing the dependencies necessary to build CLP-core in an Ubuntu Jammy x86/aarch64 + environment. * `centos-stream-9-binaries`: Builds the CLP-core binaries in the built CentOS Stream 9 container and runs core's unit tests. * `manylinux_2_28-x86_64-binaries`: Builds the CLP-core binaries in the built manylinux_2_28 container and runs core's unit tests. * `musllinux_1_2-x86_64-binaries`: Builds the CLP-core binaries in the built musllinux_1_2 container and runs core's unit tests. -* `package-image`: Builds a container image containing CLP's package components. +* `manylinux_2_28-x86_64-python-wheels`: Builds the `yscope-clp-core` wheels in the built + `manylinux_2_28` container. * `ubuntu-jammy-binaries`: Builds the CLP-core binaries in the built Ubuntu Jammy container and runs core's unit tests. * `ubuntu-jammy-binaries-image`: Builds an Ubuntu Jammy container image containing CLP-core's binaries built in the `ubuntu-jammy-binaries` job. +* `ubuntu-jammy-lint`: Runs C++ linting checks in the built ubuntu-jammy container. * `ubuntu-jammy-integration-tests-core`: Runs CLP-core's integration tests using the binaries built in the `ubuntu-jammy-binaries` job, and then uploads the logs from the tests. +* `package-image`: Builds the CLP package container image. +* `package-image-multiarch-manifest`: On pushes to or scheduled runs on `main`, merges the per-arch + tags produced by `package-image` into a single multi-arch manifest. +* `spider-worker-image`: Builds a container image containing CLP-core and `clp-tdl-package`. When the PR or commit doesn't change any of the files that affect CLP's dependencies (or the dependency container images), then the dependency container images won't be rebuilt; instead the @@ -116,9 +153,45 @@ details, see [GitHub-hosted runners][gh-hosted-runners]. This workflow builds CLP-core on macOS and runs its unit tests. +## clp-docs + +This workflow validates that the docs site can be built. + +## clp-docs-generated-code-checks + +This workflow generates the OpenAPI docs and validates that they don't differ from the committed +OpenAPI docs. + ## clp-lint -This workflow runs linting checks on the codebase. +This workflow runs all JavaScript, Python, and YAML linting checks on the codebase. + +:::{note} +C++, Rust, and Helm linting checks are run in the `clp-artifact-build`, `clp-rust-checks`, and +`clp-package-helm` workflows, respectively. +::: + +## clp-package-helm + +This workflow contains two jobs for linting, building, and publishing the Helm chart: + +* `lint` runs Helm linting checks on the chart. +* `publish` builds the chart; then on pushes to `main` and semantic-version branches, the job + publishes the built chart to the `gh-pages` branch. + +## clp-pr-title-checks + +This workflow validates pull request titles against the Conventional Commits specification. + +## clp-rust-checks + +This workflow validates Rust's lock files, runs all Rust linting checks, and runs all Rust unit +tests. + +## clp-s-generated-code-checks + +This workflow generates the KQL and SQL ANTLR parsers and validates that they don't differ from the +committed parsers. ## clp-uv-checks From bd7e70e9e759ee4c184b09de37bbc692e11dc43a Mon Sep 17 00:00:00 2001 From: Kirk Rodrigues <2454684+kirkrodrigues@users.noreply.github.com> Date: Mon, 24 Aug 2026 18:54:11 -0400 Subject: [PATCH 2/2] Address CodeRabbit's comments. --- docs/src/dev-docs/tooling-gh-workflows.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/src/dev-docs/tooling-gh-workflows.md b/docs/src/dev-docs/tooling-gh-workflows.md index 216d0964ef..8739a79a97 100755 --- a/docs/src/dev-docs/tooling-gh-workflows.md +++ b/docs/src/dev-docs/tooling-gh-workflows.md @@ -98,7 +98,7 @@ Arrows between jobs indicate a dependency. The jobs are as follows: arch-suffixed tag (e.g. `:main-amd64`). * `musllinux_1_2-deps-image-merge`: On push to `main`, merges the per-arch tags produced by `musllinux_1_2-deps-image` into a single multi-arch `:main` manifest. -* `ubuntu-jammy-x86-deps-image` / `ubuntu-jammy-aarch64-deps-image`: Builds a container image +* `ubuntu-jammy-x86_64-deps-image` / `ubuntu-jammy-aarch64-deps-image`: Builds a container image containing the dependencies necessary to build CLP-core in an Ubuntu Jammy x86/aarch64 environment. * `centos-stream-9-binaries`: Builds the CLP-core binaries in the built CentOS Stream 9 container @@ -117,8 +117,8 @@ Arrows between jobs indicate a dependency. The jobs are as follows: * `ubuntu-jammy-integration-tests-core`: Runs CLP-core's integration tests using the binaries built in the `ubuntu-jammy-binaries` job, and then uploads the logs from the tests. * `package-image`: Builds the CLP package container image. -* `package-image-multiarch-manifest`: On pushes to or scheduled runs on `main`, merges the per-arch - tags produced by `package-image` into a single multi-arch manifest. +* `package-image-multiarch-manifest`: When run on `main`, merges the per-arch tags produced by + `package-image` into a single multi-arch manifest. * `spider-worker-image`: Builds a container image containing CLP-core and `clp-tdl-package`. When the PR or commit doesn't change any of the files that affect CLP's dependencies (or the