From 75803810658bd0f939b6f0deee181f4bc3a49f92 Mon Sep 17 00:00:00 2001 From: aryansingh0012 Date: Thu, 30 Jul 2026 15:39:17 +0530 Subject: [PATCH 1/3] docs: Add packaging tooling comparison matrix for AI SDLC evaluation --- .../adr_ai_sdlc_speckit_tooling.md | 154 ++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 tools/decision_records/adr_ai_sdlc_speckit_tooling.md diff --git a/tools/decision_records/adr_ai_sdlc_speckit_tooling.md b/tools/decision_records/adr_ai_sdlc_speckit_tooling.md new file mode 100644 index 00000000000..15f5e63f2f4 --- /dev/null +++ b/tools/decision_records/adr_ai_sdlc_speckit_tooling.md @@ -0,0 +1,154 @@ + + +# DR-009-Infra: AI Agent Context Packaging Tooling Selection + +* **Date:** 2026-07-31 +* **Issue:** [#3115](https://github.com/eclipse-score/score/issues/3115) + +```{dec_rec} AI Agent Context Packaging Tooling Selection +:id: dec_rec__infra__ai_packaging_tooling +:status: proposed +:tracking: https://github.com/eclipse-score/score/issues/3115 +:version: 1 +:context: Infrastructure +:decision: APM is the primary AI agent context packaging tool; Lola is the fallback; OKIT is not recommended +``` + +--- + +## 1. Context / Problem + +S-CORE contributors use multiple AI coding assistants (GitHub Copilot, Claude Code, Cursor, etc.). +Agent context — skills, instructions, prompts, MCP server references — is today copy-pasted per +repository and per tool, unversioned and unaudited. + +A packaging layer is needed that: + +- Distributes agent context declaratively and reproducibly across all contributors regardless of IDE. +- Provides supply-chain controls (lockfile integrity, SBOM, policy allow-lists) consistent with + S-CORE's existing `sbom-tool` and safety-oriented process. +- Integrates with CI as a merge gate (audit, drift detection). + +### Visual Overview + +```{mermaid} +flowchart LR + subgraph Sources["Package Sources (git / marketplace)"] + R1[eclipse-score/mcp-servers] + R2[eclipse-score/tooling] + R3[community skills repos] + end + + subgraph PM["Package Manager (one of three)"] + APM["APM\napm.yml + apm.lock.yaml\npolicy · SBOM · hashes"]:::primary + Lola["Lola\n.lola-req\nno lockfile hashes"]:::fallback + OKIT["OKIT\nflat copy\nno versioning"]:::avoid + end + + subgraph Out["Per-Contributor Output"] + C1[".github/copilot-instructions.md"] + C2["CLAUDE.md"] + C3[".cursor/rules"] + C4["...other IDE configs"] + end + + Sources --> APM + Sources --> Lola + Sources --> OKIT + APM --> Out + Lola --> Out + OKIT --> Out + + classDef primary fill:#2e7d32,color:#fff,stroke:#1b5e20 + classDef fallback fill:#f57c00,color:#fff,stroke:#e65100 + classDef avoid fill:#c62828,color:#fff,stroke:#b71c1c +``` + +## 2. Options Considered + +### 2.1 APM — `microsoft/apm` + +Manifest + lockfile package manager for AI agent context (instructions, skills, prompts, hooks, +plugins, MCP servers) across 8 coding assistants. Uses `apm.yml` manifest and `apm.lock.yaml` +lockfile with content hashes. + +**Pros:** + +- Policy engine (`apm-policy.yml`): org-level allow-lists, tighten-only inheritance, CI audit gates. +- Lockfile with content hashes; `apm.lock export --format cyclonedx|spdx` SBOM export. +- Content-security scanning (hidden-Unicode / prompt-injection detection on install). +- Broadest agent coverage: 8 assistants (Copilot, Claude Code, Cursor, OpenCode, Codex, Gemini, Windsurf, Kiro). +- Drift detection; active Microsoft-org project (3.3k★, 71 releases). +- Directly complements `eclipse-score/mcp-servers` and `sbom-tool`. + +**Cons:** + +- Newest governance/policy features are the least battle-tested part of the tool. +- Single-vendor stewardship (Microsoft) even though built on open standards. +- Adds a new onboarding step for contributors. + +### 2.2 Lola — `LobsterTrap/lola` + +Lighter, community-governed AI skill/context package manager (Go+Python) with marketplace-based +distribution. Declarative `.lola-req` file (pip-requirements style). + +**Pros:** + +- Simpler mental model; lower adoption barrier. +- Vendor-neutral governance (GOVERNANCE.md, OpenSSF Best Practices badge). +- Covers 6 assistants (Claude Code, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, OpenCode). + +**Cons:** + +- No lockfile content-hash integrity, no policy enforcement, no SBOM, no content-security scanning. +- Narrower agent coverage than APM (6 vs. 8). +- Smaller community (109★, 8 releases); missing enterprise governance features. + +### 2.3 OKIT — `Mumme-IT/okit` + +Minimal, dependency-free (stdlib-only) Python CLI: clones a repo, copies `skills/` and `agents/` +files into whichever tool directories are detected on `PATH`. + +**Pros:** + +- Zero dependencies; trivially auditable codebase. +- Very low learning curve (three commands). + +**Cons:** + +- No dependency resolution, versioning, lockfile, policy enforcement, or content-security scanning. +- Single contributor (1★, 1 fork, 40 commits, no releases); bus factor of one. +- Not defensible as shared infrastructure for a multi-org Eclipse project. + +## 3. Conclusion + +**Primary: APM.** It is the only candidate with an actual governance and supply-chain-security +model (policy enforcement, lockfile integrity hashes, SBOM export, drift detection, content-security +scanning) — essential when agent context is executable-in-effect and S-CORE is a safety-relevant +open-source project accepting third-party contributions. + +**Fallback: Lola.** Kept on the radar as a lighter-weight, vendor-neutral alternative if APM's +Microsoft stewardship or scope becomes a concern for Eclipse Foundation governance. + +**Not recommended: OKIT.** Pre-production maturity (bus factor 1, no governance). Revisit if the +project matures significantly. + +| Tool | S-CORE Fit | Recommendation | +|------|-----------|----------------| +| APM | High | **USE** — primary packaging tool | +| Lola | Medium | **WATCH** — fallback only | +| OKIT | Low | **DO NOT USE** at this maturity | + +> **Note:** Status remains **proposed** until a proof-of-concept pilot validates integration with +> S-CORE's Bazel/Sphinx/sphinx-needs infrastructure and CI pipeline. From a289d10a3148bbec0ec0b95fc7ca3dc77f3ebed2 Mon Sep 17 00:00:00 2001 From: aryansingh0012 Date: Mon, 3 Aug 2026 18:50:49 +0530 Subject: [PATCH 2/3] docs: add AI tooling decision records --- docs/design_decisions/DR-009-infra.rst | 166 +++++ docs/design_decisions/DR-010-infra.rst | 583 ++++++++++++++++++ .../adr_ai_sdlc_speckit_tooling.md | 154 ----- 3 files changed, 749 insertions(+), 154 deletions(-) create mode 100644 docs/design_decisions/DR-009-infra.rst create mode 100644 docs/design_decisions/DR-010-infra.rst delete mode 100644 tools/decision_records/adr_ai_sdlc_speckit_tooling.md diff --git a/docs/design_decisions/DR-009-infra.rst b/docs/design_decisions/DR-009-infra.rst new file mode 100644 index 00000000000..cd00ddd7532 --- /dev/null +++ b/docs/design_decisions/DR-009-infra.rst @@ -0,0 +1,166 @@ +.. + Copyright (c) 2026 Contributors to the Eclipse Foundation + + See the NOTICE file(s) distributed with this work for additional + information regarding copyright ownership. + + This program and the accompanying materials are made available under the + terms of the Apache License Version 2.0 which is available at + https://www.apache.org/licenses/LICENSE-2.0 + + SPDX-License-Identifier: Apache-2.0 + +DR-009-Infra: AI Agent Context Packaging Tooling Selection +========================================================== + +- **Date:** 2026-07-31 +- **Issue:** `#3115 `_ + +.. dec_rec:: AI Agent Context Packaging Tooling Selection + :id: dec_rec__infra__ai_packaging_tooling + :status: proposed + :version: 1 + :context: Infrastructure + :decision: APM is the primary AI agent context packaging tool; Lola is the fallback; OKIT is not recommended + +Context / Problem +----------------- + +S-CORE contributors use multiple AI coding assistants (GitHub Copilot, Claude Code, Cursor, etc.). +Agent context — skills, instructions, prompts, MCP server references — is today copy-pasted per +repository and per tool, unversioned and unaudited. + +A packaging layer is needed that: + +- Distributes agent context declaratively and reproducibly across all contributors regardless of IDE. +- Provides supply-chain controls (lockfile integrity, SBOM, policy allow-lists) consistent with + S-CORE's existing ``sbom-tool`` and safety-oriented process. +- Integrates with CI as a merge gate (audit, drift detection). + +Visual Overview +~~~~~~~~~~~~~~~ + +.. mermaid:: + + flowchart LR + subgraph Sources["Package Sources (git / marketplace)"] + R1[eclipse-score/mcp-servers] + R2[eclipse-score/tooling] + R3[community skills repos] + end + + subgraph PM["Package Manager (one of three)"] + APM["APM\napm.yml + apm.lock.yaml\npolicy · SBOM · hashes"] + Lola["Lola\n.lola-req\nno lockfile hashes"] + OKIT["OKIT\nflat copy\nno versioning"] + end + + subgraph Out["Per-Contributor Output"] + C1[".github/copilot-instructions.md"] + C2["CLAUDE.md"] + C3[".cursor/rules"] + C4["...other IDE configs"] + end + + Sources --> APM + Sources --> Lola + Sources --> OKIT + APM --> Out + Lola --> Out + OKIT --> Out + +Options Considered +------------------ + +APM — ``microsoft/apm`` +~~~~~~~~~~~~~~~~~~~~~~~ + +Manifest + lockfile package manager for AI agent context (instructions, skills, prompts, hooks, +plugins, MCP servers) across 8 coding assistants. Uses ``apm.yml`` manifest and ``apm.lock.yaml`` +lockfile with content hashes. + +**Pros:** + +- Policy engine (``apm-policy.yml``): org-level allow-lists, tighten-only inheritance, CI audit gates. +- Lockfile with content hashes; ``apm.lock export --format cyclonedx|spdx`` SBOM export. +- Content-security scanning (hidden-Unicode / prompt-injection detection on install). +- Broadest agent coverage: 8 assistants (Copilot, Claude Code, Cursor, OpenCode, Codex, Gemini, Windsurf, Kiro). +- Drift detection; active Microsoft-org project (3.3k stars, 71 releases). +- Directly complements ``eclipse-score/mcp-servers`` and ``sbom-tool``. + +**Cons:** + +- Newest governance/policy features are the least battle-tested part of the tool. +- Single-vendor stewardship (Microsoft) even though built on open standards. +- Adds a new onboarding step for contributors. + +Lola — ``LobsterTrap/lola`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Lighter, community-governed AI skill/context package manager (Go+Python) with marketplace-based +distribution. Declarative ``.lola-req`` file (pip-requirements style). + +**Pros:** + +- Simpler mental model; lower adoption barrier. +- Vendor-neutral governance (GOVERNANCE.md, OpenSSF Best Practices badge). +- Covers 6 assistants (Claude Code, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, OpenCode). + +**Cons:** + +- No lockfile content-hash integrity, no policy enforcement, no SBOM, no content-security scanning. +- Narrower agent coverage than APM (6 vs. 8). +- Smaller community (109 stars, 8 releases); missing enterprise governance features. + +OKIT — ``Mumme-IT/okit`` +~~~~~~~~~~~~~~~~~~~~~~~~ + +Minimal, dependency-free (stdlib-only) Python CLI: clones a repo, copies ``skills/`` and ``agents/`` +files into whichever tool directories are detected on ``PATH``. + +**Pros:** + +- Zero dependencies; trivially auditable codebase. +- Very low learning curve (three commands). + +**Cons:** + +- No dependency resolution, versioning, lockfile, policy enforcement, or content-security scanning. +- Single contributor (1 star, 1 fork, 40 commits, no releases); bus factor of one. +- Not defensible as shared infrastructure for a multi-org Eclipse project. + +Conclusion +---------- + +**Primary: APM.** It is the only candidate with an actual governance and supply-chain-security +model (policy enforcement, lockfile integrity hashes, SBOM export, drift detection, content-security +scanning) — essential when agent context is executable-in-effect and S-CORE is a safety-relevant +open-source project accepting third-party contributions. + +**Fallback: Lola.** Kept on the radar as a lighter-weight, vendor-neutral alternative if APM's +Microsoft stewardship or scope becomes a concern for Eclipse Foundation governance. + +**Not recommended: OKIT.** Pre-production maturity (bus factor 1, no governance). Revisit if the +project matures significantly. + +.. list-table:: + :header-rows: 1 + :widths: 10 15 30 + + * - Tool + - S-CORE Fit + - Recommendation + * - APM + - High + - **USE** — primary packaging tool + * - Lola + - Medium + - **WATCH** — fallback only + * - OKIT + - Low + - **DO NOT USE** at this maturity + +.. note:: + + Status remains **proposed** until a proof-of-concept pilot validates integration with + S-CORE's Bazel/Sphinx/sphinx-needs infrastructure and CI pipeline. diff --git a/docs/design_decisions/DR-010-infra.rst b/docs/design_decisions/DR-010-infra.rst new file mode 100644 index 00000000000..d99f47b2f57 --- /dev/null +++ b/docs/design_decisions/DR-010-infra.rst @@ -0,0 +1,583 @@ +.. + Copyright (c) 2026 Contributors to the Eclipse Foundation + + See the NOTICE file(s) distributed with this work for additional + information regarding copyright ownership. + + This program and the accompanying materials are made available under the + terms of the Apache License Version 2.0 which is available at + https://www.apache.org/licenses/LICENSE-2.0 + + SPDX-License-Identifier: Apache-2.0 + +DR-010-Infra: AI SDLC / SpecKit Tooling Evaluation for Eclipse S-CORE +====================================================================== + +- **Issue:** `#3115 `_ — Evaluate AI SDLC / SpecKit Tooling +- **Date:** 2026-07-31 + +.. dec_rec:: AI SDLC / SpecKit Tooling Evaluation + :id: dec_rec__infra__ai_sdlc_tooling + :status: proposed + :version: 1 + :context: Infrastructure + :decision: Do not adopt any evaluated framework wholesale; build an S-CORE-owned harness based on SpecKit + Sphinx-Needs traceability + +**Decision driver:** Can a tool create a traceable chain from +**Requirements → Specifications → ADRs → Code → Tests → Documentation** +while enabling AI agents to participate in a **governed, reproducible, ASPICE-compatible** workflow? + +1. Evaluation Method +-------------------- + +Each framework was evaluated against ten categories: + +1. Requirements Engineering +2. Specification Management +3. Traceability +4. AI Agent Support +5. Governance +6. Reproducibility +7. Open Source Sustainability +8. S-CORE Integration +9. ASPICE Alignment +10. Long-Term Maintainability + +**Scoring scale:** + +.. list-table:: + :header-rows: 1 + :widths: 10 30 + + * - Score + - Meaning + * - 1 + - Poor + * - 2 + - Limited + * - 3 + - Acceptable + * - 4 + - Strong + * - 5 + - Excellent + +Evaluation prioritized **enterprise-scale collaborative engineering** and **compliance evidence** +over individual developer productivity. Evidence was drawn from each project's source repository +(README, license, package metadata, configuration) rather than marketing claims. + +**Tools evaluated:** + +.. list-table:: + :header-rows: 1 + :widths: 20 50 + + * - Tool + - Source + * - Syspilot + - https://github.com/enthali/syspilot + * - BMAD Method + - https://github.com/bmad-code-org/BMAD-METHOD + * - SpecKit + - https://github.com/github/spec-kit · https://speckit.org + * - Pharaoh + - https://github.com/useblocks/pharaoh-skills + +2. Executive Summary +-------------------- + +**No single tool fully satisfies the S-CORE decision criterion.** +The strongest capabilities are distributed across the four frameworks: + +- **SpecKit** provides the strongest structured **specification-first workflow** and the best open-source sustainability. +- **Syspilot** provides **AI-assisted Sphinx-Needs traceability** and focused change-impact context. +- **Pharaoh** provides the strongest **requirements-centric, ASPICE-aligned concepts** — but the repository is **archived**. +- **BMAD Method** provides broad **agent collaboration and agile workflows**, but is not traceability- or ASPICE-first. + +Therefore, an **adaptation strategy** (combine strengths, own the governance in S-CORE) is preferred +over direct adoption of any single framework. + +3. Why S-CORE Cannot Adopt One Tool Directly +-------------------------------------------- + +None of the evaluated frameworks fully satisfy all S-CORE requirements. +The strongest capabilities are distributed: + +- **SpecKit** provides structured specification workflows, but no native Sphinx-Needs / ADR / ASPICE semantics. +- **Syspilot** provides AI-assisted traceability, but is an early research project bound to Copilot + Jarvis. +- **Pharaoh** provides the strongest requirements-centric, safety-aligned concepts, but its repository is archived and read-only. +- **BMAD** provides agent collaboration workflows, but lacks a traceability-first artifact model. + +Because the required capabilities — specification workflow, traceability, ASPICE evidence, and +long-term maintainability — are **not present together in any one tool**, an adaptation strategy +is preferred over direct adoption. + +4. Tool-by-Tool Analysis +------------------------ + +4.1 Syspilot +~~~~~~~~~~~~ + +Syspilot is built directly on **Sphinx-Needs**, making it the most naturally aligned with +S-CORE's documentation stack. Its core idea is strong: AI agents follow **deterministic requirement +links** rather than searching the whole repository probabilistically +("the map, not the flashlight"). + +**Strengths** + +- Native Sphinx-Needs orientation. +- Strong traceability mindset: user story → requirements → design specs. +- Change-impact analysis producing focused AI context (``O(affected)``, not ``O(total)``). +- Manager/engineer agent roles cover PM, change management, quality, design, implementation, + UAT, verification, documentation, MECE, trace, and release. + +**Weaknesses** + +- README explicitly labels it an **early research project** with possible breaking changes. +- Hard dependency on VS Code, GitHub Copilot, and the ``enthali.jarvis-core`` extension + for multi-agent orchestration. +- Governance/reproducibility are promising but not yet enterprise-hardened. + +**S-CORE view:** Good **pilot candidate** for the traceability layer; not mature enough +to be the sole adopted framework. + +4.2 BMAD Method +~~~~~~~~~~~~~~~ + +BMAD is a broad AI-assisted agile development framework, rich in agents and workflows across +brainstorming, PRDs, architecture, UX, development, and testing. It is more about +**AI collaboration patterns** than regulated engineering evidence. + +**Strengths** + +- Mature open-source packaging (npm ``bmad-method`` v6.10.0), MIT license, active ecosystem. +- Large workflow surface (34+ workflows) and role-based agents. +- Strong CI-friendly tooling (lint, tests, validation scripts) and non-interactive install for CI/CD. +- Useful for product/architecture ideation and implementation support. + +**Weaknesses** + +- Not Sphinx-Needs native; not ASPICE native. +- Traceability is not the central artifact model. +- Requirement → design → code → test linkage is not provable without extra tooling. + +**S-CORE view:** **Not recommended** as the core SDLC governance tool. Keep optional for +facilitation/ideation. + +4.3 SpecKit +~~~~~~~~~~~ + +SpecKit is the strongest candidate for a **governed specification-first workflow**: +``constitution → specify → plan → tasks → analyze → implement``, with a CLI, templates, +presets, extensions, and project-local overrides. + +**Strengths** + +- Specification generation is the **core concept**; specs become versioned repo artifacts. +- Strong governance primitives: constitution, templates, checklists, phase gates. +- Good reproducibility via templates, deterministic commands, and CLI-managed structure. +- MIT license (GitHub, Inc.), broad agent support (30+), reducing vendor lock-in. +- Security-conscious build posture (ruff subprocess-shell lint locks). + +**Weaknesses** + +- Not built for Sphinx, Sphinx-Needs, Bazel, ADRs, or ASPICE. +- Traceability is workflow-level, not automatically ASPICE-grade. +- Requirement IDs, Sphinx-Needs links, ADR links, and test evidence require + S-CORE-specific templates/extensions. + +**S-CORE view:** **Best foundation** for the specification + AI workflow layer, if extended +with S-CORE templates and Sphinx-Needs integration. + +4.4 Pharaoh +~~~~~~~~~~~ + +Pharaoh is conceptually the closest match to S-CORE's needs. Built for Sphinx-Needs projects, +it offers a full V-model skill chain: requirement/architecture/test/FMEA drafting and review, +traceability, MECE, change impact, codelinks, lifecycle checks, standard conformance, decisions, +quality gates, and reproducibility checks. + +**Strengths** + +- Strongest Sphinx-Needs alignment and traceability model. +- Explicit ASPICE 4.0 / ISO 26262-8 §6 / ISO/SAE 21434 conformance concepts. +- Advisory vs **enforcing** mode via ``pharaoh.toml``, with required-link rules and codelink support. +- Review metadata (``:reviewer:``, ``:approved_by:``), lifecycle/status gates, and + reproducibility diff checks. + +**Weaknesses** + +- **The repository is archived (read-only since 2026-07-28) and no longer maintained** — + verified directly on GitHub. +- Functionality has moved to commercial-adjacent **ubCode / ubTrace** (stated free for + open-source), which changes the governance/adoption profile. +- "The AI is the runtime" — flexible, but harder to certify/stabilize. + +**S-CORE view:** **Best reference design, risky direct adoption.** +Reuse its concepts; do not depend on the archived repo. + +.. note:: + + **Verified evidence:** ``useblocks/pharaoh-skills`` shows "This repository was archived by + the owner on Jul 28, 2026. It is now read-only." (24 stars, 3 contributors, latest release + v1.2.1). Skills are stated to have moved into ubCode and ubTrace. + +5. Scoring Matrix +----------------- + +Scores: 1 = Poor, 2 = Limited, 3 = Acceptable, 4 = Strong, 5 = Excellent. + +.. list-table:: + :header-rows: 1 + :widths: 35 15 10 15 15 + + * - Category + - Syspilot + - BMAD + - SpecKit + - Pharaoh + * - Requirements Engineering + - 4 + - 3 + - 4 + - 5 + * - Specification Management + - 3 + - 3 + - 5 + - 4 + * - Traceability + - 5 + - 2 + - 3 + - 5 + * - AI Agent Support + - 4 + - 5 + - 4 + - 4 + * - Governance + - 3 + - 3 + - 4 + - 4 + * - Reproducibility + - 3 + - 2 + - 4 + - 4 + * - Open Source Sustainability + - 2 + - 4 + - 5 + - 2 + * - S-CORE Integration + - 5 + - 2 + - 3 + - 5 + * - ASPICE Alignment + - 3 + - 2 + - 3 + - 5 + * - Long-Term Maintainability + - 2 + - 4 + - 5 + - 2 + * - **Total** + - **34** + - **30** + - **40** + - **40** + +**Interpretation:** SpecKit and Pharaoh tie numerically for different reasons — SpecKit is +maintainable and workflow-ready; Pharaoh is domain-aligned but archived. For the ADR, that +qualitative difference matters more than the raw total. + +6. ASPICE Impact Assessment +--------------------------- + +.. list-table:: + :header-rows: 1 + :widths: 30 15 15 15 15 + + * - ASPICE Area + - Syspilot + - BMAD + - SpecKit + - Pharaoh + * - SYS.1 Requirements elicitation + - Medium + - Medium + - Medium/High + - High + * - SYS.2 System requirements + - High + - Low/Medium + - Medium + - High + * - SYS.3 System architecture + - Medium + - Medium/High + - Medium + - High + * - SWE.1 Software requirements + - High + - Low/Medium + - Medium + - High + * - SWE.2 Software architecture + - Medium + - Medium/High + - Medium + - High + * - SWE.3 Detailed design / unit construction + - Medium + - Medium + - Medium + - Medium/High + * - SWE.4 Unit verification + - Medium + - Medium + - Medium + - High (if linked to needs/tests) + * - SWE.5 Integration testing + - Medium + - Medium + - Medium + - Medium/High + * - SWE.6 Qualification testing + - Medium + - Low/Medium + - Medium + - High + +**Key point:** ASPICE alignment depends on **traceability evidence**, not just generated +documents. Pharaoh and Syspilot are closest because they start from Sphinx-Needs trace links. +SpecKit can support ASPICE only after S-CORE adds requirement IDs, link rules, review states, +approval metadata, and documentation-pipeline integration. + +7. Integration Architecture +--------------------------- + +7.1 Workflow (data flow) +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. mermaid:: + + flowchart LR + A[Sphinx-Needs Requirements] --> B[SpecKit Specification Layer] + B --> C[Architecture / ADR Generation] + C --> D[Implementation Tasks] + D --> E[Code + Bazel Build] + E --> F[Tests + CI Evidence] + F --> G[Sphinx Documentation] + G --> A + + A --> H[Traceability / Impact Analysis] + C --> H + D --> H + E --> H + F --> H + +7.2 Target State (conceptual stack) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: text + + +--------------------------------+ + | AI Agents | + +--------------------------------+ + | SpecKit Workflow Layer | + +--------------------------------+ + | ADRs + Sphinx-Needs | + +--------------------------------+ + | Code + Bazel + CI/CD | + +--------------------------------+ + | S-CORE Repository | + +--------------------------------+ + +7.3 Capability Mapping +~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 35 50 + + * - S-CORE Need + - Recommended Tooling Role + * - Requirements as managed artifacts + - Sphinx-Needs + Syspilot/Pharaoh-style skills + * - Specification workflow + - SpecKit + * - ADR generation + - S-CORE SpecKit extension or Pharaoh-style decision skill + * - Traceability graph + - Sphinx-Needs + codelinks / ``needs.json`` validation + * - AI agent governance + - Versioned prompts/templates in Git + * - Reproducibility + - SpecKit CLI/templates + CI validation + * - ASPICE evidence + - Sphinx-Needs reports, CI checks, review metadata + * - Documentation output + - Sphinx build pipeline + +8. Risks and Mitigations +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 40 15 45 + + * - Risk + - Impact + - Mitigation + * - Pharaoh repo archived (verified 2026-07-28) + - High + - Do not adopt directly; reuse concepts or evaluate ubCode/ubTrace separately + * - Syspilot early research status + - High + - Pilot only; do not make it mandatory infrastructure + * - SpecKit lacks ASPICE/Sphinx-Needs model + - Medium + - Build S-CORE templates/extensions for ASPICE work products + * - BMAD lacks traceability core + - Medium + - Use only for facilitation, not compliance evidence + * - AI output nondeterminism + - High + - Version prompts, templates, model settings, inputs, and generated outputs + * - Weak auditability + - High + - Store all artifacts in Git; require review states and trace links + * - Vendor/tool lock-in + - Medium + - Prefer repo-native Markdown/RST/YAML over IDE-only state + +9. Recommendation +----------------- + +Adopt a **S-CORE-owned AI SDLC harness** based on **SpecKit + Sphinx-Needs traceability**, +rather than adopting Syspilot, BMAD, SpecKit, or Pharaoh wholesale. + +- **SpecKit — Adopt/adapt** as the base specification-driven workflow. +- **Syspilot — Pilot/adapt** for Sphinx-Needs traceability and focused AI context. +- **Pharaoh — Reference design only** (repo archived); reassess ubCode/ubTrace separately. +- **BMAD — Optional** for early ideation/planning where no compliance evidence is required. + +10. Decision +------------ + +**Do not adopt any evaluated framework as mandatory S-CORE infrastructure.** Instead: + +1. **Prototype SpecKit-based specification workflows** integrated with the S-CORE repository. +2. **Reuse Sphinx-Needs traceability patterns** from Syspilot and Pharaoh (concepts, not the archived repo). +3. **Keep BMAD optional** for developer productivity and facilitation only. +4. **Re-evaluate after pilot results**, including a separate assessment of ubCode/ubTrace + as the maintained Pharaoh successor. + +**Expected S-CORE outcome:** a repository-native, Sphinx-compatible AI SDLC process where +requirements, specs, ADRs, plans, code links, tests, reviews, and documentation are all +versioned, reviewable, and traceable — with AI agents participating only through governed +templates and reproducible workflows. + +11. Evidence Sources +-------------------- + +Evidence is drawn from the **full local repositories** (agent/skill definitions, schemas, +templates, configuration), not only READMEs. + +Syspilot (``syspilot-main/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Claim + - Concrete source in repo + * - Sphinx-Needs based, US → REQ → SPEC traceability + - ``syspilot/agents/syspilot.trace.agent.md`` (upward/downward tracing, link validation, + orphan detection via ``get_need_links.py``) + * - Multi-agent architecture (13 agents) + - ``syspilot/agents/`` — ``pm``, ``cm``, ``qm``, ``design``, ``implement``, ``uat``, + ``verify``, ``docu``, ``mece``, ``trace``, ``release``, ``setup``, ``installer`` + * - Copilot + Jarvis dependency; per-agent model pinning + - Agent frontmatter (``model: Claude Haiku 4.5 (copilot)``), README requirements + (``enthali.jarvis-core``) + * - Early research status; MIT + - ``README.md`` ("Early Research Project"), ``LICENSE`` (MIT, Copyright 2026 Georg) + * - Install/bootstrap model + - ``syspilot/bootstrap.json``, ``syspilot/{skills,sphinx,templates,prompts}/`` + +BMAD Method (``BMAD-METHOD-main/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Claim + - Concrete source in repo + * - MIT + trademark notice; npm v6.10.0 + - ``LICENSE``, ``package.json`` (``"version": "6.10.0"``) + * - CI tooling / quality gates + - ``package.json`` scripts (``lint``, ``test:*``, ``validate:refs``, ``validate:skills``), + ``test/`` suite + * - Module ecosystem & workflow surface + - ``bmad-modules.yaml``, + ``src/bmm-skills/{1-analysis,2-plan-workflows,3-solutioning,4-implementation}/`` + * - Not traceability/ASPICE-first + - No Sphinx-Needs schema or trace-link model present in ``src/`` skills + +SpecKit (``spec-kit-main/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Claim + - Concrete source in repo + * - MIT (GitHub, Inc.); spec-driven CLI + - ``LICENSE``, ``pyproject.toml`` (``specify-cli``), ``spec-driven.md`` + * - Specification-first templates & gates + - ``templates/{spec,plan,tasks,constitution,checklist}-template.md`` (prioritized user + stories, acceptance scenarios, phase gates) + * - Extensible / low lock-in; 30+ agents + - ``extensions/``, ``presets/``, ``workflows/``, ``integrations/``, ``.specify/`` + override stack + * - Security-conscious build posture + - ``pyproject.toml`` ruff rules S602/S604/S605 (shell-injection lockdown) + +Pharaoh (``pharaoh-skills-main/``) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Claim + - Concrete source in repo + * - Sphinx-Needs V-model; 70+ atomic skills + - ``skills/`` (e.g. ``pharaoh-req-draft``, ``pharaoh-arch-review``, + ``pharaoh-vplan-draft``, ``pharaoh-fmea``, ``pharaoh-flow``, ``pharaoh-trace``) + * - Explicit ASPICE/ISO conformance engine + - ``skills/pharaoh-standard-conformance/SKILL.md`` + (``iso26262 | aspice40 | iso21434``, per-indicator pass/fail JSON) + * - Advisory vs enforcing governance; required links + - ``pharaoh.toml.example`` (``strictness``, ``require_verification``, + ``required_links = ["req -> spec", "spec -> impl", "impl -> test"]``, codelinks) + * - Schema-validated artifacts & IDs + - ``schemas/`` (``artefact-catalog``, ``checklists-frontmatter``, + ``id-conventions``, ``workflows``) + * - Ships an S-CORE example project + - ``examples/score/.pharaoh/project/`` + * - MIT license + - ``LICENSE`` (MIT, Copyright 2026 useblocks GmbH) + * - **Repository archived (read-only) since 2026-07-28** + - GitHub — ``useblocks/pharaoh-skills`` (verified); README deprecation note + points to ubCode/ubTrace diff --git a/tools/decision_records/adr_ai_sdlc_speckit_tooling.md b/tools/decision_records/adr_ai_sdlc_speckit_tooling.md deleted file mode 100644 index 15f5e63f2f4..00000000000 --- a/tools/decision_records/adr_ai_sdlc_speckit_tooling.md +++ /dev/null @@ -1,154 +0,0 @@ - - -# DR-009-Infra: AI Agent Context Packaging Tooling Selection - -* **Date:** 2026-07-31 -* **Issue:** [#3115](https://github.com/eclipse-score/score/issues/3115) - -```{dec_rec} AI Agent Context Packaging Tooling Selection -:id: dec_rec__infra__ai_packaging_tooling -:status: proposed -:tracking: https://github.com/eclipse-score/score/issues/3115 -:version: 1 -:context: Infrastructure -:decision: APM is the primary AI agent context packaging tool; Lola is the fallback; OKIT is not recommended -``` - ---- - -## 1. Context / Problem - -S-CORE contributors use multiple AI coding assistants (GitHub Copilot, Claude Code, Cursor, etc.). -Agent context — skills, instructions, prompts, MCP server references — is today copy-pasted per -repository and per tool, unversioned and unaudited. - -A packaging layer is needed that: - -- Distributes agent context declaratively and reproducibly across all contributors regardless of IDE. -- Provides supply-chain controls (lockfile integrity, SBOM, policy allow-lists) consistent with - S-CORE's existing `sbom-tool` and safety-oriented process. -- Integrates with CI as a merge gate (audit, drift detection). - -### Visual Overview - -```{mermaid} -flowchart LR - subgraph Sources["Package Sources (git / marketplace)"] - R1[eclipse-score/mcp-servers] - R2[eclipse-score/tooling] - R3[community skills repos] - end - - subgraph PM["Package Manager (one of three)"] - APM["APM\napm.yml + apm.lock.yaml\npolicy · SBOM · hashes"]:::primary - Lola["Lola\n.lola-req\nno lockfile hashes"]:::fallback - OKIT["OKIT\nflat copy\nno versioning"]:::avoid - end - - subgraph Out["Per-Contributor Output"] - C1[".github/copilot-instructions.md"] - C2["CLAUDE.md"] - C3[".cursor/rules"] - C4["...other IDE configs"] - end - - Sources --> APM - Sources --> Lola - Sources --> OKIT - APM --> Out - Lola --> Out - OKIT --> Out - - classDef primary fill:#2e7d32,color:#fff,stroke:#1b5e20 - classDef fallback fill:#f57c00,color:#fff,stroke:#e65100 - classDef avoid fill:#c62828,color:#fff,stroke:#b71c1c -``` - -## 2. Options Considered - -### 2.1 APM — `microsoft/apm` - -Manifest + lockfile package manager for AI agent context (instructions, skills, prompts, hooks, -plugins, MCP servers) across 8 coding assistants. Uses `apm.yml` manifest and `apm.lock.yaml` -lockfile with content hashes. - -**Pros:** - -- Policy engine (`apm-policy.yml`): org-level allow-lists, tighten-only inheritance, CI audit gates. -- Lockfile with content hashes; `apm.lock export --format cyclonedx|spdx` SBOM export. -- Content-security scanning (hidden-Unicode / prompt-injection detection on install). -- Broadest agent coverage: 8 assistants (Copilot, Claude Code, Cursor, OpenCode, Codex, Gemini, Windsurf, Kiro). -- Drift detection; active Microsoft-org project (3.3k★, 71 releases). -- Directly complements `eclipse-score/mcp-servers` and `sbom-tool`. - -**Cons:** - -- Newest governance/policy features are the least battle-tested part of the tool. -- Single-vendor stewardship (Microsoft) even though built on open standards. -- Adds a new onboarding step for contributors. - -### 2.2 Lola — `LobsterTrap/lola` - -Lighter, community-governed AI skill/context package manager (Go+Python) with marketplace-based -distribution. Declarative `.lola-req` file (pip-requirements style). - -**Pros:** - -- Simpler mental model; lower adoption barrier. -- Vendor-neutral governance (GOVERNANCE.md, OpenSSF Best Practices badge). -- Covers 6 assistants (Claude Code, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, OpenCode). - -**Cons:** - -- No lockfile content-hash integrity, no policy enforcement, no SBOM, no content-security scanning. -- Narrower agent coverage than APM (6 vs. 8). -- Smaller community (109★, 8 releases); missing enterprise governance features. - -### 2.3 OKIT — `Mumme-IT/okit` - -Minimal, dependency-free (stdlib-only) Python CLI: clones a repo, copies `skills/` and `agents/` -files into whichever tool directories are detected on `PATH`. - -**Pros:** - -- Zero dependencies; trivially auditable codebase. -- Very low learning curve (three commands). - -**Cons:** - -- No dependency resolution, versioning, lockfile, policy enforcement, or content-security scanning. -- Single contributor (1★, 1 fork, 40 commits, no releases); bus factor of one. -- Not defensible as shared infrastructure for a multi-org Eclipse project. - -## 3. Conclusion - -**Primary: APM.** It is the only candidate with an actual governance and supply-chain-security -model (policy enforcement, lockfile integrity hashes, SBOM export, drift detection, content-security -scanning) — essential when agent context is executable-in-effect and S-CORE is a safety-relevant -open-source project accepting third-party contributions. - -**Fallback: Lola.** Kept on the radar as a lighter-weight, vendor-neutral alternative if APM's -Microsoft stewardship or scope becomes a concern for Eclipse Foundation governance. - -**Not recommended: OKIT.** Pre-production maturity (bus factor 1, no governance). Revisit if the -project matures significantly. - -| Tool | S-CORE Fit | Recommendation | -|------|-----------|----------------| -| APM | High | **USE** — primary packaging tool | -| Lola | Medium | **WATCH** — fallback only | -| OKIT | Low | **DO NOT USE** at this maturity | - -> **Note:** Status remains **proposed** until a proof-of-concept pilot validates integration with -> S-CORE's Bazel/Sphinx/sphinx-needs infrastructure and CI pipeline. From da32256d65692eef24c0bbb23d69016a83594239 Mon Sep 17 00:00:00 2001 From: aryansingh0012 Date: Mon, 17 Aug 2026 14:50:48 +0530 Subject: [PATCH 3/3] docs: remove DR-009, superseded by DR-011 in separate PR Signed-off-by: aryansingh0012 --- docs/design_decisions/DR-009-infra.rst | 166 ------------------------- 1 file changed, 166 deletions(-) delete mode 100644 docs/design_decisions/DR-009-infra.rst diff --git a/docs/design_decisions/DR-009-infra.rst b/docs/design_decisions/DR-009-infra.rst deleted file mode 100644 index cd00ddd7532..00000000000 --- a/docs/design_decisions/DR-009-infra.rst +++ /dev/null @@ -1,166 +0,0 @@ -.. - Copyright (c) 2026 Contributors to the Eclipse Foundation - - See the NOTICE file(s) distributed with this work for additional - information regarding copyright ownership. - - This program and the accompanying materials are made available under the - terms of the Apache License Version 2.0 which is available at - https://www.apache.org/licenses/LICENSE-2.0 - - SPDX-License-Identifier: Apache-2.0 - -DR-009-Infra: AI Agent Context Packaging Tooling Selection -========================================================== - -- **Date:** 2026-07-31 -- **Issue:** `#3115 `_ - -.. dec_rec:: AI Agent Context Packaging Tooling Selection - :id: dec_rec__infra__ai_packaging_tooling - :status: proposed - :version: 1 - :context: Infrastructure - :decision: APM is the primary AI agent context packaging tool; Lola is the fallback; OKIT is not recommended - -Context / Problem ------------------ - -S-CORE contributors use multiple AI coding assistants (GitHub Copilot, Claude Code, Cursor, etc.). -Agent context — skills, instructions, prompts, MCP server references — is today copy-pasted per -repository and per tool, unversioned and unaudited. - -A packaging layer is needed that: - -- Distributes agent context declaratively and reproducibly across all contributors regardless of IDE. -- Provides supply-chain controls (lockfile integrity, SBOM, policy allow-lists) consistent with - S-CORE's existing ``sbom-tool`` and safety-oriented process. -- Integrates with CI as a merge gate (audit, drift detection). - -Visual Overview -~~~~~~~~~~~~~~~ - -.. mermaid:: - - flowchart LR - subgraph Sources["Package Sources (git / marketplace)"] - R1[eclipse-score/mcp-servers] - R2[eclipse-score/tooling] - R3[community skills repos] - end - - subgraph PM["Package Manager (one of three)"] - APM["APM\napm.yml + apm.lock.yaml\npolicy · SBOM · hashes"] - Lola["Lola\n.lola-req\nno lockfile hashes"] - OKIT["OKIT\nflat copy\nno versioning"] - end - - subgraph Out["Per-Contributor Output"] - C1[".github/copilot-instructions.md"] - C2["CLAUDE.md"] - C3[".cursor/rules"] - C4["...other IDE configs"] - end - - Sources --> APM - Sources --> Lola - Sources --> OKIT - APM --> Out - Lola --> Out - OKIT --> Out - -Options Considered ------------------- - -APM — ``microsoft/apm`` -~~~~~~~~~~~~~~~~~~~~~~~ - -Manifest + lockfile package manager for AI agent context (instructions, skills, prompts, hooks, -plugins, MCP servers) across 8 coding assistants. Uses ``apm.yml`` manifest and ``apm.lock.yaml`` -lockfile with content hashes. - -**Pros:** - -- Policy engine (``apm-policy.yml``): org-level allow-lists, tighten-only inheritance, CI audit gates. -- Lockfile with content hashes; ``apm.lock export --format cyclonedx|spdx`` SBOM export. -- Content-security scanning (hidden-Unicode / prompt-injection detection on install). -- Broadest agent coverage: 8 assistants (Copilot, Claude Code, Cursor, OpenCode, Codex, Gemini, Windsurf, Kiro). -- Drift detection; active Microsoft-org project (3.3k stars, 71 releases). -- Directly complements ``eclipse-score/mcp-servers`` and ``sbom-tool``. - -**Cons:** - -- Newest governance/policy features are the least battle-tested part of the tool. -- Single-vendor stewardship (Microsoft) even though built on open standards. -- Adds a new onboarding step for contributors. - -Lola — ``LobsterTrap/lola`` -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Lighter, community-governed AI skill/context package manager (Go+Python) with marketplace-based -distribution. Declarative ``.lola-req`` file (pip-requirements style). - -**Pros:** - -- Simpler mental model; lower adoption barrier. -- Vendor-neutral governance (GOVERNANCE.md, OpenSSF Best Practices badge). -- Covers 6 assistants (Claude Code, Copilot CLI, Copilot VS Code, Cursor, Gemini CLI, OpenCode). - -**Cons:** - -- No lockfile content-hash integrity, no policy enforcement, no SBOM, no content-security scanning. -- Narrower agent coverage than APM (6 vs. 8). -- Smaller community (109 stars, 8 releases); missing enterprise governance features. - -OKIT — ``Mumme-IT/okit`` -~~~~~~~~~~~~~~~~~~~~~~~~ - -Minimal, dependency-free (stdlib-only) Python CLI: clones a repo, copies ``skills/`` and ``agents/`` -files into whichever tool directories are detected on ``PATH``. - -**Pros:** - -- Zero dependencies; trivially auditable codebase. -- Very low learning curve (three commands). - -**Cons:** - -- No dependency resolution, versioning, lockfile, policy enforcement, or content-security scanning. -- Single contributor (1 star, 1 fork, 40 commits, no releases); bus factor of one. -- Not defensible as shared infrastructure for a multi-org Eclipse project. - -Conclusion ----------- - -**Primary: APM.** It is the only candidate with an actual governance and supply-chain-security -model (policy enforcement, lockfile integrity hashes, SBOM export, drift detection, content-security -scanning) — essential when agent context is executable-in-effect and S-CORE is a safety-relevant -open-source project accepting third-party contributions. - -**Fallback: Lola.** Kept on the radar as a lighter-weight, vendor-neutral alternative if APM's -Microsoft stewardship or scope becomes a concern for Eclipse Foundation governance. - -**Not recommended: OKIT.** Pre-production maturity (bus factor 1, no governance). Revisit if the -project matures significantly. - -.. list-table:: - :header-rows: 1 - :widths: 10 15 30 - - * - Tool - - S-CORE Fit - - Recommendation - * - APM - - High - - **USE** — primary packaging tool - * - Lola - - Medium - - **WATCH** — fallback only - * - OKIT - - Low - - **DO NOT USE** at this maturity - -.. note:: - - Status remains **proposed** until a proof-of-concept pilot validates integration with - S-CORE's Bazel/Sphinx/sphinx-needs infrastructure and CI pipeline.