Skip to content

Profiles: reviewed safety facts on the 10 pilot pages (Q4 plan E2) - #347

Merged
YugantM merged 2 commits into
mainfrom
e2-safety-facts
Oct 8, 2026
Merged

YugantM merged 2 commits into
mainfrom
e2-safety-facts

Conversation

@YugantM

@YugantM YugantM commented Oct 6, 2026

Copy link
Copy Markdown
Owner

Q4 plan E2 (hosted-service facts), with the list settled by the E1 audit (#344). Draft: these are statements about companies on public pages, so the plan says you review every line before it merges. Not deployed, not merged.

What it adds

  • safety_facts.json: 50 facts for Composio, Cline, Firecrawl, Dify, E2B, Claude Code, Codex, LiveKit Agents, Freebuff and Kilo Code. Topics: Who runs it, What it can reach, Where your data goes, Security practice.
    • Each fact has a source URL and an evidence class: declared (the publisher's own statement, shown as PUBLISHER, not verified by us) or observed (we checked it on 6 Oct, shown as CHECKED). The source class (from the code) exists but isn't used yet.
  • Profile section "What X can reach, and who runs it", above "How X could raise its score". It uses existing CSS classes only, and renders only on pages that have facts.
  • load_safety_facts() / safety_fact_problems() in fetch_and_build.py. A malformed file renders nothing rather than breaking the render, and the test suite rejects one before it ships.
  • Dockerfile copies safety_facts.json (the predeploy test enforces it).

Things worth your eye

  • Freebuff: its privacy policy says prompts and messages may be analysed to select and measure ads, including by deriving interest categories. Quoted closely, with the training carve-out included.
  • Kilo Code: its terms now name Anaconda, Inc. with its affiliate Kilo Code Inc. as the operator.
  • Composio: the cloud stores connected apps' OAuth tokens and API keys, with keys Composio holds by default. This is likely what "is composio safe" searchers most want to know. It's stated with Composio's own mitigations (redaction, ZDR, customer-managed keys).
  • Dify: "21 security advisories, 14 in the last 12 months". This is factual, but it reads differently from a certification line, so check you're comfortable with it.
  • Codex: only sandbox facts. OpenAI's privacy page refused automated fetches (403), so there's no data-retention line rather than an unsourced one.
  • No absence claims about certifications ("has no SOC 2"). The only absence facts are things we observed ourselves (no security.txt, no SECURITY.md).

Checks

  • tests/test_safety_facts.py:
    • the shipped file is valid and every key is a real listing slug;
    • topic order and labels are correct;
    • a malformed file is rejected whole;
    • a profile without facts is byte-identical to the old template (also confirmed against origin/main's template directly);
    • a profile with facts shows each fact, its class and source, and no empty headings.
  • scripts/lastmod_diff.py: 9 of 2,430 pages re-dated, all pilot profiles. Freebuff is the 10th: it's still pending in local data but scored in prod (56.3, C), so it will show there.
  • ./scripts/gates.sh all green. Rendered Composio checked at desktop and 375 px: no overflow, chips fit their column.

Not in this PR (next, per the audit)

  • The E3 access facts for local tools (Desktop Commander, Ollama, Open WebUI, AnythingLLM…) and the hero verdict and FAQ changes. Both build on this mechanism.
  • E4 (OSV advisory history) awaits your decision.

🤖 Generated with Claude Code

yugant and others added 2 commits October 8, 2026 10:27
safety_facts.json holds 50 sourced facts for the E2 pilot (Composio, Cline,
Firecrawl, Dify, E2B, Claude Code, Codex, LiveKit Agents, Freebuff, Kilo
Code, the list #344 settled by search demand): who runs the product, what
it can reach, where data goes, and its security practice. Each fact names
its source and evidence class (PUBLISHER = the vendor's own statement,
not verified; CHECKED = observed by us on 2026-10-06).

The profile renders them in a box above "How X could raise its score",
using existing styles only, and only on pages that have facts: every
other profile is byte-for-byte unchanged (tested against the old
template), and lastmod_diff reports 9 of 2,430 pages re-dated locally
(the 10th, Freebuff, is pending in local data but scored in prod).
Facts never touch scores or ranks. A malformed file renders nothing
rather than breaking the render; a test rejects one before it ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he rest

Owner review on 8 Oct, with the remaining products re-checked the same way:
- Composio #1: Delaware is the terms' governing law, not stated incorporation.
- Cline: the docs don't state auto-approve defaults, and the extension's
  source auto-approves file reads and edits, browser use and MCP calls by
  default (commands ask first, since #11865). New IN CODE fact citing
  AutoApprovalSettings.ts; the docs fact no longer implies everything asks.
  Provider fact limited to what the README says.
- Advisory counts now carry severities, re-counted with pagination.
- Claude Code: adds its 33 maintainer advisories (26 high), which every
  comparable page already showed; leaving them out read as favouritism.
- Dify: drops "download the reports from the console" (no reachable source).
- Codex: Auto is the preset it recommends for version-controlled folders;
  the page doesn't call it the default.
- LiveKit operator line now quotes the README; Freebuff's ad line is
  sourced from the privacy policy alone.
The legend explains IN CODE on pages that use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YugantM pushed a commit that referenced this pull request Oct 8, 2026
Severities on every maintainer advisory count (re-counted with pagination
on 8 Oct; Cherry Studio's 4 are all critical), and no "no advisories
published" lines, which can read as "no vulnerabilities". OSV-based counts
for Ollama (31) and Desktop Commander (2) re-checked, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@YugantM

YugantM commented Oct 8, 2026

Copy link
Copy Markdown
Owner Author

Line review applied (8 Oct), rebased onto main (15f79ca9, after #352):

  • Composio 🔍 New agent candidates — week of 2026-05-24 (468 found) #1: "Run by Sampark Inc, doing business as Composio. Its terms are governed by Delaware law." (governing law, not incorporation)
  • Cline: the docs don't state auto-approve defaults, and the extension's own AutoApprovalSettings.ts auto-approves file reads and edits, browser use and MCP calls by default; commands ask first (since fix(vscode): disable command auto-approval by default cline/cline#11865). Replaced with a docs fact plus an IN CODE fact citing the source. Provider line limited to what the README says.
  • Advisory lines carry severities, re-counted with pagination; no "no advisories" lines.
  • Claude Code: added its 33 maintainer advisories (26 high, 4 medium, 3 low; 23 in 12 months). Every comparable page already showed its count.
  • Dify: dropped "download reports from the console" (no reachable source).
  • Codex: Auto is the preset it recommends for version-controlled folders; its docs don't call it the default.
  • LiveKit: operator line quotes the README ("connects to LiveKit Cloud or your self-hosted server").
  • Freebuff: ad line sourced from the privacy policy alone.
  • The legend explains IN CODE only on pages that have such a fact (tested).

Observed facts re-checked 8 Oct. lastmod_diff: 9 of 2,430 pages, all pilot profiles. Gates green. #348 rebased on this, with the same advisory rules applied.

@YugantM
YugantM marked this pull request as ready for review October 8, 2026 08:29
@YugantM
YugantM merged commit 03f0ebf into main Oct 8, 2026
3 checks passed
@YugantM
YugantM deleted the e2-safety-facts branch October 8, 2026 10:29
@YugantM
YugantM restored the e2-safety-facts branch October 8, 2026 10:29
YugantM pushed a commit that referenced this pull request Oct 8, 2026
Severities on every maintainer advisory count (re-counted with pagination
on 8 Oct; Cherry Studio's 4 are all critical), and no "no advisories
published" lines, which can read as "no vulnerabilities". OSV-based counts
for Ollama (31) and Desktop Commander (2) re-checked, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@YugantM
YugantM deleted the e2-safety-facts branch October 8, 2026 10:30
YugantM added a commit that referenced this pull request Oct 8, 2026
#357) (#358)

Two-step deploy on 2026-10-08 so the #347 profiles got their lastmod before
the LASTMOD_FP_VERSION 7 migration; email digests stay dark.

Co-authored-by: yugant <hadiyalyugant@gmai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
YugantM pushed a commit that referenced this pull request Oct 9, 2026
Severities on every maintainer advisory count (re-counted with pagination
on 8 Oct; Cherry Studio's 4 are all critical), and no "no advisories
published" lines, which can read as "no vulnerabilities". OSV-based counts
for Ollama (31) and Desktop Commander (2) re-checked, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
YugantM added a commit that referenced this pull request Oct 9, 2026
* Profiles: sourced access facts for 15 local tools (Q4 plan E3)

Adds 52 facts for the local tools behind most of the remaining "is X
safe?" searches (#344): Ollama, AnythingLLM, OpenCode, Desktop Commander,
Open WebUI, Cherry Studio, Blender MCP, Headroom, Agent Reach, OpenShell,
CKAN MCP, Qwen Code, Odysseus, OnGrid and Aider. What each can reach
(shell, files, network binding, plugins that run code, approval
defaults), where data goes (telemetry defaults and opt-outs, local
storage, model providers) and security practice (published advisories
counted from GitHub or OSV, SECURITY.md, private reporting).

Data only, on top of the E2 mechanism. lastmod_diff vs E2: 7 pages
locally; the other 8 are pending in local data but scored in prod.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Safety facts (E3): advisory lines match the #347 review rules

Severities on every maintainer advisory count (re-counted with pagination
on 8 Oct; Cherry Studio's 4 are all critical), and no "no advisories
published" lines, which can read as "no vulnerabilities". OSV-based counts
for Ollama (31) and Desktop Commander (2) re-checked, unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Safety facts (E3): apply the owner's line review

Re-checked all 52 facts on 9 Oct against their sources, the paginated
GitHub advisory counts, OSV, and the code wherever a fact states a default.

- OpenCode: the docs say .env reads are denied by default, but v1.18.35
  asks; the default is now an IN CODE fact, plus a publisher fact on rules.
- Ollama: OSV's 31 entries are 16 advisories (GHSA and Go ids alias each
  other); severities added. New IN CODE fact: the Docker image binds
  0.0.0.0.
- OSV advisories added for Aider (2, low) and Blender MCP (1, medium).
- Qwen Code: usage statistics are on by default in code (IN CODE); the
  provider line lists all four sign-in methods and drops the inference.
- Cherry Studio's built-in model service relays prompts; Desktop
  Commander logs arguments unredacted; Blender MCP's socket has no auth,
  its safe mode, and telemetry may train AI models; Aider commits skip
  pre-commit hooks; Headroom's off switch is HEADROOM_OFFLINE=1;
  AnythingLLM's switch is in the sidebar.
- checked dates move to 2026-10-09; 55 facts in all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: yugant <hadiyalyugant@gmai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant