Skip to content

feat: persons v2 ($set semantics, search, detail, merge, delete) + complete GDPR coverage - #14

Merged
ayushozha merged 3 commits into
mainfrom
ayush/persons-v2
Nov 17, 2026
Merged

ayushozha merged 3 commits into
mainfrom
ayush/persons-v2

Conversation

@ayushozha

@ayushozha ayushozha commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

What and Why

Pulse's persons were a paged list with no search, no filters and no detail tabs. Identify could only overwrite traits; $set_once, $unset and person updates from events weren't supported. There was also no way to merge two people.

This adds PostHog-style persons: property-update semantics, a searchable and filterable people list, person detail with events and sessions, property editing, merge (with redirects for ids that keep sending events) and delete. It also fixes GDPR delete and export, which were missing six visitor tables.

Stacked on #13 (base ayush/cohorts-v2); retarget to main once the stack below merges.

Reference followed:

  • PostHog person property semantics: $set overwrites, $set_once sets only if absent, $unset removes, applied from identify and from any event's properties.
  • PostHog's Persons API (posthog/api/person.py): list/search with property and cohort filters, person events and sessions, property updates, merge, and delete with optional event deletion.
  • After a merge, PostHog maps the merged distinct ids to the surviving person. That's mirrored here with person_redirects.

Changes

  • Person properties. identify now accepts set_once and unset, and events may carry $set, $set_once and $unset in their data. Both use one SQL upsert, traits = ((set_once || traits) || set) - unset, which creates the person if needed.
  • POST /api/v2/persons/query:
    • search across visitor id, user id and any property value (LIKE wildcards escaped)
    • person-property and cohort filters (cohort filters require the cohorts module)
    • order by last_seen, first_seen or identified, and a total count
  • GET /api/v2/persons/{id}: profile, aliases, merged ids, pageview/event/session counts, and cohort memberships.
    • /events: newest first, filterable by event.
    • /sessions: includes a recording flag.
    • PATCH /properties: set, set_once and unset.
  • POST /api/v2/persons/{id}/merge (up to 100 people):
    • It runs in one transaction. Every visitor table's rows move to the target.
    • Unique conflicts (experiment exposures, account memberships, aliases, cohort membership) keep the target's row.
    • The merged people's properties fill gaps, and the target's values win.
    • Redirects are recorded, so events still arriving under a merged id are ingested as the target. Ingest checks a Redis hash, which is warmed from person_redirects at startup.
  • DELETE /api/v2/persons/{id} deletes the profile only, or everything with ?delete_events=true (GDPR).
  • Privacy fix: a single VISITOR_TABLES list now drives both delete and merge. GDPR delete and export previously left behind feature_flag_evaluations, guide_events, event_quality_violations, llm_traces, account_memberships and cohort memberships; they're now included.
  • Migration 034: (project_id, visitor_id, created_at) indexes on pageviews and events, plus the person_redirects table.
  • Gating: everything is behind the identity module.

Reviews and Follow-up

  • Code review: self-reviewed
  • Security review: project-scoped everywhere, LIKE escaping, identity and cohorts module gating, merge in a single transaction
  • Documentation: request shapes above. Follow-ups: person property history, split persons, and multiple group types (next PR).

Files Changed

  • migrations/034_persons.{up,down}.sql
  • services/persons.rs, routes/persons.rs (new)
  • services/identity.rs: PersonUpdate, apply_person_update
  • services/privacy.rs: VISITOR_TABLES, complete delete and export
  • routes/ingest.rs: event-level person updates, merge redirects
  • pulse-common/src/types.rs: identify set_once/unset
  • routes/mod.rs, services/mod.rs, main.rs

Scope and Non-scope

In scope: persons and the privacy completeness fix. Out of scope: groups (next PR), UI, and the excluded LLM/account features.

Validation

I ran this end to end against Postgres 18, sending events through /api/collect exactly as clients do. The checks cover:

  • $set, $set_once and $unset on identify and on events
  • search, property and cohort filters
  • detail counts against SQL, the events and sessions feeds, and property edits
  • merge (rows moved, property precedence, redirect of later events), delete, GDPR delete, and permissions

One check initially counted a cohort row belonging to a different project. It was corrected to be project-scoped and re-run, and the evidence notes this.

Persons verification

Testing

cargo test: 131 passed, including new tests for $set/$set_once/$unset extraction.

cargo test person_update

Size and Notes

About 950 changed lines.

  • Migration 034 adds two indexes on partitioned tables. That's fine for small tables; on large production data, consider creating them CONCURRENTLY on each partition first.
  • Merge rewrites history for the merged ids, which is intended and matches PostHog.

Review fixes (beac2a5)

  • Merge: $3 is bound only for the UPDATE statements, so merges between people with no aliases no longer fail.
  • Merge: conflicts on experiment_exposures and account_memberships are resolved between sources as well as against the target. The first-listed source wins.
  • Person detail: cohorts is null unless the key can read the cohorts module.

PR 14 review fixes

🤖 Generated with Claude Code

ayushozha and others added 2 commits November 16, 2026 14:52
…e) and complete GDPR coverage

- identify accepts set_once / unset; any event's $set / $set_once / $unset
  properties update (or create) the person: traits = ((set_once || traits) || set) - unset
- /api/v2/persons/query: search (visitor id, user id, property values),
  person-property and cohort filters, ordering, total count
- /api/v2/persons/{id}: profile, aliases, merged ids, event/pageview/session
  counts, cohort memberships; /events (newest first, event filter);
  /sessions (with recording flag); PATCH /properties
- POST /persons/{id}/merge: move every visitor table's rows to the target,
  resolve unique conflicts (exposures, memberships, aliases, cohorts), fill
  gaps in the target's properties, and record redirects so events that keep
  arriving under merged ids are ingested as the target (Redis hash, warmed
  from person_redirects at startup)
- DELETE /persons/{id} (profile only, or ?delete_events=true for GDPR)
- privacy: one VISITOR_TABLES list for delete and merge; GDPR delete/export
  now also cover feature_flag_evaluations, guide_events,
  event_quality_violations, llm_traces, account_memberships and cohort
  memberships (previously left behind)
- migration 034: visitor indexes on pageviews/events, person_redirects

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T22:55:07.053968Z 411e4c8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 411e4c8694

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…rts access

- Bind $3 only for the UPDATE statements in the merge loop.
- Resolve experiment_exposures/account_memberships conflicts between
  sources as well as against the target (first-listed source wins).
- GET /api/v2/persons/{id} returns cohorts: null unless the key can read cohorts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ayushozha
ayushozha changed the base branch from ayush/cohorts-v2 to main November 17, 2026 01:32
@ayushozha
ayushozha merged commit 0c24101 into main Nov 17, 2026
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant