H-6617: Create custom graph rendering pipeline inside frontend - #8952
H-6617: Create custom graph rendering pipeline inside frontend#8952indietyp wants to merge 477 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
github-actions
Bot
added
area/deps
This stack of pull requests is managed by Graphite. Learn more about stacking. |
2fbcafb to
e313dac
Compare
Merging this PR will regress 2 benchmarks
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | bit_matrix/dense/union_rows[1000] |
102.4 µs | 119.3 µs | -14.16% |
| ❌ | bit_matrix/sparse/union_rows[1000] |
126.3 µs | 144.8 µs | -12.82% |
| ⚡ | n50000_d256[32] |
1,467.9 ms | 696.5 ms | ×2.1 |
| ⚡ | n10000_d256[128] |
4 s | 1.9 s | ×2.1 |
| ⚡ | n10000_d256[32] |
1,033.5 ms | 517.7 ms | +99.63% |
| ⚡ | n10000_d256[8] |
94.3 ms | 47.9 ms | +96.69% |
| ⚡ | d256[64] |
68.3 µs | 40.7 µs | +67.74% |
| ⚡ | d1536[16] |
111.6 µs | 70.6 µs | +57.92% |
| ⚡ | d3072[16] |
221.2 µs | 140.2 µs | +57.76% |
| ⚡ | d256[16] |
19.5 µs | 12.5 µs | +56.75% |
| ⚡ | d256[15] |
18.6 µs | 12.4 µs | +50.22% |
| ⚡ | kernel/micro_4x2[256] |
4.9 µs | 3.8 µs | +27.89% |
| ⚡ | kernel/micro_4x2[1536] |
23.9 µs | 18.8 µs | +27.01% |
| ⚡ | kernel/micro_4x2[3072] |
46.8 µs | 36.8 µs | +26.99% |
| ⚡ | bit_matrix/dense/iter_row[1000] |
496.4 ns | 440.8 ns | +12.6% |
| ⚡ | kernel/dot[256] |
1.3 µs | 1.2 µs | +10.56% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bm/h-6617-experiment-with-graph-rendering-inside-of-hash-graph (6c41b64) with main (276e17d)1
Footnotes
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8952 +/- ##
==========================================
+ Coverage 59.19% 61.51% +2.32%
==========================================
Files 1389 1248 -141
Lines 135001 123443 -11558
Branches 6183 5785 -398
==========================================
- Hits 79910 75936 -3974
+ Misses 54151 46623 -7528
+ Partials 940 884 -56 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
github-actions
Bot
removed
the
area/libs
7c7fadb to
26a8078
Compare
f4af41d to
a4a2e3d
Compare
a4a2e3d to
f91ab64
Compare
f91ab64 to
efb273c
Compare
- deny_unknown_fields rejects includeDetailedData at parse, so every tile and edges request was refused - SaltileDetail carries the two-mode contract end to end: fetch options, cache entries, decode expectations - detailServes orders the cache modes: auxiliary serves a minimal load, never the reverse
- every caller passed &[], so the appended-ring machinery built nothing - fixture_dataset_extended collapses onto fixture_dataset
- every accepted document carries the keys, so the defaults served only refused versions - classifier evidence becomes required; each removal gains a refusal fixture - empty_cut_components keeps its bought Option, losing only the implied attribute
- every consumer of node sits on the operator path: is_full is cut.is_none() by definition
- classifier fit and policy report now belong to the Rust binary - audit and battery duplicate the Rust report subcommands - three common contracts lost their last consumers with them
- add window/set-operation/exists/array-slice/regex-match/jsonb-extract expressions - make binary-op and coalesce builders self-consuming with Into<Expression> - type PostgresType::Vector with an optional dimension count
- add JsonDelete, JsonTypeof, JsonPathQueryArray, JsonObjectAgg - split JsonbArrayElements into JsonArrayElements/JsonArrayElementsText, rename JsonbEach to JsonEach
- replace raw SQL constants (scope/links CTEs, ad-hoc pgvector decode) with the typed builder - split dataset/postgres/mod.rs into corpus/sql/lookup/streams/vector/vocabulary/error/card modules
- keep RunError public while its narrowed RunnerError payload stays crate-only via Error::source
- promote Placeholder/Binder/BoundStatement/SelectList/Aliased from the atlas dataset module into the shared postgres-store query AST - split Aliased into a schema-backed reference and a new Correlation for purely local/derived relations - add live-store prepare probes alongside the ported dataset queries
- replace the embedded raw-SQL constants and their custom masking tests with typed statement builders pinned to fixture renderings - retitle stray surface wording in the module docs to match the new structure
- widen math::scalar with PositiveUnitFraction and crate-wide literal macros - propagate typed scalars through relation/semantic/policy/projector/ladder pipelines - add a boundary-stability certificate to projector radius calibration
| let (client, connection) = tokio_postgres::connect( | ||
| "host=localhost user=postgres password=postgres dbname=graph", | ||
| NoTls, | ||
| ) |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.
Dataflow graph
flowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>libs/@local/graph/atlas/src/serve/hydrate/statements.rs</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/serve/hydrate/statements.rs#L845 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 845] "host=localhost user=postgres password=postgres dbname=graph"</a>"]
end
%% Intermediate
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/serve/hydrate/statements.rs#L844 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 844] tokio_postgres::connect(<br> "host=localhost user=postgres password=postgres dbname=graph",<br> NoTls,<br> )</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
File0:::invis
%% Connections
Source --> Sink
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-connection-url.
You can view more details about this finding in the Semgrep AppSec Platform.
| let (client, connection) = tokio_postgres::connect( | ||
| "host=localhost user=postgres password=postgres dbname=graph", | ||
| NoTls, | ||
| ) |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.
Dataflow graph
flowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>libs/@local/graph/atlas/src/dataset/postgres/mod.rs</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/mod.rs#L468 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 468] "host=localhost user=postgres password=postgres dbname=graph"</a>"]
end
%% Intermediate
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/mod.rs#L467 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 467] tokio_postgres::connect(<br> "host=localhost user=postgres password=postgres dbname=graph",<br> NoTls,<br> )</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
File0:::invis
%% Connections
Source --> Sink
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-connection-url.
You can view more details about this finding in the Semgrep AppSec Platform.
| let (mut client, connection) = tokio_postgres::connect( | ||
| "host=localhost user=postgres password=postgres dbname=graph", | ||
| NoTls, | ||
| ) |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.
Dataflow graph
flowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>libs/@local/graph/atlas/src/dataset/postgres/card/mod.rs</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/mod.rs#L281 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 281] "host=localhost user=postgres password=postgres dbname=graph"</a>"]
end
%% Intermediate
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/mod.rs#L280 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 280] tokio_postgres::connect(<br> "host=localhost user=postgres password=postgres dbname=graph",<br> NoTls,<br> )</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
File0:::invis
%% Connections
Source --> Sink
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-connection-url.
You can view more details about this finding in the Semgrep AppSec Platform.
| let (client, connection) = tokio_postgres::connect( | ||
| "host=localhost user=postgres password=postgres dbname=graph", | ||
| NoTls, | ||
| ) |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.
Dataflow graph
flowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>libs/@local/graph/atlas/src/dataset/postgres/card/examples.rs</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/examples.rs#L848 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 848] "host=localhost user=postgres password=postgres dbname=graph"</a>"]
end
%% Intermediate
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/examples.rs#L847 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 847] tokio_postgres::connect(<br> "host=localhost user=postgres password=postgres dbname=graph",<br> NoTls,<br> )</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
File0:::invis
%% Connections
Source --> Sink
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-connection-url.
You can view more details about this finding in the Semgrep AppSec Platform.
| let (client, connection) = tokio_postgres::connect( | ||
| "host=localhost user=postgres password=postgres dbname=graph", | ||
| NoTls, | ||
| ) |
There was a problem hiding this comment.
Semgrep identified a blocking 🔴 issue in your code:
A secret is hard-coded in the application. Secrets stored in source code, such as credentials, identifiers, and other types of sensitive data, can be leaked and used by internal or external malicious actors. It is recommended to rotate the secret and retrieve them from a secure secret vault or Hardware Security Module (HSM), alternatively environment variables can be used if allowed by your company policy.
Dataflow graph
flowchart LR
classDef invis fill:white, stroke: none
classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none
subgraph File0["<b>libs/@local/graph/atlas/src/dataset/postgres/card/associations.rs</b>"]
direction LR
%% Source
subgraph Source
direction LR
v0["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/associations.rs#L783 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 783] "host=localhost user=postgres password=postgres dbname=graph"</a>"]
end
%% Intermediate
%% Sink
subgraph Sink
direction LR
v1["<a href=https://github.com/hashintel/hash/blob/ba84db1b32c4787b8dbb37f5758058da657936c7/libs/@local/graph/atlas/src/dataset/postgres/card/associations.rs#L782 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 782] tokio_postgres::connect(<br> "host=localhost user=postgres password=postgres dbname=graph",<br> NoTls,<br> )</a>"]
end
end
%% Class Assignment
Source:::invis
Sink:::invis
File0:::invis
%% Connections
Source --> Sink
To resolve this comment:
🔧 No guidance has been designated for this issue. Fix according to your organization's approved methods.
💬 Ignore this finding
Reply with Semgrep commands to ignore this finding.
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
Alternatively, triage in Semgrep AppSec Platform to ignore the finding created by hardcoded-connection-url.
You can view more details about this finding in the Semgrep AppSec Platform.
3 participants
🌟 What is the purpose of this PR?
🔗 Related links
🚫 Blocked by
🔍 What does this change?
Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
turbo.json's have been updated to reflect this🐾 Next steps
🛡 What tests cover this?
❓ How to test this?
📹 Demo