Repository navigation
Dex Web: read uploaded Flow definitions per Flow type, isolate bad ones, load lazily - #1604
Merged
Merged
Conversation
Add UploadFlowDefinitions, ListFlowDefinitions, and GetFlowDefinitions to FlowService. server-zd keeps one FlowDefinitionRecord object per Flow type under registry/flow-definitions/<tenant_id>/types/, created with If-None-Match and replaced with If-Match by the replace rule.
Document the three FlowService RPCs, the replace rule and force, the flowDefinitions.maxDefinitionBytes key, the object layout under registry/flow-definitions/, and the 0.1.26 change log entries. Add the key to the example configuration and the dexzd-api README. Cover Flow types that contain percent signs, dots, and slashes in the tests.
…to feat/dex-web-uploaded-definitions
…shipped graphs against the limits
…d harden definition reads
…rs, and read the Connector listing once
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ait conditions and check decoded metadata
…iles without keeping large tokens
…ocument the scan limits
Take main's server-zd, protos, and generated code from the merged Flow definition storage change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--flow-rendering-dir,Target.FlowRenderingDirectory) with definitions uploaded to the Dex Server (ListFlowDefinitions/GetFlowDefinitionsfrom server-zd: store uploaded Flow Definition Graphs per Flow type #1595, through the target's FlowServiceClient). A Flow type declared by any directory file is owned by the directory. UNIMPLEMENTED means no uploaded definitions. The embedded module needs no new Target field.GET /api/flow-definitions/{flowType}, ETag and 304);GET /api/flow-definition-typeslists names;GET /api/flow-definitionsreturns metadata only. Server-side v2 routes resolve only the requested Flow type, with per-Flow-type revisions.Why
Flow Definition Graphs could only come from a directory next to Dex Web. With uploads stored on the Dex Server (#1595) and SDK Workers uploading at startup (#1599, #1602, #1603), Dex Web reads them from the server. Loading every definition on every page does not scale once Workers can upload many Flow types (agent CLIs use one Flow type per execution environment), and an audit found that one bad definition could take down the whole catalog and that very large graphs could freeze the page. Depends on #1595 (its commits appear here until it merges).
Compatibility
Dex Web browser API changes:
GET /api/flow-definitionsno longer includes graphs; v2 routes validate per-Flow-type revisions (X-Dex-Flow-Definition-Revision) instead of the catalog-wide revision;/readyzlists Flow type names and no longer fails on one invalid definition (it still fails when the directory is unreadable; its response fields are unchanged). A bad directory file now marks only itself invalid instead of failing the catalog.Testing
go test ./...in web (server source: UNIMPLEMENTED, pagination, directory precedence, isolation, limits, batch splitting on RESOURCE_EXHAUSTED, server failure with the directory still served; per-type routes with 304, 404, and encoded names; v2 routes scoped to one Flow type with stale-revision 409; Connector 404 cache and concurrency; every Flow Definition Graph in the repository within the limits; embedded Target without a directory).npm run checkin web (795 tests, including lazy loading, cache, Designs page, error boundary and limits, main line).web/scripts/run-server-zd-e2e.sh): newTestDexWebOnServerZDShowsUploadedDefinitionsuploads viaUploadFlowDefinitionsand checks display and Actions without a directory, and directory precedence.cd docs && npm run check,make copyright-check.