Repository navigation
Java and Rust SDKs: upload Flow Definition Graphs to the Dex Server - #1603
Merged
Merged
Conversation
Add UploadFlowDefinitions, ListFlowDefinitions, and GetFlowDefinitions to FlowService. server-zd keeps one FlowDefinitionRecord object per Flow type under registry/flow-definitions/<tenant_id>/types/, created with If-None-Match and replaced with If-Match by the replace rule.
Document the three FlowService RPCs, the replace rule and force, the flowDefinitions.maxDefinitionBytes key, the object layout under registry/flow-definitions/, and the 0.1.26 change log entries. Add the key to the example configuration and the dexzd-api README. Cover Flow types that contain percent signs, dots, and slashes in the tests.
…to feat/flow-definition-upload-java-rust
Workers accept flowDefinitionPaths / flow_definition_paths and flowDefinitions / flow_definitions. At start, after protocol negotiation and Attribute index sync, they upload the documents of registered Flow types in one UploadFlowDefinitions call (batched by 256 documents or 16 MiB), rewriting an absolute source.path to a relative path. Upload failures are logged as warnings and never stop the Worker; duplicate Flow types across sources fail Worker construction. The Client gains uploadFlowDefinitions / upload_flow_definitions for deploy scripts, returning per-document outcomes. Adds integration tests against dexcli dev, Developer reference sections, and 1.45 changelog entries.
Align Worker and Client Flow definition upload with the other SDKs: - A missing or unreadable path, an invalid explicit file, and a malformed generatedAt are configuration errors. Directory entries that are not Flow Definition Graph 2.x documents are skipped with an info log. In-memory documents must be 2.x documents naming their Flow type, for Workers and for Client uploads. - Parse generatedAt as strict RFC 3339 and truncate every generation time to whole milliseconds. - Rewrite source.path with OS-independent lexical rules, treating Windows drive and UNC paths as absolute, using the base name when the working directory is unknown, and encoding the replacement with minimal JSON escaping. - Share one deadline across upload batches, cancel an in-flight upload on Worker stop, log unknown outcomes separately, and report the uploader as sdk-java/<version> or sdk-rust/<version>. Add the cross-SDK conformance fixture and assert every case in both SDKs.
- Parse Flow definition documents as strict JSON; Java now rejects trailing tokens after the top-level value. - Rewrite source.path only when the document has exactly one top-level source object with exactly one string path, and send the original bytes otherwise instead of failing on duplicate keys. - Check RFC 3339 field ranges for generatedAt, including days per month and offset limits. - Validate generatedAt inside in-memory documents like file contents. - Update the conformance fixture and assert its document validity, document rewrite, and generatedAt cases in both SDKs.
Parse the RFC 3339 offset directly and subtract it from the UTC local date-time, so offsets beyond ZoneOffset's 18-hour limit are accepted like in the other SDKs.
Drop fraction digits beyond nanoseconds before parsing; they never affect the millisecond-truncated time, and the other SDKs accept them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WorkerOptions.Builder.flowDefinitionPaths(List<Path>)andflowDefinitions(List<FlowDefinitionDocument>);Client.uploadFlowDefinitions(documents, force);FlowDefinitionDocument,FlowDefinitionUploadOutcome(withUNKNOWN),FlowDefinitionUploadResult.WorkerOptions::flow_definition_pathsandflow_definitions;Client::upload_flow_definitions(documents, force);FlowDefinitionDocument, non-exhaustiveFlowDefinitionUploadOutcome(withUnknown),FlowDefinitionUploadResult. Protocol types come fromdex-protocol/build.rs, so the new RPCs need no regeneration.source.pathvalues are rewritten to a working-directory-relative path or the file name, preserving all other bytes.Why
Same as #1599 and #1602: Workers upload Flow Definition Graphs at startup so every Dex Web reads them from the Dex Server (#1595) instead of a directory next to Dex Web. Depends on #1595 (its commits appear here until it merges).
Testing
sdk-java/run-integration-tests.sh(FlowDefinitionUploadTest: directory and in-memory upload,generatedAtand mtime, registered-only filtering,source.pathrewrite byte-preserving, duplicate-source error, rejected upload still runs a Flow, Client outcomes and force; plus a fake-server UNIMPLEMENTED test) andsdk-rust/run-integration-tests.sh(flow_definition_upload_test.rs, same scenarios), against server-zd built from this branch../gradlew build javadoc test; Rust fmt, clippy-D warnings,cargo test -p dex-sdkwith doctests,make docs-check,make rust-rules-check;make copyright-check;cd docs && npm run check.