Skip to content

RDSC-5669 Publish RDI Cloud API reference#3562

Draft
ZdravkoDonev-redis wants to merge 11 commits into
mainfrom
rdsc-5669-rdi-capi-docs
Draft

RDSC-5669 Publish RDI Cloud API reference#3562
ZdravkoDonev-redis wants to merge 11 commits into
mainfrom
rdsc-5669-rdi-capi-docs

Conversation

@ZdravkoDonev-redis

@ZdravkoDonev-redis ZdravkoDonev-redis commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Keep redis/docs as a consumer of the public Cloud API file.
  • Download the combined file from the existing /v1/cloud-api-docs endpoint.
  • Test the downloaded file before a bot opens an update PR.
  • Add a semantic summary of operations, schemas, and source changes to the bot PR.
  • Remove API composition from the docs workflow. sm-cloud-api now owns that work.

This PR does not add a new public endpoint. It stays in draft until the sm-cloud-api change is merged and deployed. After that deployment, the docs workflow can update the checked-in OpenAPI file from the existing endpoint.

Related PRs

Desired flow

flowchart LR
    subgraph CORE["redis-data-integration"]
        CORE_CODE["RDI Core API code"]
        CORE_CI["CI<br/>Generate and test the OpenAPI file"]
        CORE_FILE["Versioned OpenAPI release"]

        CORE_CODE --> CORE_CI
        CORE_CI --> CORE_FILE
    end

    subgraph CLOUD_RDI["cloud-rdi-service"]
        RDI_CODE["Cloud RDI API code"]
        RDI_CI["CI<br/>Generate and test the OpenAPI file"]
        RDI_FILE["OpenAPI build output"]

        RDI_CODE --> RDI_CI
        RDI_CI --> RDI_FILE
    end

    subgraph CLOUD_API["sm-cloud-api"]
        CAPI_CODE["Cloud API code"]
        COMBINE_CI["CI<br/>Download the selected RDI files<br/>Build and test three OpenAPI files"]
        UPDATE_PR["Bot PR<br/>Update selected versions and generated files"]
        REVIEW["Review and merge"]
        DEPLOY["Deploy sm-cloud-api"]

        CAPI_CODE --> COMBINE_CI
        COMBINE_CI --> UPDATE_PR
        UPDATE_PR --> REVIEW
        REVIEW --> DEPLOY

        DEPLOY --> CAPI_DOCS["/v1/cloud-api-docs/capi"]
        DEPLOY --> RDI_DOCS["/v1/cloud-api-docs/rdi"]
        DEPLOY --> ALL_DOCS["/v1/cloud-api-docs<br/>Combined file"]
    end

    CORE_FILE -->|"Notify sm-cloud-api"| COMBINE_CI
    RDI_FILE -->|"Notify sm-cloud-api"| COMBINE_CI

    subgraph DOCS["redis/docs"]
        CORE_DOCS_CI["CI<br/>Open an RDI Core docs PR"]
        CLOUD_DOCS_CI["Daily CI<br/>Download and test the combined file"]
        CLOUD_DOCS_PR["Bot PR<br/>Update the Cloud API file"]
        DOCS_REVIEW["Review and merge"]
        STAGING["Build documentation staging"]
        PUBLISH["Publish the approved docs release"]

        CORE_DOCS_CI --> DOCS_REVIEW
        CLOUD_DOCS_CI --> CLOUD_DOCS_PR
        CLOUD_DOCS_PR --> DOCS_REVIEW
        DOCS_REVIEW --> STAGING
        STAGING --> PUBLISH
    end

    CORE_FILE --> CORE_DOCS_CI
    ALL_DOCS --> CLOUD_DOCS_CI
    PUBLISH --> CUSTOMER["redis.io API references"]
Loading

The canonical copy of this diagram is in sm-cloud-api/diagrams/rdi-public-api-documentation-flow.md.

Validation

  • All 11 OpenAPI tool unit tests pass.
  • The workflow passes actionlint and YAML parsing.
  • The validator checks the API title, paths, references, operation IDs, exact source metadata, and the exact RDI authentication pair.
  • Existing bot PR descriptions are updated when the generated API changes again.
  • The workflow only opens a PR after the downloaded combined file passes validation.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

@CLAassistant

CLAassistant commented Jun 29, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@jit-ci

jit-ci Bot commented Jun 29, 2026

Copy link
Copy Markdown

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@ZdravkoDonev-redis ZdravkoDonev-redis self-assigned this Jul 1, 2026
@ZdravkoDonev-redis
ZdravkoDonev-redis force-pushed the rdsc-5669-rdi-capi-docs branch from a347f6b to be6f28d Compare July 2, 2026 06:05
@ZdravkoDonev-redis
ZdravkoDonev-redis marked this pull request as ready for review July 2, 2026 06:11

@cmilesb cmilesb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I already mentioned it in our Slack DMs, but it doesn't seem like there's an easy way to keep the data/api file updated. Are we just going to replace the file every time the API get changed? We call the Swagger so that we can keep the docs up to date with what's actually deployed.

@paoloredis - Can you look at the updated workflow to make sure there isn't anything broken or weird?

@cmilesb
cmilesb requested a review from paoloredis July 6, 2026 18:36

@paoloredis paoloredis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for merging the RDI API reference into the RC API reference?

@cmilesb cmilesb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paoloredis This is specifically for the Redis Cloud RDI stuff, so it should go in the Redis Cloud API reference.

Comment thread .github/workflows/rc_api_sync.yaml Outdated
Comment on lines +78 to +140
def enrich_rdi_workspace_docs:
.components.schemas.WorkspaceCreateRequest.description //= "Request to provision a data integration workspace for a Redis Cloud subscription."
| .components.schemas.WorkspaceCreateRequest.properties.cidr.description //= "CIDR block for the workspace deployment network."
| .components.schemas.WorkspacesResponse.description //= "Response containing data integration workspaces in the Redis Cloud account."
| .components.schemas.WorkspacesResponse.properties.workspaces.description //= "Data integration workspaces."
| .components.schemas.WorkspaceTasksResponse.description //= "Response containing asynchronous workspace tasks."
| .components.schemas.WorkspaceTasksResponse.properties.tasks.description //= "Workspace tasks."
| .components.schemas.WorkspaceTaskResponse.description //= "Asynchronous task created by a workspace operation."
| .components.schemas.WorkspaceTaskResponse.properties.taskId.description //= "Workspace task ID."
| .components.schemas.WorkspaceTaskResponse.properties.commandType.description //= "Workspace operation that created the task."
| .components.schemas.WorkspaceTaskResponse.properties.status.description //= "Current task status."
| .components.schemas.WorkspaceTaskResponse.properties.description.description //= "Human-readable task status or failure details."
| .components.schemas.WorkspaceTaskResponse.properties.timestamp.description //= "Time when the task status was recorded."
| .components.schemas.WorkspaceTaskResponse.properties.response.description //= "Task result payload. Completed workspace tasks can include the current workspace response."
| .components.schemas.WorkspaceTaskResponse.properties._links.description //= "Links to the task and affected workspace resource."
| .components.schemas.TaskLinks.description //= "Links related to a workspace task."
| .components.schemas.TaskLinks.properties.task.description //= "Link to the workspace task."
| .components.schemas.TaskLinks.properties.resource.description //= "Link to the affected workspace resource."
| .components.schemas.TaskLink.description //= "Hypermedia link."
| .components.schemas.TaskLink.properties.href.description //= "Link URL."
| .components.schemas.TaskLink.properties.type.description //= "Linked resource type."
| .components.schemas.WorkspaceResponse.description //= "Current data integration workspace state for a subscription."
| .components.schemas.WorkspaceResponse.properties.subscriptionId.description //= "Redis Cloud subscription ID."
| .components.schemas.WorkspaceResponse.properties.bdbId.description //= "Redis database ID associated with the workspace."
| .components.schemas.WorkspaceResponse.properties.rcpClusterId.description //= "RCP cluster ID that hosts workspace resources."
| .components.schemas.WorkspaceResponse.properties.region.description //= "Cloud provider region where the workspace is deployed."
| .components.schemas.WorkspaceResponse.properties.vpcId.description //= "VPC ID used by the workspace."
| .components.schemas.WorkspaceResponse.properties.createdAt.description //= "Time when the workspace record was created."
| .components.schemas.WorkspaceResponse.properties.infrastructure.description //= "Provisioned infrastructure state and connection details."
| .components.schemas.WorkspaceResponse.properties.capabilities.description //= "Workspace features available in the current state."
| .components.schemas.InfrastructureResponse.description //= "Provisioned infrastructure state for a data integration workspace."
| .components.schemas.InfrastructureResponse.properties.state.description //= "Workspace infrastructure state."
| .components.schemas.InfrastructureResponse.properties.cidr.description //= "CIDR block assigned to the workspace deployment network."
| .components.schemas.InfrastructureResponse.properties.timestamps.description //= "Infrastructure lifecycle timestamps."
| .components.schemas.InfrastructureResponse.properties.iam.description //= "IAM values used by workspace integrations."
| .components.schemas.InfrastructureResponse.properties.networking.description //= "Workspace networking outputs."
| .components.schemas.InfrastructureResponse.properties.monitoring.description //= "Workspace monitoring endpoints."
| .components.schemas.InfrastructureResponse.properties.error.description //= "Infrastructure error details when the workspace is in a failed state."
| .components.schemas.InfrastructureTimestamps.description //= "Workspace infrastructure lifecycle timestamps."
| .components.schemas.InfrastructureTimestamps.properties.startedAt.description //= "Time when infrastructure provisioning started."
| .components.schemas.InfrastructureTimestamps.properties.updatedAt.description //= "Time when infrastructure state was last updated."
| .components.schemas.InfrastructureIam.description //= "IAM values produced for workspace integrations."
| .components.schemas.InfrastructureIam.properties.privateLinkAllowedPrincipal.description //= "AWS principal allowed to connect to the workspace PrivateLink endpoint service."
| .components.schemas.InfrastructureIam.properties.secretsRoleArn.description //= "AWS IAM role ARN used to read workspace secrets."
| .components.schemas.NetworkingOutput.description //= "Workspace networking outputs."
| .components.schemas.NetworkingOutput.properties.egressIps.description //= "Outbound IP addresses used by workspace components."
| .components.schemas.NetworkingOutput.properties.availabilityZones.description //= "Availability zones used by workspace infrastructure."
| .components.schemas.MonitoringOutput.description //= "Workspace monitoring outputs."
| .components.schemas.MonitoringOutput.properties.metricsUrls.description //= "Metrics endpoint URLs for workspace components."
| .components.schemas.MetricsUrls.description //= "Metrics endpoint URLs for RDI workspace components."
| .components.schemas.MetricsUrls.properties.rdiOperatorUrl.description //= "Metrics endpoint URL for the RDI operator."
| .components.schemas.MetricsUrls.properties.rdiProcessorUrl.description //= "Metrics endpoint URL for the RDI processor."
| .components.schemas.MetricsUrls.properties.collectorSourceUrls.description //= "Metrics endpoint URLs for deployed collector sources, keyed by source name."
| .components.schemas.InfrastructureError.description //= "Infrastructure failure details."
| .components.schemas.InfrastructureError.properties.code.description //= "Machine-readable error code."
| .components.schemas.InfrastructureError.properties.message.description //= "Human-readable error message."
| .components.schemas.InfrastructureError.properties.timestamp.description //= "Time when the error was recorded."
| .components.schemas.InfrastructureError.properties.details.description //= "Additional structured error details."
| .components.schemas.Capabilities.description //= "Workspace capabilities available in the current state."
| .components.schemas.Capabilities.properties.validateSecrets.description //= "Whether the workspace can validate secret ARNs."
| .components.schemas.Capabilities.properties.validatePrivateLink.description //= "Whether the workspace can validate PrivateLink endpoint service names."
| .components.schemas.Capabilities.properties.readyForPipeline.description //= "Whether the workspace is ready to create and deploy RDI pipelines."
| .components.schemas.Capabilities.properties.createPrivateLink.description //= "Whether the workspace can create PrivateLink endpoints.";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is all of this stuff added here and is not part of the rdi json file in the first place?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal was that CAPI exposes functionality not just of the cloud database but all applications as well. So for example, the RDI API is accessible under the CAPI Pro subscriptions path, so on one side it makes sense to document the RDI API in the same place.

If you think we should separate the docs reference, we can also discuss that approach.

TBH I don't have a strong opinion.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not against merging the two, but this json manipulation is going to be hard to maintain. Why are all these changes needed for something that's published on https://api.redislabs.com/v1/cloud-api-docs/rdi ? Can the file not be published there with these changes already in place?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me rethink the approach and I'll get back.

@ZdravkoDonev-redis
ZdravkoDonev-redis force-pushed the rdsc-5669-rdi-capi-docs branch from 63e664e to ff0e0a1 Compare July 21, 2026 13:31
@ZdravkoDonev-redis
ZdravkoDonev-redis marked this pull request as draft July 21, 2026 13:32
@ZdravkoDonev-redis
ZdravkoDonev-redis force-pushed the rdsc-5669-rdi-capi-docs branch from ff0e0a1 to 34d2776 Compare July 21, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants