Skip to content

feat(cloudflare): add Cloudflare Workflows support#304

Open
userlaurin wants to merge 264 commits into
spcl:masterfrom
userlaurin:feature/cloudflare-workflows
Open

feat(cloudflare): add Cloudflare Workflows support#304
userlaurin wants to merge 264 commits into
spcl:masterfrom
userlaurin:feature/cloudflare-workflows

Conversation

@userlaurin

Copy link
Copy Markdown
Contributor

Implement Cloudflare Workflows deployment, generation, invocation, and
regression support for the workflow benchmark suite.

Add a TypeScript workflow generator that translates SeBS FSM definitions into
Cloudflare Workflows, including Map and Parallel fan-out via child workflows,
Durable Object fan-in coordination, dispatcher container calls, retry handling,
and R2-backed storage support.

Add Cloudflare workflow packaging and deployment logic, wrangler workflow
templates, workflow triggers, and workflow resource/config handling for
container-based execution. Extend container readiness handling with
max_instances-aware health checks and configurable chunk_size/max_instances
settings.

Update Cloudflare Python/Node wrappers and regression selection so supported
workflow benchmarks can run on Cloudflare containers.

Regression: Cloudflare Python 3.12 workflow suite passes 13/13.

ldzgch (MacOS) and others added 30 commits March 18, 2026 10:25
When running the handler as a thread and then awaiting said thread
(asyncio.to_thread), we can run asyncio.run() in the subsequent call
stack.
…er orchestrator, worker orchestrator proxy for r2 and durable objects to be used in container
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 206 files, which is 56 over the limit of 150.

To get a review, narrow the scope:
• coderabbit review --type committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 5bf696f7-4aa4-44f8-9412-7673448f0897

📥 Commits

Reviewing files that changed from the base of the PR and between a7a0532 and b625fae.

📒 Files selected for processing (206)
  • .github/workflows/_regression-job.yml
  • .github/workflows/regression.yml
  • .mypy.ini
  • benchmarks-data
  • benchmarks/000.microbenchmarks/040.server-reply/config.json
  • benchmarks/100.webapps/110.dynamic-html/config.json
  • benchmarks/100.webapps/120.uploader/config.json
  • benchmarks/100.webapps/120.uploader/nodejs/cloudflare/function.js
  • benchmarks/100.webapps/120.uploader/python/cloudflare/function.py
  • benchmarks/100.webapps/130.crud-api/config.json
  • benchmarks/200.multimedia/210.thumbnailer/config.json
  • benchmarks/200.multimedia/220.video-processing/config.json
  • benchmarks/300.utilities/311.compression/config.json
  • benchmarks/300.utilities/311.compression/nodejs/cloudflare/function.js
  • benchmarks/300.utilities/311.compression/nodejs/cloudflare/package.json
  • benchmarks/400.inference/411.image-recognition/config.json
  • benchmarks/400.inference/411.image-recognition/python/cloudflare/requirements.txt.3.11
  • benchmarks/400.inference/411.image-recognition/python/cloudflare/requirements.txt.3.12
  • benchmarks/500.scientific/501.graph-pagerank/config.json
  • benchmarks/500.scientific/502.graph-mst/config.json
  • benchmarks/500.scientific/503.graph-bfs/config.json
  • benchmarks/500.scientific/504.dna-visualisation/config.json
  • benchmarks/600.workflows/610.gen/config.json
  • benchmarks/600.workflows/610.gen/definition.json
  • benchmarks/600.workflows/610.gen/input.py
  • benchmarks/600.workflows/610.gen/python/few_people.py
  • benchmarks/600.workflows/610.gen/python/get_astros.py
  • benchmarks/600.workflows/610.gen/python/many_people.py
  • benchmarks/600.workflows/610.gen/python/map_astros.py
  • benchmarks/600.workflows/610.gen/python/process_astros.py
  • benchmarks/600.workflows/610.gen/python/requirements.txt
  • benchmarks/600.workflows/6100.1000-genome/LICENSE
  • benchmarks/600.workflows/6100.1000-genome/README.md
  • benchmarks/600.workflows/6100.1000-genome/config.json
  • benchmarks/600.workflows/6100.1000-genome/definition.json
  • benchmarks/600.workflows/6100.1000-genome/input.py
  • benchmarks/600.workflows/6100.1000-genome/python/frequency.py
  • benchmarks/600.workflows/6100.1000-genome/python/individuals.py
  • benchmarks/600.workflows/6100.1000-genome/python/individuals_merge.py
  • benchmarks/600.workflows/6100.1000-genome/python/mutation_overlap.py
  • benchmarks/600.workflows/6100.1000-genome/python/requirements.txt
  • benchmarks/600.workflows/6100.1000-genome/python/sifting.py
  • benchmarks/600.workflows/6101.1000-genome-individuals/config.json
  • benchmarks/600.workflows/6101.1000-genome-individuals/definition.json
  • benchmarks/600.workflows/6101.1000-genome-individuals/input.py
  • benchmarks/600.workflows/6101.1000-genome-individuals/python/individuals.py
  • benchmarks/600.workflows/6101.1000-genome-individuals/python/requirements.txt
  • benchmarks/600.workflows/620.func-invo/config.json
  • benchmarks/600.workflows/620.func-invo/definition.json
  • benchmarks/600.workflows/620.func-invo/input.py
  • benchmarks/600.workflows/620.func-invo/python/gen.py
  • benchmarks/600.workflows/620.func-invo/python/process.py
  • benchmarks/600.workflows/6200.trip-booking/config.json
  • benchmarks/600.workflows/6200.trip-booking/definition.json
  • benchmarks/600.workflows/6200.trip-booking/input.py
  • benchmarks/600.workflows/6200.trip-booking/python/cancel_flight.py
  • benchmarks/600.workflows/6200.trip-booking/python/cancel_hotel.py
  • benchmarks/600.workflows/6200.trip-booking/python/cancel_rental.py
  • benchmarks/600.workflows/6200.trip-booking/python/confirm.py
  • benchmarks/600.workflows/6200.trip-booking/python/reserve_flight.py
  • benchmarks/600.workflows/6200.trip-booking/python/reserve_hotel.py
  • benchmarks/600.workflows/6200.trip-booking/python/reserve_rental.py
  • benchmarks/600.workflows/630.parallel-sleep/config.json
  • benchmarks/600.workflows/630.parallel-sleep/definition.json
  • benchmarks/600.workflows/630.parallel-sleep/input.py
  • benchmarks/600.workflows/630.parallel-sleep/python/generate.py
  • benchmarks/600.workflows/630.parallel-sleep/python/process.py
  • benchmarks/600.workflows/631.parallel-download/config.json
  • benchmarks/600.workflows/631.parallel-download/definition.json
  • benchmarks/600.workflows/631.parallel-download/input.py
  • benchmarks/600.workflows/631.parallel-download/python/generate.py
  • benchmarks/600.workflows/631.parallel-download/python/process.py
  • benchmarks/600.workflows/640.selfish-detour/config.json
  • benchmarks/600.workflows/640.selfish-detour/definition.json
  • benchmarks/600.workflows/640.selfish-detour/input.py
  • benchmarks/600.workflows/640.selfish-detour/python/measure.py
  • benchmarks/600.workflows/640.selfish-detour/python/package.sh
  • benchmarks/600.workflows/640.selfish-detour/python/selfish-detour.c
  • benchmarks/600.workflows/650.vid/LICENSE
  • benchmarks/600.workflows/650.vid/README.md
  • benchmarks/600.workflows/650.vid/config.json
  • benchmarks/600.workflows/650.vid/definition.json
  • benchmarks/600.workflows/650.vid/input.py
  • benchmarks/600.workflows/650.vid/python/analyse.py
  • benchmarks/600.workflows/650.vid/python/decode.py
  • benchmarks/600.workflows/650.vid/python/requirements.txt
  • benchmarks/600.workflows/650.vid/python/summarize.py
  • benchmarks/600.workflows/660.map-reduce/config.json
  • benchmarks/600.workflows/660.map-reduce/definition.json
  • benchmarks/600.workflows/660.map-reduce/input.py
  • benchmarks/600.workflows/660.map-reduce/python/map.py
  • benchmarks/600.workflows/660.map-reduce/python/reduce.py
  • benchmarks/600.workflows/660.map-reduce/python/shuffle.py
  • benchmarks/600.workflows/660.map-reduce/python/split.py
  • benchmarks/600.workflows/670.auth/config.json
  • benchmarks/600.workflows/670.auth/definition.json
  • benchmarks/600.workflows/670.auth/input.py
  • benchmarks/600.workflows/670.auth/python/auth.py
  • benchmarks/600.workflows/670.auth/python/requirements.txt
  • benchmarks/600.workflows/680.excamera/LICENSE
  • benchmarks/600.workflows/680.excamera/README.md
  • benchmarks/600.workflows/680.excamera/config.json
  • benchmarks/600.workflows/680.excamera/definition.json
  • benchmarks/600.workflows/680.excamera/input.py
  • benchmarks/600.workflows/680.excamera/python/encode.py
  • benchmarks/600.workflows/680.excamera/python/rebase.py
  • benchmarks/600.workflows/680.excamera/python/reencode.py
  • benchmarks/600.workflows/680.excamera/python/requirements.txt
  • benchmarks/600.workflows/680.excamera/python/split.py
  • benchmarks/600.workflows/690.ml/config.json
  • benchmarks/600.workflows/690.ml/definition.json
  • benchmarks/600.workflows/690.ml/input.py
  • benchmarks/600.workflows/690.ml/python/generate.py
  • benchmarks/600.workflows/690.ml/python/package.sh
  • benchmarks/600.workflows/690.ml/python/requirements.txt
  • benchmarks/600.workflows/690.ml/python/train.py
  • benchmarks/wrappers/aws/python/handler_workflow.py
  • benchmarks/wrappers/aws/python/storage.py
  • benchmarks/wrappers/azure/python/fsm.py
  • benchmarks/wrappers/azure/python/handler_workflow.py
  • benchmarks/wrappers/azure/python/main_workflow.py
  • benchmarks/wrappers/azure/python/run_subworkflow.py
  • benchmarks/wrappers/azure/python/run_workflow.py
  • benchmarks/wrappers/azure/python/storage.py
  • benchmarks/wrappers/cloudflare/nodejs/build.js
  • benchmarks/wrappers/cloudflare/nodejs/container/handler.js
  • benchmarks/wrappers/cloudflare/nodejs/container/nosql.js
  • benchmarks/wrappers/cloudflare/nodejs/container/package.json
  • benchmarks/wrappers/cloudflare/nodejs/container/storage.js
  • benchmarks/wrappers/cloudflare/nodejs/container/worker.js
  • benchmarks/wrappers/cloudflare/nodejs/handler.js
  • benchmarks/wrappers/cloudflare/nodejs/nosql.js
  • benchmarks/wrappers/cloudflare/nodejs/postprocess.js
  • benchmarks/wrappers/cloudflare/nodejs/request-polyfill.js
  • benchmarks/wrappers/cloudflare/nodejs/storage.js
  • benchmarks/wrappers/cloudflare/python/container/handler.py
  • benchmarks/wrappers/cloudflare/python/container/nosql.py
  • benchmarks/wrappers/cloudflare/python/container/storage.py
  • benchmarks/wrappers/cloudflare/python/handler.py
  • benchmarks/wrappers/cloudflare/python/nosql.py
  • benchmarks/wrappers/cloudflare/python/storage.py
  • benchmarks/wrappers/gcp/python/handler_workflow.py
  • benchmarks/wrappers/gcp/python/storage.py
  • benchmarks/wrappers/local/python/storage.py
  • benchmarks/wrappers/openwhisk/python/storage.py
  • configs/example.json
  • configs/python.json
  • configs/systems.json
  • dockerfiles/cloudflare/Dockerfile.manage
  • dockerfiles/cloudflare/nodejs/Dockerfile.build
  • dockerfiles/cloudflare/nodejs/Dockerfile.function
  • dockerfiles/cloudflare/python/Dockerfile.build
  • dockerfiles/cloudflare/python/Dockerfile.function
  • dockerfiles/cloudflare_nodejs_installer.sh
  • dockerfiles/cloudflare_python_installer.sh
  • docs/build.md
  • docs/platforms.md
  • docs/storage.md
  • docs/workflows.md
  • pyproject.toml
  • requirements.aws.txt
  • requirements.azure.txt
  • requirements.gcp.txt
  • requirements.txt
  • sebs/aws/aws.py
  • sebs/aws/config.py
  • sebs/aws/generator.py
  • sebs/aws/triggers.py
  • sebs/aws/workflow.py
  • sebs/azure/azure.py
  • sebs/azure/config.py
  • sebs/azure/function.py
  • sebs/azure/triggers.py
  • sebs/benchmark.py
  • sebs/cli.py
  • sebs/cloudflare/__init__.py
  • sebs/cloudflare/cli.py
  • sebs/cloudflare/cloudflare.py
  • sebs/cloudflare/config.py
  • sebs/cloudflare/containers.py
  • sebs/cloudflare/function.py
  • sebs/cloudflare/generator.py
  • sebs/cloudflare/kvstore.py
  • sebs/cloudflare/pyodide_packages.py
  • sebs/cloudflare/r2.py
  • sebs/cloudflare/resources.py
  • sebs/cloudflare/templates/wrangler-container.toml
  • sebs/cloudflare/templates/wrangler-worker.toml
  • sebs/cloudflare/templates/wrangler-workflow.toml
  • sebs/cloudflare/triggers.py
  • sebs/cloudflare/workers.py
  • sebs/cloudflare/workflow.py
  • sebs/config.py
  • sebs/experiments/config.py
  • sebs/faas/config.py
  • sebs/faas/fsm.py
  • sebs/faas/function.py
  • sebs/faas/system.py
  • sebs/gcp/config.py
  • sebs/gcp/gcp.py
  • sebs/gcp/generator.py
  • sebs/gcp/triggers.py
  • sebs/gcp/workflow.py
  • sebs/regression.py
  • sebs/sebs.py
  • sebs/utils.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@userlaurin
userlaurin marked this pull request as ready for review June 21, 2026 16:40
=Laurin added 13 commits July 13, 2026 18:43
Use the documented virtual-host outbound handler for container R2/KV access, remove legacy proxy and multipart paths, and add the storage latency benchmark and analysis tooling.
# Conflicts:
#	benchmarks/wrappers/cloudflare/nodejs/container/nosql.js
#	benchmarks/wrappers/cloudflare/nodejs/container/storage.js
#	benchmarks/wrappers/cloudflare/python/container/nosql.py
#	benchmarks/wrappers/cloudflare/python/container/storage.py
Use the documented virtual-host outbound handler for container R2/KV access, remove legacy proxy and multipart paths, and add the storage latency benchmark and analysis tooling.
# Conflicts:
#	benchmarks/wrappers/cloudflare/python/container/storage.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants