Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/architecture/adapter-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ The canonical recipe:

5. **Runtime wiring is a single `install_setup_hook` call** at app
startup (see [Runtime wiring](#runtime-wiring) below). The hook
captures whatever pre-start state the adapter needs, allocates +
registers host surfaces, and (for escalate-trigger adapters)
sets the bridge on `GpuContext`.
captures whatever pre-start state the adapter needs and
allocates + registers host surfaces. Nothing is installed on
`GpuContext` — every escalate op it answers is always present.

That's the full shape. Every in-tree adapter follows it, with the
only meaningful axis of variation being the **handle type** (DMA-BUF
Expand Down
24 changes: 13 additions & 11 deletions docs/architecture/adapter-runtime-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,8 @@ Every surface adapter's host-side wiring runs through
[`Runner::install_setup_hook`][hook]. The hook fires exactly
once per `start()`, after `GpuContext::init_for_platform_sync` has
created the live `GpuContext` but before any processor's `setup()`
runs — the window where adapter bridges and pre-allocated host
surfaces have to be in place.
runs — the window where pre-allocated host surfaces have to be in
place.

[hook]: ../../runtime/streamlib-engine/src/core/runtime/runtime.rs

Expand Down Expand Up @@ -309,20 +309,22 @@ The shape of what the hook does varies by seam:
adapter — and that submit signals `produce_done`. The cdylib
signals `consume_done` from `end_read_access`.

The graphics / ray-tracing kernel bridges still follow the older shape
(`gpu.set_graphics_kernel_bridge`, `set_ray_tracing_kernel_bridge`) for
adapters that escalate kernel dispatch through the host RHI. Compute and CPU
readback have no bridge: `register_compute_kernel` / `run_compute_kernel`,
`run_cpu_readback_copy` / `try_run_cpu_readback_copy` and
`open_cpu_readback_staging` are always-present `GpuContext` capabilities
served by the escalate handler directly.
No kernel dispatch flavour uses a bridge. Compute, graphics, ray tracing
and CPU readback are always-present `GpuContext` capabilities served by the
escalate handler directly, with no installation step and no runtime-absent
case: `create_or_reuse_compute_kernel` / `compute_kernel_by_id`,
`create_or_reuse_graphics_kernel` / `graphics_kernel_by_id`,
`create_or_reuse_ray_tracing_kernel` / `ray_tracing_kernel_by_id`,
`register_acceleration_structure` / `acceleration_structure_by_id`, and
`run_cpu_readback_copy` / `try_run_cpu_readback_copy` /
`open_cpu_readback_staging`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

The hook is the canonical opt-in registration point for adapters that
need pre-start GpuContext access. Application authors call
`install_setup_hook` exactly once per adapter they wire.

Per-acquire host work on behalf of a subprocess is not an adapter
concern. Compute dispatch and CPU readback are `GpuContext`
concern. Kernel dispatch and CPU readback are `GpuContext`
capabilities reached the same way by every caller, with no
installation step and no runtime-absent case.

Expand Down Expand Up @@ -430,7 +432,7 @@ With `install_setup_hook` the model is:
1. Add the adapter crate as a Cargo dep.
2. Call `runtime.install_setup_hook(...)` exactly once at app
startup, doing the adapter's required pre-start work (allocate
host surfaces, register in surface-share, set bridge if needed).
host surfaces, register in surface-share).

The cost: one extra line of wiring per adapter at the application's
`main.rs`. Compile-time presence is not enough — you have to
Expand Down
32 changes: 30 additions & 2 deletions docs/architecture/graphics-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ declaration, the kernel:
per-stage descriptor sets, and validates that:
- Declared `bindings` match the merged shader declaration (kind +
stage visibility).
- A spec that carries a `name` spells the slot the way the SPIR-V
does.
- No two stages name the same slot differently — bindings are
resolved by name, so one slot spelled two ways cannot be bound.
- Push-constant size and stage visibility match.
- Only descriptor set 0 is used (multi-set is out of scope).
- Stage classification matches the SPIR-V (a Vertex stage's blob
Expand All @@ -31,6 +35,13 @@ declaration, the kernel:
Mismatches surface as a `Result::Err` at *kernel creation*, not as
undefined GPU behavior at first draw.

Reflection also **adopts the shader's own name onto each spec**:
wherever the SPIR-V carries an `OpName` for a slot, the spec the
kernel stores and `bindings()` returns comes back with that name,
whether or not the caller supplied one. The numeric binding is what
the descriptor set is built from; the name is what a by-name draw
resolves against.

2. **Builds the descriptor-set layout, descriptor pool, descriptor-set
ring, pipeline layout, graphics pipeline (with on-disk pipeline
cache), and a default linear-clamp sampler.** None of this is your
Expand Down Expand Up @@ -105,8 +116,10 @@ serial-dispatch contract).

2. **Wire the shaders into `build.rs`.** Append vertex + fragment
entries to the `shaders` array in `runtime/streamlib-engine/build.rs`. The
build script invokes `glslc -O` per-stage and writes SPIR-V into
`OUT_DIR`. SPIR-V is read at compile time via
build script invokes `glslc -g -O` per-stage and writes SPIR-V into
`OUT_DIR`. The `-g` is not optional: `-O` strips every `OpName`, and
a binding whose name is gone cannot be bound by name. SPIR-V is read at
compile time via
`include_bytes!(concat!(env!("OUT_DIR"), "/<name>.<stage>.spv"))`.
Do not commit `.spv` files to the source tree — they're build
artifacts.
Expand All @@ -119,6 +132,13 @@ serial-dispatch contract).
GraphicsBindingSpec::sampled_texture(0, GraphicsShaderStageFlags::FRAGMENT),
];

// Or assert the shader's spelling too — creation fails if the SPIR-V
// names slot 0 anything else:
let bindings = vec![
GraphicsBindingSpec::sampled_texture(0, GraphicsShaderStageFlags::FRAGMENT)
.with_name("input_texture"),
];

let pipeline_state = GraphicsPipelineState {
topology: PrimitiveTopology::TriangleList,
vertex_input: VertexInputState::None, // gl_VertexIndex
Expand All @@ -133,6 +153,14 @@ serial-dispatch contract).
};
```

A `GraphicsBindingSpec` is `(binding, kind, stages, name)` — the
`sampled_texture` / `storage_buffer` / `uniform_buffer` /
`storage_image` constructors leave `name` as `None`, and `with_name`
fills it. A caller that knows the shader's name but not its slot
declares a `GraphicsBindingDeclaration` (name + kind + stages, no
slot) instead; it is reconciled against reflection by name, stage by
stage, and the slot comes back from the shader.

4. **Create the kernel via `GpuContext::create_graphics_kernel`** at
setup time and store the `Arc<VulkanGraphicsKernel>`:

Expand Down
37 changes: 33 additions & 4 deletions docs/architecture/ray-tracing-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,12 @@ binding declaration, the kernel:
[`rspirv-reflect`](https://docs.rs/rspirv-reflect), merges the
per-stage descriptor sets, and validates that:
- Declared `bindings` match the merged shader declaration (kind +
stage visibility, including the new
stage visibility, including the
`RayTracingBindingKind::AccelerationStructure` variant).
- A spec that carries a `name` spells the slot the way the SPIR-V
does, and no two stages name the same slot differently —
bindings are resolved by name, so one slot spelled two ways
cannot be bound.
- Push-constant size matches the largest declared push-constant
range across all stages.
- Only descriptor set 0 is used (multi-set is out of scope).
Expand All @@ -46,6 +50,13 @@ binding declaration, the kernel:
Mismatches surface as `Result::Err` at *kernel creation*, not as
undefined GPU behavior at first trace.

Reflection also **adopts the shader's own name onto each spec**:
wherever the SPIR-V carries an `OpName` for a slot, the spec the
kernel stores and `bindings()` returns comes back with that name,
whether or not the caller supplied one. The numeric binding is what
the descriptor set is built from; the name is what a by-name trace
resolves against.

2. **Builds the descriptor-set layout, descriptor pool, descriptor
set, pipeline layout, ray-tracing pipeline, and shader-binding
table.** None of this is your code anymore. The SBT is laid out
Expand Down Expand Up @@ -82,9 +93,11 @@ binding declaration, the kernel:
2. **Wire the shaders into `build.rs`.** Add an entry per stage to
the `rt_shaders` array in `runtime/streamlib-engine/build.rs`. RT shaders
are compiled with `--target-env=vulkan1.2 --target-spv=spv1.4`
so `SPV_KHR_ray_tracing` opcodes are available; the helper
handles the per-stage `-fshader-stage=rgen|rmiss|rchit|...`
flag. SPIR-V is read via
so `SPV_KHR_ray_tracing` opcodes are available, and with `-g -O`;
the helper handles the per-stage
`-fshader-stage=rgen|rmiss|rchit|...` flag. The `-g` is not
optional: `-O` strips every `OpName`, and a binding whose name is
gone cannot be bound by name. SPIR-V is read via
`include_bytes!(concat!(env!("OUT_DIR"), "/<name>.spv"))`.

3. **Declare stages, groups, bindings, and push constants as data.**
Expand All @@ -108,8 +121,24 @@ binding declaration, the kernel:
RayTracingBindingSpec::acceleration_structure(0, RayTracingShaderStageFlags::RAYGEN),
RayTracingBindingSpec::storage_image(1, RayTracingShaderStageFlags::RAYGEN),
];

// Or assert the shader's spelling too — creation fails if the SPIR-V
// names slot 0 anything else:
let bindings = vec![
RayTracingBindingSpec::acceleration_structure(0, RayTracingShaderStageFlags::RAYGEN)
.with_name("top_level_acceleration_structure"),
];
```

A `RayTracingBindingSpec` is `(binding, kind, stages, name)` — the
`acceleration_structure` / `storage_image` / `storage_buffer` /
`uniform_buffer` / `sampled_texture` constructors leave `name` as
`None`, and `with_name` fills it. A caller that knows the shader's
name but not its slot declares a `RayTracingBindingDeclaration`
(name + kind + stages, no slot) instead; it is reconciled against
reflection by name, stage by stage, and the slot comes back from the
shader.

4. **Build the acceleration structures.** Triangle BLASes come from
`GpuContext::build_triangles_blas(label, vertices, indices)` —
vertices are interleaved `[x, y, z, x, y, z, …]` (R32G32B32_SFLOAT,
Expand Down
2 changes: 1 addition & 1 deletion docs/plan/changes/python-kernel-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Bare patterns — the ship gate greps each line verbatim as a fixed string.
- REMOVED: importing a foreign DMA-BUF is not reachable from a Python processor yet
The refusal (`python_processor_context.rs:745-759`) and its stub entry (`_engine.pyi:381`).
- REMOVED: writable: false
The `writable: false` texture arm (`surface_export_staging.rs:373`), which is the anchor this
The `writable: false` texture arm (`surface_export_staging.rs:394`), which is the anchor this
bullet was re-cut to when #1774 implemented the readback half — the shared literal it named
before ("device export is read-only") no longer distinguishes the two halves, because #1774
reworded that refusal to name the surface. Only the texture arm retires, with the
Expand Down
10 changes: 10 additions & 0 deletions packages/test-fixtures/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ fn main() {
}
}

/// `glslc -O` strips every `OpName`, and a binding with no reflected name
/// cannot be dispatched against by name. The engine's own `build.rs` applies
/// this uniformly for that reason; a fixture compiled without it would be the
/// one blob in the tree whose bindings cannot be bound.
#[cfg(target_os = "linux")]
const KEEP_BINDING_NAMES: &str = "-g";

#[cfg(target_os = "linux")]
fn compile_cpu_ref_doubler() {
use std::path::{Path, PathBuf};
Expand All @@ -23,6 +30,7 @@ fn compile_cpu_ref_doubler() {
let dst: PathBuf = Path::new(&out_dir).join("cpu_ref_doubler.spv");
let status = Command::new("glslc")
.arg("-fshader-stage=compute")
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down Expand Up @@ -57,6 +65,7 @@ fn compile_graphics_kernel_smoke() {
let dst: PathBuf = Path::new(&out_dir).join(dst_name);
let status = Command::new("glslc")
.arg(stage_arg)
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down Expand Up @@ -104,6 +113,7 @@ fn compile_ray_tracing_kernel_smoke() {
.arg(stage_arg)
.arg("--target-env=vulkan1.2")
.arg("--target-spv=spv1.4")
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down
Loading
Loading