Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/architecture/adapter-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,9 @@ The canonical recipe:

5. **Runtime wiring is a single `install_setup_hook` call** at app
startup (see [Runtime wiring](#runtime-wiring) below). The hook
captures whatever pre-start state the adapter needs, allocates +
registers host surfaces, and (for escalate-trigger adapters)
sets the bridge on `GpuContext`.
captures whatever pre-start state the adapter needs and
allocates + registers host surfaces. Nothing is installed on
`GpuContext` — every escalate op it answers is always present.

That's the full shape. Every in-tree adapter follows it, with the
only meaningful axis of variation being the **handle type** (DMA-BUF
Expand Down
32 changes: 21 additions & 11 deletions docs/architecture/adapter-runtime-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,8 @@ Every surface adapter's host-side wiring runs through
[`Runner::install_setup_hook`][hook]. The hook fires exactly
once per `start()`, after `GpuContext::init_for_platform_sync` has
created the live `GpuContext` but before any processor's `setup()`
runs — the window where adapter bridges and pre-allocated host
surfaces have to be in place.
runs — the window where pre-allocated host surfaces have to be in
place.

[hook]: ../../runtime/streamlib-engine/src/core/runtime/runtime.rs

Expand Down Expand Up @@ -309,20 +309,30 @@ The shape of what the hook does varies by seam:
adapter — and that submit signals `produce_done`. The cdylib
signals `consume_done` from `end_read_access`.

The graphics / ray-tracing kernel bridges still follow the older shape
(`gpu.set_graphics_kernel_bridge`, `set_ray_tracing_kernel_bridge`) for
adapters that escalate kernel dispatch through the host RHI. Compute and CPU
readback have no bridge: `register_compute_kernel` / `run_compute_kernel`,
`run_cpu_readback_copy` / `try_run_cpu_readback_copy` and
`open_cpu_readback_staging` are always-present `GpuContext` capabilities
served by the escalate handler directly.
No kernel dispatch flavour uses a bridge. Compute, graphics, ray tracing
and CPU readback are `GpuContext` capabilities served by the escalate handler
directly, with no installation step and no runtime-absent case. The set is
Linux-only: the kernel and acceleration-structure methods are
`#[cfg(target_os = "linux")]`, and every op in it answers `… is only available
on Linux` off it. The methods are `create_or_reuse_compute_kernel` /
`compute_kernel_by_id`, `create_or_reuse_graphics_kernel` /
`graphics_kernel_by_id`, `create_or_reuse_ray_tracing_kernel` /
`ray_tracing_kernel_by_id` and `register_acceleration_structure` /
`acceleration_structure_by_id`. CPU readback has no `GpuContext` method of its
own name — `run_cpu_readback_copy`, `try_run_cpu_readback_copy` and
`open_cpu_readback_staging` are escalate ops, and they land on
`GpuContextLimitedAccess`'s `refill_surface_export_staging` /
`try_refill_surface_export_staging`,
`copy_surface_export_staging_back_to_surface` /
`try_copy_surface_export_staging_back_to_surface` and
`surface_export_staging` + `share_surface_export_staging`.

The hook is the canonical opt-in registration point for adapters that
need pre-start GpuContext access. Application authors call
`install_setup_hook` exactly once per adapter they wire.

Per-acquire host work on behalf of a subprocess is not an adapter
concern. Compute dispatch and CPU readback are `GpuContext`
concern. Kernel dispatch and CPU readback are `GpuContext`
capabilities reached the same way by every caller, with no
installation step and no runtime-absent case.

Expand Down Expand Up @@ -430,7 +440,7 @@ With `install_setup_hook` the model is:
1. Add the adapter crate as a Cargo dep.
2. Call `runtime.install_setup_hook(...)` exactly once at app
startup, doing the adapter's required pre-start work (allocate
host surfaces, register in surface-share, set bridge if needed).
host surfaces, register in surface-share).

The cost: one extra line of wiring per adapter at the application's
`main.rs`. Compile-time presence is not enough — you have to
Expand Down
32 changes: 30 additions & 2 deletions docs/architecture/graphics-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ declaration, the kernel:
per-stage descriptor sets, and validates that:
- Declared `bindings` match the merged shader declaration (kind +
stage visibility).
- A spec that carries a `name` spells the slot the way the SPIR-V
does.
- No two stages name the same slot differently — bindings are
resolved by name, so one slot spelled two ways cannot be bound.
- Push-constant size and stage visibility match.
- Only descriptor set 0 is used (multi-set is out of scope).
- Stage classification matches the SPIR-V (a Vertex stage's blob
Expand All @@ -31,6 +35,13 @@ declaration, the kernel:
Mismatches surface as a `Result::Err` at *kernel creation*, not as
undefined GPU behavior at first draw.

Reflection also **adopts the shader's own name onto each spec**:
wherever the SPIR-V carries an `OpName` for a slot, the spec the
kernel stores and `bindings()` returns comes back with that name,
whether or not the caller supplied one. The numeric binding is what
the descriptor set is built from; the name is what a by-name draw
resolves against.

2. **Builds the descriptor-set layout, descriptor pool, descriptor-set
ring, pipeline layout, graphics pipeline (with on-disk pipeline
cache), and a default linear-clamp sampler.** None of this is your
Expand Down Expand Up @@ -105,8 +116,10 @@ serial-dispatch contract).

2. **Wire the shaders into `build.rs`.** Append vertex + fragment
entries to the `shaders` array in `runtime/streamlib-engine/build.rs`. The
build script invokes `glslc -O` per-stage and writes SPIR-V into
`OUT_DIR`. SPIR-V is read at compile time via
build script invokes `glslc -g -O` per-stage and writes SPIR-V into
`OUT_DIR`. The `-g` is not optional: `-O` strips every `OpName`, and
a binding whose name is gone cannot be bound by name. SPIR-V is read at
compile time via
`include_bytes!(concat!(env!("OUT_DIR"), "/<name>.<stage>.spv"))`.
Do not commit `.spv` files to the source tree — they're build
artifacts.
Expand All @@ -119,6 +132,13 @@ serial-dispatch contract).
GraphicsBindingSpec::sampled_texture(0, GraphicsShaderStageFlags::FRAGMENT),
];

// Or assert the shader's spelling too — creation fails if the SPIR-V
// names slot 0 anything else:
let bindings = vec![
GraphicsBindingSpec::sampled_texture(0, GraphicsShaderStageFlags::FRAGMENT)
.with_name("input_texture"),
];

let pipeline_state = GraphicsPipelineState {
topology: PrimitiveTopology::TriangleList,
vertex_input: VertexInputState::None, // gl_VertexIndex
Expand All @@ -133,6 +153,14 @@ serial-dispatch contract).
};
```

A `GraphicsBindingSpec` is `(binding, kind, stages, name)` — the
`sampled_texture` / `storage_buffer` / `uniform_buffer` /
`storage_image` constructors leave `name` as `None`, and `with_name`
fills it. A caller that knows the shader's name but not its slot
declares a `GraphicsBindingDeclaration` (name + kind + stages, no
slot) instead; it is reconciled against reflection by name, stage by
stage, and the slot comes back from the shader.

4. **Create the kernel via `GpuContext::create_graphics_kernel`** at
setup time and store the `Arc<VulkanGraphicsKernel>`:

Expand Down
37 changes: 33 additions & 4 deletions docs/architecture/ray-tracing-kernel.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,12 @@ binding declaration, the kernel:
[`rspirv-reflect`](https://docs.rs/rspirv-reflect), merges the
per-stage descriptor sets, and validates that:
- Declared `bindings` match the merged shader declaration (kind +
stage visibility, including the new
stage visibility, including the
`RayTracingBindingKind::AccelerationStructure` variant).
- A spec that carries a `name` spells the slot the way the SPIR-V
does, and no two stages name the same slot differently —
bindings are resolved by name, so one slot spelled two ways
cannot be bound.
- Push-constant size matches the largest declared push-constant
range across all stages.
- Only descriptor set 0 is used (multi-set is out of scope).
Expand All @@ -46,6 +50,13 @@ binding declaration, the kernel:
Mismatches surface as `Result::Err` at *kernel creation*, not as
undefined GPU behavior at first trace.

Reflection also **adopts the shader's own name onto each spec**:
wherever the SPIR-V carries an `OpName` for a slot, the spec the
kernel stores and `bindings()` returns comes back with that name,
whether or not the caller supplied one. The numeric binding is what
the descriptor set is built from; the name is what a by-name trace
resolves against.

2. **Builds the descriptor-set layout, descriptor pool, descriptor
set, pipeline layout, ray-tracing pipeline, and shader-binding
table.** None of this is your code anymore. The SBT is laid out
Expand Down Expand Up @@ -82,9 +93,11 @@ binding declaration, the kernel:
2. **Wire the shaders into `build.rs`.** Add an entry per stage to
the `rt_shaders` array in `runtime/streamlib-engine/build.rs`. RT shaders
are compiled with `--target-env=vulkan1.2 --target-spv=spv1.4`
so `SPV_KHR_ray_tracing` opcodes are available; the helper
handles the per-stage `-fshader-stage=rgen|rmiss|rchit|...`
flag. SPIR-V is read via
so `SPV_KHR_ray_tracing` opcodes are available, and with `-g -O`;
the helper handles the per-stage
`-fshader-stage=rgen|rmiss|rchit|...` flag. The `-g` is not
optional: `-O` strips every `OpName`, and a binding whose name is
gone cannot be bound by name. SPIR-V is read via
`include_bytes!(concat!(env!("OUT_DIR"), "/<name>.spv"))`.

3. **Declare stages, groups, bindings, and push constants as data.**
Expand All @@ -108,8 +121,24 @@ binding declaration, the kernel:
RayTracingBindingSpec::acceleration_structure(0, RayTracingShaderStageFlags::RAYGEN),
RayTracingBindingSpec::storage_image(1, RayTracingShaderStageFlags::RAYGEN),
];

// Or assert the shader's spelling too — creation fails if the SPIR-V
// names slot 0 anything else:
let bindings = vec![
RayTracingBindingSpec::acceleration_structure(0, RayTracingShaderStageFlags::RAYGEN)
.with_name("top_level_acceleration_structure"),
];
```

A `RayTracingBindingSpec` is `(binding, kind, stages, name)` — the
`acceleration_structure` / `storage_image` / `storage_buffer` /
`uniform_buffer` / `sampled_texture` constructors leave `name` as
`None`, and `with_name` fills it. A caller that knows the shader's
name but not its slot declares a `RayTracingBindingDeclaration`
(name + kind + stages, no slot) instead; it is reconciled against
reflection by name, stage by stage, and the slot comes back from the
shader.

4. **Build the acceleration structures.** Triangle BLASes come from
`GpuContext::build_triangles_blas(label, vertices, indices)` —
vertices are interleaved `[x, y, z, x, y, z, …]` (R32G32B32_SFLOAT,
Expand Down
2 changes: 1 addition & 1 deletion docs/plan/changes/python-kernel-surface.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ Bare patterns — the ship gate greps each line verbatim as a fixed string.
- REMOVED: importing a foreign DMA-BUF is not reachable from a Python processor yet
The refusal (`python_processor_context.rs:745-759`) and its stub entry (`_engine.pyi:381`).
- REMOVED: writable: false
The `writable: false` texture arm (`surface_export_staging.rs:373`), which is the anchor this
The `writable: false` texture arm (`surface_export_staging.rs:394`), which is the anchor this
bullet was re-cut to when #1774 implemented the readback half — the shared literal it named
before ("device export is read-only") no longer distinguishes the two halves, because #1774
reworded that refusal to name the surface. Only the texture arm retires, with the
Expand Down
10 changes: 10 additions & 0 deletions packages/test-fixtures/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ fn main() {
}
}

/// `glslc -O` strips every `OpName`, and a binding with no reflected name
/// cannot be dispatched against by name. The engine's own `build.rs` applies
/// this uniformly for that reason; a fixture compiled without it would be the
/// one blob in the tree whose bindings cannot be bound.
#[cfg(target_os = "linux")]
const KEEP_BINDING_NAMES: &str = "-g";

#[cfg(target_os = "linux")]
fn compile_cpu_ref_doubler() {
use std::path::{Path, PathBuf};
Expand All @@ -23,6 +30,7 @@ fn compile_cpu_ref_doubler() {
let dst: PathBuf = Path::new(&out_dir).join("cpu_ref_doubler.spv");
let status = Command::new("glslc")
.arg("-fshader-stage=compute")
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down Expand Up @@ -57,6 +65,7 @@ fn compile_graphics_kernel_smoke() {
let dst: PathBuf = Path::new(&out_dir).join(dst_name);
let status = Command::new("glslc")
.arg(stage_arg)
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down Expand Up @@ -104,6 +113,7 @@ fn compile_ray_tracing_kernel_smoke() {
.arg(stage_arg)
.arg("--target-env=vulkan1.2")
.arg("--target-spv=spv1.4")
.arg(KEEP_BINDING_NAMES)
.arg("-O")
.arg(Path::new(src))
.arg("-o")
Expand Down
Loading
Loading