This document describes the technical architecture of Rerun.
BUILD.mdTESTING.mdrerun_py/README.md- build instructions for Python SDKCODE_OF_CONDUCT.mdCODE_STYLE.mdCONTRIBUTING.mdRELEASES.md
It all starts with logging. You can log rich data (point clouds, images, etc.) with either our Python SDK or our Rust SDK.
The logging SDK:s encodes the data using Apache Arrow (see more below).
The logging data can be written to disk as .rrd files, or transmitted over gRPC to either a Rerun Viewer or a Rerun Server.
The Rerun Viewer is where log data is visualized. It is usually run as a native app, but can also be compiled to WebAssembly (Wasm) and run in a browser.
The easiest way to launch the Viewer is directly from the logging API with rr.init("rerun_example_app", spawn=True). However, the standalone Viewer can also be run from the command line, for example to view an .rrd file: rerun mydata.rrd.
You can try running the Viewer in a browser using rr.serve() in Python, or using rerun --web-viewer mydata.rrd.
The web viewer consists of just a few small files - a thin .html, a .wasm blob, and an auto-generated .js bridge for the wasm. These files are served using the re_web_viewer_server crate.
The web viewer can load .rrd files (just drag-drop them into the browser), or read logging data streamed over gRPC.
.rrd ("Rerun Data") is just a bunch of log messages appended one after the other to a file.
NOTE: .rrd files do not yet offer full backwards or forwards compatibility. However, the current version of Rerun will always be able to open .rrd files generated by the previous version. This is not guaranteed to work for files generated with older versions.
Apache Arrow is a language-independent columnar memory format for arbitrary data. We use it to encode the log data when transmitting it over the network or storing it in an .rrd file. We also use it in our in-RAM data store, re_chunk_store.
In Rust, we use the arrow crate.
The Rerun Viewer uses the wgpu graphics API. It provides a high-performance abstraction over Vulkan, Metal, D3D12, D3D11, OpenGLES, WebGL and WebGPU. This lets us write the same code graphics code for native as for web.
On web builds, we use WebGPU when available on the Web, but automatically fall back to a WebGL based emulation layer (with a more limited feature set).
We have written our own high-level rendering crate on top of wgpu, called re_renderer.
The GUI in the Rerun Viewer is using egui, a cross-platform, immediate mode GUI.
We use eframe, the egui framework, to run egui on both native and web.
Wasm (short for WebAssembly) is a binary instruction format supported by all major browser. The Rerun Viewer can be compiled to Wasm and run in a browser.
Threading support in Wasm is nascent, so care must be taken that we don't spawn any threads when compiling for wasm32.
Wasm has no access to the host system, except via JS calls (something that may change once WASI rolls out), so when compiling for wasm32 you can NOT use the Rust standard library to:
- Access files
- Read environment variables
- Get the current time (use
instantinstead) - Use networking (use
ehttp,reqwest, orewebsockinstead) - etc
The Rerun Viewer uses an immediate mode GUI, egui. This means that each frame the entire GUI is being laid out from scratch.
In fact, the whole of the Rerun Viewer is written in an immediate mode style. Each rendered frame it will query the in-RAM data store, massage the results, and feed it to the renderer.
The advantage of immediate mode is that is removes all state management. There is no callbacks that are called when some state has already changed, and the state of the blueprint is always in sync with what you see on screen.
Immediate mode is also a forcing function, forcing us to relentlessly optimize our code. This leads to a very responsive GUI, where there is no "hiccups" when switching data source or doing time scrubbing.
Of course, this will only take us so far. In the future we plan on caching queries and work submitted to the renderer so that we don't perform unnecessary work each frame. We also plan on doing larger operation in background threads. This will be necessary in order to support viewing large datasets, e.g. several million points. The plan is still to do so within an immediate mode framework, retaining most of the advantages of stateless code.
Here is an overview of the crates included in the project, and how they depend on each other:
Each band is one folder under crates/, with rerun_py and rerun_c folded into the top band.
Every arrow points downwards, because the folders are layers: a crate may only depend on its own folder or one below it.
scripts/check_crate_layers.py enforces that in CI.
A dev-dependency is exempt, since a test may reach anywhere.
See crates/README.md for the layers as a list, and for which one a new crate belongs in.
The tables below give a one-line description of every crate, one table per folder.
Only the arrows inside a band are drawn.
The diagram and the crate tables below are both generated from cargo metadata, so neither has to be written by hand.
A crate's description is its description field in its own Cargo.toml, so there is only one place to write it.
Regenerate both after adding, removing, or renaming a crate:
pixi run crate-graphRun that command yourself whenever you add or remove a crate.
Nothing checks it for you: pixi run crate-graph-check is disabled in CI, because graphviz lays the same graph out slightly differently on macOS and on Linux, so whoever regenerated the diagram last decided whether the check passed.
Pass --edges all to scripts/generate_crate_graph.py for a version with every dependency drawn, cross-band ones included.
crates/tests — only tests depend on these, so they sit at the top of the diagram.
| Crate | Description |
|---|---|
| re_redap_tests | Official test suite for the Rerun Data Protocol |
| re_test_context | A common context used for tests. |
| re_test_mocks | In-process server doubles (MockOtlpCollector, MockPostHog) used by tests that need to capture outbound OTel/PostHog traffic. |
| re_test_viewport | A viewport used for tests |
| re_web_tests | Discovers and runs Rerun web tests |
crates/top — the entry points for our users, and the viewer app itself.
| Crate | Description |
|---|---|
| re_sdk | Rerun logging SDK |
| re_viewer | The Rerun viewer |
| re_viewer_mcp | MCP server that allows llm agents to use the Rerun Viewer. |
| re_web_viewer_server | Serves the Rerun web viewer (Wasm and HTML) over HTTP |
| rerun | The Rerun Rust SDK: log images, point clouds, etc, and visualize them effortlessly |
| rerun-cli | The Rerun command-line tool: the rerun binary, including the native Viewer |
| rerun_c | Rerun C SDK |
| rerun_py | The Rerun Python SDK |
crates/views — the visualizations a user can put in the viewport.
A sibling of crates/panels: neither depends on the other.
| Crate | Description |
|---|---|
| re_view_bar_chart | A view that shows a single bar chart. |
| re_view_dataframe | A view that shows the data contained in entities in a table. |
| re_view_graph | A view that shows a graph (node-link diagram). |
| re_view_map | A view that shows a map. |
| re_view_spatial | Views that show entities in a 2D or 3D spatial relationship. |
| re_view_state_timeline | A view that shows state transitions as horizontal lanes over time. |
| re_view_tensor | A view dedicated to visualizing tensors with arbitrary dimensionality. |
| re_view_text_document | A simple view that shows a single text box. |
| re_view_text_log | A view that shows text entries in a table and scrolls with the active time. |
| re_view_time_series | A view that shows plots over Rerun timelines. |
crates/panels — the panels the app is assembled from, and the widgets they are built out of.
A sibling of crates/views: neither depends on the other.
| Crate | Description |
|---|---|
| re_agent_ui | Chat UI for driving a coding agent (Claude Code, Codex, …) over the Agent Client Protocol |
| re_blueprint_tree | The UI for the blueprint tree in the left panel. |
| re_chunk_store_ui | Display the contents of chunk stores. |
| re_memory_view | Flamegraph visualization for memory usage trees |
| re_recording_panel | The UI for the recording panel. |
| re_redap_browser | The UI and communication to implement the in-viewer redap server browser. |
| re_selection_panel | The UI for the selection panel. |
| re_time_panel | The time panel of the Rerun Viewer, allowing to control the displayed timeline & time. |
| re_viewport | The central viewport panel of the Rerun viewer. |
crates/viewer_support — the UI and rendering machinery the views are built on.
| Crate | Description |
|---|---|
| re_arrow_ui | Show arrow data in a tree of rerun list_items and format arrow with syntax highlighting. |
| re_component_fallbacks | Handles registering various component fallbacks to FallbackProviderRegistry. |
| re_component_ui | Provides ui editors for Rerun component data for registration with the Rerun Viewer component ui registry. |
| re_context_menu | Support crate for context menu and actions. |
| re_data_ui | Provides ui elements for Rerun component data for the Rerun Viewer. |
| re_dataframe_ui | Rich table widget over DataFusion. |
| re_gamepad | Gamepad input handling for the Rerun viewer. |
| re_plot | Shared utilities for plot-based views. |
| re_renderer | A wgpu based renderer for all your visualization needs. |
| re_renderer_examples | Examples for the re_renderer crate. |
| re_time_ruler | Time ruler widget shared by the time panel and time-aware views. |
| re_ui | Rerun GUI theme and helpers, built around egui |
| re_view | Types & utilities for defining view classes and communicating with the viewport. |
| re_viewer_context | Rerun viewer state that is shared with the viewer's code components. |
| re_viewport_blueprint | The data model describing the layout of the viewport. |
crates/store_app — the queryable state a viewer or a server works with.
| Crate | Description |
|---|---|
| re_dataframe | High-level query APIs |
| re_datafusion | High-level query APIs |
| re_entity_db | In-memory storage of Rerun entities |
| re_query | High-level query APIs |
| re_server | A Rerun server implementation backed by an in-memory store |
| re_tf | Rerun spatial transform processing |
crates/data_flow — getting data in and out: clients, servers, and file importers.
| Crate | Description |
|---|---|
| re_data_source | Handles loading of Rerun data |
| re_grpc_client | Client for the legacy StoreHub API (re_grpc_server). |
| re_grpc_server | Server for the legacy StoreHub API |
| re_hdf5 | Core HDF5-to-chunk loading logic for Rerun |
| re_importer | Handles importing of Rerun data from file using importer plugins |
| re_lenses | Lenses are an API for extracting, transforming, and restructuring component data. |
| re_lerobot | Core LeRobot-dataset-to-chunk loading logic for Rerun |
| re_mcap | Convert MCAP into Rerun-compatible data |
| re_mp4_reader | Core mp4-to-chunk loading logic for Rerun |
| re_parquet | Core parquet-to-chunk loading logic for Rerun |
| re_redap_client | Official gRPC client for the Rerun Data Protocol |
crates/store — the data model, and the in-memory store that holds it.
| Crate | Description |
|---|---|
| re_chunk | A chunk of Rerun data, encoded using Arrow. Used for logging, transport, storage and compute. |
| re_chunk_optimizer | Analysis and memory-bounded optimization of Rerun chunk layouts |
| re_chunk_store | A storage engine for Rerun's Chunks |
| re_lenses_core | Core lens types and composable Arrow array transformations. |
| re_log_channel | An in-memory channel of Rerun data messages. |
| re_log_encoding | Encode/decode and serialize/deserialize RRD streams |
| re_log_types | The basic building blocks of the Rerun log format: entity paths, timelines, store ids, and log messages. |
| re_protos | Rerun remote gRPC/protobuf API types |
| re_sdk_types | The standard Rerun data types, component types, and archetypes. |
| re_sorbet | Rerun arrow metadata definitions |
| re_types_core | The core traits and types that power Rerun's data model. |
| re_uri | Parsing and constructing Rerun URIs |
crates/build — crates that run at build time.
| Crate | Description |
|---|---|
| re_build_tools | build.rs helpers for generating build info |
| re_dev_tools | Various tools for rerun development |
| re_protos_builder | Code generation for Rerun's Protobuf and gRPC definitions. |
| re_type_definitions | Rerun's type definitions: a subset of Rust that generates the Rust, Python and C++ SDKs |
| re_types_builder | Generates code for Rerun's SDKs from the type definitions. |
| re_types_builder_macros | The #[rerun_type] attribute macro used by Rerun's IDL definitions. |
| re_types_builder_prelude | The vocabulary that Rerun's IDL definitions are written against. |
crates/utils — small crates that depend on nothing outside crates/utils.
| Crate | Description |
|---|---|
| re_agent | Talk to a coding agent (Claude Code, Codex, …) over the Agent Client Protocol |
| re_analytics | Rerun's analytics SDK |
| re_arrow_util | Helpers for working with arrow. |
| re_async | Async runtime abstractions for native and WebAssembly targets. |
| re_auth | Authentication helpers for Rerun |
| re_backoff | Utilities for retrying operations with backoff. |
| re_build_info | Information about the build. Use together with re_build_tools |
| re_byte_size | Calculate the heap-allocated size of values at runtime. |
| re_byte_size_derive | Derive macro for the SizeBytes trait from re_byte_size. |
| re_capabilities | Capability tokens for the Rerun code base. |
| re_case | Case conversions, the way Rerun likes them |
| re_crash_handler | Detect panics and signals, logging them and optionally sending them to analytics. |
| re_error | Helpers for handling errors. |
| re_format | Miscellaneous tools to format and parse numbers, durations, etc. |
| re_grpc_headers | Rerun gRPC header conventions: well-known header names, the RerunVersionInterceptor, and the tower Layer machinery that propagates them across requests and responses. |
| re_int | Small numeric helper traits: saturating integer casts and unsigned absolute value |
| re_log | Helpers for setting up and doing text logging in the Rerun crates. |
| re_memory | Run-time memory tracking and profiling. |
| re_mutex | A wrapper around parking_lot::Mutex useful for debugging deadlocks. |
| re_perf_telemetry | In and out of process performance profiling utilities for Rerun & Redap |
| re_quota_channel | A channel that applies backpressure based on byte size. |
| re_ros_msg | Parse and decode ROS messages |
| re_rvl | Support for compressed depth data using the RVL format. |
| re_span | An integer range that always has a non-negative length |
| re_string_interner | Yet another string interning library |
| re_tracing | Helpers for tracing/spans/flamegraphs and such. |
| re_tuid | 128-bit Time-based Unique Identifier |
| re_video | Rerun video processing utilities. |
| re_web | Utilities for interacting with Web APIs |
In order to get a dependency graph for our in-house crates and their docs, we recommend you run:
cargo install cargo-depgraph
cargo depgraph --all-deps --workspace-only --all-features --dedup-transitive-deps | dot -Tpng > /tmp/rerun-deps.png
open /tmp/rerun-deps.png
and:
cargo doc --no-deps --open
and then browse through the re_* crates.