Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions src/rtichoke/probs_distribution.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ def create_probs_histogram(
"""Create a standalone Prediction Distribution browser chart.

This function generates a canonical prediction distribution chart displaying
predicted probability histograms or risk percentile rank distributions,
predicted probability histograms or prediction percentile rank distributions,
decomposed by observed binary outcome, with interactive threshold and PPCR
operating-point controls.

In Probability Threshold mode, the x-axis represents predicted probability
intervals. In PPCR (Predicted Positives Condition Rate) mode, the x-axis
displays the producer-owned risk-rank distribution (percentile strata).
displays the producer-owned rank distribution (percentile strata).
Bars in each bin are decomposed by observed binary outcome (positives and
negatives). Moving the operating-point control changes the classification
partition and updates performance metrics attached to each operating point.
Expand Down Expand Up @@ -65,7 +65,7 @@ def create_probs_histogram(
>>> reals = np.array([0, 0, 1, 1])
>>> chart = rtichoke.create_probs_histogram(probs=probs, reals=reals)

Create a prediction distribution chart in PPCR / Risk Percentile mode:
Create a prediction distribution chart in PPCR mode:

>>> chart = rtichoke.create_probs_histogram(
... probs=probs,
Expand Down
23 changes: 19 additions & 4 deletions src/rtichoke/summary_report/summary_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,20 @@ def create_summary_report_times(
) -> Path:
"""Create a canonical browser time-dependent model-performance summary report.

The generated time-dependent browser report contains five top-level sections:

1. Event Probability
2. Calibration
3. Discrimination
4. Utility
5. Performance Table

Compared with the static browser Summary Report, it intentionally omits:

* Prevalence section (replaced by Event Probability over time);
* Prediction Distribution section;
* AUROC summary metric.

Parameters
----------
probs : Dict[str, np.ndarray]
Expand All @@ -92,7 +106,8 @@ def create_summary_report_times(
List of heuristic configurations for censoring and competing events.
Defaults to ``[{"censoring_heuristic": "adjusted", "competing_heuristic": "adjusted_as_negative"}]``.
by : float, optional
Step size for probability thresholds / PPCR. Defaults to 0.01.
Step size for probability thresholds / Predicted Positives Condition Rate (PPCR).
Defaults to 0.01.
output_file : str or pathlib.Path, optional
HTML destination file path. Defaults to ``"summary_report_times.html"``.

Expand Down Expand Up @@ -332,9 +347,9 @@ def create_summary_report(
is an explicit opt-in path that uses Python's existing production
calculations, canonical standalone component builders, canonical ReportSpec
assembly, and the vendored ``rtichoke_viz`` ``renderReport()`` composer.
For ``renderer="browser"``, the static Discrimination section includes
Prediction Distribution by Probability Threshold and Prediction Distribution
by PPCR / Risk Percentile.
For ``renderer="browser"``, the static browser Summary Report contains six
top-level sections: Prevalence, Prediction Distribution, Calibration,
Discrimination, Utility, and Performance Table.

Parameters
----------
Expand Down
72 changes: 65 additions & 7 deletions user_guide/06-summary-reports.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,72 @@ create_summary_report(
)
```

The browser path uses the same production calculations as the standalone Python components, converts those results with the existing canonical component builders, assembles a canonical ReportSpec, and delegates report composition to the vendored immutable `rtichoke_viz v0.5.0` `renderReport()` implementation.
The browser path uses the same production calculations as the standalone Python components, converts those results with existing canonical component builders, assembles a canonical ReportSpec, and delegates report composition to the vendored immutable `rtichoke_viz` renderer.

The first public browser report contains, in deterministic order:
The Summary Report browser backend writes a single self-contained, offline-ready HTML artifact with all required shared renderer JavaScript and CSS embedded directly into the file. The browser backend returns the generated HTML `pathlib.Path`; the default historical R backend retains its existing `None` return behavior.

1. PerformanceTable;
2. ROC-v2;
3. calibration-v2.
## Static browser Summary Report

The generated HTML is accompanied by `rtichoke-viz.js` and `rtichoke-viz.css` in the same directory, so those three files should be kept together when moving the report. The browser backend returns the generated HTML `pathlib.Path`; the default historical R backend retains its existing `None` return behavior.
The static browser Summary Report contains six top-level sections in the following exact order:

The browser backend does not replace Quarto or the historical R backend, and it is not the default. Existing Plotly, Matplotlib, table, standalone browser-chart, and time-dependent APIs are unchanged by this opt-in report path.
1. **Prevalence**: Displays population prevalence summary metrics.
2. **Prediction Distribution**: Displays decomposed predicted probability histograms with interactive operating-point controls, offered in two views/groups:
* **By Probability Threshold**
* **By Predicted Positives Condition Rate (PPCR)**
3. **Calibration**: Displays model calibration curves in two views:
* **Smooth**
* **Discrete**
4. **Discrimination**: Displays summary metrics (AUROC) and curve visualizations across both operating-point dimensions (**By Probability Threshold** and **By Predicted Positives Condition Rate (PPCR)**) in exact component order:
* **ROC**
* **Lift**
* **Precision-Recall**
* **Gains**
5. **Utility**: Displays Decision Curve and Interventions Avoided curves.
6. **Performance Table**: Displays full performance metrics across operating points, grouped **By Probability Threshold** and **By Predicted Positives Condition Rate (PPCR)**.

## Time-dependent Summary Report

For survival and time-to-event outcomes, `create_summary_report_times()` generates a canonical time-dependent browser summary report at fixed time horizons:

```python
import numpy as np
from rtichoke import create_summary_report_times

probs = {
"Model A": np.array(
[0.03, 0.08, 0.12, 0.18, 0.25, 0.32, 0.40, 0.50, 0.62, 0.75, 0.88, 0.96]
)
}
reals = np.array([0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1])
times = np.array([10, 12, 15, 8, 20, 25, 30, 18, 22, 14, 19, 28])

create_summary_report_times(
probs,
reals,
times,
fixed_time_horizons=[15.0],
output_file="summary_report_times.html",
)
```

The time-dependent browser report contains five top-level sections in exact order:

1. **Event Probability**: Displays population event probabilities at specified fixed time horizons (derived using time-dependent estimators accounting for censoring and competing events).
2. **Calibration**: Displays smooth and discrete time-dependent calibration curves.
3. **Discrimination**: Displays time-dependent curve visualizations grouped **By Probability Threshold** and **By Predicted Positives Condition Rate (PPCR)** in exact component order:
* **ROC**
* **Lift**
* **Precision-Recall**
* **Gains**
4. **Utility**: Displays time-dependent Decision Curve and Interventions Avoided curves.
5. **Performance Table**: Displays time-dependent performance tables grouped **By Probability Threshold** and **By Predicted Positives Condition Rate (PPCR)**.

### Architectural differences from static reports

The time-dependent summary report intentionally differs from the binary static report in three key ways:

* **No Prevalence section**: Replaced by the time-horizon **Event Probability** section to account for censoring and competing events over time.
* **No Prediction Distribution section**: Prediction distribution histograms are omitted in time-dependent reports.
* **No AUROC summary metric**: AUROC summary metric cards are omitted in time-dependent reports.

The browser backend does not replace Quarto or the historical R backend. `renderer="browser"` is an explicit opt-in parameter for `create_summary_report()`, whereas `create_summary_report_times()` generates a browser summary report directly without requiring a `renderer` argument. Existing Plotly, Matplotlib, table, and standalone browser-chart APIs remain unchanged.
Loading