Discussing the Ophys QC Pipeline

[Dedalus9]

I am starting a dedicated discussion thread to discuss the quality control pipeline for the ophys data.

My GitHub repository can be found here: https://github.com/Dedalus9/openscope-ophys-qc.git. In it, I include notebooks that show how to load the data, what's included in the data, and some preliminary work for analyzing the impact of motion artifacts on data quality.

[Dedalus9]

Ophys QC Analysis.pptx

These were my takeaways from our meeting on 6-2-26:
Qualitative Comparisons

• Generate figures of a few ROIs that were heavily impacted by motion alongside examples with minimal motion impact for qualitative comparison
• Include example frames showing how somas and dendrites appear in field of view

Metric Descriptions and Concerns

• Every metric in the pipeline should include a plain-language description of what it measures and why it matters
• Include lateral (x/y) drift and axial (z) drift as explicit, reportable metrics
• Soma/Dendrite probability may be problematic metrics: the boundary between a small soma and a large dendrite process may not be clean.
• Investigate relationship between animal behavior and motion-induced ROI degradation.

New Figures to Add

• Histogram of the average number of frames any individual ROI is in frame across a session. This captures how often cells drift in and out of the focal plane.
• Distribution of SNR across all ROIs computed across a set of sessions.

Validation Metrics: Open Questions

• We need to establish what our SNR metric should be. For instance, candidate SNR metrics include representational quality and noise level of pyramidal neurons relative to background.
• We need to define and agree on SNR metrics.

[davisgrubin]

I’m planning to compare three ROI-level SNR/event-quality metrics:

1. CaImAn-style exceptional-event score (From my PI @farznaj 's postdoc)
   Basically estimate baseline/noise, then ask how unlikely the strongest run of consecutive positive samples would be under the
   noise model. This is useful for identifying ROIs with at least one statistically convincing event-like transient.
   Code here:
   https://github.com/farznaj/imaging_decisionMaking_exc_inh/blob/master/imaging/evaluate_components.py

2. 95th/50th percentile robust event SNR
   This uses SNR = (P95(dF/F) - P50(dF/F)) / robust residual noise SD, as implemented in utils_2p/roi_labels.py. This gives a more direct amplitude-to-noise measure of calcium-like dF/F fluctuations across the trace.
   https://github.com/najafi-laboratory/2p_imaging/blob/main/utils_2p/roi_labels.py#L122-L158

3. Allen-event distribution-fit metric
   https://github.com/davisgrubin/openscope-ophys-qc/blob/main/src/mesoscope_snr.py
   Would test whether detected Allen event amplitudes fit an exponential distribution, and whether residual/error around that model is Gaussian. Know this isn't a pure dF/F SNR measure and brings events into play but think it makes sense.

I plan to calculate/plot distributions of these metrics across ROIs for a given session, and then extend to multiple planes/sessions (or multiple sessions / days) if useful. I’ll sort ROIs by high and low values for each metric and pull example traces for the extremes. I’ll also look for ROIs where the metrics disagree, since those cases should be especially informative. I should have the first plots ready by tomorrow.

If you have any feedback on the units of analysis for plotting (either across all sessions, days, planes etc...) or if there's something these don't capture that another metric would feel free to suggest!

[jeromelecoq]

Hi Davis, thanks for this update.
Will you be presenting these at our meeting next Tuesday?

Thanks!

[davisgrubin]

Yes I can do July 7th if that works with the current schedule?

[davisgrubin]

Hi Jerome, just confirming if there is a meeting tomorrow or not (did not see one here )? I have run the proposed metrics on one session (notebook w plots here) and was planning on running on multiple sessions before the meeting, also happy to present next Tuesday 7/14 if that works?

[ido4848]

I'm trying to do a similar QC analysis for the SLAP 2 data (#152), and I wonder which of these metrics are also relevant in the SLAP 2 situation.

[davisgrubin]

Hi Ido, I ended up using an event based metric that I presented on in this meeting, there was some discussion on applicability to dendrite/soma differences and assumptions there:
https://allenneuraldynamics.github.io/openscope-community-predictive-processing/meetings/2026-07-21/

There is also a notebook here with a description of the event based metric and how it was calculated:
https://github.com/davisgrubin/openscope-ophys-qc/blob/main/notebooks/EventBackgroundQC.ipynb
A quick summary is that we take the OASIS style/event amplitudes (inferred spikes) that come with the Allen data, define time around them as "event" windows and everything else as "non-event" to get a baseline, then the number of SDs from that baseline a given event has. Then each ROI is classified/clustered based on its proportion of events that are 2SD, 2-4SD, >4SD from that baseline. As a heuristic it does seem to capture some interesting differences in classes depth of layer.

There's also an interactive visualization where you can see the layers (working on integrating this into the MyST format discussed last meeting, atm have to download or "Open in App" if you have sharepoint). multiplane-ophys_839909_event_cluster_review (1).html

I'd say without looking at the SLAP2 data myself I'm not sure what exact metrics will be useful for it, the above is just a heuristic that seems to reflect some reasonable differentiation between ROIs in the data; I will be comparing some more traditional metrics Sarah suggested in the same meeting I linked to above. I'm not entirely sure but since this metric is kind of biased towards large somatic spikes as "signal" (and dendritic ROIs were proposed as an explanation for lower SD categories in deeper layers) I don't know how well it separate out different signal levels among dendritic ROIs.

[Dedalus9]

Introduction to the Design of the Receptive Field (RF) Sessions

Here is a receptive field analysis of the OpenScope ophys data: optical physiology data from two-photon calcium imaging. The data come from OpenScope receptive field (RF) mapping ophys sessions. For each subject, session, and imaging plane, I analyze the receptive fields of the segmented ROI activity. The stimulus is a visual receptive-field mapping stimulus. Each condition is one combination of screen $x$ position, screen $y$ position, and stimulus orientation. There are 9 possible $x$ and $y$ positions and $3$ possible stimulus orientations, which gives 243 RF stimulus conditions per ophys session.

This figure shows the visual stimulus grid, i.e., the 9 $x$ positions, 9 $y$ positions, and $3$ orientations.

Expectations

I look at receptive field (RF) mapping sessions in the primary visual cortex (VISp) and lateral visual cortex (VISl). Each RF condition corresponds to the X and Y position of the stimulus as well as its orientation. My analysis focuses on population-level feature tuning. If the data is clean and the RF indeed evokes population-level responses tuned for the spatial position of the stimulus and its orientation, then I expect the following:

1. If the recorded population is tuned to the RF stimulus features, similar RF conditions should evoke more similar population responses than dissimilar RF conditions.

Support

• This is consistent with Stringer et al. (2019). They argue that visual cortical population codes are constrained by smoothness, such that similar stimuli should evoke related population responses.
• This is consistent with Marshel et al. (2011). They show that the population-responses of mouse visual cortex are tuned to visual features.

2. If the recorded population shows multiplexed tuning, population responses should respond to multiple RF stimulus features rather than single features.

Support

• This is consistent with Marshel et al. (2011). They show that mouse visual cortical populations contain diverse tuning profiles across features such as orientation, direction, spatial frequency, and temporal frequency.

Results

The data type used is the ophys-derived neural response data. Specifically, Iuse the inferred spike-rate or deconvolved event responses estimated from calcium imaging traces. For each ophys session and imaging plane, we compute the average response of each ROI response to each RF condition, giving us an RF response matrix (ROI x Condition).

I then computed PCA using this matrix to find the dominant population-level RF-response patterns across ROIs. The figures below are generated from subject 832700 from a session recorded on 01-24-26:

PCA Figures

PCA Loadings for VISp_0

This is an example of the PCA loadings for plane VISp_0, computed from the spike-rate RF tuning matrix. The bars show the PCA loading assigned to each stimulus condition or marginal condition. The full xytheta plot shows loadings for each orientation-X-Y condition. The X-marginal plot averages loadings across stimulus Y separately for each orientation; the Y-marginal plot averages loadings across stimulus X separately for each orientation; and the orientation-marginal plot averages loadings across both X- and Y-conditions.

Stimulus-Space RF Tuning

These figures show stimulus-space RF tuning PCA loadings for spike-rate responses. Each row shows one imaging plane (VISp_0-VISp_3, VISl_4-VISl_7), and each column shows one component (PC2-PC5). I marginalized each PC’s condition loadings over a stimulus dimension to isolate an axes-pair: X-Y, X-orientation, and Y-orientation. I also marginalized over X- and Y- conditions to isolate the orientation axes.

In each panel, the color corresponds to the projection of the stimulus space onto the respective marginalized PCA loading. Positive and negative values correspond to stimulus coordinates which contribute in opposite directions to that PC.

Theta Marginal

PC Loadings

Stimulus Map

Y-Marginal

X-Marginal

Consistent with Our Expectations?

Qualitatively, these figures show that population-level responses are multiplexed. They also show that similar stimulus conditions evoke similar population-level responses. This validates that the data is clean enough to yield results which are consistent with what we'd expect given the literature.

I will include an analysis which quantifies this across all sessions. I will be editing this post to include this cross-session/cross-subject analysis later this afternoon.

@maierav Are these results consistent with your analysis of receptive fields?

Works Cited

Marshel, James H., Marina E. Garrett, Ian Nauhaus, and Edward M. Callaway. “Functional Specialization of Seven Mouse Visual Cortical Areas.” Neuron, vol. 72, no. 6, 2011, pp. 1040–1054. https://doi.org/10.1016/j.neuron.2011.12.004.

Stringer, Carsen, Marius Pachitariu, Nicholas Steinmetz, Matteo Carandini, and Kenneth D. Harris. “High-Dimensional Geometry of Population Responses in Visual Cortex.” Nature, vol. 571, no. 7765, 2019, pp. 361–365. https://doi.org/10.1038/s41586-019-1346-5.

[Dedalus9]

Introduction: Individual ROI visualizations

Here are individual-ROI receptive-field figures for the ophys data. They were generated using PC loadings computed from the dataset which corresponds to subject 832700, recorded on 01-24-26.

Methodology

For each PC, I selected the top 9 ROIs within each plane, separately for the largest positive scores and the most negative scores. In each figure, rows correspond to imaging planes, ordered by VISp then VISl, and columns correspond to ROI rank within that plane. Positive-score panels are sorted from highest to lower positive score, while negative-score panels are sorted from most negative to less negative score.

For each PC and score sign, I plotted four receptive-field summaries by marginalizing over different stimulus dimensions. These views show complementary aspects of each ROI’s tuning, including its X-Y spatial receptive field, X-by-orientation structure, Y-by-orientation structure, and orientation tuning after marginalizing over spatial position.

How this supports my PCA analysis and validates the data

I am only showing the ROIs for a few of these PCs to avoid overwhelming this discussion thread with figures. Because the PCA analysis identified population-level modes, I expected ROIs with strong positive or negative scores on a given PC to show tuning patterns related to that PC’s stimulus-space loading structure. This appears to be the case: PCs with more distributed or mixed loading structure are associated with ROIs showing more diverse or multiplexed RF tuning, whereas simpler PCs are associated with more similar ROI-level RF structure.

Figures

PC2 Per-ROI: Receptive Fields Marginalized Over Orientation Tuning

Note that the x-axis corresponds to tuning with respect to the x-position of the stimulus; the y-axis corresponds to tuning with respect to the y-position of the stimulus

PC2 Per-ROI: Receptive Fields Marginalized Over Y-position Tuning

Note that the x-axis corresponds to tuning with respect to the x-position of the stimulus; the y-axis corresponds to orientation-tuning

PC2 Per-ROI: Receptive Fields Marginalized Over X-position Tuning

Note that the y-axis corresponds to tuning with respect to the y-position of the stimulus; the x-axis corresponds to orientation-tuning

PC5 Per-ROI: Receptive Fields Marginalized Over Orientation Tuning

Note the receptive for VISl5 and VISl7 and Compare with population-level stimulus tuning in original post
Note that the x-axis corresponds to tuning with respect to the x-position of the stimulus; the y-axis corresponds to tuning with respect to the y-position of the stimulus.

[Dedalus9]

Within-Subject Stability of Representational Geometry of Each Layer

I sought to determine whether each subject's RF representational geometry was stable across recording dates. By representational geometry I refer to the way in which a given population of neurons organizes its activity around stimulus conditions. For instance, if a population exhibits a pattern of activity across stimulus conditions, how do we expect for that pattern to vary across recording dates?

To answer this question, I relied on central kernel alignment (CKA). CKA is a method used in machine learning for comparing representational geometry across machine learning models. It is useful particularly since it doesn't depend on comparing ROIs. Rather, it only depends on comparing the organization of activity with respect to stimulus conditions. For this reason, even if the ROIs differ between recording sessions, planes, or subjects, the comparison isn't invalidated by this fact.

In the figure below, each line corresponds to a subject. The higher the mean within-subject session-pair CKA the more similar that layer's representational geometry is across recording sessions.

Across-Subject Stability of Representational Geometry of Each Layer

For this analysis, I asked how similar are the representational geometry of a given layer across subjects, after averaging across recording sessions. The higher the Mean CKA between subject-average Grams is, the more similar is the representational geometry across subjects.

Next Steps

I will repeat this analysis across all modalities. I will also repeat the PCA analysis across all modalities, i.e., NeuroPixels and Slap2

[Dedalus9]

CKA Values

I generated this plot given feedback from last Tuesday's meeting. I will generate it for the other data modalities as well. I believe it's consistent with what we'd expect if the data were clean. That is, that the representational geometry of planes within a given cortical area (i.e., whether it's VISp or VISl) are more similar than they are to other cortical areas.

For each subject, within-session CKA values were averaged across all recording sessions for every pair of VISp and VISl imaging planes. Planes were ordered by layer depth from superficial to deep, and each subject was shown as an 8-by-8 heatmap containing both within-area similarity and cross-area VISp-VISl similarity.

Each subplot shows a different subject. For each heatmap, each cell shows the mean CKA for that pair of planes. Higher CKA values indicate more similar population-level representations between the two planes; lower values indicate less similar representations.

[Dedalus9]

Neuropixels Across-Subject Similarity

This figure compares RF representational geometry across mice using the Neuropixel recordings collected during receptive-field stimulus presentations. For each subject and visual layer group, RF responses were summarized as session-averaged representational Gram matrices, then cross-subject CKA was computed between those subject-average Grams; lines show mean CKA across subject pairs and shading shows the 95% CI.

[Dedalus9]

Neuropixels Within-Subject Similarity

This Neuropixels within-subject CKA figure was generated from OpenScope RF stimulus recordings by grouping sorted single-unit responses by visual area and atlas-assigned cortical layer, computing a session-level RF representational Gram matrix for each subject/session/layer group, and then comparing pairs of recording sessions from the same subject using CKA. Each colored line is one subject’s within-subject session-pair CKA across layers, the black line is the layer-wise mean across subjects with 95% CI shading, and the gray bar is the overall area mean. The x-axis is ordered by layer depth