Skip to content

Allow a metadata-aware splitter as cv_class (used directly, not wrapped as the inner groups-CV) #1104

Description

@bruAristimunha

Summary

CrossSubjectEvaluation (and the CrossSubjectSplitter/CrossSessionSplitter wrappers)
always treat cv_class as the inner CV: they instantiate it and call
split(X=index, y, [groups=subject]), forwarding only the subject column. There is no
public way to plug in a metadata-aware top-level splitter — one that needs split(y, metadata)
and other columns (e.g. session) to build the fold.

Why this matters

We want a single-target cross-subject fold that restricts the test set to one session while
excluding the target's other sessions from training (test = target@session, train = other
subjects). That decision needs the session column, which the inner-CV path never passes.

A splitter that already follows the moabb convention (subclasses BaseCrossValidator, declares
metadata_columns = ("subject", "session"), implements split(self, y, metadata)) cannot be
passed via cv_class:

import pandas as pd
from moabb.evaluations.splitters import CrossSubjectSplitter
# MySplitter: BaseCrossValidator with metadata_columns=("subject","session") and split(self, y, metadata)
md = pd.DataFrame({"subject": [1, 1, 2, 2], "session": ["0", "1", "0", "1"]})
list(CrossSubjectSplitter(cv_class=MySplitter, target=2, test_session="1").split(y=None, metadata=md))
# TypeError: MySplitter.split() got an unexpected keyword argument 'X'

Current workaround

Subclass the evaluation and override _create_splitter to use cv_class directly:

class TargetSubjectEvaluation(CrossSubjectEvaluation):
    def _create_splitter(self):
        return self.cv_class(**self.cv_kwargs) if self.cv_class else super()._create_splitter()

Proposed

The evaluation already inspects cv_class.metadata_columns in __init__ (to extend
additional_columns). Extend that: when cv_class declares metadata_columns, use it directly
as the top-level splitter — call split(y, metadata) — instead of wrapping it as the inner
groups-CV. Backward compatible (group CVs without metadata_columns keep wrapping), and it lets
metadata-driven folds be configured through the public cv_class / cv_kwargs API with no subclass.

Happy to open a PR if this direction sounds good.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions