Clarification on the acquisition of calibration assets

[bruno-f-cruz]

Since this has comeup a few times in the last month (with @hagikent and https://github.com/AllenNeuralDynamics/aind-scientific-computing/issues/483#issuecomment-3706464075) I wanted to ask for help understanding what the intended pattern for calibration assets is.

My understanding is that calibrations exist as a separate subject_details type in the subject schema:

aind-data-schema/src/aind_data_schema/core/subject.py

Line 23 in 58069c8

subject_details: Discriminated[MouseSubject | HumanSubject | CalibrationObject] = Field(

However, it is not clear to me how this information interacts:

• required subject_id field in acquisition.json:
  

  aind-data-schema/src/aind_data_schema/core/acquisition.py

  Line 323 in 58069c8

  subject_id: str = Field(default=..., title="Subject ID", description="Unique identifier for the subject")

• required subject_id field in aind-data-transfer-service (https://github.com/AllenNeuralDynamics/aind-data-transfer-service/blob/2deb43b29e0ea7365a94c03b0935e808604463c5/src/aind_data_transfer_service/models/core.py#L181) @jtyoung84
• required subject in the watchdog application supported by SIPE (https://github.com/AllenNeuralDynamics/aind-watchdog-service/blob/503c36403eb1add3fb3fabc2a47ab7a1e8eaf887/src/aind_watchdog_service/models/manifest_config.py#L57) @patricklatimer
• subject name in data asset (I was trying to use a previous reference but it seems broken https://docs.allenneuraldynamics.org/en/latest/metadata/data_conventions.html, so not sure about this one?)

Can someone clarify and try to figure out if we need to change the specification in any of these tools? Thanks!

[patricklatimer]

To clarify the watchdog bit, the only thing subject_id is used for is to pass into the data-transfer-service, no other actual need for it.

[dbirman]

For watchdog the only change we need is for you to be able to accept a string and not constrain it to an integer.

[jtyoung84]

subject_id and acq_datetime are required fields for the main branch of aind-metadata-mapper. However, it looks like they are being made optional for the next major release. I also made a note in this issue that it would still be useful for a user to pass a session_id to the transfer service so we can track how the session is being processed end-to-end. I reckon we could make it an optional field though.

[jtyoung84]

Actually, they might also be optional in the main branch of aind-metadata-mapper too. We can deprecate the fields in aind-data-transfer-service, but I'm not sure of a timeline. Our Q1 roadmap has been defined, but we could probably get this in Q2 unless there is an urgent need.

[bruno-f-cruz]

I am ok either way. I like the session_id since it may be relevant for the logging infrastructure we are discussing too.
Regarding the rest of the API I first want to make sure that @saskiad and @dbirman don't already have something else in place to handle this in a different way, before making large changes to the infrastructure. thanks!

[dbirman]

I think our preference is that you use the string "calibration" as the subject_id. We'll clarify this in the AIND docs.

[dbirman]

We'll probably validate this in V3, but for now in V2 I will start raising a warning if someone provides a CalibrationObject and tries to use a subject_id other than "calibration".