Annotations may disappear if records are reloaded

Field annotations may disappear from the VRE interface if records are reloaded from the Explorer in two cases:

Annotations to fields that are missing from the original records disappear if the original record later includes these fields
Annotations to fields that are present in the original record disappear if the values of the field change in the source database

The annotations will remain present in the triplestore and are also loaded to the frontend, but the frontend won't render them.

I investigated this issue and I think the two subissues might have different causes.

I assigned myself on the assumption that I might be able to fix this today, but I think it is a bit too involved for this narrow time frame. I might still do it later but for now I relinquish it. My approach would to add a behavioral regression test for one scenario, step with the debugger until I understand what's going wrong, fix it and then repeat with the other scenario.

@tijmenbaarda

Regarding the second subissue, #349 (when the field is originally present on the record), I am quite sure what is going on:

The RecordDetailView creates a FlatterFields, in which every {field, value} pair is represented as a separate model. This means there may be multiple models representing the same field. In addition, it includes {field} models for those fields that don't have a value in the originating catalog. The FlatterFields is passed to a RecordFieldsView instance.
The RecordFieldsView creates a FieldView for each of its models. In other words, there are one or more FieldViews per field, depending on the number of values for a field in the original record. Each FieldView also gets a collection of annotations matching the {field, value} model. This matching is based on the annotationMatchesField function.
Each FieldView then renders as one or more rows in the table of fields-with-annotations.

The disappearance happens in step 2. If an annotation has an originalText, and the record from the catalog no longer includes that text as a value of the annotated field, then annotationMatchesField returns false. Hence, the annotation in question is never passed to a FieldView.

This same mechanism does not explain the other subissue (#348, when the field was originally missing on the record). To the contrary: purely based on the logic above, I would actually expect annotations that lack an originalText to be repeated in every FieldView with matching fieldname. I have not yet investigated why this is not the case. However, it might be worth postponing this until #349 has been fixed, because I think that will require a minor redesign.

I'm calling this a redesign, because I think we may need to make changes at all three view levels. We currently take an asymmetric approach, where the contents of the original record together with the set of possible fields dictate the contents of the FlatterFields and hence determine what FieldViews will be rendered. Annotations that cannot be matched to any of those FieldViews are simply never rendered. I think we need a type of three-way merge instead, between the possible fields, the field-value pairs present in the original record, and the field annotations, in which either side is still rendered even if it cannot be matched to the other side.

My current idea on how to approach this, building on top of #353, is as follows. Instead of using FlatterFields, we go "back" to something more akin to FlatFields, where every possible field is represented by exactly one model. Each model contains a potentially empty list of original values for the given field in the record, as well as a potentially empty list with all annotations (corrections/additions) for that field, which is purely matched on the fieldname.

Each of these models is displayed by an instance of FieldView, which will need to be supercharged a bit because it might have to represent multiple values in the original record. It should display:

One row for each original field value that does not have a matching correction. Each of these rows should have a pencil for making a correction.
One row for each correction that still matches an original field value, also with a pencil for editing.
One row for each correction that no longer matches an original field value. Instead of a pencil, it should have a broken link icon. When the user clicks the broken link icon, they should get information about the original value that is no longer present and options to either delete the correction or re-link it to a newer field value in the original record.
One row for each addition, again with a pencil icon for editing.
A final empty row with a + icon, that can be pressed in order to add more additions.

@tijmenbaarda feedback welcome.