!!! FEATURE: Extensible meta data properties - #11
Open
bwaidelich wants to merge 44 commits into
Open
Conversation
Major rewrite to support extensible, configurable meta data properties
Member
Author
|
NOTE: This is Work in Progress. Before anything gets merged (and potentially released as new major version) we will evaluate if we can re-implement support for the previous features (such as metadata extraction, ...) or whether this new functionality should be moved to a separate package |
Sebobo
reviewed
Apr 16, 2026
| } | ||
|
|
||
| public function setMetaDataPropertyValue( | ||
| string $assetId, |
Member
There was a problem hiding this comment.
I think we always need a full asset identity consisting of the asset id and the asset source id.
I'm also currently checking all code in the Media.Ui to provide both or at least the asset source id to support new implementations like @nezaniel builds.
Member
Author
There was a problem hiding this comment.
Member
Author
There was a problem hiding this comment.
nevermind – I think, I'll go for two new VOs in this package. They probably need to be stored in separate columns anyways (in the DBAL implementation) and the URI representation is just an implementation detail anyways
Member
Author
There was a problem hiding this comment.
I have realized that in the form of a MetaDataAssetReference VO now.
If already installed, make sure to run
./flow doctrine:migrationexecute 20260415145934 --direction down
./flow doctrine:migrationexecute 20260415145934 --direction up
and simplify property value implementation
…`DimensionSpacePointProvider`
bwaidelich
commented
Apr 17, 2026
… parent dimension fallback
Member
Author
|
@Sebobo @dlubitz @c4ll-m3-j4ck (mentioning you as fellow contributors) – I'm sorry for the silence around this one. I did not lose track, but focus.. |
|
I can help here. I've planned some more Neos time in August anyways. |
Sebobo
reviewed
Jul 31, 2026
| { | ||
| foreach ($this->assetRepository->findAll() as $asset) { | ||
| /** @var Asset $asset */ | ||
| $title = $asset->getTitle(); |
Member
There was a problem hiding this comment.
Is the title being ignored here?
Sebobo
reviewed
Jul 31, 2026
|
|
||
| public function migrateExistingAssetPropertiesCommand(): void | ||
| { | ||
| foreach ($this->assetRepository->findAll() as $asset) { |
Member
There was a problem hiding this comment.
Need to check what happens with assets from other asset sources
`createAllPresetCombinations()` built its coordinates from the preset identifiers, while everything else in the adapter works with dimension values: the fallback chain matches on `values[0]` and the default dimension space point is built from the `default` of each dimension. For the common configuration where a preset is named after its primary value the two coincide, but as soon as they differ - a preset "german" with the values ["de"] - the dimension space points returned by `getDimensionSpacePoints()` were rejected by `isDimensionSpacePointValid()`, and the default dimension space point was not among them. Presets without values are now skipped as well, as they could never be matched.
Reading the label raised an "Undefined array key" warning for properties that are declared without a `ui` section. The label already falls back to the property name when it is NULL.
Properties declared with `globalScope: true` now have a single value that is shared by all dimensions instead of the flag being ignored. Such values are stored under the reserved dimension hash `global`, so that for a given asset and property the storage holds either one shared value or one value per dimension space point, never both. Reads always look up the scope a property is configured for, which makes values of the respective other shape unreachable rather than wrong after a configuration change. The four read methods of the MetaDataManager are replaced by one, because they were three views of the same answer rather than three questions: `MetaDataPropertyValue` now carries the own value (for editing, without shine-through), the inherited value and its origin (for the translation hint) and the effective value (for rendering) side by side. This also halves the number of queries an inspector needs. Resolution rules move out of the storage: it no longer picks a winner via a MySQL specific `FIELD()` ordering but returns all values of the requested scope, leaving fallback priority to the MetaDataManager. Adds `assetmetadata:repair` to consolidate values whose scope contradicts the current configuration, plus unit and functional test suites.
Adds regression tests for 7d4c8f1: a configuration whose preset identifiers differ from the primary values of those presets, asserting that dimension space points are enumerated by value, that every enumerated point is considered valid and includes the default one, and that presets without values are skipped. All three fail when 7d4c8f1 is reverted.
Adds `MetaDataManager::findAssets()`, returning the references of all assets that have a matching metadata value. The criteria of a `MetaDataAssetFilter` are all optional: asset source id, dimension space point, search term and the properties to search in. A value only counts if it is the one `getMetaDataPropertyValue()` would return for the filter's dimension space point, so the search agrees with what an editor working in that dimension sees: an asset whose caption is inherited from a fallback dimension is found, one whose inherited caption is overridden by a non matching value of its own is not. Properties with a global scope are matched on their shared value regardless of the dimension space point, just like they are read regardless of it. As everywhere else in this package, an omitted dimension space point means the *default* one rather than "any dimension" - a search is always carried out as seen from one dimension space point. Resolving the precedence per asset in PHP would mean a query per candidate, so it happens in SQL: the manager hands the storage the fallback chain ordered, from the most to the least specific dimension space point, and the storage applies that ranking rather than deriving one. The "order is meaningless" rule therefore moves from the interface docblock onto the method it actually describes. !!! `MetaDataStorage` gained a method, so third party implementations of that interface have to be extended.
Allows a single metadata property to be read from Fusion without fetching all
of them first:
caption = ${AssetMetaData.getMetaDataProperty(asset, 'caption', {language: 'de'})}
Like `getMetaData()` it returns the effective value, i.e. with dimension
fallbacks applied, and empty coordinates mean the default dimension.
`allowsCallOfMethod()` now allows every method, so that helper methods added
in the future are usable from Fusion without having to be listed twice.
Values used to be written and read as provided, so the `type` a property is declared with was exposed to consumers but never applied. It is now enforced in both directions: on the way in, so that nothing but a value of that type is ever stored, and on the way out, so that `MetaDataPropertyValue::$value` means what its `string|int|bool|null` signature says. Unambiguous conversions are applied, so that callers which only ever have strings - the command line, form input, Fusion - do not have to cast: "42" is a valid integer, "true", "on" and "yes" are a valid boolean, as are their negative counterparts. Anything else is rejected with an `InvalidArgumentException` rather than silently turned into a wrong value - "abc" is not 0. Reading is deliberately more forgiving, because it meets values that were written before a property was given its current type: a value that cannot be interpreted reads as NULL. Such a value is skipped rather than treated as an empty one, so that it does not shadow a fallback that is still readable. Booleans are stored as `1`/`0` and integers in decimal, which is what was already written before, so existing values stay readable.
There are two kinds of tests now: one for the `MetaDataManager`, with its dependencies as test doubles, and one per implementation of those dependencies. The manager tests no longer round trip through an in-memory storage. Each states the values the storage holds and asserts what resolves from them, so a test says what a rule *is* rather than demonstrating it through a write followed by a read. Writes are asserted as the calls they make. The repair tests work the same way and record those calls in order, which finally states outright that promotions happen before the deletions of the rows they came from. What a storage does with a lookup is its own business and is covered once per implementation. The tests of the DBAL adapter therefore also took over what used to be tested through the manager: the shadowing of values further down the fallback chain, the escaping of LIKE wildcards, distinctness, ordering and the separation of the localized and the global scope - plus the isolation of assets and asset sources, which was never a rule of the manager to begin with. They also cover the `MetaDataStorageMaintenance` surface that `assetmetadata:repair` is built on, including the values of unconfigured dimensions and undefined properties that reads can never return. `MetaDataConfigurationProviderYamlAdapter` had no test at all and has one now. The adapter stays MySQL specific, so its tests remain the only ones that need a database.
They describe where dimensions come from and what happens when the scope of a property is changed, so they belong to the configuration section as a whole rather than under the subsection about property types that was inserted above them.
…a-properties FIX: Add getOwnValue method to MetaDataPropertyValue for fusion access
This allows to disable property definitions that are configured
elsewhere:
Neos:
MetaData:
metaDataProperties:
'copyright': ~
…thub.com/neos/metadata into feature/global-scope-metadata-properties
…a-properties FEATURE: Allow nullable ui configuration
bwaidelich
marked this pull request as ready for review
August 6, 2026 14:09
Member
|
Thanks for all the changes, is this now a finalish reviewable state on which I can fully build upon in the Media.Ui ? |
|
@Sebobo I think there are some things left. But this shouldn't effect the integration into Media.UI. |
Sebobo
reviewed
Aug 31, 2026
| `asset_source_id` VARCHAR(255) DEFAULT NULL, | ||
| `asset_id` VARCHAR(40) DEFAULT NULL, | ||
| `property_name` VARCHAR(40) NOT NULL, | ||
| `property_value` VARCHAR(250) NOT NULL, |
Member
There was a problem hiding this comment.
250 characters is not enough. I already had some files in my own website with a longer caption and the migration command failed.
Neos used longtext but that is too much. So I discussed with @dlubitz that we should use text instead for 65k characters.
Suggested change
| `property_value` VARCHAR(250) NOT NULL, | |
| `property_value` TEXT NOT NULL, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Major rewrite to support extensible, configurable meta data properties
Todos
DateTime)?– suggestion: no, let's keep it simple
ui.editorOptionsglobalScopeandtype, and find assets by metadata #13globalScopeandtype, and find assets by metadata #13globalScopeandtype, and find assets by metadata #13globalScopeandtype, and find assets by metadata #13neos/metadata-extractorto use newMedaDataManagerAssetService->emitAssetUpdatedto flush caches etc. when metadata is updatedPackage.phpofneos/metadataor "glue package"globalScopeandtype, and find assets by metadata #13 (comment) and follow-ups)