setObjectProperty reports DONE for a Condition that does not occur in the object's selector — the step changes nothing and the run stays green

[Wladefant]

Symptom

The action description shown for setObjectProperty is

Set object [<Object>] property as [<Data>] at runtime

Read literally that says "set this object's css property to this value", so the natural authoring is Condition = css, Input = <the new selector>. Authored that way the step reports success and changes nothing:

🔵 Step:2   | Object: Cell | Action: setObjectProperty
          | Input: @#table1 tbody tr:nth-child(3) td:nth-child(1) | Condition: css
[DONE]   | Setting Object Property for css with #table1 tbody tr:nth-child(3) td:nth-child(1)
           for Object [[Project] Tables - Cell] 🟢

The object keeps its original selector. Every following step acts on the original element. The steps pass, the test case passes, and the screenshot the engine files as evidence shows the original element.

Condition is not a property name — it is a placeholder token that must already occur inside the object's attribute value. That is a perfectly reasonable design, but nothing tells an author they have used it the other way, and the failure is silent in the dangerous direction.

Root cause

setObjectProperty records whatever it is given and always reports DONE — DynamicObject.java#L60-L82:

} else {
    setProperty(Condition, Data);
}
String text = String.format(
    "Setting Object Property for %s with %s for Object [%s - %s]",
    Condition, Data, Reference, ObjectName
);
Report.updateTestLog(Action, text, Status.DONE);

setProperty (L84-L98) just puts the pair into AutomationObject.dynamicValue. There is no check that the key means anything, and no failure path other than empty input.

The pair is consumed by AutomationObject.getRuntimeValue (L901-L916), called on each attribute value as the locator is built (L1040):

for (String Key : dynamicValue.get(pageName).get(objectName).keySet()) {
    value = value.replace(Key, dynamicValue.get(pageName).get(objectName).get(Key));
}

So the operation is a plain substring replacement inside the attribute value. "css" does not occur in #table1 tbody tr:nth-child(1) td:nth-child(1), String.replace returns the value unchanged, and the step that reported DONE has done nothing.

Two consequences follow from the same line:

1. A Condition that occurs nowhere is a no-op reported as success. That is this report.
2. A Condition that occurs by accident silently corrupts the selector. The key is matched anywhere in the raw attribute value, not against a delimited token, so a name that happens to appear as a substring of a selector is replaced there too. (Read off the source; not separately reproduced.)

Reproduction

Windows 11, JDK 17.0.12, a build of release/3.1.0, headless Chromium, against a real public page (https://the-internet.herokuapp.com/tables, whose #table1 has four rows: Smith, Bach, Doe, Conway). On a throwaway copy of the bundled Projects/CLIDemo sample, run through the legacy CLI (-project_location … -release R1 -testset … -run -quit).

Object repository page Tables:

page: Tables
scope: PROJECT
elements:
  Cell:
    css: "#table1 tbody tr:nth-child(1) td:nth-child(1)"      # row 1 -> Smith
  CellTemplated:
    css: "#table1 tbody tr:nth-child(__ROW__) td:nth-child(1)"

Three test cases, all with the same intent — make the object point at row 3, whose first cell is Doe.

Test case	Step 2	Result
PlaceholderAuthoring — placeholder token, on CellTemplated	setObjectProperty Input=@3 Condition=__ROW__	PASS 4/4. storeElementTextinVariable reads Doe; assertVariable @%Actual%=Doe passes. The feature works exactly as designed.
PropertyNameAuthoring — property-name reading, on Cell	setObjectProperty Input=@#table1 tbody tr:nth-child(3) td:nth-child(1) Condition=css	PASS 3/3. Step 2 [DONE] … 🟢; step 3 assertElementIsVisible [PASS] [Cell] is visible ✅. Nothing indicates anything went wrong.
PropertyNameMeasured — identical to the above, plus a read-back	same	Step 3 storeElementTextinVariable reads Smith. Step 4 assertVariable @%Actual%=Doe → [FAIL] Variable value is Smith but expected value is Doe ❌

The third case is the measurement, and it is the engine's own message: after a step that reported DONE for repointing the object at row 3, the object still resolves to row 1.

The second case is the one that matters in practice. It is what an author actually writes — repoint the object, then assert it is there — and it is green on the wrong element. The screenshot the engine filed for that green step draws its highlight around Smith, i.e. the element the author had just written a step to move away from. A wrong element, a passing step, and a screenshot of the wrong element stored as the evidence for it.

PlaceholderAuthoring is included deliberately: the mechanism is not broken. Only the contract is undiscoverable, and misuse of it is indistinguishable from use.

Suggested fix

The cheapest version needs no new plumbing, because the information is already in one place:

1. In getRuntimeValue, notice when a registered key matched nothing. value.contains(Key) is the whole test. When a key is registered for an object and occurs in none of that object's attribute values, report it — a warning at minimum, ideally a failed step, naming the token, the object and the value it was tested against. That converts every instance of this mis-authoring into a first-run failure with an actionable message, and it also catches the accidental-substring case from the other direction.
2. Or validate at set time in setObjectProperty, since Reference and ObjectName are exactly what setProperty already keys the map by. Reporting FAILNS there instead of DONE is a one-branch change, though it moves the check earlier than the point where the value is actually known.
3. Independently, reword the description, because the current one is what leads to the wrong reading in the first place. Something closer to what the action does — "Replace the placeholder [<Condition>] in [<Object>]'s locator with [<Data>] at runtime" — makes the contract visible in the place the author is looking when they choose the action.

(1) and (3) together seem worth more than either alone: the wording stops most of it happening, the check catches the rest.

Happy to prepare a PR for (1) and (3) if that is a direction you'd take.

A shape, if it is useful

Two other reports open at the moment end the same way from unrelated code: #320, where an ambiguous css selector inside a frame resolves to .first() and the step passes on the wrong element, and #319, where the legacy CLI exits 0 for a crashed run, a failing run and a passing run alike. With this one, that is three independent places where a wrong outcome is reported as a good one — and a green result on wrong behaviour is the only kind of defect that cannot be found by looking at the report. Individually they are three small fixes; they may be cheaper to think about as one theme than separately. Mentioned only for that reason.

Environment

Windows 11, JDK 17.0.12, headless Chromium, a build of release/3.1.0. Every source citation above was read off the file at branch head 7bc763b6; DynamicObject.java was additionally fetched from this repository over the API and matched the local copy byte for byte (blob 6cd4d964). The reproduction ran with the plugin search path pointed at an empty directory, so no plugin was loaded (Plugin search directory source=env, path=…, exists=true); the throwaway project was deleted afterwards.