[x86] [sysv] returned structures more than register width should be passed in rdi

[VisualEhrmanntraut]

Struct `Class::method`(class Class* `this`)

where sizeof(Struct) > REG_SIZE must be handled as

void `Class::method`(Struct& ret, class Class* `this`)

[bdash]

Can you please provide an example where you're seeing the wrong behavior?

In a test I constructed we look to be correctly handling the different return value locations for types larger than 8 bytes (16 byte struct returned in rax:rdx, larger structs returned indirectly via rdi with this shifting to rsi).

sret_test.cpp
sret_test.o.zip

[VisualEhrmanntraut]

mac-x86_64, BN 6.0.10489-dev

nothing happens when I set the return type myself

[bdash]

Are you able to share that binary, or some other simplified binary, that reproduces the problem for you?

[VisualEhrmanntraut]

the code you posted reproduces the issue for me

Screen.Recording.2026-08-18.at.19.41.59.mov

[bdash]

In that case this remained typed with an explicit @ rdi location. If you remove that explicit location, the body looks as you'd expect. The screenshot you shared does not seem to have the explicit @ rdi location, so I think something different is going on there.

[bdash]

Wrote up #8429 about the fact this ends up with an explicit @ rdi location after changing the return type.

[VisualEhrmanntraut]

it looks like no storage was assigned for some reason

[VisualEhrmanntraut]

and it looks like the return type is not propagated

[bdash]

Are you able to share a binary that reproduces the problem as originally described?

[VisualEhrmanntraut]

one moment, I'll fetch one

[VisualEhrmanntraut]

test.cpp

test.zip

create the structure test and set the return type to it

[bdash]

Thanks, I can reproduce the problem with that binary after 1) applying the return type, and 2) removing the explicit @ rdi location from this:

100000420    struct test cls::foo(class cls* this)

10000042b        struct test result
10000042b        struct test* var_18 = &result
10000042f        class cls* this_1 = this
10000043a        class cls* this
10000043a        _memset(__b: this, __c: 0, __len: 0x30)
100000448        return result

It looks like something in the function's analysis state is not being invalidated correctly when the return value / parameter locations change.

These are the types I applied when testing this:

struct test {
  uint8_t pad[0x30];
};

class cls { };