Skip to content

Keep the declared source_field when deconstructing a FK - #2284

Merged
waketzheng merged 5 commits into
tortoise:developfrom
aksgupta98:fix/fk-source-field-deconstruct-2283
Sep 17, 2026
Merged

waketzheng merged 5 commits into
tortoise:developfrom
aksgupta98:fix/fk-source-field-deconstruct-2283

Conversation

@aksgupta98

Copy link
Copy Markdown
Contributor

Description

ForeignKeyField and OneToOneField lose a declared source_field when they are deconstructed, so makemigrations writes the wrong column name.

Apps._init_relations() moves the declared source_field onto the generated <field>_id backing field, and then reuses the attribute on the relation field to hold that backing field's name. Model._meta and migrations/writer.py both depend on that. But deconstruct() read the same attribute as if it still held the column name.

This reads the column name back off the backing field instead.

Fields that declare no source_field are unaffected. Their backing field holds the <field>_id default, which is exactly what was emitted before.

Motivation and Context

Fixes #2283.

Given this model:

consumer = fields.ForeignKeyField("models.Parent", source_field="account_id")
  • generate_schemas() creates the column account_id, which is correct.
  • makemigrations writes source_field='consumer_id', so a database built from that migration gets consumer_id.

The two paths disagree and neither reports an error. A project that uses generate_schemas() in development and migrations in deployment ends up with a different column name in each, and any hand-written SQL or backfill then works in one environment and fails in the other.

How Has This Been Tested?

Four new tests in tests/fields/test_fk.py. They use the existing SourceFields and Event models, so no new test models were needed:

  • a ForeignKeyField with source_field deconstructs to the declared column
  • a OneToOneField with source_field does the same
  • a ForeignKeyField without source_field still gets the <field>_id default
  • the deconstructed value always matches the backing field's column and appears in _meta.db_fields

Three of the four fail on develop and pass with this change. The fourth covers the default path and passes either way, to catch a regression there.

Full suite on SQLite: 1932 passed, 159 skipped, 2 xfailed, 0 failed.

Two modules were skipped locally because they need asyncmy / aiomysql, which I do not have installed: tests/backends/test_connection_params.py and tests/test_default.py. Neither touches this code path.

ruff format --check, ruff check and mypy are all clean on the changed files. The three ruff check warnings in tortoise/fields/relational.py are pre-existing on develop.

Environment: Python 3.14, SQLite.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

🤖 Generated with Claude Code

Apps._init_relations() moves a ForeignKeyField's declared source_field onto
the generated `<field>_id` backing field, then reuses the attribute on the
relation field to hold that backing field's name. Model._meta and the
migration writer both depend on that.

deconstruct() read the attribute as if it were still the column name, so
makemigrations wrote `<field>_id` instead of the declared column. A schema
built from migrations then disagreed with one built by generate_schemas(),
and neither path reported an error.

Read the column name back off the backing field instead. Fields that
declared no source_field are unaffected: the backing field holds the
`<field>_id` default, which is what was already emitted.

Fixes tortoise#2283

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codspeed

codspeed Bot commented Sep 15, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 24 untouched benchmarks


Comparing aksgupta98:fix/fk-source-field-deconstruct-2283 (6c02d6a) with develop (8477e47)

Open in CodSpeed

@waketzheng

Copy link
Copy Markdown
Contributor

Thanks for the PR! The fix is well-targeted and the root cause is clearly explained.

A few suggestions before merge:

  1. Test robustness: the new tests hardcode "fk_sometable" and "o2o_sometable". It would be more resilient to derive the expected value dynamically from the test model, or to introduce a dedicated test model for this scenario. This avoids breakage if the existing test models are refactored later.

  2. Edge case coverage: could you add a test for a ForeignKeyField whose declared source_field is identical to the field name itself (e.g. fk = ForeignKeyField(..., source_field="fk"))? Just to confirm the logic doesn't accidentally override it.

  3. Pre-initialization deconstruct: a quick test that calls deconstruct() on a field before Tortoise.init_models() would confirm the getattr(self, "model", None) guard behaves as expected.

Reviewer feedback on tortoise#2284:

- The tests keyed off SourceFields, whose shape they do not own. Add
  FKSourceFields, used only by these tests, so a later refactor of another
  test model cannot break them.
- Add a case for a declared source_field equal to the field name itself,
  confirming the backing field name does not override it.
- Add a case calling deconstruct() on a field that was never attached to a
  model, covering the `model` guard.

Also assert every deconstructed value is a real column in _meta.db_fields,
so the reported name is checked against the schema rather than restated.

No change to the fix itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@aksgupta98

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All three addressed in e330acd. The fix itself is unchanged — this is test work only.

1. Test robustness

Added FKSourceFields to tests/testmodels.py, used only by these tests, so they no longer depend on the shape of SourceFields. It is self-referencing, so all four cases fit in one table:

Field Case Column
renamed renamed column renamed_column
same source_field equal to the field name same
plain no source_field declared plain_id
o2o renamed OneToOneField o2o_column

I went with a dedicated model rather than deriving the expected value at runtime: deriving it from fields_map[...].source_field would just restate what the implementation reads, so the assertion would hold even if the fix were wrong.

2. source_field equal to the field name

Covered by test_deconstruct_fk_source_field_equal_to_field_name. Not overridden — fk = ForeignKeyField(..., source_field="same") keeps the column same while field.source_field is the backing name same_id. This already worked before the test existed; there was simply nothing pinning it.

3. Pre-initialisation deconstruct

Covered by test_deconstruct_before_init_models_keeps_declared_source_field. A field never attached to a model has model is None, the guard skips, and the declared value is reported unchanged.

Also added

test_deconstruct_source_field_is_always_a_real_column asserts every deconstructed value appears in _meta.db_fields, so the reported name is checked against the generated schema rather than against the implementation.

Verification

  • 6 deconstruct tests, up from 4. Four fail on develop and pass here. The other two — the default-column case and the pre-init case — pass either way by design, guarding against the fix breaking those paths.
  • Full suite on SQLite: 1934 passed, 0 failed.
  • ruff format, ruff check and mypy clean. The 14 pre-existing RUF012/FURB167 warnings in tests/testmodels.py are unchanged; the new model adds none.

@waketzheng

Copy link
Copy Markdown
Contributor

Thanks for the update. The fix looks good overall, and the root cause analysis is clear. A few remaining points:

  1. CHANGELOG reference number

We use PR number. Please use #2284 instead of #2283.

  1. Variable naming: backing

I noticed that backing is used as a variable name here, but this doesn't match the naming conventions elsewhere in the codebase. Other places consistently use more descriptive names like backward_key, backward_fk_fields, or BackwardFKRelation. While the term "DB-Backing Field" appears in documentation, backing alone as a variable name is ambiguous. It's not immediately clear whether it refers to the backing _id field or something else. Would it be clearer to use something like backing_field instead of backing?

  1. Test organization

The newly added test_ functions are currently module-level. If tests/fields/test_fk.py already groups related tests into classes, consider moving the new tests into a single class (e.g. TestDeconstructSourceField) for better organization and discoverability. If the file is purely function-based, keeping them as module-level functions is fine, but grouping them would still improve readability.

Everything else looks good: the fix logic, the CHANGELOG entry placement, and the test coverage. Thanks for including tests for both the FK and O2O cases.

- CHANGELOG now references the PR (tortoise#2284) rather than the issue.
- Rename `backing` to `backing_field` so it reads as the `<field>_id`
  field the column name was moved onto.
- Group the deconstruct tests into `TestDeconstructSourceField`,
  following the class style used in tests/test_source_field.py.

No change to the fix itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@aksgupta98

Copy link
Copy Markdown
Contributor Author

Thanks — all three are done in 3d5e809. The fix itself is unchanged.

1. CHANGELOG number

Now (#2284). I checked the neighbouring entries and they do use PR numbers, so that was my mistake.

2. backing → backing_field

Renamed in all three places. You are right that backing on its own does not say what it holds.

3. Test grouping

The file is purely function-based, so there was no existing class to follow. I used the class style from tests/test_source_field.py instead: a TestDeconstructSourceField class with a model attribute and self.model in each test. The # ==== banner comment I had added is gone, since the class now does that job.

I also shortened the method names, as the class name already carries "deconstruct":

Before Now
test_deconstruct_fk_keeps_declared_source_field test_fk_keeps_declared_source_field
test_deconstruct_o2o_keeps_declared_source_field test_o2o_keeps_declared_source_field
test_deconstruct_fk_without_source_field_uses_default_column test_fk_without_source_field_uses_default_column
test_deconstruct_fk_source_field_equal_to_field_name test_source_field_equal_to_field_name
test_deconstruct_before_init_models_keeps_declared_source_field test_before_init_models_keeps_declared_source_field
test_deconstruct_source_field_is_always_a_real_column test_source_field_is_always_a_real_column

One more change

The class docstring referenced (#2283). After your point about the CHANGELOG, a bare number there read as ambiguous, so it now says (issue #2283).

Verification

  • The six tests pass. With the fix reverted, four of them fail, so moving them into a class did not weaken them.
  • Full suite on SQLite: 1936 passed, 0 failed.
  • ruff format, ruff check and mypy clean. No new warnings in the files I touched.

Happy to squash the three commits into one if you would prefer that before merge.

@waketzheng

Copy link
Copy Markdown
Contributor

Nice. A thing of beauty is a joy for ever.

@waketzheng
waketzheng merged commit 219870b into tortoise:develop Sep 17, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

makemigrations writes the wrong source_field for ForeignKeyField (value is overwritten during _init_relations)

2 participants