feat: N+1 elimination via select_related, async support, and docs overhaul by arunsureshkumar · Pull Request #25 · strollby/graphene-mongo

arunsureshkumar · 2026-05-24T15:24:01Z

Summary

Replace dataloader-based N+1 mitigation with select_related pre-fetching — resolves all reference fields in a single aggregation pipeline
Add async MongoEngine support (AsyncMongoengineObjectType, AsyncMongoengineConnectionField) using aobjects queryset
Overhaul documentation structure to match graphene-python.org pattern (quickstart, types, prefetching, relay, telemetry, examples)
Expand CI MongoDB version matrix to 4.4–8.3

Key changes

graphene_mongo/base/ — shared ObjectType factory and connection field base
graphene_mongo/synchronous/ — sync-only implementation
graphene_mongo/asynchronous/ — async implementation extending sync
docs/ — restructured into focused sub-pages per section

Test plan

Run make test — all 179 tests pass across Python 3.10–3.14 and MongoDB 4.4–8.0
Verify sync queries return correct data with pre-fetched references
Verify async queries work with AsyncMongoengineObjectType
Build docs with sphinx-build -b html docs docs/_build/html — no warnings

Replace poetry-based tooling with uv: convert pyproject.toml to PEP 621 format, use hatchling as build backend, move dev deps to PEP 735 dependency-groups, and wire the mongoengine git source via [tool.uv.sources]. Update Makefile to use `uv run` and `uv build`. Replace poetry.lock with uv.lock.

Drop setup.cfg, MANIFEST.in, and README.rst — all superseded by the uv/ hatchling setup in pyproject.toml. Migrate coverage omit config from setup.cfg into [tool.coverage.run] in pyproject.toml. Add .venv/ to .gitignore.

Relocate graphene_mongo/tests/ → tests/ and convert all relative `from ..` imports to absolute `from graphene_mongo.` imports. Update Makefile and pyproject.toml paths accordingly.

Mirror the mongoengine project structure: separate sync and async tests into tests/synchronous/ and tests/asynchronous/. Shared fixtures stay in tests/conftest.py; models.py and types.py stay at tests/ root. Update all relative imports accordingly.

Remove @pytest.mark.asyncio and async/await from all tests in synchronous/ — they test MongoengineObjectType and MongoengineConnectionField which are fully sync, so test functions and resolvers should be plain def using schema.execute(). Also fix MongoengineObjectType.get_node in types.py: it was declared async def but its body is entirely synchronous, causing schema.execute() to receive an unawaited coroutine instead of the document instance.

Introduce base/, synchronous/, asynchronous/ subdirectories matching the mongoengine project structure: base/ transport-agnostic shared code advanced_types.py, converter.py, registry.py, utils.py, field_resolvers/ synchronous/ sync API (MongoengineObjectType, MongoengineConnectionField) types.py, fields.py asynchronous/ async API (AsyncMongoengineObjectType, etc.) types.py, fields.py, dataloader.py, utils.py get_dataloader and DATALOADER_CONTEXT_ATTRIBUTE move from base/utils.py to asynchronous/utils.py to keep base/ free of async-specific code. Public API via __init__.py is unchanged. Tests updated to use new direct module paths.

- tests/asynchronous/utils.py: with_local_async_registry decorator - tests/asynchronous/types.py: non-relay AsyncMongoengineObjectType variants in a private local registry to avoid conflicting with relay nodes - tests/asynchronous/test_query.py: async basic query tests (9 tests) - tests/asynchronous/test_mutation.py: async mutation tests using asave/aobjects - tests/asynchronous/test_inputs.py: async input mutation tests - tests/asynchronous/test_types.py: AsyncMongoengineObjectType metaclass tests - test_relay_query.py: convert remaining sync objects/save calls to aobjects/asave

- Add get_select_related_paths() to walk queried GraphQL fields and build __-separated paths for mongoengine select_related() - Apply select_related in get_queryset() and get_node() for both async and sync fields, resolving all queried ReferenceFields in a single aggregation - Skip count() DB call when no pagination args present (saves 1 query for unpaginated relay connections) - Remove aiodataloader-based MongoDataLoader/ModelLoader and get_dataloader() helper; field resolvers now make direct async DB calls as fallback - Remove dataloader_resolver classmethod from AsyncMongoengineObjectType - Strip _async suffix from all async test function names - Add test_select_related.py for both async and sync with query-count assertions verifying N+1 elimination and count-skip optimisation

- Add tox.ini with tox-uv runner targeting py310–py314 - Remove aiodataloader dependency (dataloader removed in previous commit) - Rename dependency group dev → test to match tox dependency_groups convention

… dep - Append TOX_ENV_NAME to DB_NAME in conftest so parallel tox runs use separate databases (pattern from mongoengine test suite) - Pin mongoengine git dep in tox.ini since tool.uv.sources is not inherited by tox-created venvs

- Rewrite ci.yml: Python × MongoDB (7.0, 8.0) matrix using uv + tox, replaces old poetry-based single-version workflow - Remove lint.yml: linting consolidated into ci.yml - Rewrite publish.yml: use uv build + trusted publishing (id-token) - Add docker-compose.yml for local MongoDB (mongo:latest) - Revert mongo port factors from tox.ini — MongoDB version matrix belongs in CI (GitHub Actions spins up the server), not tox

FastAPI (new): - Library domain (Author, Book with ReferenceField) - AsyncMongoengineConnectionField with filter_fields and pagination - Async create/delete mutations - Lifespan for async DB connection - GraphQL Playground at GET /graphql - Tests with httpx AsyncClient + mongomock Flask: replace abandoned flask-graphql with native async Flask route (Flask 3.x) Falcon: rewrite for Falcon 4.x ASGI — async on_get/on_post, MongoLifespan middleware for startup/shutdown, fix await-in-sync bugs from old version Requirements: update all three to current versions (Django 5, Falcon 4, Flask 3, mongoengine 0.28+, pytest 8+)

Replace requirements.txt + pytest.ini in each example with a single pyproject.toml. References graphene-mongo via path = "../.." and mongoengine via the git source for local development.

- index.rst: add project overview, features list, installation, quick start for both sync and async APIs; update toctree - tutorial.rst: modernize Flask tutorial — uv setup, Flask 3.x async view, add mutations, filtering/pagination examples; remove flask-graphql references - async_tutorial.rst (new): FastAPI tutorial covering AsyncMongoengineObjectType, AsyncMongoengineConnectionField, async mutations, N+1 elimination explanation - fields.rst: replace bare list with table, field descriptions, code examples for ReferenceField, EmbeddedDocument, ListField, geo, FileField, inheritance, self-reference, and filter_fields operators - conf.py: update version to 0.5.0, copyright, author

…le-query resolution - Add get_related_field_filter_args() to walk the GraphQL AST and extract filter args from connection sub-fields (e.g. articles(headline: "Hello")) - Apply extracted filters via MongoEngine's filter() before select_related so the pipeline builder pushes conditions inside the $lookup stage rather than fetching all docs and post-filtering in Python - Remove Python-level list filtering from default_resolver; pre-loaded select_related data is returned directly - Delete standalone test_select_related.py files; coverage merged into test_relay_query.py with explicit query-count assertions - Remove stale GenericLazyReferenceField docstring references (field no longer exists in MongoEngine)

Count query is only needed when `last` is used (tail-offset computation) or when `pageInfo` is explicitly queried (hasNextPage/hasPreviousPage). For first/after/before without pageInfo the count is now skipped, reducing those queries from 2 to 1. Also tighten all count >= 1 assertions to exact values across mutation, query, and relay_query tests, and add explicit tests for each count-query boundary (after+pageInfo, before+pageInfo, first-without-pageInfo). Fix redundant collection meta on Child/AnotherChild subclasses (removed to silence MongoEngine SyntaxWarning).

Move all shared properties and resolver helpers from MongoengineConnectionField/AsyncMongoengineConnectionField into a new BaseMongoengineConnectionField in graphene_mongo/base/fields.py. Extracted helpers (_hydrate_args, _qs_accessor, _apply_select_related, _build_args_copy, _prepare_resolver_inputs, _collect_required_fields, _transform_qs_args) eliminate ~480 lines of duplication. The _qs_accessor hook lets async cleanly override model.aobjects without duplicating get_queryset. Also aligns sync has_previous_page to gate on requires_page_info, matching async behaviour.

Move construct_fields, construct_self_referenced_fields, and the create_graphene_generic_class factory into graphene_mongo/base/types.py. The unified factory accepts executor, registry factories, and connection field class as keyword params, eliminating ~130 lines of duplication between the sync and async type systems. sync/types.py and async/types.py now just call the factory with their specific params and set get_node (sync vs async) via classmethod attribute assignment to avoid triggering __init_subclass_with_meta__.

Project requires Python 3.10+, so the guard around EnumField registration was never needed at runtime.

- Remove from __future__ imports (absolute_import, unicode_literals) — no-ops since Python 3 - Replace OrderedDict with plain dict in base/utils.py and base/types.py (dicts are insertion-ordered since Python 3.7) - Remove unused get_type_for_document() dead function from base/utils.py - Remove unused GraphQLResolveInfo import from base/utils.py - Remove unused LazyReference import from union_resolver.py - Rename UnionFieldResolver.resolver/resolver_async to reference_resolver/reference_resolver_async to match the naming convention used by ListFieldResolver and DynamicReferenceFieldResolver

Every public and private class, method, and module-level function now has a docstring that describes its purpose, all input arguments (with types), and return values — making the library self-documenting for new contributors. Files covered: base/utils.py, base/fields.py, base/field_resolvers/*, synchronous/fields.py, synchronous/types.py, asynchronous/fields.py, asynchronous/types.py.

…grammar Add noinspection comment for @classmethod applied outside a class body in the async factory function. Minor docstring grammar corrections (e.g. comma, spacing).

Two reference sections were missing from the README: - Meta options table covering all class Meta attributes with types and descriptions - MongoEngine → GraphQL field type mapping table

select_related pre-fetches GenericReferenceField and ListField(GenericReferenceField) at the queryset level, so the ThreadPoolExecutor / asyncio.gather in ListFieldResolver and the per-document fetch in UnionFieldResolver never ran in practice. DynamicReferenceFieldResolver is kept: select_related only reaches references directly off the root queryset; nested ReferenceField (e.g. editor → company) and references inside custom get_queryset return LazyReference objects that graphene cannot serialize without this fallback. 179/179 tests pass.

…es all pre-fetching All three field resolvers are now removed. The mongoengine pipeline_builder fix (passing embedded_list_path=full_path when recursing into ListField items) enables select_related("list_field__nested_ref") to correctly hydrate nested references inside list elements via $map rather than writing a _missing_reference marker via $addFields on an array. Additional fixes: - get_select_related_paths: unwrap Relay edges→node when recursing into connection sub-fields so nested refs (e.g. beforeChild→parent) are visible - get_queryset: apply _apply_select_related even when a custom get_queryset callback returns a QuerySet/AsyncQuerySet directly - Update stale count assertions to reflect that all referenced documents are now pre-fetched in a single aggregation (no lazy deref extra queries)

…_related applied

…n resolver When a resolver on the Query class returns a QuerySet to a connection field, chained_resolver now applies _apply_select_related before passing it to default_resolver. This means returning a queryset from any custom resolver automatically gets query-driven pre-fetching, consistent with get_queryset.

…nc support - Add built-in OpenTelemetry span emission for connection fields and Node lookups - Implement automatic select_related pre-fetching driven by GraphQL selection set - Add async Motor-backed QuerySet support via AsyncMongoengineObjectType - Add telemetry.py wiring to all four example applications - Expand test coverage for converter, telemetry, and relay queries - Update pyproject.toml with telemetry optional extra and dev dependencies

- Restructure flat RST pages into subdirectory layout matching graphene-python.org pattern - Split content into focused sub-pages: quickstart/, types/, prefetching/, relay/, telemetry/, examples/ - Use rubric directives for non-TOC headings to keep sidebar clean - Revert field type mapping from tables back to definition lists - Remove all uv references; use pip throughout - Move telemetry framework wiring into per-framework example pages - Add sphinx_graphene_theme zip URL to docs/requirements.txt

- Remove unused nullcontext import from telemetry.py - Remove unused service_name assignments in example telemetry files - Fix noqa directives in django settings_test.py - Add noqa: E402 for mid-file imports in test_relay_query files - Install tox and tox-uv via --with flag in CI test step

…ware instead Telemetry belongs at the GraphQL execution layer via graphene middleware, not in individual MongoEngine integration libraries. - Delete graphene_mongo/base/telemetry.py - Remove field_span and node_span wrappers from fields and types - Remove telemetry optional dependency from pyproject.toml - Remove telemetry.py from all four framework examples - Remove OpenTelemetry sections from example docs - Remove telemetry/ docs section - Bump mongoengine to v0.30.0-alpha.3 in pyproject.toml and tox.ini

- Push on all branches with concurrency cancellation - Add MongoDB 8.3 to test matrix - Switch linting to pre-commit via uv sync --only-group dev - Switch tests to uv sync --only-group test + uv run tox - Add tox and tox-uv to test dependency group - Add enable-cache to all setup-uv steps - Bump mongodb-github-action to 1.12.1 - Add MONGODB_URI env var support to conftest

- Bump mongoengine to v0.30.0-alpha.5 (ships ZonedDateTimeField) - Add ZonedDateTimeType / ZonedDateTimeInputType to advanced_types - Register ZonedDateTimeField converter → graphene.Field(ZonedDateTimeType) - ZonedDateTimeType resolvers handle both raw dict and to_python datetime - Filter args: ZonedDateTimeType maps to graphene.DateTime scalar - filter_fields operators (gte/lte/gt/lt/in/nin/all) transparently rewrite to the utc subfield via _hydrate_args (_to_utc handles lists too) - Fix construct_fields crash when Dynamic field is in non_required_fields (EmbeddedDocumentField inputs now honour non_required_fields correctly) - Fix _field_args None filter values from complex nested list fields Test suite (229 → 246 tests): - Rename test_deep_relay_query.py → test_relay_query_deep.py (grouping) - Harden mongo_capture.py: async with support, command_count property - Add test_nested_embedded_input_create (sync + async) - Add geo __near filter tests: arg existence + live $near query - Add filter_fields invalid lookup tests (schema ok, query fails) - Add enum field query/filter tests, empty pageInfo, projection tests - Add ZonedDateTimeField tests: query, exact filter, range, in operator - Add test_utils: self-referential and relay edge→node unwrapping paths - Add tests/README.md with full coverage map

mak626 · 2026-05-25T04:05:39Z

+
+
+@shareable  # Support Graphene Federation v2
+class ZonedDateTimeType(graphene.ObjectType):


It would be better if this is a scalar as follows.

The `AwareDateTime [Pydantic name]` scalar type represents a DateTime value with extended timezone information as specified by RFC 9557 (IXDTF). Example input/output: "2024-05-16T12:00:00+09:00[Asia/Tokyo]"

https://datatracker.ietf.org/doc/html/rfc9557

Timezone database [Based on IANA]
This repo is autoupdated
https://github.com/vvo/tzdb/blob/main/raw-time-zones.json
Use rawFormat for display purposes, and use name for timezone string

Javascript Native support
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Temporal/ZonedDateTime/from

// Inbuild support const zdt = Temporal.ZonedDateTime.from("2021-07-01T12:34:56-04:00[America/New_York]"); const legacyDate = new Date(zdt.epochMilliseconds); console.log(legacyDate.toISOString()); # For internal use

….30.0-alpha.6)

…DTF) AwareDateTimeField now serialises to a single RFC 9557 IXDTF scalar string: "2024-06-15T14:30:00+05:30[Asia/Kolkata]" The string carries the local wall-clock time, the correct UTC offset (DST-aware), and the IANA timezone annotation in brackets. JavaScript clients can parse it natively with Temporal.ZonedDateTime.from(). Filter inputs accept the full IXDTF format or any plain RFC 3339 string with a UTC offset; both are normalised to UTC before the MongoEngine query.

Extends the 10-level reference-chain test suite with three new field-type combinations not previously covered: 1. ListField(GenericReferenceField) on a Document (DeepL8.generic_refs) 2. ListField(ReferenceField) inside an EmbeddedDocument (DeepEmbedWithRef.list_refs) — exposed as a relay connection; pre-loaded via select_related path "embed__list_refs" 3. Nested EmbeddedDocumentField within EmbeddedDocumentField (DeepNestedEmbed inside DeepEmbedWithRef) — ref resolved via "embed__nested__ref_item" in a single aggregation pipeline Adds DeepNestedEmbed model and registers DeepNestedEmbedType / DeepNestedEmbedAsyncType before DeepEmbedWithRef* in nodes files so the EmbeddedDocumentField converter sees the inner type at class-creation time. All 256 tests pass.

mak626 · 2026-06-04T19:04:59Z

+    @staticmethod
+    def serialize(value):
+        """Serialise a stored AwareDateTimeField value to an IXDTF string."""
+        if isinstance(value, dict):


this must never occur, mongo provides only datetime objects to python layer. couple more fixes, i will raise a PR

mak626 · 2026-06-04T19:14:51Z

+        Temporal.ZonedDateTime.from("2024-05-16T12:00:00+09:00[Asia/Tokyo]")
+
+    Accepts as input:
+        - Full IXDTF string: "2024-06-15T14:30:00+05:30[Asia/Kolkata]"


We accept only Full IXDTF format.

Plain RFC 3339 string with offset: "2024-06-15T09:00:00+00:00" , frontend can pass +05:30 and our parser would make it UTC as fallback, which defeats the purpose and can cause confusions on usage

UTC string: "2024-06-15T09:00:00Z", this would have been fine but is not a valid IXDTF format to accept as input.

will raise a PR

arunsureshkumar force-pushed the feat/async-select-related branch from 5a1c1a7 to 728ad1d Compare May 24, 2026 15:27

arunsureshkumar added 29 commits May 24, 2026 20:58

chore: remove obsolete build/config files

081dfb0

Drop setup.cfg, MANIFEST.in, and README.rst — all superseded by the uv/ hatchling setup in pyproject.toml. Migrate coverage omit config from setup.cfg into [tool.coverage.run] in pyproject.toml. Add .venv/ to .gitignore.

refactor: move tests/ to project root

b85d840

Relocate graphene_mongo/tests/ → tests/ and convert all relative `from ..` imports to absolute `from graphene_mongo.` imports. Update Makefile and pyproject.toml paths accordingly.

chore: add tox.ini and clean up pyproject.toml

219cabc

- Add tox.ini with tox-uv runner targeting py310–py314 - Remove aiodataloader dependency (dataloader removed in previous commit) - Rename dependency group dev → test to match tox dependency_groups convention

chore: update uv.lock after removing aiodataloader and setuptools

026556a

chore: convert examples to uv (pyproject.toml)

a5fd03e

Replace requirements.txt + pytest.ini in each example with a single pyproject.toml. References graphene-mongo via path = "../.." and mongoengine via the git source for local development.

chore: remove unused imports

ca5849a

chore: remove dead sys.version_info >= (3, 6) guard in converter

f379438

Project requires Python 3.10+, so the guard around EnumField registration was never needed at runtime.

docs: remove RST markup from docstrings to fix warnings

dfe89be

chore: suppress false-positive classmethod warning and fix docstring …

236ab78

…grammar Add noinspection comment for @classmethod applied outside a class body in the async factory function. Minor docstring grammar corrections (e.g. comma, spacing).

docs: add Meta options and field type mapping tables to README

ce031ac

Two reference sections were missing from the README: - Meta options table covering all class Meta attributes with types and descriptions - MongoEngine → GraphQL field type mapping table

style: fix indentation in convert_field_to_list

580fcb6

arunsureshkumar added 7 commits May 24, 2026 20:58

docs: add How It Works section explaining query-driven pre-fetching

66b0f53

docs: clarify that get_queryset returning a QuerySet also gets select…

21c58af

…_related applied

chore: expand MongoDB version matrix to 4.4–8.0

34bba7a

fix: install ruff via --with flag in linting CI step

4e7f5ad

arunsureshkumar force-pushed the feat/async-select-related branch from 728ad1d to 4e7f5ad Compare May 24, 2026 15:28

arunsureshkumar force-pushed the feat/async-select-related branch from 66fc7b7 to ad93e4c Compare May 24, 2026 15:37

arunsureshkumar added 11 commits May 24, 2026 22:26

chore: remove accidental worktree submodule from index

b00b1f6

refactor: remove telemetry test files

67955e0

chore: bump actions/checkout to v6 and setup-uv to v8 for Node.js 24

73c83bd

fix: use full version tag astral-sh/setup-uv@v8.1.0

4ebd763

chore: add pre-commit hook with ruff

6aa18a0

chore: bump mongoengine to v0.30.0-alpha.4

546a2c2

chore: bump examples to mongoengine v0.30.0-alpha.5 and clean async docs

d3405a0

chore: bump tox.ini to mongoengine v0.30.0-alpha.5

298f3c3

mak626 reviewed May 25, 2026

View reviewed changes

arunsureshkumar added 3 commits May 25, 2026 12:41

chore: rename ZonedDateTimeField → AwareDateTimeField (mongoengine v0…

60c20de

….30.0-alpha.6)

mak626 reviewed Jun 4, 2026

View reviewed changes

fix: ensure extra_args handle unresolved federated types.

5d3c0e9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: N+1 elimination via select_related, async support, and docs overhaul#25

feat: N+1 elimination via select_related, async support, and docs overhaul#25
arunsureshkumar wants to merge 52 commits into
feat/async-mongoenginefrom
feat/async-select-related

arunsureshkumar commented May 24, 2026 •

edited

Loading

Uh oh!

mak626 May 25, 2026 •

edited

Loading

Uh oh!

mak626 Jun 4, 2026

Uh oh!

mak626 Jun 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants



		@shareable # Support Graphene Federation v2
		class ZonedDateTimeType(graphene.ObjectType):

Uh oh!

Conversation

arunsureshkumar commented May 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key changes

Test plan

Uh oh!

mak626 May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mak626 Jun 4, 2026

Choose a reason for hiding this comment

Uh oh!

mak626 Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

arunsureshkumar commented May 24, 2026 •

edited

Loading

mak626 May 25, 2026 •

edited

Loading

mak626 Jun 4, 2026 •

edited

Loading