Skip to content

Fix memory leak and add Cython 3.0 support - #8

Open
niko-zvt wants to merge 1 commit into
poine:masterfrom
niko-zvt:memory-leak-and-cython3-support
Open

Fix memory leak and add Cython 3.0 support#8
niko-zvt wants to merge 1 commit into
poine:masterfrom
niko-zvt:memory-leak-and-cython3-support

Conversation

@niko-zvt

Copy link
Copy Markdown

Summary

This PR fixes two critical issues in pysbpl:

  1. Memory leak (~500 MB per operation) caused by missing destructors
  2. Cython 3.0 incompatibility causing compilation failures

Both fixes must be applied together because the memory leak fix adds C++ operators that require the Cython 3.0 migration to compile correctly.


Problem 1: Memory Leak

Issue

Each pathfinding operation leaks ~300-500 MB of virtual memory because C++ objects are created with new but never freed. Over thousands of operations, this accumulates 10-25 GB and eventually crashes with MemoryError.

Root Cause

Missing __dealloc__() methods in Cython wrapper classes:

  • EnvironmentNAVXYTHETALAT: Leaks C++ environment object (~300 MB) and malloc'd mapdata
  • ARAPlanner: Leaks C++ planner object (~200 MB)

Solution

Added proper __dealloc__() destructors following Cython documentation.

Problem 2: Cython 3.0 Incompatibility

Issue

The setup.py uses deprecated Cython.Distutils.build_ext API from Cython 0.x, which fails to process the # distutils: language = C++ directive correctly. This causes compilation errors:

Error compiling Cython file:
pysbpl/sbpl.pyx:68:23: Operation only allowed in c++

Root Cause

Cython.Distutils is deprecated since Cython 3.0. The C++ language directive is ignored, causing compilation in C mode. C++ operators (new, del) are only available in C++ mode.

Solution

Migrated to modern Cython.Build.cythonize() API:


Backward Compatibility

100% backward compatible for users:

  • No API changes
  • No changes to method signatures
  • No changes to public interfaces
  • Existing code continues to work identically

Build requirement:

  • Requires Cython == 3.0.11 for compilation

List of changes:

  • Migrate to Cython 3.0.11 and modern cythonize() API
  • Add pyproject.toml to fix build isolation issues
  • Implement proper dealloc() methods to prevent memory leaks
  • Remove deprecated Cython.Distutils and -Wstrict-prototypes workaround
  • Update to Python 3 syntax (print statements)
  • Add macOS troubleshooting for rpath issues
  • Bump version to 0.0.2

- Migrate to Cython 3.0.11 and modern cythonize() API
- Add pyproject.toml to fix build isolation issues
- Implement proper __dealloc__() methods to prevent memory leaks
- Remove deprecated Cython.Distutils and -Wstrict-prototypes workaround
- Update to Python 3 syntax (print statements)
- Add macOS troubleshooting for rpath issues
- Bump version to 0.0.2
@niko-zvt

Copy link
Copy Markdown
Author

@poine @schmittlema
Hi everyone, I ran into a memory leak issue when running the library in multiprocessing mode with hundreds of calls. I’ve implemented a fix that resolves the problem in my tests. I’d really appreciate it if you could take a look - happy to make any adjustments if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant