Skip to content

CUDA: CPU fallback for GPUs the build has no code for (#1315), plus fixes from #1316-#1321 - #1324

Merged
cdcseacave merged 13 commits into
developfrom
fix/issue-1315-cuda-compat
Nov 3, 2026
Merged

cdcseacave merged 13 commits into
developfrom
fix/issue-1315-cuda-compat

Conversation

@cdcseacave

@cdcseacave cdcseacave commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Fixes #1315. Also supersedes #1316, #1317, #1318, #1319, #1321, #1322 and #1323, which reported the problems fixed here; thanks to @ConnorBaker and @Tobias-Fischer, who are credited as co-authors of the matching commits.

There is one commit per problem.

CUDA: use the CPU when the build has no code for the GPU (#1315, #1316)

Cause. The v2.4.0 Windows CUDA binaries contain native code only for sm_89, the build machine's GPU, and no PTX (cuobjdump). The default became CMAKE_CUDA_ARCHITECTURES=native in 2.4.0, and it was 75 in 2.3.0. On any other GPU, the first __constant__ upload fails with named symbol not found (code 500) and the application exits. CUDA_FORCE_PTX_JIT=1 reproduces it exactly on an RTX 4070. On develop the same run exits silently: errors raised in .cu files went to fprintf(stderr), which never reaches the log.

Fix.

  • CMake resolves the architectures being compiled (native, all, all-major or a list). initDevices() skips a device the build cannot run on, logs the rebuild command, and densification continues on the CPU.
  • OpenMVS_MAX_CUDA_COMPATIBILITY is removed. Distributed binaries pass -DCMAKE_CUDA_ARCHITECTURES=all-major instead. The SiftGPU port defaults to all-major.
  • "No CUDA device" is now an expected condition, not an assert. It covers Fall back to the CPU when no CUDA device is available #1316's sandbox case, and AllocateCudaPool() initializes the devices itself instead of probing through the PatchMatch constructor.
  • CUDA_CHECK logs the caller's file, line and expression to the log file.
  • PatchMatch stores its cameras in __constant__ byte storage in every build. Release used a Camera array, which nvcc 13.4 + GCC 16 reject. This was checked for speed: only the geometric-consistency kernels' SASS changes, and the last geometric pass got faster (fountain-P11, full resolution, 6 interleaved runs: median 3.13 s vs 3.77 s).
  • The helpers left over from the runtime-loaded PTX modules are removed.

Verified on an RTX 4070:

Build / run Result
sm_89 build uses the GPU
sm_75-only build CUDA device 0 (NVIDIA GeForce RTX 4070) has compute capability 8.9, but this build contains CUDA code only for sm_75; rebuild with -DCMAKE_CUDA_ARCHITECTURES=89 (or all-major), then the CPU densifies (exit 0)
CUDA_VISIBLE_DEVICES=-1 CPU, exit 0
CUDA_FORCE_PTX_JIT=1 logs PatchMatchCUDA.cu:715: cudaMemcpyToSymbolAsync(...)

Tests.exe 0, 1 and 2 pass, test 2 also with no device.

Review follow-up. An architecture-specific target (ex. 90a-real) now matches only its exact compute capability; family-specific f targets keep the same-major rule.

densify: DMapCache::UseImage() never ejects the depth-map it provides (#1321)

The ejection now stops at the requested depth-map, and UseImage() asserts it on return. This makes every caller safe without having to protect the image before loading it. The PR adds DMapCacheKeepsRequestedImageTest.

build: stop overwriting CMAKE_EXE_LINKER_FLAGS with -stdlib=libc++ (#1318)

The line is deleted: the flag is already in CMAKE_CXX_FLAGS, which CMake passes on every C++ link.

common: support OpenCV 5 (#1317)

These are #1317's changes, checked against the OpenCV 5.0.0 headers. OpenCV 5 also widens CV_CN_SHIFT from 3 to 5, so multi-channel cv::Mat type codes differ between OpenCV 4 and 5. The only untyped cv::Mat OpenMVS archives are the single-channel SfM descriptors, whose codes are the same in both versions, so scene files stay compatible.

build: make the installed CMake package usable and relocatable (#1319)

build/PackageTest is a minimal project using an installed OpenMVS, and CI now builds it against the install tree on all three platforms. It hit these problems, all fixed:

  • --prefix was ignored for the libraries: install directories are now relative.

  • The config was not found on Windows: it now installs to lib/cmake/OpenMVS everywhere.

  • No dependency was looked up. Each exported lookup now goes through find_exported_dependency(), which records it for the config, so the list cannot drift from the build.

  • The targets were missing the include directory, the C++ standard and OPENMVS_SHARED.

  • *.hpp and MVS/CUDA headers were not installed.

  • The Tests helper was exported.

  • The Python bindings were installed to /openmvs: they now go to the prefix's site-packages (Lib/site-packages on Windows, lib/pythonX.Y/site-packages elsewhere).

  • Clang builds use libc++: the installed Common target now passes -stdlib=libc++ to the projects using it.

Verified locally: a consumer built against cmake --install --prefix <scratch> loads a scene.

build: use the nlohmann-json package instead of a bundled copy (#1323)

libs/IO/json.hpp was nlohmann JSON 3.5.0 (2018), while vcpkg already builds 3.12.0 for tinygltf. IO now links nlohmann_json::nlohmann_json, also recorded in the installed package, and the bundled copy is deleted. Package managers such as conda-forge use their own copy with no option to set, and only one JSON version is ever compiled in.

viewer: support GLAD 2 alongside GLAD 1 (#1322)

GLAD 1 is unmaintained and missing on some platforms (ex. conda-forge linux-aarch64), but vcpkg ships only GLAD 1, so both are supported. A GLAD 2 CMake package (glad_add_library, gl:compatibility=4.6 so the GL 4.3 debug output stays available) is preferred when found. Common.h is the only place choosing the header and the loader call; the other Viewer headers get OpenGL through it. The GLAD 1 path is built and started locally and in CI; the GLAD 2 path follows #1322, which was tested in a stand-alone project.

viewer: Linux install fixes

  • The update-mime-database install step broke cmake_install.cmake (missing newline between commands); it now also runs in $DESTDIR and the prefix given at install time.
  • The .desktop entry's Exec path is configured at install time, so it points at the binary under the actual --prefix.

ci

  • The Windows package test passes the vcpkg and install paths quoted (bash dropped the backslashes).
  • vcpkg moved to the 2026.10.16 release.

🤖 Generated with Claude Code

cdcseacave and others added 5 commits November 3, 2026 00:02
Release binaries built with the default CMAKE_CUDA_ARCHITECTURES=native carry
native code only for the GPU of the build machine (v2.4.0: sm_89, no PTX), so
on any other GPU the first kernel or __constant__ access failed mid-run with
"named symbol not found (code 500)" and the application exited.

- CMake resolves the architectures the CUDA code is compiled for (native,
  all, all-major or an explicit list) and passes them to initDevices(), which
  skips a device it cannot run on: native code needs the same major version
  and an equal or lower minor one, PTX an equal or lower version. When no
  device is left the log says why and how to rebuild, and densification
  continues on the CPU.
- OpenMVS_MAX_CUDA_COMPATIBILITY is removed: distributed binaries pass the
  standard -DCMAKE_CUDA_ARCHITECTURES=all-major (or set CUDAARCHS) instead.
  The SiftGPU port defaults to all-major, as vcpkg builds it once for every
  machine the binaries run on.
- No usable CUDA device is an expected condition, not an error: initDevices()
  asks the runtime for the device count instead of the asserting cuInit()
  (the runtime initializes the driver), and AllocateCudaPool() initializes
  the devices itself instead of probing through a PatchMatch constructor that
  then asserts a device exists.
- CUDA_CHECK() reports through the log, also from .cu files (their fprintf to
  stderr never reached the log, so develop exited silently), with the
  location and the expression of the failed call instead of the header's.
  The driver-API reporter initializes the strings cuGetErrorName/String leave
  untouched for an unknown code.
- PatchMatch keeps its cameras in __constant__ byte storage in every build
  (Release declared a Camera array, which nvcc 13.4 + GCC 16 reject as
  dynamic initialization); the SASS only differs in the geometric-consistency
  kernels, and the last geometric pass is faster (fountain-P11, full
  resolution, 6 interleaved runs: median 3.13 s vs 3.77 s, other passes equal).
- Remove the helpers left from the runtime-loaded PTX modules: the arch hint
  block, ensureCudaResult, align, MemDevicePtr/MemDeviceArr, stream smart
  pointers and the unused ArrayRT accessors and typedefs.

Includes the fixes proposed in #1316.

Co-authored-by: Connor Baker <ConnorBaker01@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UseImage() loads a depth-map, then ejects the least recently used ones until
the cache fits its budget; with a budget smaller than that depth-map the
ejection reached the one just loaded, so the caller got a released map (a
crash in fusion on 2.4.0, a silently skipped reference since the IsEmpty()
check). The ejection now stops at the requested depth-map, which stays loaded
even if it alone exceeds the budget, and UseImage() asserts it on return, so
every caller is safe without protecting the image beforehand.

Adds DMapCacheKeepsRequestedImageTest. Supersedes #1321, which reported it.

Co-authored-by: Tobias Fischer <info@tobiasfischer.info>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With Clang the executable linker flags were replaced by "-stdlib=libc++",
dropping any the user or the toolchain had set (ex. conda's), so two C++
runtimes could end up in the same process. The flag is redundant there: it is
already in CMAKE_CXX_FLAGS, which CMake passes on every C++ link.

Supersedes #1318, which reported it.

Co-authored-by: Tobias Fischer <tobias.fischer@qut.edu.au>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- OpenCV 5 adds the unsigned depths CV_32U and CV_64U and defines
  cv::DataType<unsigned> and cv::DataType<uint64_t> itself; the OpenMVS
  specializations (mapping them onto CV_32S and CV_64F) are now defined only
  for OpenCV < 5.
- OpenCV 5 moved DistanceTypes (cv::DIST_L2 used by ImportROMA2) into the new
  geometry module, which opencv2/opencv.hpp does not include.

Checked against the OpenCV 5.0.0 headers (core/traits.hpp, core/hal/interface.h,
geometry/2d.hpp, opencv.hpp); nothing changes for OpenCV 4. Note that OpenCV 5
widens CV_CN_SHIFT from 3 to 5, so the type code of a multi-channel cv::Mat
differs between the two versions; the only untyped cv::Mat OpenMVS archives
are the single-channel SfM descriptors, whose codes are the same in both, so
the scene files stay compatible.

Supersedes #1317.

Co-authored-by: Tobias Fischer <tobias.fischer@qut.edu.au>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A project using an installed OpenMVS (build/PackageTest, now built by the CI
against the install tree) failed at every step:
- `cmake --install --prefix` was ignored for the libraries: the install
  directories were made absolute from CMAKE_INSTALL_PREFIX at configure time.
  They are now relative to the prefix, resolved at install time.
- find_package(OpenMVS) did not find the config on Windows, installed in
  <prefix>/CMake/OpenMVS; it is now in lib/cmake/OpenMVS on every platform,
  and OpenMVSConfig.cmake computes its paths from its own location.
- The config looked up none of the packages the exported targets link
  against. Each such lookup now goes through find_exported_dependency(), which
  records it (with the <Package>_ROOT hint given to the build) for the config,
  so the consumer finds exactly what OpenMVS was built with, whatever the
  options, instead of a hand-maintained list that drifts; the gflags binding
  the ceres target needs is recorded the same way. PNG, JPEG and TIFF are
  linked through their imported targets instead of absolute paths.
- The exported targets carried no include directory, did not require the C++
  standard the headers were compiled with (the BS::thread_pool layout depends
  on it) and did not define OPENMVS_SHARED, which the headers need to import
  the symbols of the shared libraries; Common now exports all three.
- The headers were incomplete: *.hpp (BS_thread_pool.hpp, json.hpp) and the
  MVS/CUDA headers were not installed.
- The Tests helper is no longer exported: installers leaving it out made the
  imported targets fail to load.

OpenMVS_LIBRARIES and OpenMVS_BINARIES now name the namespaced targets.

Supersedes #1319, which reported the missing dependencies and include
directory.

Co-authored-by: Tobias Fischer <tobias.fischer@qut.edu.au>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Installed targets omit required libc++ flags, risking consumer ABI and link failures, while stripped CUDA architecture qualifiers can admit incompatible GPUs and cause aborts instead of CPU fallback.

2 open issues
What changed in this PR

This PR improves OpenMVS portability and reliability by adding CPU fallback for unsupported CUDA devices and addressing the build, packaging, and depth-map cache issues reported in #1315–#1321.

Changes:

  • Detect CUDA architecture compatibility before device selection, improve error logging, and simplify camera constant storage.
  • Keep requested depth maps loaded under cache pressure, with a regression test.
  • Add OpenCV 5 compatibility, preserve linker flags, and improve installed CMake packages with a CI consumer build.
File Description
ports/​siftgpu/​source/​CMakeLists.txt Default distributed SiftGPU builds to all-major.
libs/​SFM/​CMakeLists.txt Record exported SfM dependencies.
libs/​MVS/​SceneDensify.h Update CUDA pool documentation.
libs/​MVS/​SceneDensify.cpp Initialize CUDA before pool creation and log CPU fallback.
libs/​MVS/​PatchMatchCUDA.cu Use aligned byte storage for constant cameras in every build.
libs/​MVS/​PatchMatchCUDA.cpp Require CUDA initialization before construction.
libs/​MVS/​DMapCache.h Export the cache class and declare eviction protection.
libs/​MVS/​DMapCache.cpp Prevent eviction of the requested depth map.
libs/​MVS/​CMakeLists.txt Record dependencies and install CUDA headers.
libs/​Math/​CMakeLists.txt Record optional SuiteSparse dependencies.
libs/​IO/​CMakeLists.txt Use imported image-library targets and install .hpp headers.
libs/​Common/​UtilCUDADevice.h Route runtime errors through centralized logging.
libs/​Common/​UtilCUDA.h Initialize error strings and remove unused helpers.
libs/​Common/​UtilCUDA.cpp Filter incompatible GPUs and handle unavailable devices.
libs/​Common/​Types.inl Avoid conflicting OpenCV 5 type specializations.
libs/​Common/​Types.h Include OpenCV 5 geometry definitions.
libs/​Common/​CMakeLists.txt Export consumer requirements and embed CUDA architecture metadata.
docs/​wiki/​Building.md Document CUDA targeting and installed-package usage.
CMakeLists.txt Resolve CUDA targets and generate relocatable package configuration.
build/​Utils.cmake Preserve linker flags and record exported dependencies.
build/​Templates/​OpenMVSConfig.cmake.in Resolve dependencies and expose namespaced targets.
build/​PackageTest/​main.cpp Exercise installed MVS and SfM libraries.
build/​PackageTest/​CMakeLists.txt Add a minimal installed-package consumer.
apps/​Tests/​TestsMVS.h Declare the cache regression test.
apps/​Tests/​TestsMVS.cpp Test requested-map retention under cache pressure.
apps/​Tests/​Tests.cpp Register the cache regression test.
apps/​Tests/​CMakeLists.txt Stop exporting the test executable.
.github/​workflows/​continuous_integration.yml Build the package consumer against the install tree.

Review effort: Balanced

# BS::thread_pool layout) and on the library type: tell them we are building shared
# libraries, so they pick dllimport and the cross-DLL-friendly path for things like
# boost exception-unwinding (see Types.h)
TARGET_COMPILE_FEATURES(Common PUBLIC cxx_std_${CMAKE_CXX_STANDARD})

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 56495bf: when built with Clang, the installed Common target now carries -stdlib=libc++ as an INSTALL_INTERFACE compile option (C++ only) and link option, so the projects using the package pick the same standard library. The build tree already gets it from CMAKE_CXX_FLAGS.

Comment thread CMakeLists.txt Outdated
Comment on lines +250 to +251
if(NOT CMAKE_MATCH_3 STREQUAL "-virtual")
LIST(APPEND _CUDA_ARCHS_SASS ${CMAKE_MATCH_1})

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 40df18d: architecture-specific targets (ex. 90a-real) now go to a separate _CUDA_ARCHS_SASS_EXACT list and _isComputeCapabilitySupported() accepts them only for the exact compute capability. Family-specific f targets keep the same-major, equal-or-higher-minor rule, and suffixed PTX was already excluded from the forward-compatible PTX.

cdcseacave and others added 4 commits November 3, 2026 03:01
An `a` target (ex. 90a-real) was recorded as generic native code, so
initDevices() accepted any GPU of the same major version with an equal or
higher minor, which cannot load it. Architecture-specific SASS is now
listed separately and matched exactly; family-specific (`f`) targets keep
the generic same-major rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PYTHON_INSTALL_PATH was never set, so `cmake --install` tried to write the
bindings to /openmvs. It now defaults to the site-packages folder of a
Python installation or virtual environment, relative to the prefix.

Clang builds use libc++, so the installed Common target now tells the
projects using OpenMVS to compile and link against it as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
libs/IO/json.hpp was nlohmann JSON 3.5.0 from 2018, while vcpkg already
builds 3.12.0 for tinygltf. IO now links nlohmann_json::nlohmann_json
(recorded as a dependency of the installed package) and the bundled copy
is removed, so package managers use their own copy with no option to set.

Co-Authored-By: Tobias Fischer <info@tobiasfischer.info>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GLAD 1 is unmaintained and missing on some platforms (ex. conda-forge
linux-aarch64), while vcpkg ships only GLAD 1. The Viewer now prefers a
GLAD 2 CMake package (glad_add_library, gl:compatibility=4.6 so the GL 4.3
debug output stays available) and falls back to GLAD 1. Common.h is the
only place choosing the header and the loader call; the other Viewer
headers get OpenGL through it.

Co-Authored-By: Tobias Fischer <info@tobiasfischer.info>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cdcseacave and others added 4 commits November 3, 2026 03:26
The install(CODE) script put execute_process() and if() on one line, which
CMake rejects, so `cmake --install` failed on every Linux build with the
Viewer. It also ran in the configure-time prefix; it now uses the prefix
and DESTDIR given at install time, and belongs to the desktop component.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tall prefix

The .desktop Exec path was configured with the configure-time
CMAKE_INSTALL_PREFIX, so installing with --prefix left the menu entry
pointing to a binary that is not there. The file is now configured at
install time, with the prefix given then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bash drops the backslashes of an unquoted Windows path, so the package
test could not find the vcpkg toolchain on the Windows runner; pass the
paths through quoted shell variables instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cdcseacave
cdcseacave merged commit 01c4e5e into develop Nov 3, 2026
9 checks passed
@cdcseacave
cdcseacave deleted the fix/issue-1315-cuda-compat branch November 3, 2026 08:11
cdcseacave added a commit that referenced this pull request Nov 6, 2026
The user's request of 2026-11-05: "next merge latest remote develop, remove all chached results / features / matches, etc, and rerun all test datasets as there were made improvements to SiftGPU" (T/briefs-20261017/develop-merge-20261106.md).
develop brings 17 commits since 9fabec1: SiftGPU parallel decoding, weak-texture detector, moment orientation and adaptive grid selection (#1320), the CUDA CPU fallback (#1324), MVS refine/texture/split and mesh algorithms moved to halfmesh (#1314, #1299, #1300, #1302), viewer (#1310), densify (#1309, #1298), SGM (#1306, #1305), scene split (#1304), the MSVC build split of Scene.cpp (#1303), InterfacePolycam output paths (#1245), TIFF write (#1246), and C++17 minimum with vcpkg tinyxml2/nlohmann-json (#1301).

Conflicts (one line each in T/records-2026-11-06-merge/resolution.md):
- vcpkg.json: tinyxml2 added; tinynpy stays out (this branch removed the NPZ import at 1380970).
- libs/SFM/CMakeLists.txt: develop's find_exported_dependency for TinyEXIF, PoseLib, Ceres; no TinyNPY.
- libs/SFM/StarInitializer.cpp: this branch's ComputePairScales kept (develop edited the inline trimmed mean that ComputePairScales replaced; same statistic, so the result is unchanged).
- libs/SFM/FeaturesExtractor.cpp: develop's adaptive SiftGPU selection (SelectFeatures), followed by this branch's ReleaseFeatures (clears the dense-keypoint boundary).
- libs/Common/List.h: this branch's std::vector constructor without the removed _SUPPORT_CPP11 guard; both cListTest additions.
- apps/CreateStructure/CreateStructure.cpp: develop's --siftgpu-darkness-adaptivity and --siftgpu-cuda options with this branch's --match-mode 4 / --match-sequence-overlap text; --siftgpu-cuda gates SiftGPU only, and RoMa2's GPU switch stays as it was.
- apps/InterfacePolycam/InterfacePolycam.cpp: develop's output-path handling (a superset of this branch's ensureValidPath fix).
- apps/Tests/Tests.cpp, TestsMVS.cpp: both sides' tests (PlatformArrGrowOwnershipTest and develop's MeshRefineStep, window-stats, clean-per-vertex and image-resize tests).
- docs/design/HierarchicalSFM.md: this branch's content placed in develop's 7-section design-record layout.
- docs/design/SphericalCameraSupport.md: develop's rewrite.
- docs/pipeline-traces.md: deleted, as develop did (this branch's edits to it are already in docs/pipelines.md).
Clean merges that needed a fix because develop removed a definition this branch's new code uses: MAYBEUNUSED -> [[maybe_unused]] (libs/SFM/SceneBlocks.cpp); ../IO/json.hpp -> <nlohmann/json.hpp> (libs/SFM/RoMa2Matcher.cpp, apps/Tests/TestsSFM.cpp).

Verification: Release build (Ninja Multi-Config; the reconfigure installed tinyxml2, nlohmann-json, halfmesh 0.5.1 and develop's siftgpu port) has no warning in libs/SFM (40 of 40 SFM objects rebuilt; the 4 warnings in the build are develop's -Wclass-memaccess in libs/MVS/MeshHalfMesh.cpp). Release ctest passes 3 of 3 (SFMPipelineTest 313.7 s). Debug build with ASSERTs on (a dev copy of the merge tree with a temporary test dispatcher, never committed; T/records-2026-11-06-merge/debug-tests.txt): 39 of 40 selected SFM tests pass with no ASSERT (StarEvidence, StarReferenceView, TripletStarInit, NearDuplicateYardstick, PairScales, DenseKeypointBoundary, FundamentalPoseTwoCameras, the ROMA2/RoMa2, spherical, CubeMap and SceneBlocks tests, ...), and so do the unit tests (cListTest, PlatformArrGrowOwnership, develop's MVS tests). The one failure, ROMA2ReconstructTest, ran out of GPU memory while another job held 35 GB of the GPU; its rerun and the smoke run are in T/records-2026-11-06-merge/smoke.log. The SceneAnalyzeSFM binary, run against the merged libraries, writes byte-identical CSVs for b68e320e dup-20261105-base. Snapshot T6/bin-merge-4eaec0b-develop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DensifyPointCloud v2.4.0: CUDA error "named symbol not found (code 500)" on GTX 1660 Ti (compute 7.5) — works on v2.3.0

2 participants