Repository navigation
Refine/texture/split improvements, one-command large-scene pipeline, mesh algorithms moved to halfmesh - #1313
Conversation
…the sample The default image-gradient stencil (central differences, OPTREFINE::nImageGradient 1) is linear, so it commutes with the bilinear interpolation: differencing image B sampled one pixel either side of the warped point equals sampling the precomputed stencil image. Both backends now compute it at the sample (MeshRefine::CentralGradient, CUDA computePhotoPixel) and no longer build or keep the two per-view float gradient images; only the stencils that do not commute (0 = 3x5, 2 = Sobel) still precompute them. Same-input bench (EPFL Herz-Jesu-P8 / fountain-P11 / Herz-Jesu-P25, CUDA): F1 0.5186 / 0.3490 / 0.6457 vs 0.5174 / 0.3479 / 0.6473 (float rounding re-rolls the stepper trajectory, mean +0.0002), host peak RAM 1.28 / 1.64 / 2.51 GB vs 1.60 / 2.17 / 3.41 GB; device memory per view 21 -> 13 B/px. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ComputeWindowStats built six full-image products and their 7x7 box sums for every pair-direction, and the CPU refinement runs one pair-direction per thread, so the per-pair scratch was multiplied by the thread count. The sums are now built per band of 64 rows plus the window halo: a band row's window reads only rows of its band. Gate rejections leave the mask only after the next band has summed its halo rows, so every sum is taken over the warp mask as before (the result differs only by float rounding). Same-input bench, CPU backend, Herz-Jesu-P8: peak RAM 4.1-4.3 GB vs 8.3-9.1 GB, F1 0.5171 / 0.5188 vs 0.5185 / 0.5180 (run-to-run order noise of the CPU path), wall unchanged within noise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ot twice it ListCameraFaces rendered every view at twice the full image size, so each OpenMP thread held a face map and a depth map of 4x the image pixels (8 bytes each). One raster pixel per image pixel is what the view's texture resolution is anyway. Hold-out texture bench (bench/eval_texture.py, every 4th view left out of texturing and scored, measured together with the view-selection term of the next commit): PSNR HJ8 26.552 vs 26.582, F11 28.795 vs 28.785, HJ25 24.517 vs 24.509 dB; TextureMesh peak RAM HJ25 3.0 vs 5.8 GB, F11 2.2 vs 2.5 GB. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to the face's best The view-selection data term was the blurred image-gradient magnitude summed over the face's pixels times cos^2 of the view angle, normalized by one global 95th percentile. It is now (as in Acute3D's FinishMesh): - the number of the face's pixels the view sees, i.e. the texture resolution the view offers (already falling with distance and obliqueness, so no cos^2 factor); - divided by 10 for a view that sees under 90% of the face's projection (with more than 2 pixels missing): such a view would copy the occluder into the face's texture; - normalized per face by its best view (cost 0), and views below 0.2 of the best are not candidates. The per-view Sobel + 15x15 Gaussian gradient image is no longer computed. Hold-out texture bench (every 4th view left out of texturing and scored, sharpening off, same mesh): PSNR / SSIM HJ8 26.58 / .9231 vs 26.38 / .9205, F11 28.79 / .9451 vs 28.66 / .9438, HJ25 24.51 / .8988 vs 24.39 / .8938, Ignatius (in the T&T crop) 17.47 / .7527 vs 16.89 / .7458. Each component alone is worse than the gradient term except the partial-visibility penalty; the combination wins on all four. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…f the empty colour The texels no patch covers were left at the empty colour (orange by default), which a mipmapped lookup averages into the texels along every patch border once the texture is minified. They are now filled by push-pull over a box pyramid of the covered texels (each empty texel takes the normalized average of the coarsest covered neighbourhood around it); covered texels never change, and the unseen faces' patch keeps the empty colour. The atlas log line also reports the total patch pixels. Hold-out texture bench, HJ8, rendered at 1/8 with mip level 3: PSNR 26.716 vs 26.675 dB, SSIM .9051 vs .9044; identical at full resolution. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TextureMesh sharpened every atlas (--sharpness-weight 0.5, unsharp mask sigma 1.5), which moves the texture away from the photos it was taken from: its gradient energy rose to 1.25-1.36x the photos' against 1.08-1.17x unsharpened. The default is now 0 (the option stays for a deliberate look). Hold-out texture bench (every 4th view left out of texturing and scored), default vs no sharpening: PSNR / SSIM HJ8 25.84 / .9088 vs 26.01 / .9198, F11 28.25 / .9332 vs 28.40 / .9429, HJ25 24.15 / .8811 vs 24.24 / .8926, Ignatius 16.81 / .7234 vs 16.89 / .7458 -- every scene and every in-sample cell too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe TextureMesh as implemented: full-resolution face-view raster, visible-pixel view quality with the partial-visibility penalty, per-face normalized LBP labeling, seam leveling, packing with push-pull gutter fill; parameters, invariants, the hold-out validation table and the rejected alternatives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…els^2) TextureMesh's view labeling is loopy belief propagation with a Potts smoothness term, whose messages the generic loop computed over every (sender label, receiver label) pair, re-summing the sender's incoming messages for each. Under Potts the sender energy of label k is one of two values, E0[k] (same label, smoothness 0) or E1[k] (smoothness pottsCost), so the message of receiver label j is min(E0[k=j], min_k E1[k]); LBPInference::SetSmoothCostPotts selects this path. The energies are summed in the generic loop's order, so the messages are bit-identical. Ignatius (591k faces, 263 views): labeling 12.3 s vs 42.7 s. The view-selection stage now logs its face-view listing and labeling times separately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scene::Split now takes a memory budget (GB) instead of a sampling area: the depth-map samples are bisected recursively into K equal parts (longest axis, quantile floor(K/2)/K) and K is the smallest count whose largest sub-scene's predicted fusion and Delaunay memory fits, with a cost model fitted on measured runs. DensifyPointCloud --sub-scene-area becomes --sub-scene-memory (<0: 85% of the physical memory). MvsMergeChunks.py merges the sub-scene meshes by face centroid in each sub-scene's box. T&T Church, 12 GB: 4 sub-scenes peaking 10.0-11.5 GB (fusion), mesh F1 0.6085 vs 0.5967 for the area split's 5 sub-scenes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each patch is packed by its footprint (its faces rasterized in the patch rectangle plus the 2 px border) on a 4x4-pixel block grid: bottom-left first fit over per-row free intervals, largest first, either orientation, the search for a footprint size resuming where the last one of that size stopped. The page width is tried around the square root of the footprint area and the page is cropped to the extent used (rounded to a power of two or the multiple), so pages need not be square; only footprint pixels are copied and push-pull fills the rest. Hold-out runs: atlas HJ8/F11/HJ25/Ignatius 64/64/64/64 -> 32/32/32/32 MTexels, PSNR equal at full resolution and +0.01..+0.03 dB at mip 3, atlas stage time unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ListCameraFaces counted the visible pixels (and summed their colours) of every face of a view inside the critical section shared by all the view threads; the counts now go to per-thread arrays and only the faces the view sees are appended to the shared face data, inside the critical section. Same face data; Ignatius TextureMesh wall 51.5-51.8 s -> 49.0-49.7 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pair-directions of an evaluation are issued grouped by reference view: each adds its per-pixel photometric terms into the view's pixel sums and sets its bit (up to 32 directions per group), then the faces the view owns are accumulated once, instead of once per direction. A vertex counts the distinct directions its faces bring through a 64-bit (group, bits) stamp, so photoCount is unchanged; only the float summation order moves. Per-evaluation time -20% (F11), -28% (HJ25), -52% (Truck) at the finest scale, Truck wall 82.8 -> 71.5 s; identical accept/reject sequences, F1 equal (Truck 0.6681/0.6682 under one alignment, EPFL within 0.0005). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rasterizer now writes only the face map. The reference view's depth map is recomputed from its face map once per accumulation group into one scratch buffer (kernelFaceDepth), and the partner view's depth at the warp tap from the face there (pixelDepth: the rasterizer's projection and pixelBary). Output byte-identical (HJ8, Truck); device memory Truck 8.85 -> 6.86 GB (-4 bytes per pixel per view), wall 66.5 -> 68 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ence LBPInference kept two std::vectors of messages per directed edge, grown one label at a time; the messages now live in two flat buffers sized once the labels are final (same arithmetic, same order). FaceViewSelection releases each face's view list as soon as its data costs are set, so the face-view lists and the inference are never held together. TextureMesh peak RAM Ignatius 3.87 -> 3.44 GB, Truck 2.95 -> 2.73 GB; wall -4..-5 %; hold-out PSNR unchanged (HJ8 26.553, F11 28.797). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no scale the refinement loop never prepares a view, and the final simplification then read the unprepared views (segfault on both backends). Both entry points now reject nScales == 0 with an error, like the other externally controlled knobs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The crash came from the post-refinement simplification, which ran after the scale loop unconditionally and projected the mesh into views that only a scale prepares (InitImages). With no scale nothing is refined and nothing is prepared, so the mesh is now returned as it came in, as it was before the simplification step existed; the views contract is asserted in ListCameraFaces on both backends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the library started returning true after estimating the depth-maps only, the app took the success branch and saved scene_dense.mvs/.ply holding whatever point-cloud the scene had (the sparse one, or the virtual cloud of a cameras-only scene) as if it were the dense result. The app now stops after the depth-maps on success, and fails only on a failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sysinfo's freeram leaves the reclaimable page cache out and reads near zero on any machine that has been reading files, while Windows and macOS report the available memory. The fusion caches and the sub-scene budget size themselves from it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…needed MvsScalablePipeline.py scene.mvs now runs the whole dense chain: the depth-maps, the memory-budgeted split, per sub-scene fusion/mesh/refine, the merge of the point-clouds and meshes, and the texturing of the merged mesh with all the images. The budget defaults to this machine: the available physical memory less a reserve of 10% of the total (at least 2 GB). A scene that fits is not split and goes through the modules as an unsplit run would (its fusion from the saved depth-maps is byte-identical). MvgMvsPipeline.py runs its dense steps through it by default (--sub-scene-memory off restores the plain calls). Scene::Split returns false on no depth-map samples and leaves the chunks empty when the scene fits; DensifyPointCloud then exits successfully. MvsMergeChunks.py merges point-clouds too, keeping every vertex property (the per-point view lists included). README, wiki Usage (Large Scenes) and the design document describe the pipeline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, PushPullFill) The SHA512 is a placeholder until the v0.5.0 tag is published: vcpkg reports the archive's hash on the first install, which goes here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The footprint packer and the push-pull fill moved to halfmesh (PackFootprints, PushPullFill), -250 lines here. A packed patch is now turned 90 degrees counter-clockwise instead of transposed, so the UV winding is preserved; pixel (x,y) maps to (y,w-1-x). Same atlas sizes and texture quality (Ignatius and HJ8: 32 MPixel pages, HJ8 PSNR 26.598 vs 26.599 dB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s normal SmoothNormalFaces normalized the zero sum of admissible neighbour normals (an isolated face, or one on a crease sharper than the gradient on every side) into a NaN normal, which view selection then compared against. Such a face now keeps its own normal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ries removed Mesh::Subdivide (the refinement's per-scale 1-to-4 split with red-green closure) and Mesh::SamplePoints moved to halfmesh (SubdivideFaces, SamplePoints); the CUDA refine output and a seeded --sample-mesh cloud are byte-identical through the move. GetEdgeFaces, GetFaceFaces, GetEdgeAdjacentFace, GetAdjVertexFaces and the disabled angle-weighted vertex normals had no caller and are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…halfmesh LoadOBJ/SaveOBJ, IsWatertight, ComputeVolume, Join and SubMesh delegate to halfmesh 0.5.0 through MeshHalfMesh.cpp; libs/IO/OBJ is deleted. - OBJ: 26 M faces save in 0.82 s instead of 29.8 s and load in 0.99 s instead of 30.2 s; PLY -> OBJ -> PLY is bit-exact (the old writer kept six decimals), the old reader failed on its own untextured output, polygons, v//vn and relative indices; textures follow --export-texture-lossless. - IsWatertight is const and checks closed, edge-manifold, consistent orientation, without building adjacency caches. - ComputeVolume(plane): TransformScene --compute-volume measures an open mesh against its ground plane by prisms (fill and cut), so its boundary need not lie exactly on the plane. - Join keeps textures; SubMeshes converts the mesh once for all chunks (Save(chunks), SplitMeshPerTextureBlob). - The halfmesh port depends on fast-float. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cdcseacave
left a comment
There was a problem hiding this comment.
Summary
This branch moves the OBJ codec, volume, watertightness, join, sub-mesh, subdivision, point sampling, and texture-footprint packing onto halfmesh 0.5.0, replaces the area-based scene split with a memory budget, and adds a Python driver that fuses, meshes, refines, merges, and textures a scene that does not fit in one machine. CUDA refinement accumulates once per reference view, and the LBP Potts path uses flat messages. The numeric cores that the new tests cover line up with those tests. The defects that remain are in the chunk merge, the halfmesh port hash, leveled-volume reporting, Mesh::Join on an empty mesh, texture-blob indexing, and the scalable driver when a later step is run without the mesh stage.
Issue counts by severity
- bugs: 6
- suggestions: 3
- nits: 1
| if faces is None: | ||
| # a point belongs to the box it lies in | ||
| local = xyz @ rot.T | ||
| keep = np.all((local >= lo) & (local < hi), axis=1) |
There was a problem hiding this comment.
[bug] The keep test is half-open on the upper side (local < hi for points, and the same test on face centroids at line 179). A shared cut is stored as the left chunk's maximum and the right chunk's minimum (SceneSplit.cpp:202), so an element on that plane is kept by the right chunk. The root box's maximum face on each axis has no neighbor. Points and face centroids that sit exactly on that outer face match no chunk and are dropped. ExportChunks stores the box as an identity ROI (SceneSplit.cpp:304), and OBB::Intersects is closed, so those elements are inside the region the C++ side keeps. A wall or a dense point that lies on the scene maximum of X, Y, or Z disappears from the merged result. docs/design/SceneSplit.md describes the half-open rule as sending a boundary element to exactly one box.
Suggestion: Close the upper bound on any side that is the root box's maximum, so only a shared cut stays half-open.
There was a problem hiding this comment.
Fixed in 9c68938: the keep test stays half-open on a shared cut and is now closed on the scene box's own upper faces (the per-axis maximum of the sub-scene boxes), so every boundary point or centroid goes to exactly one box. SceneSplit.md §2.6 updated.
| REPO cdcseacave/halfmesh | ||
| REF v${VERSION} | ||
| SHA512 d6096ffeb71e5375e08d00f8eb808a4d04323b38d01214160e608d5bf0a538269d0fc6ac1824c03fe45ecbfcf3a61083e308af31cc17c2db6f17088306b1b34f | ||
| SHA512 0 |
There was a problem hiding this comment.
[bug] vcpkg_from_github for halfmesh v0.5.0 sets SHA512 to 0. vcpkg rejects that hash, so a configure that uses this overlay port cannot download the source this branch depends on. The previous portfile carried a real digest.
Suggestion: Replace 0 with the SHA512 of the v0.5.0 GitHub archive.
There was a problem hiding this comment.
Not addressable yet: the v0.5.0 tag does not exist until cdcseacave/halfmesh#11 is merged and tagged (CI fails on the 404 for that archive). The SHA512 gets set from the tag's archive right after; the PR description flags this as the blocker.
| // exactly on the plane, and a part below the ground counts as cut | ||
| REAL fill, cut; | ||
| const REAL volume(mesh.ComputeVolume(Planef(Point3f::EVec(up), 0.f), fill, cut)); | ||
| VERBOSE("Volume above the ground plane %g, below it %g", fill, -cut); |
There was a problem hiding this comment.
[bug] ComputeLeveledVolume logs the part below the ground plane as -cut and returns fill + cut (line 309). TransformScene prints that return value as "Mesh volume" (apps/TransformScene/TransformScene.cpp:376). A run with 10 above and 2 below prints "above 10, below -2" and then "Mesh volume: 12". Adding the two logged numbers gives 8. Mesh::ComputeVolume documents the return as the sum of the two positive parts (libs/MVS/Mesh.h:281), and the octahedron test expects a positive below (apps/Tests/TestsMVS.cpp:260). The CLI help calls the same measurement "fill above, cut below".
Suggestion: Log cut with the sign that the returned sum uses, so the above and below lines add up to the mesh volume the tool reports.
There was a problem hiding this comment.
Fixed in 0f33515: the part below the plane is logged as the positive value that enters the returned sum, so the two logged parts add up to the reported volume.
| if (mesh.IsEmpty()) | ||
| return *this; | ||
| halfmesh::Mesh halfMesh; | ||
| halfmesh::ConvertMesh(std::move(*this), halfMesh); |
There was a problem hiding this comment.
[bug] Mesh::Join moves *this into a half-mesh and joins. Mesh.h:160 states that an attribute survives only when both meshes carry it. An empty destination carries no vertex colors, so the first join drops colors that exist only on the incoming mesh. The previous Mesh::Join assigned *this = mesh when the destination was empty and then concatenated vertexColors. apps/Viewer/Scene.cpp:1279 BuildMergedVisibleMesh releases the output, strips texture and normals from each visible layer, leaves vertexColors in place, and calls mesh.Join(layerMesh) at line 1292. The merged view therefore loses vertex colors on every layer.
Suggestion: When the destination is empty, take the incoming mesh whole so a one-sided attribute is kept, then join any further layers that carry the same attribute.
There was a problem hiding this comment.
Not a bug: halfmesh::Mesh::Join copies the other mesh whole when the destination has no vertices (*this = other), so the first viewer layer keeps its colors. 0f33515 documents this in Mesh.h and adds a bridge test (empty mesh joined with a colored one keeps vertexColors).
| os.path.join(SUBSCENES, 'scene_XXXX.mvs')] | ||
| run([*merge, os.path.join(SUBSCENES, 'scene_XXXX_dense.ply'), stem + '_dense.ply'], stem + '_dense.ply') | ||
| suffix = os.path.basename(meshes)[len('scene_0000'):] | ||
| for name in sorted({'_dense_mesh', suffix}) if 'mesh' in steps else (): |
There was a problem hiding this comment.
[bug] Per-chunk mesh merges run only when 'mesh' is in steps. reconstruct_scene still runs refine whenever 'refine' is in steps (line 102) and passes -m <base>_dense_mesh.ply (line 103). Texture (line 149) then reads mesh + '.ply'. A run that includes refine or texture and omits mesh never writes that ply, and the refined chunks stay unmerged. MvgMvsPipeline.py:548 builds that step list from whichever of steps 18-21 the user named, so --steps 18 20 21 hits this path.
Suggestion: Merge the mesh products the selected steps actually write, and skip refine's -m input when the mesh step did not run.
There was a problem hiding this comment.
Fixed in 9c68938: each run merges the products its steps wrote (_dense / _dense_mesh / _dense_mesh_refined), and texture reads the refined mesh when the run refines. Refine keeps -m <base>_dense_mesh.ply: without the mesh step that file is the earlier run's mesh, which is refine's only input, so dropping it would leave refine with nothing to refine.
| submeshes.emplace_back(std::move(submesh)); | ||
| std::vector<FaceIdxArr> chunks(texturesDiffuse.size()); | ||
| FOREACH(idxFace, faceTexindices) { | ||
| FaceIdxArr& chunk = chunks[faceTexindices[idxFace]]; |
There was a problem hiding this comment.
[bug] SplitMeshPerTextureBlob sizes chunks to texturesDiffuse.size() and indexes it with faceTexindices[idxFace]. A face whose texture index is outside that range is an out-of-bounds access. The previous loop appended a face only when its index equalled a texture id, so an unmatched face was left out. The viewer calls this from apps/Viewer/Renderer.cpp:366. PLY load stores face.texId as written (Mesh.cpp:721) and appends one image per TextureFile comment, so a file whose ids do not match the comment count reaches this subscript.
Suggestion: Skip a face whose texture index is outside texturesDiffuse.
There was a problem hiding this comment.
Fixed at the producer in 0f33515: LoadPLY refuses a file whose face texture index has no matching TextureFile comment, and SplitMeshPerTextureBlob asserts the contract instead of skipping faces (a silently dropped face would hide the bad input). The bridge test loads such a PLY and expects the failure.
| ("estimate-scale", boost::program_options::value(&OPT::fEstimateScale)->default_value(0.f), "estimate the point-scale for the dense point-cloud (scale multiplier, 0 - disabled)") | ||
| ("estimate-segmentation", boost::program_options::value(&OPT::nEstimateSegmentation)->default_value(0), "estimate segmentation of the dense point-cloud based on the image segmentation masks; num views to agree (0 - disabled, <0 - only segmentation)") | ||
| ("sub-scene-area", boost::program_options::value(&OPT::fMaxSubsceneArea)->default_value(0.f), "split the scene in sub-scenes such that each sub-scene surface does not exceed the given maximum sampling area (0 - disabled)") | ||
| ("sub-scene-memory", boost::program_options::value(&OPT::fMaxSubsceneMemory)->default_value(0.f), "split the scene in sub-scenes whose depth-map fusion and mesh reconstruction each fit this much memory, in GB (0 - disabled, <0 - fit this machine: the physical memory available now, less a reserve of 10% of the total and at least 2GB)") |
There was a problem hiding this comment.
[suggestion] --sub-scene-area is removed. Boost rejects an unknown option, so a script that still passes the area flag fails at parse time. The replacement is --sub-scene-memory, in GB. docs/design/SceneSplit.md:92 records the removal.
Suggestion: Keep the old name as a hidden option that fails with a message naming --sub-scene-memory.
There was a problem hiding this comment.
Keeping it removed: Boost already stops at parse time naming the unknown option, so nothing runs with a wrong setting, and no script in the repo passes it. A hidden alias would be a permanent option kept only to print a message.
| bestParts = numParts; | ||
| numStalled = 0; | ||
| } else if (++numStalled >= 4) { | ||
| VERBOSE("warning: no split fits %gGB, the smallest sub-scene peak reached is %.1fGB (%d sub-scenes)", maxMemory, bestMax, bestParts); |
There was a problem hiding this comment.
[suggestion] After four part counts fail to shrink the largest box by 1%, Split warns and still returns true (line 293) with that over-budget partition. DensifyPointCloud.cpp:473 treats true as success and ExportChunks writes the boxes. The following fuse and mesh steps then run on chunks the budget said would not fit.
Suggestion: Return false from the stall path so the caller stops before exporting an over-budget split.
There was a problem hiding this comment.
Keeping this behavior; it is the documented design (SceneSplit.md §2.3). The stall means the depth-maps the images bring set a floor no split goes below, and the kept partition has the smallest peak any split reaches. Failing there would leave the scene unreconstructable at this budget, while the warning names the reachable peak. The auto budget already keeps a reserve of 10% of RAM and at least 2 GB. A budget that can never fit, below a sub-scene's fixed cost, is now refused up front (9c68938).
| return mesh | ||
|
|
||
| chunks = [] | ||
| if memory != 0 and 'densify' in steps: |
There was a problem hiding this comment.
[suggestion] chunks is filled only when memory != 0 and 'densify' is in steps. --resume is checked inside that same branch (line 109). A later run with --steps mesh,refine,texture leaves chunks empty and reconstructs the whole scene, including when subscenes/scene_XXXX.mvs is already on disk. MvgMvsPipeline.py:548 enters this driver only when step 18 is selected, so steps 19-21 alone take the same whole-scene path.
Suggestion: When the sub-scene files already exist, use them for the later steps even if this run does not include densify.
There was a problem hiding this comment.
Fixed in 9c68938: without densify, the driver reuses subscenes/scene_XXXX.mvs from the earlier run, and MvgMvsPipeline routes any of steps 18-21 through the driver, not only runs that include 18.
| costs[l] = BoxCost(leaves[l].idxs, chunkImages); | ||
| break; | ||
| } | ||
| ++numParts; |
There was a problem hiding this comment.
[nit] The part-count search adds one part at a time and runs Partition over every sample on each try, until the budget fits or the search stalls.
Suggestion: Jump the part count by the ratio of the current largest cost to the budget, and partition once per candidate.
There was a problem hiding this comment.
Partly done in 9c68938. The search now starts at the larger of the two stages' point-only bounds (the mesh bound is the tighter one, about 1.5x the old start). Both are true lower bounds. A cost-ratio jump is not safe here, for two reasons. First, near the depth-map floor the ratio stays above 1, so the part count would grow geometrically and the 1% stall test would no longer bound it. Second, the 30% image-contribution threshold can make the cost fall faster than 1/K, so a jump can overshoot the smallest count that fits. Each try is cheap next to loading the depth-maps: on HJ25 the whole Split takes 212 ms.
…se the sub-scenes - MvsMergeChunks.py: the keep test stays half-open on a shared cut but is closed on the scene box's own upper faces, so a point or face centroid lying there is no longer dropped from the merged result. - MvsScalablePipeline.py: without the densify step, the sub-scenes an earlier run wrote are reused; each run merges only the products its steps wrote, and the texture step reads the refined mesh when the run refines. MvgMvsPipeline.py routes any of steps 18-21 through it, not only runs that include step 18. - Scene::Split refuses a budget not above a sub-scene's fixed cost (3.8 GB) instead of searching hundreds of parts, and starts the part-count search at the larger of the two stages' point-only bounds (the mesh one is tighter). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut as positive - LoadPLY refuses a face whose texture index has no TextureFile comment (the per-texture split used to index past texturesDiffuse); SplitMeshPerTextureBlob asserts the contract. - ComputeLeveledVolume logs the part below the ground plane with the sign of the returned sum, so the two logged parts add up to the reported volume. - Mesh::Join documents that an empty mesh takes the given one whole (halfmesh already does); the bridge test covers it and the PLY texture-index check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…streams Scene::Split counts each image's valid depth-map samples before the scene ROI test: the fusion loads the whole map, so a tight ROI no longer drops most of the depth term (HJ25 with a 20% ROI slab: 4.9 GB predicted, was 4.1 GB for the same 25 full maps). The ROI-filtered counts still place the images in the boxes. MvsMergeChunks.py: - renumbers the per-point view lists from each sub-scene's images to the scene's (--scene, matched by image ID; required when the points carry views); - takes its inputs from the sub-scenes there are, so a stale product of an earlier, larger split is ignored, and a missing one is an error; - reads properties in any order (a scalar such as label after the lists); - streams the kept records and faces to disk, one sub-scene at a time, so its memory follows the largest sub-scene, not the scene. MvsScalablePipeline.py --dry-run no longer deletes the existing sub-scenes. MvgMvsPipeline.py keeps the driver command apart from the step command (the NATIVE preset ran CreateStructure twice and skipped the dense steps; an SfM-only preset raised IndexError), routes nothing with --sub-scene-memory off, and quotes the passthrough options so a Windows path with spaces reaches the module intact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The receiver labels found their energy by scanning the sender's labels, O(L1*L2) per message; a per-thread label-indexed table makes the message O(L1+L2) as documented, with the same values. A unit test compares the Potts path with the generic one on random graphs with partly shared label sets (labels and energies equal exactly). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…xport Scene::Split returns the predicted peak memory of its largest sub-scene (of the whole scene when it is not split), 0 on failure, and the cost model is Scene::PredictSubsceneMemory. ExportChunks creates the output folder, so DensifyPointCloud --sub-scene-memory no longer fails on a missing one and a scene that is not split gets none. Tests.exe 2 keeps the depth-maps on disk for SceneSplitTest and deletes them right after it: the scene stays whole in a large budget, a ROI slab holding a quarter of the samples keeps the whole depth-maps of its images in the prediction (the ROI-filtered count it replaced fails this check), and a forced split exports three sub-scenes with their boxes into a new folder (fails without the folder creation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v0.5.0 tag is released (cdcseacave/halfmesh 7083a14), so the port now checks the archive it downloads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ComputeEnergy sums through an OpenMP reduction whose order is not fixed (libgomp), so two equal labelings gave energies differing in the last bit on the Linux CI. The energy is a function of the labels, which are still compared after every iteration; a broken Potts lookup still fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
halfmesh (C++20) and OpenMVS both instantiate BS::thread_pool, whose layout depends on the standard: from C++23 on it stores its tasks in std::move_only_function. With OpenMVS on C++23 the program held two definitions of the same class; on Linux the executable's copy of the worker interposed halfmesh's, so the pool halfmesh::PackAtlas builds (halfmesh 0.5.0, through Mesh::TransferTexture) ran its init function through the wrong layout and the unit tests segfaulted (gdb on the CI runner: BS::thread_pool<0>::worker from libs/Common, pool constructed in halfmesh::PackAtlas). OpenMVS uses no C++23 feature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The version variables move above PROJECT(), which now takes
VERSION ${OpenMVS_VERSION}, so CMake's PROJECT_VERSION and
CMAKE_PROJECT_VERSION carry it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… too halfmesh v0.5.0 (re-tagged) picks the newest standard the compiler supports by the same rule as build/Utils.cmake, so the two agree on BS::thread_pool's layout without pinning OpenMVS to C++20. The port pins the new tag archive. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cdcseacave
left a comment
There was a problem hiding this comment.
PR Review: Acute3D-derived refine/texture/split improvements, scalable pipeline, and halfmesh 0.5.0 migration
Summary of Changes Reviewed
This branch introduces comprehensive improvements across several core modules and scripts:
-
RefineMesh: CPU row-band window sums (cutting peak RAM by ~50%), on-sample central-difference image gradient calculation without precomputed gradient maps, single photometric accumulation per reference view in CUDA with bitmask-directed tracking, elimination of per-view depth maps in CUDA, and
--scales 0early return. -
TextureMesh: View selection scoring by visible resolution normalized to face best view (with penalties for partial occlusions), 1:1 image resolution rasterization, footprint atlas packing and push-pull gutter filling via halfmesh 0.5.0, unsharp mask disabled by default, and
$O(L)$ Potts LBP message computation with flat message buffers. -
Large Scenes / Split & Merge: Memory-budgeted recursive bisection scene split (
Scene::Split,PredictSubsceneMemory), automated Python pipeline driver (MvsScalablePipeline.py), streaming chunk merger (MvsMergeChunks.py), and integration intoMvgMvsPipeline.py. -
Halfmesh 0.5.0 Port: Delegation of OBJ codec, watertightness, volume calculation (both closed and leveled against a plane), mesh joining, subdivision, surface point sampling, and sub-mesh extraction; removal of deprecated
libs/IO/OBJand unused mesh queries. -
Build & Standards: Unified C++ standard selection with halfmesh (C++20/C++23) to avoid
BS::thread_poolABI mismatch.
Issue Counts by Severity
- Bugs / Critical Edge Cases: 1
- Logical / Usability Issues: 2
- Static Analysis / Code Quality: 2
- Minor / QoL: 2
Detailed Findings
1. [Bug] 0-face sub-scene mesh treated as point-cloud in chunk merger, causing merge failure
- File:
scripts/python/MvsMergeChunks.py:171-172andscripts/python/MvsMergeChunks.py:230-234 - Description:
Inread_ply(path):If a sub-scene mesh contains 0 faces (if len(elems) == 1 or elems[1][1] == 0: return vertices, None
elems[1][1] == 0, e.g. empty space or water/sky region with no reconstructed geometry),read_plyreturnsvertices, None.
Inmain():If any earlier chunk contained faces (vertices, faces = read_ply(path) if is_mesh is None: is_mesh, props = faces is not None, vertices.props elif is_mesh != (faces is not None): sys.exit('%s: meshes and point-clouds can not be merged together' % path)
is_mesh == True), encountering a chunk with 0 faces (faces is None) triggersis_mesh != (faces is not None)and aborts execution withmeshes and point-clouds can not be merged together. Conversely, if chunk 0 has 0 faces,is_meshis set toFalse(treating the mesh as a point cloud), causing all subsequent chunks with faces to fail. - Recommendation:
Determine mesh status by checking whether the face element exists (len(elems) > 1 and elems[1][0] == 'face'), rather than whetherelems[1][1] == 0. Whenelems[1][1] == 0, returnvertices, np.empty((0, 3), dtype=np.uint32).
2. [Logical / Usability] Multi-step runs in MvsScalablePipeline.py revert to unrefined mesh when texturing
- File:
scripts/python/MvsScalablePipeline.py:149 - Description:
InMvsScalablePipeline.py:If a user runs the pipeline in steps (e.g. runningif 'texture' in steps: # the (merged) mesh is textured from all the images: the refined one when this run refines mesh = stem + ('_dense_mesh_refined' if 'refine' in steps else '_dense_mesh') module(texture, 'texture', stem + '.mvs' if len(chunks) > 1 else stem + '_dense.mvs', mesh + '_texture.mvs', '-m', mesh + '.ply')
--steps densify,mesh,refinefirst, and subsequently running--steps texture),'refine' in stepsisFalse.
Consequently,meshis set tostem + '_dense_mesh', andTextureMeshtexturesscene_dense_mesh.plyinstead of the newly producedscene_dense_mesh_refined.ply, ignoring the refinement step. - Recommendation:
Check if the refined mesh exists on disk as well:mesh = stem + ('_dense_mesh_refined' if ('refine' in steps or os.path.exists(os.path.join(work, stem + '_dense_mesh_refined.ply'))) else '_dense_mesh')
3. [Logical / Edge Case] Single sub-scene produced if chunk pruning leaves 1 chunk in Scene::Split
-
File:
libs/MVS/SceneSplit.cpp:271-277andlibs/MVS/SceneSplit.cpp:309-314 -
Description:
The check:is performed before the post-partition pruning that removes images completely contained by larger chunks. If pruning subsequently empties one of the chunks, leaving exactly 1 chunk:if (chunks.size() < 2) { chunks.Empty(); if (peakCost <= maxMemory) VERBOSE("Scene fits in %.1fGB (predicted %.1fGB): not split", maxMemory, peakCost); return peakCost; }
RFOREACH(c, chunks) if (chunks[c].images.empty()) chunks.RemoveAt(c);
SplitreturnspeakCostwithchunks.size() == 1.
DensifyPointCloudthen exports a single sub-scenesubscenes/scene_0000.mvs.
However,MvsScalablePipeline.pyexpects either 0 sub-scenes (fits whole) or$\ge 2$ sub-scenes (if len(chunks) <= 1: reconstruct_scene(stem)), meaning a single exported subscene is left orphaned while the driver reconstructs the whole scene directly. -
Recommendation:
Re-checkif (chunks.size() < 2) { chunks.Empty(); ... }after the empty-chunk removal loop.
4. [Static Analysis] Uninitialized member variable Node::numParts in SceneSplit.cpp
- File:
libs/MVS/SceneSplit.cpp:184 - Description:
CodeQL / MSVC PREfast flags warningtype.6onNode::numParts:
Variable 'MVS::Scene::Split::<lambda_3>::()::Node::numParts' is uninitialized. Always initialize a member variable (type.6).
WhenNode left, right;is declared on line 213,numPartsis initially uninitialized before assignment. - Recommendation:
Initialize member default:struct Node { std::vector<SIdx> idxs; AABB3f aabb; int numParts{0}; };.
5. [Code Quality] In-place mutation during list iteration in MvgMvsPipeline.py
- File:
scripts/python/MvgMvsPipeline.py:536-541 - Description:
Instep_options(cstep):for anOpt in STEPS[cstep].opt: if anOpt in opt: idx = STEPS[cstep].opt.index(anOpt) del STEPS[cstep].opt[idx:idx+2]
STEPS[cstep].optis mutated in place during iteration, which can cause elements to be skipped if consecutive matching options occur, and permanently modifies the step's default option list across calls. - Recommendation:
Iterate over a shallow copy (list(STEPS[cstep].opt)) or build a new list without in-place deletion.
6. [Minor] Case-sensitive option handling between MvgMvsPipeline.py and MvsScalablePipeline.py
- File:
scripts/python/MvgMvsPipeline.py:550andscripts/python/MvsScalablePipeline.py:58-64 - Description:
If a user passes--sub-scene-memory OFFor--sub-scene-memory Auto,MvgMvsPipeline.pydoes a case-sensitive checkCONF.sub_scene_memory != 'off', andMvsScalablePipeline.pychecksargs.memory == 'off'. A non-lowercase string falls through tofloat(args.memory), raising aValueError. - Recommendation:
Normalize with.lower()when checking'off'or'auto'.
7. [Minor] Unchecked loadMVSInterface return value in MvsMergeChunks.py
- File:
scripts/python/MvsMergeChunks.py:209-210 - Description:
IfloadMVSInterface(scene)fails to parse the file or encounters an invalid format, it prints an error and returnsNone. The subsequentobb = data.get('obb')raises an unhandledAttributeError: 'NoneType' object has no attribute 'get'. - Recommendation:
Addif not data: sys.exit('error: failed loading %s' % scene)before accessingdata.
Verification & Strengths
-
LBP Optimization: Potts model optimization in
$O(L)$ with thread-local lookup slots verified against generic$O(L^2)$ path inTestLBPPotts(). Flat message array avoids per-edge vector allocations. -
CUDA Refine Performance & Correctness: Reference view grouping with 32-bit directional accumulation masks and
countDirectionsCAS bit-tracking is clean and race-free. Recomputing depth viapixelDepthremoves per-view depth maps while maintaining exact agreement withpixelBary. -
Halfmesh 0.5.0 Integration: Volume calculation (open mesh against ground plane by prisms) verified in
MeshHalfMeshBridgeTest. Exact round-trip preservation of OBJ geometry and materials confirmed. -
Tests:
Tests.exe 0andTests.exe 1run cleanly and pass with 0 assertion failures.
Dropping the images another chunk already covers could empty a chunk, whose box was then removed and never reconstructed; the budget is met before that pruning, so such a chunk now keeps its images and no chunk is ever empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er run left - MvsMergeChunks.py tells meshes from point-clouds by the face element, so a sub-scene mesh with no faces merges instead of aborting, and an unreadable scene exits with a message. - Both drivers texture the refined mesh when the run refines, the unrefined one when it only rebuilds the mesh, else the refined one if it exists. - MvgMvsPipeline.py no longer skips an overridden default option that follows another; the memory budget options accept any case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The review fixes: PLY index validation and per-page UVs, glTF vertex colors only when every primitive has them, pinched chart boundary loops, explicit contracts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This PR covers three things:
There is one commit per feature, and each message carries its own measurements.
RefineMesh
--scales 0returns the input mesh. Before, it crashed: the post-refine simplification read views that only a scale prepares. The views contract is now asserted.TextureMesh
Hold-out bench: every 4th view is left out of texturing and scored.
halfmesh::PackFootprints), and the gutter is filled by push-pull (halfmesh::PushPullFill). The atlas drops from 64 to 32 MTexels on all four scenes, with PSNR equal or better (up to +0.03 dB at mip 3).Large scenes
Scene::Splittakes a memory budget: the depth-map samples are bisected until the largest predicted sub-scene fits. On T&T Church at 12 GB, mesh F1 is 0.6085 vs 0.5967 with the old area split.MvsScalablePipeline.py scene.mvsruns depth-maps → split → per-sub-scene fusion / mesh / refine → merge → texture. The budget defaults to this machine's available memory, so a scene that fits is not split and its fusion is byte-identical to an unsplit run.MvgMvsPipeline.pyroutes its dense steps through it by default.docs/design/SceneSplit.md.--fusion-mode 1no longer saves a bogusscene_dense.MemAvailable.Mesh algorithms moved to halfmesh
MeshHalfMesh.cppnow delegates Subdivide, SamplePoints, the OBJ codec, IsWatertight, ComputeVolume, Join and SubMesh(es);libs/IO/OBJand the unused adjacency queries are deleted.v//vnand relative indices.--export-texture-lossless, so they are PNG by default.TransformScene --compute-volumemeasures an open mesh against its ground plane by prisms, and reports fill and cut separately.SmoothNormalFaces. A face with no admissible neighbour no longer gets a NaN normal.--sample-meshcloud.Tests
Tests.exe0, 1 and 2 pass in RelWithDebInfo and in Debug (_HEADLESS_DEBUG, 0 ASSERTs).🤖 Generated with Claude Code