Repository navigation
Interface
OpenMVS pipeline needs as input a set of camera poses and the corresponding undistorted images, plus the sparse point-cloud generated by the Structure-from-Motion pipeline. There are several ways to generate the necessary input:
-
Run the native SfM module,
CreateStructure, directly on a folder of images. It produces a.sfmscene that can additionally be exported as a.mvsproject with the--export-mvsflag, feeding the downstream MVS pipeline with zero external dependencies. Video can be ingested first viaExtractKeyframes, including 360° / equirectangular video. -
The most generic way is to generate a native OpenMVS project. In order to do this, just copy and include libs/MVS/Interface.h (self contained) into your project, fill the structure with your data and save it to a file using the included serialization support. As an example, see OpenMVG exporter openMVG_main_openMVG2openMVS.
-
OpenMVG output is supported by OpenMVS. In order to convert a project from OpenMVG to OpenMVS either use the OpenMVG integrated exporter openMVG_main_openMVG2openMVS, or at the building stage make sure to point the CMake tool to the OpenMVG installation folder to generate an importer.
-
COLMAP dense output (undistorted images and project files) is supported by OpenMVS through the
InterfaceCOLMAPapp. -
BlocksExchange XML open exchange format used by Bentley iTwin Capture Modeler / ContextCapture and Agisoft Metashape is supported through
InterfaceMetashape. -
Polycam projects are supported through
InterfacePolycam. -
MVSNet-style scenes (pre-computed depth predictions plus camera poses) are supported through
InterfaceMVSNet.
The input camera poses should contain full calibration: intrinsics and extrinsics. The intrinsics are represented as a standard calibration matrix K composed of the focal-lengths fx and fy and the principal point cx and cy. K values can be in pixels, but in order to be more generic to the input images, they should be normalized by 1/MAX(width,height) of the corresponding image. Using a normalized K matrix, makes the project immune to possible manual re-scaling of the images after the SfM process. The extrinsics represent a pose using a rotation matrix R and a position C. In order to project a 3D point X from world coordinates to image coordinates K*R(X-C) is used. The projection in image coordinates uses the convention that the center of a pixel is defined at integer coordinates, i.e. the center is at (0, 0) and the top left corner is at (-0.5, -0.5). All matrices are expected in row-major format.
OpenMVS supports both pinhole and spherical (equirectangular) cameras. Pinhole cameras use the standard K * R * (X - C) projection described above. Spherical cameras are modelled as equirectangular panoramas whose image width is exactly twice the image height; projection uses longitude/latitude angles directly and K is simply the identity matrix. Spherical images are processed natively during Structure-from-Motion, feature matching, pose estimation and bundle adjustment. For the downstream MVS modules (DensifyPointCloud, ReconstructMesh, RefineMesh, TextureMesh) each spherical image is internally converted to a six-face cube-map of virtual pinhole views via libs/SFM/SphereCubeMap.h, so the pinhole-based dense reconstruction stack works unchanged.
OpenMVS supports central and non-central cameras. In order to accomplish this, the following structure is used:
- Platforms
- Cameras
- Poses
- Images
- fileName
- platformID
- cameraID
- poseID
- ID
The camera poses are represented by two arrays of Platform and Image structures.
The array of platforms contains one element for each camera (central or non-central) used in the project. A Platform is represented by two arrays of Camera and Pose structures. The array of cameras contains only one camera in the case of a central camera, or multiple cameras for representing a non-central camera. For the non-central case, each camera should contain along with the camera matrix also the pose relative to the platform.
The array of images contains one element for each undistorted image contained by the project. An Image contains the image file name, plus three IDs representing the platformID, cameraID and poseID indices into the corresponding platform, camera and pose arrays. Images with missing calibration can be represented by filling poseID with NO_ID special value. Optionally, ID can be set to a desired ID used to reference this image outside OpenMVS project.
The sparse point-cloud is represented as an array of points, each one containing the position, the list of image IDs seeing it, and optionally the color and normal.
The output is a point-cloud and/or a triangle mesh, exported by default in the PLY file format, and a texture image, exported by default as PNG.
Additional export formats:
-
OBJ — meshes (with
.mtlmaterial + texture sidecars when textured). -
glTF 2.0 — both meshes and point-clouds, in either the ASCII
.gltfform (with binary buffer sidecars) or the self-contained binary.glbform. Vertex colors / normals (point-cloud), faces and the diffuse texture map (textured mesh) round-trip. Implemented inMVS::Mesh::SaveGLTF/MVS::PointCloud::SaveGLTFon top of the header-onlytiny_gltflibrary; load/save format is auto-selected by file extension. Both sides write the glTF-mandated z-up to y-up rotation as a matrix on the root node (never baked into the vertex buffer) and undo it on load, so the file is spec-conformant y-up, viewers show the model upright, and export/re-import is an exact identity - the two matrices are signed permutations, so their product is exactly the identity. The mesh side gets this from halfmesh; the point-cloud side applies the same convention inMVS::PointCloud::SaveGLTF/LoadGLTF, which also flattens the node hierarchy on load, so a mesh and a point-cloud exported from the same scene stay in the same frame as each other. It writes the diffuse textures beside the file, PNG unless a lossy export was requested. Caveat for old files: glTF written by OpenMVS before this change carried no node transform at all, so its z-up data now reads back rotated - nothing in such a file distinguishes it from a conformant y-up one. Re-export it if you need it. This applies to point-clouds as well as meshes. -
Potree 2.0 — point-cloud only. Saves an LOD octree (
metadata.json+hierarchy.bin+octree.bin) into a directory, ready to be streamed by the Potree web viewer. Implemented inMVS::PointCloud::SavePotree(libs/MVS/PointCloud.cpp) using the LOD octree builder inlibs/Common/OctreeLOD.h. A simple Python helper,scripts/python/potree_server.py, serves the result through a self-contained Potree.js HTML page for local inspection.
TransformScene --convert 1 --export-type {ply|obj|glb|gltf|potree} is the swiss-army CLI that drives any of the above formats from an existing .mvs project, and the Viewer accepts .gltf / .glb as drag-and-drop inputs and can re-export to .ply / .obj / .glb.