New Features in Meshroom 2023.1

1. Image Masking
2. CCTAG nodes: What is new?
3. Color Checker
4. New Documentation and Notes tab in nodes
5. Save and load templates
6. Computation speed
7. Adobe DNG Color Profile and Lens Camera Profiles databases

1. Image Masking

The Input from the ImageMasking connects to CameraInit. The output of the ImageMasking node can be connected to FeatureExtraction (use mask for in camera estimation) Masking Input and the PrepareDenseScene Masking Input (applies the undistort to the mask and combines it with the RGB to a RGBA image).

You can get the setting by hovering over the parameter (the hue info is missing in the gui at the moment):

HSV

HSV mode is used for colour keying

Green : default values

White : Hue=AnyValue, Tolerance=1, minSaturation=0, maxSaturation=0.1, minValue=0.8, maxValue=1

Black : Hue=AnyValue, Tolerance=1, minSaturation=0, maxSaturation=0.1, minValue=0, maxValue=0.2

Blue : Hue=0.68, Tolerance=0.2, minSaturation=0.1, maxSaturation=0.1, minValue=0.1, maxValue=1

Make a test run and check the output. Black areas will be masked out. Adjust the values as and disable "invert" when necessary

The node will compute an image mask like this:

For Blue for example, disable the "invert" option to mask out the background Used options here were:

AutoGreyscaleThreshold

Split depth images in two parts using https://en.wikipedia.org/wiki/Otsu%27s_method using opencv https://docs.opencv.org/4.x/d7/d4d/tutorial_py_thresholding.html

1.1 Combine CCTAG detection and Masking

You can do the cctag detection with a parallel node similar to the workflow described in the section "Improve CCTAG detection" here https://github.com/alicevision/Meshroom/wiki/CCTAG-scaling

add the ImageMasking node to the default node graph
create a parallel branch for CCTAG detection only
use both branches as input in the StructureFromMotion node

To check the FeatureExtraction results double click on the node. Disable the preview before double clicking on the node from the other branch, so the viewer updates correctly

*in the node configuration Note Tab you can rename your nodes and add comments or a custom colour

Here is the Meshroom.mg file content (copy-paste in text file and save it as .mg. Then in Meshroom go to File->Advanced->Import Template): https://gist.github.com/natowi/52eb673a75f9fbf6181b86bd879ada5a

First posted here: https://github.com/alicevision/Meshroom/discussions/1993

2. CCTAG - What is new?

With MR2023 it is now possible to debug the markers detected in the scene by writing all marker numbers and their relative distance to each other into the node log of the SMDistance node:

So in combination this is good for testing the scaling process:

Read more about CCTAG usage here https://github.com/alicevision/Meshroom/wiki/CCTAG-scaling

3. Color Checker

One of the new (Beta) nodes in Meshroom allow for Color Checker detection and correction. This is really useful when working with RAW images in photogrammetry and panorama pipelines. (Note this is a beta feature and some functionality is not yet implemented)

Color Checker Pipeline

A: CameraInit->ColorCheckerDetection->ColorCheckerCorrection->PrepareDenseScene

B: once the ColorCheckerDetection node is computed successfully, the detected Checker can be enabled in the viewer

C: there will be a red bar around the detected checker

D: and a visual reference will be added to the view, so you can compare the colors

E: by moving over the image, the rgb values will be shown in the upper right corner

(Images taken from Triplegangers' Free Scan Sample, License: CC BY-NC-ND 4.0)

ColorCheckerDetection: (BETA)

Performs Macbeth color checker chart detection.

Outputs:

• the detected color charts position and colors
• the associated tranform matrix from "theoric" to "measured" assuming that the "theoric" Macbeth chart corners coordinates are: (0, 0), (1675, 0), (1675, 1125), (0, 1125)

Dev notes:

Fisheye/pinhole is not handled ColorCheckerViewer is unstable with multiple color chart within a same image

ColorCheckerCorrection: (BETA)

• Performs color calibration from Macbeth color checker chart.
• The node assumes all the images to process are sharing the same colorimetric properties.
• All the input images will get the same correction.
• If multiple color charts are submitted, only the first one will be taken in account.

Dev Notes:

Color Correction is done with OpenCV (COLORCHECKER_Macbeth)

4. New Documentation and Notes tab in nodes

Commonly used nodes will now include a short documentation (WIP)

In the notes Tab, we can now add comments and change the name and color of the node. This is really useful when trying different iterations for a node.

5. Save and load templates

You can now save your favourite graph as template and even share it with others*. (*open the mg. file as text file to make sure no personal data like file paths are included) This is really useful for complex testing pipelines or custom workflows, so you don´t have to recreate everything.

6. computation speed

Major improvements of the depth map quality, performances and scalability. The full resolution can now be computed on most of the standard GPUs. FeatureExtraction is now using DSP-SIFT by default for the 3D Reconstruction pipeline. [nodes] DepthMap: depth map improvements https://github.com/alicevision/Meshroom/pull/1818 Depth map refactoring https://github.com/alicevision/Meshroom/pull/680

The default pipeline is not faster compared to previous versions, but there are improvements on the quality of the reconstruction. dspsift in Feature extraction is used by default, which provides much better results that the previous default sift. However this has an impact on the computation time. It is impressive, that the computation time for depthmap was reduced by ~50% compared to the 2019 version. As an result, Meshroom can now run on Nvidia GPUs with less resources.

If you are looking for speed over quality, enable advanced options in the node and run FeatureExtraction with sift + gpu instead of dspsift

(tests run with the monstree dataset)

7. Adobe DNG color Profile and Lens Camera Profiles databases

Add support for Lens Camera Profiles (LCP) https://github.com/alicevision/Meshroom/pull/1771

Enhanced interpretation of RAW images, including new support for Adobe DNG Color Profile and Lens Camera Profiles databases (if installed on your workstation).

Adobe CameraRaw - What's new? Download latest Adobe DNG Converter

Windows Download

MacOS Download

You can find LCP profiles for your camera under: %ALLUSERSPROFILE%\Adobe\CameraRaw\LensProfiles\1.0"

https://rawpedia.rawtherapee.com/How_to_get_LCP_and_DCP_profiles

Everyone can create LCP profiles using Adobe's Lens Profile Creator.

(sidenote: it would be interesting to convert lensfun xml to adobe lcp for use in Meshroom: https://lensfun.github.io/manual/v0.3.95/lensfun-convert-lcp.html)

---

Release 2023.2

New Photometric Stereo nodes

Input images need to be located in a folder that starts with 'ps_' to be grouped together using their pose ID. The photometric stereo cannot run otherwise

It is possible to use the <- and -> arrows in the node settings to position the circle on the sphere.

[ui] Display an icon on nodes that have viewable outputs