Core Solver - Add Ceres Solver #174
Comments
david-cattermole
added
core solver
|
Ceres Solver has been added as a dependency in v0.4.0, however Ceres is not yet supported as a solver back-end. Progress will continue to add support as a Solver back-end. |
|
What kind of features listed in the bullet points at #144 (comment) are already covered by OpenMVG? |
|
Hello @bhack, OpenMVG provides Fundamental Matrix solving, Essential Matrix Solving and Unconstrained Bundle Adjustment (via Ceres Solver). These have been used in the Camera solver in mmSolver, although I'm not very happy with my implementation as I don't find it very robust, and can provide vastly different results each run of the solver. OpenMVG does not provide 2D pattern tracking, only libmv can provide that. OpenMVG only provides photogrammetry-style matching using SIFT-like image feature detection. 2D pattern tracking is a non-goal of MM Solver (at least in the immediate future), so not having that feature is a non-issue. David |
Do you have any specific plan to improve on this? I see that you also import tracks from 3Dequalizer. Have you compared with its own solver? |
|
I do not have a specific plan, however I have a general idea of what needs to be improved. Generally, the main issue with quality is caused because the bundle adjustment step is only performed every 5 camera poses that are approximated. Changing this value to 1 improves the quality overall but results in a much slower camera solve. I have experimented with changing the camera solver algorithm to find a good compromise. The bundle adjustment performance can probably be improved by using the Bundle Adjustment function inside OpenMVG (which uses Ceres). This is not yet implemented, and the Camera solver is currently using the mmSceneGraph and the mmSolver core for bundle adjustment. Yes, importing 2D Marker data is supported from 3DEqualizer and Blender (which is using libmv for 2D tracking and solving) and any other software that can export the .uv file format (which is pretty simple). I test against 3DEqualizer solved camera output, or any other software, as long as the markers and bundles match with a low error/deviation and is physically plausible, I consider that correct. Generally my goal is to solve an image sequence of say 250 frames with 20 to 40 markers/bundles in under 1 second. You can see the current camera solver implementation here: Thanks, |
|
Thanks for the details.. |
|
mmSolver has a test suite to test for correctness, but there isn't any performance benchmark tests (other than me being annoyed that tests take too long to finish). Anecdotally, 3DEqualizer's camera solver is the fastest and most robust solver for SfM tasks that I've used. I am not intending mmSolver to compete directly with 3DEqualizer's camera solver, instead mmSolver provides a different type of solver and focuses on getting results that 3DEqualizer cannot solve; for example the core of mmSolver allows arbitrary transform hierarchies with connections between objects, utilizing the Maya scene graph and the ability to solve arbitrary attributes. The mmSolver camera solver is intended to provide an even lower number of minimum points (only 5 points needed per-frame, as opposed to 3DEqualizer's 6 points per-frame), and is also intended to be used to solve complex camera moves that need to be stitched together, and combined with the mmSolver core solver with arbitrary attributes and transform hierarchies. |
|
Ok thanks for the details. I subscribe to this ticket to check if we have some updates for the OpenMVG/Ceres bundle adjustment. |
Add the Ceres library to bake a "LMDIF" solver, which will perform Levenberg-Marquardt using "finite differences". This change also requires the "solver verison" and "solver_type" to be exposed to the Python API, so that the tests can run using Ceres. Right nwo Ceres is similar to the "cminpack_lmdif" in terms of performance, and does not yet match "cminpack_lmder". As a result, the new "ceres_lmdif" will not be made default until the solver can be proven to provide better results and integrates nicely, just like "cminpack_lm*" solver types. GitHub issue #174.
"LMDER" stands for "Levenberg-Marquardt analyic DERivatives". This solver is intended to parallel the "cminpack_lmder", which has historically been the faster solver in mmSolver. We have not yet proven that "ceres_lmder" is always (or mostly) better than "cminpack_lmder", therefore the default has not been changed. This also turns off "use_nonmonotonic_steps", which I've tested and seems to produce worse results in simple cases. GitHub issue #174.
The frames that can or cannot be solved are validated and culled before a proper solve can be started. This is intended to avoid invalid solves from attempting to be solved, and overall produce a more robust solver. We also move the logic of whether the solver is valid from the Python "actionstate.py" file to the C++ core, which gives us more information to make better and more informed decisions about solver validity. This is partially related to gitHub issue #174.
Feature
Ceres Solver is a state-of-the-art non-linear least squares solver and would likely show performance improvements over the currently supported CMinpack and LevMar solvers.
Ceres solver will be installed as part of #144, and so we should make sure mmSolver can use it as a solver type if available.
This will involve writing a new wrapper around our internal "solverFunc" for Ceres, and providing the
maya.cmds.mmSolverPython command with the ability to use Ceres solver.Software Versions
The text was updated successfully, but these errors were encountered: