Skip to content

Refactor: Migrate to vismatch package and clean up codebase#155

Draft
Vincentqyw wants to merge 46 commits intomainfrom
dev
Draft

Refactor: Migrate to vismatch package and clean up codebase#155
Vincentqyw wants to merge 46 commits intomainfrom
dev

Conversation

@Vincentqyw
Copy link
Copy Markdown
Owner

Summary

This PR introduces major refactoring to simplify the codebase by leveraging the vismatch pip package for image matching algorithms, instead of maintaining them as git submodules.

Key Changes

  • Remove all git submodules: Eliminated imcui/third_party/ directory containing ~40 algorithm implementations
  • Remove hloc module: Deleted entire imcui/hloc/ directory (8000+ lines of code)
  • Migrate to vismatch: All matchers are now dynamically loaded from the vismatch package
  • Simplify configuration: Reduce config files from 548+ lines to minimal YAML
  • Clean up dependencies: Trimmed requirements.txt from 43 lines to 17 lines

Additional Improvements

  • Fix match_threshold parameter not being passed to dense matchers (LoFTR, RoMa)
  • Add support for keypoint_threshold parameter
  • Update documentation to clarify matchers are maintained in vismatch repository
  • Improve config path resolution (check current dir, then fallback to package default)

Breaking Changes

  • All matching algorithms must now be installed via vismatch package
  • New matchers should be added to the vismatch repository instead of this WebUI

Test plan

  • Verify UI loads and displays available matchers
  • Test sparse matcher (superpoint-lightglue)
  • Test dense matcher (loftr) with match_threshold parameter
  • Verify config file loading
  • Run existing tests

🤖 Generated with Claude Code

Vincentqyw and others added 16 commits March 18, 2026 10:05
Add inbrowser=False to prevent Gradio from automatically opening
a browser on launch, which was causing proxy-related errors
in certain environments.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Remove config/app.yaml, use imcui/config/app.yaml as default
- Improve config path resolution: check current dir first, then fallback
- Simplify matcher_zoo config (548 lines -> minimal)
- Update tests to use dynamically loaded matcher_zoo from vismatch
- Update CLAUDE.md with CLI options documentation
- Update README.md with new installation instructions

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
删除未使用的依赖:
- datasets, e2cnn, einops, gdown, h5py, huggingface_hub, imageio, Jinja2, kornia
- pandas, pycolmap, pytorch-lightning, ray, roma, scikit-image, scikit-learn
- scipy, seaborn, shapely, tensorboardX, torchmetrics, torchvision, yacs, psutil

精简后从 43 行减少到 17 行

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Add acknowledgment for vismatch repository by @gmberton
- Update README.md to clarify this WebUI no longer maintains matchers
- Update CLAUDE.md directory structure (remove hloc/third_party references)
- Update API usage examples to use dynamically loaded matchers
- Simplify configuration precedence (remove obsolete paths)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Update _init_models() in core.py to pass max_keypoints and match_threshold
- Update set_model() to support threshold parameter
- Add keypoint_threshold support in utils.py get_model()
- Include keypoint_threshold in cache key for proper model caching

This ensures dense matchers (LoFTR, RoMa, etc.) receive the match_threshold
parameter, and matchers like zippypoint/silk receive keypoint_threshold.
@Vincentqyw
Copy link
Copy Markdown
Owner Author

Vincentqyw commented Mar 21, 2026

Linux: Model loading fails after the first successful import. Specifically, loftr loads without issues, but se2loftr, xoftr, ASpanFormer, and MatchAnything cannot be imported.

pytest tests/ -s
image

Error details:

se2loftr:

  File "/usr/local/lib/python3.13/site-packages/vismatch/__init__.py", line 175, in get_matcher
    from vismatch.im_models import se2loftr
  File "/usr/local/lib/python3.13/site-packages/vismatch/im_models/se2loftr.py", line 11, in <module>
    from configs.loftr.outdoor.loftr_ds_e2_dense_8rot import cfg as rot8_cfg
  File "/usr/local/lib/python3.13/site-packages/vismatch/third_party/Se2_LoFTR/configs/loftr/outdoor/loftr_ds_e2_dense_8rot.py", line 6, in <module>
    cfg.LOFTR.RESNETFPN.NBR_ROTATIONS = 8
    ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 141, in __getattr__
    raise AttributeError(name)
AttributeError: RESNETFPN

xoftr:

  File "/app/imcui/ui/config.py", line 142, in get_model
    return get_matcher(model_name, device=DEVICE, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/vismatch/__init__.py", line 180, in get_matcher
    from vismatch.im_models import xoftr
  File "/usr/local/lib/python3.13/site-packages/vismatch/im_models/xoftr.py", line 10, in <module>
    from src.xoftr import XoFTR
ModuleNotFoundError: No module named 'src.xoftr'

ASpanFormer:

  File "/app/imcui/ui/config.py", line 142, in get_model
    return get_matcher(model_name, device=DEVICE, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/vismatch/__init__.py", line 185, in get_matcher
    from vismatch.im_models import aspanformer
  File "/usr/local/lib/python3.13/site-packages/vismatch/im_models/aspanformer.py", line 14, in <module>
    from src.ASpanFormer.aspanformer import ASpanFormer
ModuleNotFoundError: No module named 'src.ASpanFormer'

MatchAnything:

  File "/app/imcui/ui/config.py", line 142, in get_model
    return get_matcher(model_name, device=DEVICE, **kwargs)
  File "/usr/local/lib/python3.13/site-packages/vismatch/__init__.py", line 172, in get_matcher
    return matchanything.MatchAnythingMatcher(device, variant="roma", *args, **kwargs)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/vismatch/im_models/matchanything.py", line 55, in __init__
    self._load_model()
    ~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/vismatch/im_models/matchanything.py", line 68, in _load_model
    cfg.merge_from_file(str(MATCHANYTHING_DIR.joinpath("configs", "models", "roma_model.py")))
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 212, in merge_from_file
    cfg = self.load_cfg(f)
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 363, in load_cfg
    return cls._load_cfg_from_file(cfg_file_obj_or_str)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 374, in _load_cfg_from_file
    return cls._load_cfg_py_source(file_obj.name)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 390, in _load_cfg_py_source
    module = _load_module_from_file("yacs.config.override", filename)
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 554, in _load_module_from_file
    spec.loader.exec_module(module)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 1023, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/local/lib/python3.13/site-packages/vismatch/third_party/MatchAnything/imcui/third_party/MatchAnything/configs/models/roma_model.py", line 2, in <module>
    cfg.ROMA.RESIZE_BY_STRETCH = True
    ^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/yacs/config.py", line 141, in __getattr__
    raise AttributeError(name)
AttributeError: ROMA
 
SSL certificate issue when downloading model (click to expand) Windows: image image image

Vincentqyw and others added 10 commits March 21, 2026 18:14
- Add .mypy_cache, .pytest_cache, and .claude/settings.local.json to .gitignore
- Update CLAUDE.md with new directory structure and installation instructions
- Add top-level utility exports (get_default_config_path, get_example_data_path, get_version)
- Refactor app.py to use new package-level imports for HuggingFace Spaces compatibility
Excludes 82MB example datasets from PyPI distribution to reduce package size.
Adds auto-download from HuggingFace on first run, with IMCUI_DATA_DIR env
variable and -d flag support for local datasets.
- Remove stored example datasets from .gitignore and project structure
- Add auto-download from HuggingFace to user cache on first run
- Update CLAUDE.md with new datasets management documentation
- Simplify CLI options (-d flag now auto-downloads if not found)
Simplify import statements by exporting ImageMatchingApp from the main
imcui package. Both entry points (CLI and app.py) now use unified imports
from the top level instead of importing from imcui.ui.app_class.

This provides a cleaner API for users:
- from imcui import ImageMatchingApp (new, recommended)
- from imcui.ui import ImageMatchingApp (still works)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Break down imcui/ui/utils.py into focused modules:
- config_utils.py: configuration and path management
- image_utils.py: image processing utilities
- matching.py: matching logic
- geometry.py: geometry utilities
- examples.py: example data handling

Also rename app_class.py -> image_matching_app.py for clarity.
Update all imports across the codebase to use new module structure.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Fix matching.py to import ModelCache from model_cache (as ARCSizeAwareModelCache)
- Fix __init__.py to import process_ransac_matches (not proc_ransac_matches)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Document new UI module structure after utils.py split
- Add table of modular components (config.py, matching.py, geometry.py, etc.)
- Update exports documentation for imcui.ui
- Fix deprecated file references

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Integrate Rerun SDK for advanced 3D point cloud visualization
- Fix COLMAP camera initialization and API compatibility issues
- Add Rerun component to Gradio interface for interactive 3D viewing
- Improve color-based point cloud generation from matched keypoints
- Add proper error handling and path validation for Rerun recording
- Update dependencies to include gradio_rerun and rerun packages
- Refactor Rerun API usage to follow official documentation
- Implement dynamic point cloud scaling and coordinate axis visualization
- Add comprehensive HTML fallback for Rerun component failures

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
@Vincentqyw Vincentqyw added the enhancement New feature or request label Mar 21, 2026
Vincentqyw and others added 2 commits March 22, 2026 12:23
- Add .readthedocs.yaml configuration for automatic documentation builds
- Create comprehensive documentation structure in docs/ directory
- Add documentation for installation, quickstart, CLI, API, configuration, and development
- Configure Sphinx with autoapi support and modern theme
- Update pyproject.toml with docs optional dependencies
- Include multiple output formats: HTML, PDF, EPUB

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
…ation structure

- Update API function references from imcui.api to imcui.ui
- Correct configuration file structure to use defaults section
- Fix RANSAC method names (CV2_USAC_MAGSAC instead of MAGSAC)
- Update configuration parameters and threshold values
- Add comprehensive troubleshooting guide
- Fix typo in installation documentation (warning directive)
- Update configuration examples across all documentation files

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Vincentqyw and others added 4 commits March 22, 2026 13:00
Improve documentation based on vismatch repository standards:

Sphinx Configuration:
- Add sphinx.ext.linkcode extension for GitHub source code linking
- Implement linkcode_resolve function to link docs to GitHub
- Expand intersphinx mapping (add cv2 and torch references)

Documentation Structure:
- Reorganize index.rst into three sections: Getting Started, Configuration & Models, Development & API
- Add comprehensive models.rst page with model categories and details
- Improve documentation navigation and organization

README Improvements:
- Clarify documentation access links
- Reorganize online demo section for better visibility
- Remove redundant requirements section

These changes align documentation standards with vismatch repository while maintaining IMCUI-specific content.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
…umentation

- Removed the readthedocs documentation link from README.md
- Added a new comprehensive API documentation file (docs/api.mdx) with complete API reference
- The new documentation includes detailed module and class documentation for the entire codebase
- This provides better local documentation coverage and reduces external dependencies
Updated the footer social media links in docs.json to reflect current profiles:
- Changed X (Twitter) handle from 'realcat' to 'AlphaRealcat'
- Replaced LinkedIn link with personal website link pointing to Hugging Face profile
Remove 'integrations' section containing algolia and analytics keys
as they are not supported by Mintlify docs.json schema. Also update
huggingface social link type to 'website' and fix X handle.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Remove missing logo configuration and add root path redirect to fix
404 error on Mintlify deployment. Normalize redirect paths and fix
Claude Code link to use proper URL.

Changes:
- Remove logo config (files don't exist, will use text logo)
- Add "/" redirect to "introduction" for homepage
- Normalize redirect paths (remove leading slash in destinations)
- Fix Claude Code link from "claude-code" to "https://claude.ai/code"

This should resolve the 404 issue on https://realcat.mintlify.app/
in monorepo configuration.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Vincentqyw and others added 3 commits March 22, 2026 15:10
MAJOR FIXES:
- Add frontmatter to all documentation pages (installation, api, cli,
  configuration, development, models, troubleshooting) to resolve 404 errors
- Remove missing logo configuration
- Fix link formatting consistency across all documentation files

NAVIGATION IMPROVEMENTS:
- Install/Launch sections now default to expanded (no collapsed accordions)
- Unified internal link format (removed leading slashes)
- Fixed all navigation anchors in docs.json
- Added root path redirect from "/" to "introduction"

TECHNICAL DETAILS:
- All pages now have proper frontmatter with title and description
- Internal links use consistent format: "page-name" instead of "/page-name"
- AccordionGroup components now use defaultOpenAll prop for better UX

This should fix all 404 errors on mintlify.app deployment including:
- https://realcat.mintlify.app/installation
- https://realcat.mintlify.app/api
- All other documentation pages

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Add draft PR checks to all CI/CD workflows at job level to avoid running
unnecessary builds and tests for work-in-progress pull requests.

Affected workflows (job-level if conditions):
- ci.yml: Skip CI build job for draft PRs
- pip.yml: Skip multi-Python version build job for draft PRs
- docker-publish.yml: Add draft check to job (tag-triggered workflow)

Logic: if: github.event.pull_request.draft == false || github.event_name == 'push'

This saves CI/CD resources and reduces noise for WIP PRs while still
allowing testing via workflow_dispatch or direct pushes.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Mintlify requires absolute paths (with leading slashes) for
navigation.global.anchors href values, they cannot be relative paths.

Changed:
- /installation (was: installation)
- /quickstart (was: quickstart)
- /api (was: api)

This fixes the Mintlify deployment validation error:
"Must be a valid URL or relative path"

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Vincentqyw and others added 4 commits March 22, 2026 15:33
- Move docs.json to root for better Mintlify integration
- Add comprehensive API documentation for all core modules:
  * config-utils.mdx - Configuration utilities
  * core.mdx - Core API functions
  * geometry.mdx - Geometry estimation
  * image-utils.mdx - Image processing utilities
  * matching.mdx - Matching algorithms
  * visualization.mdx - Visualization tools
- Update navigation paths to use docs/ prefix
- Add GitHub CTA button to top navigation
- Remove favicon config (Mintlify default)
- Update multiple doc files with new structure
- Add gitignore entries for development artifacts

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Convert all HTML comments to MDX JSX-style comments to fix parsing errors:
- bug_report.md - 2 comment conversions
- feature_request.md - 2 comment conversions
- question.md - 3 comment conversions
- PULL_REQUEST_TEMPLATE.md - 2 comment conversions

This resolves MDX parsing errors caused by HTML comment syntax in templates.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Move docs/api/index.mdx to docs/api.mdx for simpler routing
- Update navigation to use docs/api instead of docs/api/index
- Remove global anchors section from docs.json
- Streamline API documentation structure for improved navigation

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Update redirect destination from docs/api/index to docs/api to match
the flattened API documentation structure.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
…uction

Removed unnecessary circular redirect for /docs/api path that was causing
404 errors on production while working fine on localhost. The page is
properly configured in navigation structure, so the redirect is redundant.

- Removed circular redirect: source "/docs/api" -> destination "docs/api"
- Added .mintignore file to exclude build artifacts and development files
- Verified build validation passes with no broken links

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Enhanced documentation with detailed architecture diagrams and resolved icon
display issues across all documentation pages.

- Added Mermaid system overview diagrams in introduction.mdx and development.mdx
- Created detailed project structure visualization
- Integrated project color scheme for consistent branding
- Fixed invisible icons by replacing with supported Lucide icons
- Updated project name to "IMCUI" in docs.json
- Removed duplicate .mintignore file

Icon fixes include:
- Architecture cards: box, monitor, sliders_horizontal, code
- Quickstart steps: upload, search, option, check
- Performance cards: check-circle, heart
- Algorithm selection: check-square, hard-drive, star
- Navigation cards: file-text, alert-triangle, search, play

Diagrams support both light and dark modes with proper styling.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Updated project documentation files to reference the new Mintlify documentation site:
- Added documentation badge to README.md badge section
- Added documentation link to CLAUDE.md project overview
- Added documentation files info to CLAUDE.md important files section

Documentation: https://imcui.mintlify.app

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Add a new "Swap Image" button that allows users to quickly swapping
the two input images without having to re-upload them. This improves
user experience when working with image pairs.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
- Improve model cache configuration with better defaults
- Update config.py with optimized settings
- Enhance UI performance with improved caching strategies

Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant