|
| 1 | +# Contributing |
| 2 | + |
| 3 | +## Adding a New Method |
| 4 | +Let's for example add a matcher called new_matcher. |
| 5 | +1. Copy the template to create a new file: `cp vismatch/TEMPLATE.py vismatch/im_models/new_matcher.py` |
| 6 | +2. If the method requires external modules (for example the offical repository of new_matcher), use `git submodule add`: for example, I used this command to add the LightGlue module |
| 7 | + ```bash |
| 8 | + git submodule add https://github.com/cvg/LightGlue vismatch/third_party/LightGlue |
| 9 | + ``` |
| 10 | + This command automatically modifies `.gitmodules` (you should not modify `.gitmodules` manually!), and when cloning the repository it will automatically clone also the LightGlue repo in `vismatch/third_party`. |
| 11 | + |
| 12 | +3. In `vismatch/im_models/new_matcher.py` you only need to implement the method `_forward`, which takes two image tensors as input and returns 6 objects: `[mkpts0, mkpts1, kpts0, kpts1, desc0, desc1]`. The template has more details on how to implement the class. |
| 13 | + |
| 14 | +4. Open `vismatch/__init__.py` and add the model name (all lowercase) to the `available_models` list. Add an `elif` case to instantiate the class, as for the other matchers. |
| 15 | + |
| 16 | +5. If it requires additional dependencies, add them to `requirements.txt` or to the `[project.optional-dependencies]` of `pyproject.toml`. |
| 17 | + |
| 18 | +6. Format the code with [ruff](https://github.com/astral-sh/ruff) |
| 19 | + ``` |
| 20 | + ruff format . |
| 21 | + ruff check --fix . |
| 22 | + ``` |
| 23 | + |
| 24 | +7. Test your model. Make sure the model weights are downloaded automatically in the code, either with huggingface_hub (if on HF), gdown (if on GDrive), or py3_wget (any other platform or HTTP URL). |
| 25 | + ``` |
| 26 | + # Run this and have a look at the generated images in outputs_new_matcher |
| 27 | + python vismatch_match.py --matcher new_matcher --out_dir outputs_new_matcher |
| 28 | + # Run this and make sure it passes the test |
| 29 | + python vismatch_test.py --matcher new_matcher |
| 30 | + ``` |
| 31 | + Now submit a PR! |
| 32 | + |
| 33 | +```{note} |
| 34 | +As authors update their model repos, consider updating the submodule reference here using the below: |
| 35 | +To update a submodule to the head of the remote, run |
| 36 | +```bash |
| 37 | +git submodule update --remote vismatch/third_party/[submodule_name] |
| 38 | +``` |
| 39 | + |
| 40 | + |
| 41 | +## Optional: add docs |
| 42 | +You can create the file `docs/source/model_specific/new_matcher.md` to explain how the model is used. This is especially useful if the model has multiple hyperparameters. |
| 43 | + |
| 44 | +## Optional: adding model weights to the Hugging Face Hub |
| 45 | + |
| 46 | +Although not mandatory, we encourage authors to upload their models to the Hugging Face Hub, under the [image matching organization](https://huggingface.co/vismatch). This will increase model visibility and help track usage of each model. |
| 47 | + |
| 48 | +Here are the steps that we took to add the ELoFTR model: |
| 49 | + |
| 50 | +1. Dowloaded the model from Google Drive (any other storage) |
| 51 | +```py |
| 52 | +!pip install -q pytorch_lightning # Needed for the ELoFTR download |
| 53 | +from pathlib import Path |
| 54 | +from safetensors.torch import save_file |
| 55 | +from huggingface_hub import upload_file |
| 56 | +import gdown |
| 57 | +import torch |
| 58 | + |
| 59 | +weights_src = "https://drive.google.com/file/d/1jFy2JbMKlIp82541TakhQPaoyB5qDeic/view" |
| 60 | +model_path = "eloftr_outdoor.ckpt" |
| 61 | +gdown.download(weights_src, output=model_path, fuzzy=True) |
| 62 | +``` |
| 63 | + |
| 64 | +2. Although weights can be uploaded as PyTorch file, safetensor is preferred. Save the state dict as a [safetensor](https://huggingface.co/docs/safetensors/en/index) |
| 65 | +```py |
| 66 | +state_dict = torch.load(model_path, map_location=torch.device("cpu"), weights_only=False)["state_dict"] |
| 67 | +save_file(state_dict, "eloftr_outdoors.safetensors") |
| 68 | +``` |
| 69 | + |
| 70 | +3. Upload the safetensor file to the Hub (you can upload it to your personal account and later transfer to the organization) |
| 71 | +``` |
| 72 | +from huggingface_hub import HfApi |
| 73 | +
|
| 74 | +api = HfApi() |
| 75 | +api.upload_file( |
| 76 | + path_or_fileobj="eloftr_outdoors.safetensors", |
| 77 | + path_in_repo="eloftr_outdoors.safetensors ", |
| 78 | + repo_id="{your_personal_repo}/eloftr", # personal repository |
| 79 | +) |
| 80 | +``` |
| 81 | + |
| 82 | +You can access the weights here: https://huggingface.co/{your_personal_repo}/eloftr |
| 83 | + |
| 84 | +You can now see in this [PR](https://github.com/alexstoken/vismatch/pull/46) how we can move holistically to the Hub. |
0 commit comments