- Documentation: https://face-rhythm.readthedocs.io
- Preprint: Hakim et al. (2025), bioRxiv
- Issues / support: GitHub Issues
A Python package that turns videos of facial or other behavior into a small set of interpretable behavioral components.
Why use face-rhythm?
- Unsupervised. No labels, no model zoo.
- Interpretable. Each component is a (space × frequency × time) factor you can plot and read off directly.
Interactive notebooks:
demo_pipeline.ipynb— end-to-end demo on a single session. Start here.demo_set_rois_multisession.ipynb— draw and align ROIs across multiple sessions of the same subject.demo_event_alignment.ipynb— align extracted factors to event timestamps and view trial-averaged traces.
Command line for batch runs across many sessions:
python scripts/run_pipeline_basic.py --path_params params.json --directory_save /path/to/project/scripts/params_pipeline_basic.json is a ready-to-edit template.
Python API: see Quick start below, or the full API reference.
conda create -n face_rhythm python=3.12
conda activate face_rhythm
python -m pip install --upgrade pipActivate the env (conda activate face_rhythm) each time you use
face-rhythm.
Linux:
conda install -c conda-forge 'torchcodec=*=cpu*' ffmpeg libstdcxx-ngmacOS:
conda install -c conda-forge 'torchcodec=*=cpu*' ffmpegWindows: skip this step. torchcodec doesn't explicitly support Windows. Installing it often works, but is not guaranteed. Unless you need ultrafast GPU speeds, just use the 'decord' backend, instead.
pip install face-rhythmFor headless servers, GPU acceleration, and installation troubleshooting, see the installation docs.
git clone https://github.com/RichieHakim/face-rhythm.gitimport json
import face_rhythm as fr
with open("params_pipeline_basic.json", "r") as f:
params = json.load(f)
params["project"]["directory_project"] = "/path/to/new/project/"
params["paths_videos"]["directory_videos"] = "/path/to/videos/"
params["ROIs"]["initialize"]["path_file"] = "/path/to/ROIs.h5"
results = fr.pipelines.pipeline_basic(params)Copy scripts/params_pipeline_basic.json
as a template, edit the three paths, and run. Results land in the project
directory as HDF5 files plus summary plots.
pip install --upgrade face-rhythmTo update the cloned notebooks/scripts: cd face-rhythm && git pull.
- Read the video frames (
face_rhythm.helpers.BufferedVideoReader). - Draw ROIs that pick (a) where to track and (b) what region to crop
(
face_rhythm.rois). - Track a dense grid of points via optical flow
(
face_rhythm.point_tracking). - Compute a spectrogram for each point's trajectory
(
face_rhythm.spectral_analysis). - Factorize the (points × frequency × time) tensor with non-negative TCA
(
face_rhythm.decomposition).
face-rhythm runs on CPU by default. Install the CPU setup above first.
PyTorch compute: set project.use_GPU: true in your params. Check CUDA
with:
python -c "import torch; print(torch.cuda.is_available())"OpenCV CUDA: build OpenCV plus opencv_contrib with CUDA enabled, then
make sure that build is the cv2 imported in this env. Useful links:
OpenCV CUDA build options
and opencv_contrib.
NVDEC video decoding: (uses experimental libraries). On Linux/NVIDIA systems, try a CUDA torchcodec package, then pass device='cuda' when constructing video readers:
conda install -c conda-forge 'torchcodec=*=cuda130*' ffmpeg libstdcxx-ngUse cuda126*, cuda129*, or cuda130* to match your driver. Useful
links: TorchCodec CUDA decoding
and NVIDIA Video Codec SDK.
If you use face-rhythm in your research, please cite our preprint:
Hakim et al. (2025). Spectral envelopes of facial movements predict intention, cortical representations, and neural prosthetic control. bioRxiv. https://doi.org/10.1101/2025.09.10.675423
BibTeX and a machine-readable CITATION.cff are at the root of the repo.
Bug reports, feature requests, and pull requests are welcome. Please open an issue before submitting substantial changes.
MIT — see LICENSE.