README.md

fremor

fremor CMORizes FRE output with CMOR. It is both a conda and pip package.

Documentation is on readthedocs.

License

Apache License 2.0 — see LICENSE.md

DOI/Citation

Under construction! Sometimes the badge doesn't render, try refreshing.

Contributors

Background and Purpose

fremor is a model output rewriter (CMORizer) for FRE/FMS based models and output. It is specifically geared for standardizing NOAA-GFDL datasets for further quality control checks, assessments and data publishing pipelines in the context of CMIP7 using the CMOR library.

Relationship to fre-cli

fremor was originally the fre.cmor submodule of NOAA-GFDL/fre-cli and so stands on the shoulders of it's contributors, retaining it's general structure and lessons learned from it. Future re-integrations back into fre-cli, as a formal package dependency, are being assessed.

AI Disclaimer

AI was heavily used in the creation of this repository, primarily github's copilot with Claude (opus4.6, sonnet4.6, and haiku), and Gemini and Chat-GPT models to a lesser extent, in agent mode. Claude and Codex agents have also contributed.

Quickstart

# Initialize CMIP6Plus resources (config template + MIP tables)
fremor init -m cmip6plus -e exp_config.json -t mip-cmor-tables --fast

For an overview of required inputs and sample commands, see the CMOR Quickstart.

Installation / Access

via PPAN / modules

If you're trying to gain access to fremor functionality as quickly as possible:

# a tagged version of fremor, post-releases will never be named modules
module load fremor/X.Y.Z

## (COMING SOON) the current post-release, i.e., latest commit on main
#module load fremor/test

via conda and/or conda-forge

If you have a path to a fremor environment you can activate it like so:

conda activate some/path/to/fremor_env

If you want your own fremor environment:

# the environment will be named fremor_en
conda create -n fremor_env conda-forge::fremor

# see fremor_env in the list --> activate it by name
conda env list
conda activate fremor_env

or, if you've already activated a conda environment

conda create -n empty_env
conda activate empty_env
conda install -c conda-forge fremor

# equivalent syntax
conda install conda-forge::fremor

pip install into a virtual environment (conda/venv)

via pip install [-e] . with a checkout of the code

If you're trying to develop fremor capabilities, or edit the code to your liking in either a big or small way, this is for you. This checks out the code, creates and activates an environment, installs into the environment, and runs all unit-tests and pylint checks:

# omit --recursive if you don't want tables as submodules
git clone --recursive https://github.com/NOAA-GFDL/fremor.git
cd fremor

# create an environment and install the local checkout
conda env create -f environment.yaml
conda activate fremor
pip install -e . 

# Run tests
pytest fremor/tests/

# Run linter
pylint --rcfile pylintrc fremor/

via pip install fremor without a checkout of the code

Not all of fremor's dependencies are pip installable, so this method assumes you've grabbed the requirements, and made them available to fremor in a common context via another method (conda is still easiest). Assuming this was done correctly and all relevant python modules are installed to locations in PYTHONPATH, pip install fremor should install the package from PyPI.

Usage

as a command line interface (CLI)

The CLI entry point is fremor, a set of routines for preparing and CMORizing model output for CMIP workflows.

# The full list of subcommands
fremor init      # Initialize CMOR configuration resources: generate template user config, fetch tables
fremor find      # Find and print variables in MIP tables according to your variable lists or other input
fremor varlist   # Create a simple variable list of netCDF files in a directory
fremor config    # Generate a basic CMOR YAML configuration from a pp directory tree
fremor check     # Audit mapping coverage, and optionally input staging and vertical dimensions
fremor map       # Interactively review and edit mappings before writing them to variable-list files
fremor stage     # Recall all mapped YAML-run inputs in one dmget batch
fremor resolve   # Resolve a FRE model YAML + cmor/grids YAMLs into one combined document for inspection
fremor yaml      # Bulk routine for processing data based on a CMOR YAML config, calls fremor run many times
fremor run       # Lowest-level routine, no CMOR YAML needed, rewrites output files in a directory with CMOR

Review mappings and stage archive data

After fremor config produces a self-contained CMOR YAML file, use check to review which MIP variables are unmapped, mapped more than once, or mapped to an unknown table entry. --staging also checks that the selected input files are present and disk-resident; --dims compares a representative input file's vertical dimension with the MIP table.

fremor check -y cmor.yaml --staging --dims
fremor check -y cmor.yaml Amon --show_mapped

Use map when the report identifies mappings that need human review. It opens a terminal UI: select a MIP variable to see its table definition, select a post-processing file to preview it, then press m to stage a mapping or d to stage its removal. Press s to save all staged edits; nothing is written before then.

fremor map -y cmor.yaml Amon

If the selected inputs are archived offline, inspect the planned recall first, then submit the complete deduplicated selection in one dmget batch.

fremor stage -y cmor.yaml --dry_run
fremor stage -y cmor.yaml

The CLI offers full logging and verbosity control independent of the command chosen:

# verbosity and logging
fremor -v  ...          # INFO level logging
fremor -vv ...          # DEBUG level logging
fremor -q  ...          # ERROR level only (quiet)
fremor -l mylog.txt ... # Log to file (appends)

If you've used the previous fre cmor command, there is a direct mapping of syntax:

# past fre-cli command
fre -vv -l logfile.txt cmor <COMMAND> [OPTIONS]

# fremor equivalent
fremor -vv -l logfile.txt <COMMAND> [OPTIONS]

as a python module

Each CLI subcommand (run, yaml, etc.) maps to an API under under fremor, so the CLI functionality is equivalently available via import in scripts as a proper python module

Versioning and tags

fremor uses a post-release scheme to identify development beyond the latest tagged version. To avoid confusion with fre-workflows and fre-cli, which often demand that the version tags match, fremor's version format is X.Y.Z[.post].

New Release Procedure

see .github/PULL_REQUEST_TEMPLATE/new_release.md

Requirements

fremor's requirements are deliberately kept loose, so that it can be installed in as wide array of conda environments as possible. You'll find stricter ones in environment.yaml. Currently, the package build is using the following, slightly looser requirements:

• python>=3.11
• click>=8.2
• cmor>=3.15.0
• netCDF4>=1.7
• numpy>=2
• pyyaml

For development and testing, pylint, pytest, and pytest-cov are all highly recommended as helpful additions.

CI/CD Workflows and QA

codecov

pylint

conda env create -f environment.yaml

python==3.11:

python==3.12:

python==3.13:

python==3.14:

cmor==nightly:

conda build .

conda-forge/fremor-feedstock

readthedocs and sphinx

WCRP Compliance Checking

Under development

The wcrp_compliance_check workflow validates CMORized NetCDF outputs against WCRP project specifications using cc-plugin-wcrp, a plugin for the IOOS compliance-checker. This pipeline:

• Runs automatically on pull requests and via manual dispatch
• Executes unit tests to generate CMORized output files
• Gathers and categorizes outputs by CMIP version (CMIP6, CMIP7)
• Validates outputs using the wcrp_cmip6 compliance checker
• Uploads compliance reports as workflow artifacts (retained for 30 days)

To view compliance results from a workflow/CI run:

1. Navigate to the Actions tab in GitHub
2. Select the wcrp_compliance_check workflow run
3. Download the wcrp-compliance-reports artifact