Add uncertainty quantification (via CalibrateEmulateSample.jl) to Calibration experiments

[odunbar]

Issue

Currently, only preliminary UQ is available (from early termination of EKI methods in the calibration experiment). We would like to obtain more realistic estimated posterior distribution, and another script to run an exploratory ensemble to obtain empirical distributions of important land fluxes samples from this posterior.

Solution

CalibrateEmulateSample.jl is designed to take the output from the EKI experiment, and produce approximated posterior distributions.

Updates to Packages [possibly?]

• Ensure a CES compatible environment in e.g. .buildkite/Project.toml

We should add the following to experiments/Calibration:
A CES script:

• An initial script that is matched to an existing EKI experiment, using the inputs (i.e., prior/observational noise) and outputs (i.e., eki pairs) to produce (and store) an approximate posterior
• The script should be supplied with additional validation for the land model emulator. It is hard to assess posterior consistency, but easier to assess emulators' predictions, so we should do this stage well.
• The script should be supplied with additional validation for the posterior distribution

A Forward-ensemble launcher:

• We need to run a forward ensemble on HPC of the land-model from samples of the posterior (can we use clima calibrate tooling for this?). This run is likely different to the EKI ensemble runs.