1.0.0rc4

What's Changed

• docs fixes by @parashardhapola in #165
• Agent by @parashardhapola in #166

Full Changelog: 1.0.0rc3...1.0.0rc4

1.0.0rc3

What's Changed

• Docs restructure by @parashardhapola in #162
• scm_versioning by @parashardhapola in #163

Full Changelog: 1.0.0rc2...1.0.0rc3

1.0.0rc2

What's Changed

• Expanded tests by @parashardhapola in #157
• Harden DataStoreMerge streaming and simplify mapping diagnostics by @parashardhapola in #160
• Add N-way WNN, TEA-seq tutorial, and Seurat ChromatinAssay import. by @parashardhapola in #161

Full Changelog: 1.0.0rc1...1.0.0rc2

1.0.0rc1

This is a major, breaking rewrite toward Scarf 1.0. Analysis still centers on DataStore and Zarr-backed stores, but storage, execution, plotting, provenance, and several workflows are new or replaced.

Existing Zarr v2 stores remain readable and writable. Opening a store does not convert it.

Requires Python 3.12+. Preferred install path uses uv (installation).

See the quick start, tutorials, and API reference.

New capabilities

Provenance and reusable results

• Persisted analysis results are artifacts with provenance (operation, parameters, inputs).
• List, inspect, load, and follow lineage (list_artifacts, inspect_artifact, load_artifact, lineage).
• Related parameter branches can live in one datastore; matching requests can reuse a completed result.
• AssayState exposes the current analysis chain for an assay.

Analysis workflows

• ds.pipeline.run() runs the standard RNA recipe (filter, cell cycle, HVGs, PCA, neighbors, embeddings, clustering, optional doublets/markers/Harmony) and returns artifact refs.
• Graph construction is stage-wise and provenance-backed: run_normalization → run_pca → build_ann_index → query_neighbors → build_connectivity_map (replaces monolithic make_graph).
• Fixed mapping references for atlas-style projection and label transfer (build_mapping_reference / MappingReference), including a Symphony-style query path, label-transfer evidence, and mapping diagnostic plots.
• Doublet scoring via DataStore.run_doublet_detection.
• Gene-set activity scoring with AUCell and WAGGR.
• Multi-sink fate mapping (run_fate_mapping / FateMappingResult).
• Integration metrics on DataStore (iLISI, cLISI, proportional batch mixing, graph connectivity, cluster separability, label concordance).

Storage, scale, and remote data

• New writes use Zarr v3 with local / cloud compression profiles (fast_local, cloud).
• mem_budget and nthreads plan streaming block size and concurrency (planners, not hard RSS caps).
• Open and analyze stores on object storage (s3://, gs://, hf://) without downloading the full matrix.
• mount_datastore keeps counts in a read-only source while writing metadata and results to a separate writable store.
• Optional local_cache stages normalized expression for remote PCA/LSI.
• Optional repack of older stores with python -m scarf.tools.repack_zarr (v2→v3, profile sharding; when sharding counts, also writes feature-major countsT).

Plotting

• New public API: scarf.plotting (often import scarf.plotting as splt).
• Store-bound convenience via ds.plots.*.
• Shared themes/scales, PlotResult returns, large-embedding raster paths, plus mapping diagnostic plots.

I/O and datasets

• Seurat RDS inspect / read / import (inspect_seurat, SeuratReader, SeuratToZarr).
• Format inspectors: inspect_h5ad, inspect_mtx.
• Example and fixture datasets via Cytebase (scarf.cytebase) instead of the old OSF download helpers.

Breaking changes / removed

• Dask is no longer the matrix execution backend. Assay matrices and blockwise work use scarf.matrix.ChunkedArray. Dask and Polars are not dependencies.
• make_graph is removed. Use the stage-wise graph APIs, or ds.pipeline.run() / ds.run_harmony where appropriate.
• Legacy plotting is gone: scarf.plots, scarf.plotting._legacy, and old DataStore.plot_* methods. Use scarf.plotting or ds.plots.
• Unified mapping / layout APIs are gone: load_unified_graph, run_unified_umap, run_unified_tsne, plot_unified_layout. unified_embedding is not a public plotting entry point.
• ZarrMerge is removed. Use AssayMerge / DatasetMerge.
• Legacy dataset helpers are removed: fetch_dataset, show_available_datasets, scarf.readers.datasets, scarf.downloader. Use scarf.cytebase.
• Nabo H5 reader/writer paths are removed.
• to_polars_dataframe is removed. Use to_pandas_dataframe.
• Prenormed marker-search arguments / Assay.save_normed_for_query are removed.
• run_mapping contract changed: it is MappingReference-first; exclude_missing, run_coral, ref_mu, ref_sigma, and target_assay are gone. Use missing_feature_policy in {reference_mean, zero, error}.
• metric_integration is removed. Use metric_label_concordance.
• Many former top-level modules have no forwarding shims (scarf.harmony, scarf.plots, scarf.downloader, scarf.genomics, scarf.markers, scarf.meld_assay, and others). Prefer documented top-level / DataStore imports.
• Soft leftovers that still warn: Assay.save_aggregated_ordering → DataStore.run_pseudotime_aggregation; ATACassay.mark_prevalent_peaks → DataStore.mark_prevalent_peaks.
• If you previously used WNN, re-run integrate_assays(..., method="wnn") and redo dependent layouts/clusters. The affinity / neighbor path was corrected.
• New stores are Zarr v3; old stores stay v2 until you explicitly repack. Import does not create countsT by default; add it via repack when you want faster feature-wise stages.

Migration notes

1. Create a Python 3.12+ environment and install Scarf (installation).
2. Open an existing Zarr v2 store and smoke-test the workflow you care about. Open does not convert the store (Zarr internals).
3. Replace Dask-specific array usage with Scarf chunked-array / DataStore APIs.
4. Replace make_graph and legacy plotting with stage-wise graph APIs / pipeline.run, and scarf.plotting or ds.plots.
5. Update imports away from removed modules and helpers (scarf.plots, ZarrMerge, dataset download helpers, retired top-level paths, etc.).
6. Update mapping callers to MappingReference and the new missing-feature policy; switch metric_integration to metric_label_concordance.
7. Re-run WNN (and dependent UMAP/clusters) if you rely on multimodal graphs (CITE-seq, FAQ).
8. For large or remote work, set mem_budget / nthreads, and use remote open / mount_datastore / local_cache as needed (scale and memory, remote stores).
9. Optionally repack to Zarr v3 + profile sharding, and/or add countsT for faster feature-wise analysis:

0.32.3

What's Changed

• Update for h5ad reader by @suu-yi in #147

Full Changelog: 0.32.2...0.32.3

0.32.2

What's Changed

• Update test_datastore.py by @suu-yi in #146

Full Changelog: 0.32.1...0.32.2

0.32.1

Full Changelog: 0.32.0...0.32.1

0.32.0

What's Changed

• Marker search p-values by @suu-yi in #143

New Contributors

• @suu-yi made their first contribution in #143

Full Changelog: 0.31.4...0.32.0

0.31.4

What's Changed

• introduced max_cells parameter by @parashardhapola in #140

Full Changelog: 0.31.3...0.31.4

0.31.3

Full Changelog: 0.31.2...0.31.3