firebase documentation

CONTRIBUTING.md

@@ -17,6 +17,30 @@ project documentation. If you cannot find the documentation you're
 looking for, please file a GitHub issue with details of what
 you'd like to see documented.
 
+## Firebase Overview
+
+cellPACK Studio reads recipe data from Firebase to render the editor and polls job status during a packing. When a user submits an edited recipe, the client sends the modified recipe as JSON in the packing-request body and the backend processes it directly.
+
+### Collections Used by cellPACK Studio
+
+| Collection | What It Does | Access |
+|------------|--------------|---------------|
+| `example_packings` | Maps recipes to UI display names and editable fields | Read |
+| `editable_fields` | Defines which recipe fields users can edit | Read |
+| `recipes` | Full recipe data with references | Read |
+| `objects` | Object definitions (molecules, organelles) | Read |
+| `gradients` | Gradient definitions for spatial distributions | Read |
+| `composition` | Composition definitions | Read |
+| `job_status` | Packing job progress | Poll (read), refresh `timestamp`, cleanup |
+
+> **Note:** The `configs` collection is managed by the backend. Scheduled cleanup of `job_status` is run by this repo via `.github/workflows/cleanup.yml` (see `scripts/cleanup.ts`). For the complete database schema, see [FIREBASE_SCHEMA.md](FIREBASE_SCHEMA.md).
+
+### Getting Access
+
+- **Development:** Create your own Firebase project in test mode. See [Firebase Firestore tutorial](https://firebase.google.com/docs/firestore).
+- **Staging:** Contact the code owner for credentials, then configure your `.env` file.
+
+
 ## How to Contribute
 
 Typical steps to contribute:

FIREBASE_SCHEMA.md

@@ -0,0 +1,17 @@
+# Firebase Schema Documentation
+
+This document describes the Firestore database schema used by cellPACK. Collections are accessed by the client (cellPACK Studio), the core package (backend), or both.
+
+## Collections
+
+| Collection Name      | Purpose                                                           | Key Fields                                                                                                                                          | Read                                                                         | Write                                                        | Cleanup Policy                                | Related Process                                                  | Notes                                           |
+| -------------------- | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------ | --------------------------------------------- | ---------------------------------------------------------------- | ----------------------------------------------- |
+| **recipes**          | Stores the full-structured recipes                                | `name`, `id`, `dedup_hash`, `description`, `version`, `format_version`, `bounding_box`, `composition`, `objects`, `gradients`, `optional_gradients` | Recipe loading, UI display, user edit comparison, packing execution          | Backend uploads via admin CLI tool                           | **No cleanup** - Permanent storage            | Recipe loading, reference resolution, comparison with user edits | `optional_gradients` manually added             |
+| **configs**          | Stores packing configuration settings                             | Referenced by path (e.g., `firebase:configs/{id}`)                                                                                                  | Building packing job request, backend reads during packing                   | Backend admin uploads via CLI tool                           | **No cleanup** - Permanent storage            | Packing job configuration, submitted to Docker server            | Referenced but not directly queried by frontend |
+| **objects**          | Stores object definitions (molecules, organelles)                 | `name`, `id`, `dedup_hash`, `type`, `color`, `packing_mode`, `place_method`, `radius`, `inherit`, `gradient`, `representations`, `jitter_attempts`  | Recipe reference resolution, inheritance chain resolution, packing execution | Backend admin uploads with deduplication                     | **No cleanup** - Permanent storage            | Reference resolution in recipes, inheritance chain resolution    |                     |
+| **gradients**        | Stores gradient definitions for spatial distributions             | `name`, `id`, `dedup_hash`, `description`, `mode`, `mode_settings`, `weight_mode`, `pick_mode`, `reversed`, `invert`                                | Recipe/object reference resolution, editable field options                   | Backend admin uploads with deduplication                     | **No cleanup** - Permanent storage            | Reference resolution, gradient options for editable fields       | Can be combined (multiple per object)           |
+| **composition**      | Stores composition definitions (what objects go where)            | `name`, `id`, `dedup_hash`, `count`, `molarity`, `object`, `inherit`, `regions` (interior, surface, leaflets)                                       | Recipe reference resolution, region-based placement                          | Backend admin uploads with topological sort                  | **No cleanup** - Permanent storage            | Reference resolution, region-based object placement              |          |
+| **job_status**       | Tracks AWS packing job status                                     | `status`, `error_message`, `outputs_directory`, `result_path`, `timestamp`                                                                          | Frontend polling (every 500ms), progress monitoring, result retrieval        | Backend updates during job execution (RUNNING → DONE/FAILED); frontend refreshes `timestamp` after the final read | **30 days** - cleaned if older than 30 days         | Job monitoring, status polling, result retrieval                 |          |
+| **example_packings** | Maps example recipes to configs and defines UI metadata           | `name` (display name), `recipe` (recipe ID), `config` (config ID), `editable_fields` (array of field IDs), `result_path`, `outputs_directory`       | App startup, recipe dropdown population                                      | Populated by recipe authors via the Studio upload script | **No cleanup** - Permanent storage            | App initialization, recipe dropdown population, UI configuration | **Details for upload** - [documentation](https://github.com/mesoscope/cellpack/blob/main/docs/STUDIO_SITE.md#cellpack-studio-site) |
+| **editable_fields**  | Defines which recipe fields users can edit in UI                  | `id`, `name`, `data_type`, `input_type`, `description`, `path`, `min`, `max`, `options`, `gradient_options`, `conversion_factor`, `unit`            | App initialization, form generation, input validation                        | Defined by users through upload script                     | **No cleanup** - Permanent storage            | Dynamic form generation, input validation, gradient options      | **Details for upload** - [documentation](https://github.com/mesoscope/cellpack/blob/main/docs/STUDIO_SITE.md#cellpack-studio-site)     |
+

README.md

@@ -19,16 +19,15 @@ Front end website to interact with the cellPACK services running in AWS. This we
 ### cellPACK Server
 This client interacts with the cellPACK server, which consists of a variety of backend services hosted in AWS to run [cellPACK packings](https://github.com/mesoscope/cellpack). These AWS services include:
 * **API Gateway**: cellPACK REST API providing this client with access to needed AWS resources for running and receiving data from cellPACK jobs. Includes the following endpoints:
-  * POST /submit-packing?recipe={myrecipe}&config={myconfig}
-  * GET /logs?logStreamName={name}
-  * GET /packing-status?jobId={id}
-* **Batch**: A call to POST /submit-packing launches a new batch job to run. A call to GET /packing-status will return the status of the specified batch job. Once the job is completed, the path to the results file(s) will be added to the results table of the cellPACK Firebase database.
-* **S3**: Result files from the AWS Batch job are written to the `cellpack-demo` S3 bucket.
-* **ECR**: Docker image built from the [cellPACK github repo](https://github.com/mesoscope/cellpack) is published to the `cellpack-private` ECR repository. That image defines the container specificationsin which the batch job will run.
-* **CloudWatch**: Logs from each AWS Batch job are written to CloudWatch. These logs can be accessed via the GET /logs endpoint.
+  * POST /start-packing?recipe={myrecipe}[&config={myconfig}]
+* **ECS**: A call to POST /start-packing launches a new AWS packing job to run. Once the job is completed, the path to the results file(s) is written to the job's `job_status` entry in the cellPACK Firebase database.
+* **S3**: Result files from the AWS packing job are written to the `cellpack-demo` S3 bucket.
+* **ECR**: Docker image built from the [cellPACK github repo](https://github.com/mesoscope/cellpack) is published to the `cellpack-private` ECR repository. That image defines the container specifications in which the AWS packing job will run.
+
+### cellPACK Database
+* **Firebase Firestore**: The cellPACK database is hosted in Firebase Firestore. This database stores all recipes, objects, gradients, compositions, packing configurations, and job statuses. See [CONTRIBUTING.md](CONTRIBUTING.md#firebase-overview) for Firebase overview and [FIREBASE_SCHEMA.md](FIREBASE_SCHEMA.md) for the complete database schema.
 
 #### Resources
 * [Server Architecture Overview Diagram](https://docs.google.com/presentation/d/1eG2XCxgYNaoDIYI-M6Tzef17bGuFirZZhmfZlBFTaXc/edit#slide=id.g26c8fd413da_0_34)
-* [AWS Batch Dashboard](https://us-west-2.console.aws.amazon.com/batch/home?region=us-west-2#)
 * [Staging Firebase Database](https://console.firebase.google.com/u/0/project/cell-pack-database/firestore/databases/-default-/data/~2Fcomposition~2F9XjxZ0ApsNQqCXUbtVTc)
 * [Project Notes](https://docs.google.com/document/d/1jqIvf8DzjWgzbG-NMMJ8pdQbi2qQ7wrzLdLqyR7F0i0/edit?tab=t.0#heading=h.yg9wht4r88xr)