Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7acda2e
sfs: initial config
lucaspar Mar 19, 2026
fd0c0fd
sfs: more config
lucaspar Mar 19, 2026
485e15f
solved filer persistence issue; sfs docs
lucaspar Apr 7, 2026
1e5ac48
docs to interact with sfs using the minio cli
lucaspar Apr 7, 2026
14d0090
integrating seaweed and gateway compose stacks
lucaspar Apr 7, 2026
6fe48e5
increased automation for sfs migration
lucaspar Apr 7, 2026
f340490
wip
lucaspar Apr 9, 2026
23217ae
additional env variables for sfs integration
lucaspar Apr 14, 2026
23dd3e0
created dual storage backend + tests
lucaspar Apr 14, 2026
4403901
sfs env things
lucaspar Apr 14, 2026
f32638c
ignoring hostnames file
lucaspar Apr 14, 2026
68bba36
additional healthchecks for services
lucaspar Apr 16, 2026
4d9b883
hardening nginx config
lucaspar Apr 16, 2026
1a77c06
adjustments to automated secret generation
lucaspar Apr 16, 2026
3d98c66
trying to get sfs loading on ci
lucaspar Apr 16, 2026
352b093
object store integrity checker script
lucaspar Apr 20, 2026
52fd9a9
explicitly controlling celery concurrency
lucaspar Apr 28, 2026
09dbb76
seaweedfs: align prod deployment with sfs checklist
lucaspar May 5, 2026
af9ead3
refactor: rename SFS/MINIO settings to PRIMARY/SECONDARY
lucaspar May 5, 2026
84bf224
infra: replace MinIO with RustFS in compose files
lucaspar May 5, 2026
75ae9c8
infra: consolidate storage env files and update scripts
lucaspar May 5, 2026
35ef34d
docs: update minio/sfs references to primary/secondary
lucaspar May 5, 2026
e0cc088
docs: add deployment documentation and update gateway deploy script
lucaspar May 7, 2026
9b440de
infra: unify seaweedfs compose files across all environments
lucaspar May 7, 2026
7ddf1fd
fix: enable external network for CI compose
lucaspar May 7, 2026
38a7913
feat: add seaweedfs health check script and just recipes
lucaspar May 7, 2026
a8be586
feat: add SDS_ENV inline override to env-selection scripts
lucaspar May 7, 2026
1e7415e
addressing ci issues
lucaspar May 8, 2026
4cc6b27
infra: wire storage env into gateway compose and justfile
lucaspar May 9, 2026
c9be5a6
fix: improve deploy scripts with debug logs and uid-based ownership
lucaspar May 9, 2026
3a689a4
gwy: django management cmd to create buckets
lucaspar May 9, 2026
e20a343
feat: add seaweedfs as secondary object store for local/dev environments
lucaspar May 9, 2026
29ba6d2
fix: resolve ruff linting errors in object store tests and minio client
lucaspar May 9, 2026
01b76b9
fix: clear cluster create_index block in tests and deploy scripts for…
lucaspar May 12, 2026
83c983f
fix: update test assertion to match settings-driven default services …
lucaspar May 12, 2026
3d9559c
fix: make secondary object store optional with None-guarded operations
lucaspar May 13, 2026
54f2529
sfs: upgraded docker image
lucaspar Jun 8, 2026
19e04d7
sfs: yaml indentation
lucaspar Jun 8, 2026
027d0cf
upgraded all things
lucaspar Jun 9, 2026
39b6451
ci: several improvements
lucaspar Jun 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/gwy-code-quality.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ jobs:
- uses: actions/checkout@v6

# this ubuntu-latest version ships node 18, which was causing issues
# https://github.com/actions/setup-node#usage
- uses: actions/setup-node@v6
with:
node-version: 24
Expand All @@ -52,7 +53,7 @@ jobs:

- name: Cache prek hooks
id: cache-prek
uses: actions/cache@v4
uses: actions/cache@v5
# https://github.com/actions/cache/blob/main/examples.md#python---pip
with:
key: prek-gateway-${{ hashFiles('gateway/.pre-commit-config.yaml') }}
Expand Down Expand Up @@ -99,8 +100,8 @@ jobs:
run: |
npm install -g rust-just

- name: Deploy action
run: ./scripts/deploy.sh ci
- name: Deploy action (RustFS only — no SeaweedFS)
run: SDS_SKIP_SFS=true ./scripts/deploy.sh ci
working-directory: ./gateway

- name: Run tests
Expand Down
7 changes: 2 additions & 5 deletions .github/workflows/sdk-code-quality.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
- uses: actions/checkout@v6

# this ubuntu-latest version ships node 18, which was causing issues
# https://github.com/actions/setup-node#usage
- uses: actions/setup-node@v6
with:
node-version: 24
Expand All @@ -48,16 +49,12 @@ jobs:

- name: Cache prek hooks
id: cache-prek
uses: actions/cache@v4
uses: actions/cache@v5
# https://github.com/actions/cache/blob/main/examples.md#python---pip
with:
key: prek-${{ hashFiles('.pre-commit-config.yaml') }}
path: ~/.cache/prek/

- name: Install gateway frontend dependencies
working-directory: ./gateway
run: npm ci

- name: Install hooks
working-directory: ./sdk
run: uv run --dev prek install --install-hooks
Expand Down
9 changes: 5 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
.DS_Store

.agents
seaweedfs
**/agents.md
.config/agents/
.coverage
.coverage.*
.DS_Store
**/agents.md
agents.md
htmlcov/
seaweedfs
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ repos:

# linter for markdown files - see .rumdl.toml to configure rules
- repo: https://github.com/rvben/rumdl-pre-commit
rev: v0.2.4
rev: v0.2.5
hooks:
- id: rumdl # Lint only (fails on issues)
- id: rumdl-fmt # Auto-format and fail if issues remain
Expand Down
13 changes: 0 additions & 13 deletions gateway/.envs/example/minio.env

This file was deleted.

24 changes: 24 additions & 0 deletions gateway/.envs/example/storage.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# ====================== STORAGE ENV ======================
# PRIMARY (RustFS) — S3-compatible storage, default for local/CI
# SECONDARY (SeaweedFS) — S3-compatible object store for local/dev

# PRIMARY (RustFS) credentials
PRIMARY_ACCESS_KEY_ID=admin
PRIMARY_ENDPOINT_URL=sds-gateway-local-rustfs:9000
PRIMARY_S3_ENDPOINT_URL=http://sds-gateway-local-rustfs:9000
PRIMARY_SECRET_ACCESS_KEY=admin
PRIMARY_STORAGE_BUCKET_NAME=spectrumx
PRIMARY_STORAGE_USE_HTTPS=false

# SECONDARY (SeaweedFS) credentials
SECONDARY_ACCESS_KEY_ID=admin
SECONDARY_SECRET_ACCESS_KEY=admin
SECONDARY_ENDPOINT_URL=sds-gateway-local-sfs-s3:8333
SECONDARY_S3_ENDPOINT_URL=http://sds-gateway-local-sfs-s3:8333
SECONDARY_STORAGE_BUCKET_NAME=spectrumx
SECONDARY_STORAGE_USE_HTTPS=false

# Transition controls
OBJECT_STORE_DUAL_WRITE_STRICT=false
OBJECT_STORE_READ_FALLBACK_TO_SECONDARY_ENABLED=false
OBJECT_STORE_WRITE_BOTH_ENABLED=false
26 changes: 26 additions & 0 deletions gateway/.envs/example/storage.prod.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# ====================== STORAGE ENV (PRODUCTION) ======================
# SeaweedFS config — see seaweedfs/compose.production.yaml
# RustFS config — see gateway/compose.<env>.yaml

# PRIMARY credentials (RustFS in local and ci, SeaweedFS in prod)
PRIMARY_ACCESS_KEY_ID=admin
PRIMARY_ENDPOINT_URL=sds-gateway-prod-sfs-s3:8333
PRIMARY_S3_ENDPOINT_URL=http://sds-gateway-prod-sfs-s3:8333
PRIMARY_SECRET_ACCESS_KEY=admin
PRIMARY_STORAGE_BUCKET_NAME=spectrumx
PRIMARY_STORAGE_USE_HTTPS=false

# SECONDARY credentials (usually RustFS in prod; absent in local and ci)
SECONDARY_ACCESS_KEY_ID=minioadmin
SECONDARY_ENDPOINT_URL=prod-secondary-rustfs:9000
SECONDARY_ROOT_PASSWORD=<GENERATED SECONDARY ROOT PASSWORD>
SECONDARY_ROOT_USER=minioadmin
SECONDARY_S3_ENDPOINT_URL=http://prod-secondary-rustfs:9000
SECONDARY_SECRET_ACCESS_KEY=<SAME AS SECONDARY_ROOT_PASSWORD>
SECONDARY_STORAGE_BUCKET_NAME=spectrumx
SECONDARY_STORAGE_USE_HTTPS=false

# Transition controls
OBJECT_STORE_DUAL_WRITE_STRICT=false
OBJECT_STORE_READ_FALLBACK_TO_SECONDARY_ENABLED=false
OBJECT_STORE_WRITE_BOTH_ENABLED=false
64 changes: 0 additions & 64 deletions gateway/.github/workflows/ci.yml

This file was deleted.

Loading
Loading