Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
c0e45f5
chore: bump Go to 1.26
ynsta Apr 22, 2026
c2668bc
docs: add AGENTS.md and development guide
ynsta Apr 22, 2026
a91fcd9
refactor: rename crypto pkg and apply quick hardening fixes
ynsta Apr 22, 2026
34d5afe
refactor: reuse S3 client, bound upstream timeouts, drop redundant MD5
ynsta Apr 22, 2026
401c7ad
feat: cap PutObject bodies and switch KEK derivation to HKDF-SHA256
ynsta Apr 22, 2026
8353ea3
feat: /readyz probes upstream S3; introduce Config struct for DI
ynsta Apr 22, 2026
aa552c2
test(ci): expand unit coverage and add CI + Renovate
ynsta Apr 22, 2026
5d79371
chore: vendor dependencies
ynsta Apr 22, 2026
d261330
refactor: migrate logrus to log/slog
ynsta Apr 22, 2026
bee9277
feat: expose Prometheus metrics on /metrics
ynsta Apr 22, 2026
2dbf244
feat: emit OpenTelemetry traces and correlate with slog
ynsta Apr 22, 2026
afcb316
chore(monitoring): ship alert rules and Grafana dashboard
ynsta Apr 22, 2026
b084b93
test(e2e): add MinIO roundtrip test behind an e2e build tag
ynsta Apr 22, 2026
e89bbd6
feat: warn loudly on startup when S3PROXY_INSECURE is enabled
ynsta Apr 22, 2026
dbb7925
test(e2e): demonstrate at-rest encryption in MinIO roundtrip test
ynsta Apr 22, 2026
55a8cc7
fix(router): stop forwarding upstream checksum headers on transformed…
ynsta Apr 22, 2026
4ce20d7
fix(dockerfile): bump build stage to golang:1.26.2
ynsta Apr 22, 2026
c971709
chore(chart): expose observability in the Helm chart
ynsta Apr 22, 2026
65c5fe8
docs(agents): align Language section with updated setup-dev-project s…
ynsta May 4, 2026
83f388a
docs: refresh AGENTS.md to current iagen-dev rules
ynsta May 5, 2026
1230694
docs(agents): add API/Auth/product-validation sections, align with cu…
ynsta May 5, 2026
359168a
perf(router): release large buffers and FreeOSMemory between phases
ynsta May 19, 2026
758a2c8
Merge main into chore/modernize-and-review
ynsta May 19, 2026
dea9289
fix(chart): set default log level to Info, document observability knobs
ynsta May 19, 2026
74387e8
docs(chart): document observability surface, bump chart to 1.8.1
ynsta May 19, 2026
129b8f4
docs: full README rewrite with reference-grade sections
ynsta May 19, 2026
3b5b876
docs: refresh CONTRIBUTING, add CHANGELOG, enrich Chart.yaml, more ba…
ynsta May 19, 2026
c76a7c9
feat(chart): add values.schema.json with strict-on-known-keys validation
ynsta May 20, 2026
7545a9b
docs: refresh AGENTS.md to current iagen-dev rules
ynsta May 20, 2026
d4c518f
chore(deps): bump Go to 1.26.3 and update all dependencies
ynsta May 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
58 changes: 58 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: CI

on:
pull_request:
branches: ["main"]
push:
branches: ["main"]
workflow_dispatch:

permissions:
contents: read

jobs:
test:
name: Test
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
cache: true
- name: go test
run: go test -race -covermode=atomic -coverprofile=coverage.out ./...
- name: upload coverage
uses: actions/upload-artifact@v4
with:
name: coverage
path: coverage.out

vuln:
name: Vulnerability scan
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
cache: true
- name: govulncheck
run: |
go install golang.org/x/vuln/cmd/govulncheck@latest
govulncheck ./...

build:
name: Build
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v5
- uses: actions/setup-go@v6
with:
go-version-file: go.mod
cache: true
- name: go build
run: go build -v ./...
344 changes: 344 additions & 0 deletions AGENTS.md

Large diffs are not rendered by default.

125 changes: 125 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Changelog

All notable changes to this project are documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

Two version streams move independently:

- **Application** — versioned via Git tags (`vX.Y.Z`) and tracked by the
Helm chart's `appVersion`. Drives the published Docker image tag.
- **Chart** — `charts/s3proxy/Chart.yaml` `version`. Bumped independently
when only chart values/templates change.

## [Unreleased]

### Chart

#### Added
- `values.schema.json`: JSON Schema (draft-07) validated automatically on
`helm install` / `upgrade` / `template` / `lint`. Catches the kind of
typo/out-of-range bug that produced the `--level=-4` default. Strict on
known keys (types, enums, port range, duration patterns, threshold
ranges 0..1 for `prometheusRule.thresholds.highErrorRate`, etc.) but
permissive on unknown top-level keys so user overrides still flow through.

#### Changed
- `1.8.0` → `1.8.1`: default `--level` switched from `-4` (Debug, due to a
legacy out-of-range value) to `0` (Info), matching the binary's documented
default. Verbose logging is now opt-in via `args: ["--level=-1"]`.
- `values.yaml` heavily commented: every key annotated with its
`S3PROXY_*` mapping, OTLP precedence rules and example overrides. Removed a
duplicate `extraEnv:` key that silently shadowed the first declaration.
- `Chart.yaml` enriched with `icon`, `home`, `sources`, `keywords` and
`maintainers` so the chart renders cleanly on Artifact Hub.

### Docs
- README rewritten end-to-end: anchored table of contents, full env var matrix,
CLI flags, metric/alert tables with real PromQL, OTLP tracing notes,
security caveats (`S3PROXY_INSECURE`, `S3PROXY_DECRYPTION_FALLBACK`,
multipart blocked by default), Mermaid PUT/GET sequence diagram, Helm chart
reference table, development workflow.
- `CONTRIBUTING.md` refreshed: Go 1.26, vendor mode, `e2e` build tag,
`golangci-lint` + `govulncheck` invocations, release flow.

### Performance
- `internal/router`: release large request/response buffers and trigger
`runtime.FreeOSMemory()` between encryption phases to keep RSS bounded under
bursty PutObject load (commit `359168a`).

## [1.7.2] — 2026-04-23

### Added
- Opt-in `S3PROXY_DECRYPTION_FALLBACK=1` knob that retries failing GetObject
decryptions with an all-zero KEK, to bridge migrations away from
legacy/unencrypted objects.

## [1.7.1] — 2026-04-22

### Fixed
- Vulnerable dependency upgrades.
- Router now passes its KEK to object handlers (previously a stale package
global was used in some paths).
- Healthcheck endpoints (`/healthz`, `/readyz`) bypass the throttling
middleware so probes do not get rejected under load.
- Reduced memory copies on the object body hot path.

### Added
- `.github/workflows/golangci-lint.yml` runs the linter on every PR.
- Linter configuration `.golangci.yml` with input validation rules.

## [1.7.0] — 2025-10-30

### Added
- TLS certificate handling improvements.
- Project documentation set (`AGENTS.md`, expanded `CONTRIBUTING.md`).

### Changed
- Go toolchain upgrade.
- Dependency refresh.

## [1.6.0] — 2025-07-23

### Changed
- Bumped to Go 1.24.5.
- Helm chart version aligned with app version.

## [1.5.2] — 2025-07-22

### Changed
- Helm chart `args` defaults adjusted.

## [1.5.1] — 2025-07-22

### Fixed
- Helm chart `args` rendering bug.

## [1.5.0] — 2025-07-22

### Added
- Forward-flow request logging including client-cancel and "client closed
connection" events.
- `context.WithoutCancel` propagation on PUT to avoid aborting an in-flight
encryption stream when the client disconnects.

### Fixed
- Special-character handling in forwarded URLs.
- Checksum handling on the upstream PUT path.

## [1.4.0] — 2024-11-14

Initial published release line after the fork from
[edgelesssys/constellation](https://github.com/edgelesssys/constellation).
Earlier `v1.0.0` … `v1.3.0` tags exist for the pre-fork lineage; see the Git
log for details.

[Unreleased]: https://github.com/Intrinsec/s3proxy/compare/v1.7.2...HEAD
[1.7.2]: https://github.com/Intrinsec/s3proxy/compare/v1.7.1...v1.7.2
[1.7.1]: https://github.com/Intrinsec/s3proxy/compare/v1.7.0...v1.7.1
[1.7.0]: https://github.com/Intrinsec/s3proxy/compare/v1.6.0...v1.7.0
[1.6.0]: https://github.com/Intrinsec/s3proxy/compare/v1.5.2...v1.6.0
[1.5.2]: https://github.com/Intrinsec/s3proxy/compare/v1.5.1...v1.5.2
[1.5.1]: https://github.com/Intrinsec/s3proxy/compare/v1.5.0...v1.5.1
[1.5.0]: https://github.com/Intrinsec/s3proxy/compare/v1.4.0...v1.5.0
[1.4.0]: https://github.com/Intrinsec/s3proxy/releases/tag/v1.4.0
137 changes: 99 additions & 38 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,65 +1,126 @@
# Dev with Docker & VSCode
# Contributing to S3Proxy

Thanks for considering a contribution! This document covers the local dev loop,
the linting and security toolchain, and the conventions used in this repo.

## Setup

To set up your development environment:
The repo ships with a `vendor/` tree, so a working Go 1.26 toolchain is all you
need locally. The instructions below use a containerised Go to keep host tooling
isolated, but a native install works the same.

### Option A — containerised Go (VSCode + Docker)

1. **Clone the repository:**
```bash
git clone https://github.com/intrinsec/s3proxy.git
git clone https://github.com/Intrinsec/s3proxy.git
cd s3proxy
```
2. **Run a Docker container with Go:**
2. **Run a Go dev container:**
```bash
docker run --rm -p 4433:4433 -v $PWD:/app -it golang:1.24.5 bash
docker run --rm -p 4433:4433 -v "$PWD":/app -w /app -it golang:1.26 bash
```
This command launches a Docker container with the specified Go version, mounts your current project directory (`$PWD`) to `/app` inside the container, and exposes port `4433`.
3. **Attach VSCode to the running container:**
- In VSCode, use the "Attach to Running Container..." option.
- Select the `golang` container.
- Open the `/app` folder within the attached VSCode window. This allows you to develop within a consistent Go environment without installing Go directly on your host machine.
4. **Run the S3Proxy server:**
Mounts the working tree at `/app` and exposes the proxy port. Inside the
container, `go test`, `go build`, etc. run as usual.
3. **Attach VSCode to the container:**
- Command palette: *"Dev Containers: Attach to Running Container…"*
- Select the `golang` container, open `/app`.
4. **Run the proxy locally:**
```bash
go run s3proxy/cmd/main.go --no-tls --level=-4
export AWS_ACCESS_KEY_ID=…
export AWS_SECRET_ACCESS_KEY=…
export S3PROXY_HOST=s3.fr-par.scw.cloud
export S3PROXY_ENCRYPT_KEY="$(openssl rand -base64 32)"
go run ./s3proxy/cmd --no-tls --level=-1
```
This command starts the S3Proxy server with TLS disabled and a debug log level.
`--level=-1` enables Debug logging; defaults to Info.

### Example
### Option B — native Go

```bash
git clone # or gh repo clone Intrinsec/s3proxy
git clone https://github.com/Intrinsec/s3proxy.git
cd s3proxy
docker run --rm -p 4433:4433 -v $PWD:/app -it golang:1.25.3 bash
# In VSCode, use "Attach to Running Container..." option => select golang container => open /app folder
export AWS_ACCESS_KEY_ID=xxx
export AWS_SECRET_ACCESS_KEY=xxx
export S3PROXY_HOST=xxx
export S3PROXY_ENCRYPT_KEY=toto
go run s3proxy/cmd/main.go --no-tls --level=-4
go test ./... # unit tests
go test -tags e2e ./... # + MinIO roundtrip
go run ./s3proxy/cmd --no-tls --level=-1
```

### End-to-end smoke test

With the proxy running on `:4433` and `aws` configured against it:

```bash
echo "test" > test.txt
aws s3 cp ./test.txt s3://bucket/
aws s3 cp s3://bucket/test.txt ./test.txt
aws s3 rm s3://bucket/test.txt
aws --endpoint-url http://localhost:4433 s3 cp ./test.txt s3://bucket/
aws --endpoint-url http://localhost:4433 s3 cp s3://bucket/test.txt ./test.txt
aws --endpoint-url http://localhost:4433 s3 rm s3://bucket/test.txt
```

## Linting

`golangci-lint` is used for static code analysis to ensure code quality and consistency. It combines multiple linters into a single tool.
`golangci-lint` aggregates the linters configured in `.golangci.yml`:

**Enabled Linters:**
- `bodyclose`: Checks for unclosed HTTP response bodies.
- `gocognit`: Computes and checks the cognitive complexity of functions.
- `goconst`: Finds repeated strings that could be replaced by constants.
- `gocyclo`: Computes and checks the cyclomatic complexity of functions.
- `gosec`: Inspects code for security problems.
- `misspell`: Finds commonly misspelled English words.
- `revive`: A fast, configurable, extensible, flexible, and beautiful linter for Go.
- `staticcheck`: A Go static analysis tool that finds bugs and performance issues.
- `bodyclose` — unclosed HTTP response bodies
- `gocognit` / `gocyclo` — function complexity
- `goconst` — repeated strings that should be constants
- `gosec` — security-focused checks
- `misspell` — common English misspellings
- `revive` / `staticcheck` — general code quality

Run:

**How to Run:**
To run the linter, execute the following command in your development environment:
```bash
golangci-lint run
```
This will analyze the codebase and report any issues based on the configuration in `.golangci.yml`.

CI runs the same configuration in `.github/workflows/golangci-lint.yml`.

## Vulnerability scanning

```bash
govulncheck ./...
```

CI runs `govulncheck` against the vendored dependencies; PRs that introduce
known-vulnerable code paths will fail.

## Tests

- **Unit tests** — `go test -race ./...` (matches CI).
- **E2E (MinIO roundtrip)** — `go test -tags e2e ./...`. The `e2e` build tag
gates a test that spins up MinIO and exercises a full encrypt/decrypt
roundtrip; see `s3proxy/e2e/`.

## Helm chart

```bash
helm lint charts/s3proxy
helm template charts/s3proxy
```

When you change `values.yaml` defaults, render the chart with both the default
values and a representative override (e.g. `serviceMonitor.enabled=true`) to
confirm the templates still produce valid Kubernetes manifests.

## Conventions

- **Commit messages** — Conventional Commits (`feat:`, `fix:`, `chore:`,
`docs:`, `refactor:`, `test:`, `ci:`, `perf:`). Use the scope to point at the
affected area (e.g. `fix(router):`, `chore(chart):`).
- **Repo coding rules** — see [AGENTS.md](AGENTS.md) for the project-wide
conventions (error wrapping, slog field names, package naming, …) that are
enforced both by review and by `golangci-lint`.
- **Sign-off / co-authors** — co-authoring tools and trailers are accepted; do
not commit credentials or large binaries (the repo ignore list catches the
obvious cases but a quick `git status` before staging never hurts).

## Releasing

- The Go binary version is exposed via the Docker image tag, built by
`.github/workflows/docker-build-push.yml` on tag pushes.
- The Helm chart is published to GHCR as an OCI artifact by
`.github/workflows/helm-push.yml`. The `version:` field in
`charts/s3proxy/Chart.yaml` is the chart version; `appVersion:` tracks the
application release.
- User-facing changes go into [CHANGELOG.md](CHANGELOG.md) under the matching
release header.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.25.3 AS build-stage
FROM golang:1.26.3 AS build-stage

WORKDIR /app

Expand Down
Loading
Loading