Skip to content

Commit 2d2f201

Browse files
committed
docs: refresh README
1 parent f3b1f96 commit 2d2f201

1 file changed

Lines changed: 56 additions & 59 deletions

File tree

README.md

Lines changed: 56 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,11 @@
55
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
66

77
A headless, config-driven uploader that fans one batch of files out to many
8-
FTP, FTPS, and SFTP endpoints at once, concurrently, with retries and
9-
post-upload verification. Built for unattended automation (cron, CI,
10-
folder-watchers), not interactive use. A single static binary and one YAML file.
11-
12-
You point it at some files and a list of endpoints; it uploads every file to
13-
every endpoint in parallel, verifies each transfer, retries transient
14-
failures, and exits non-zero if anything failed so a wrapping script can react.
8+
FTP, FTPS, and SFTP endpoints concurrently. You point it at files and a list of
9+
endpoints; it uploads every file to every endpoint in parallel, verifies each
10+
transfer, retries transient failures, and exits non-zero if anything failed so
11+
a wrapping script can react. Run it from cron, CI, or a folder-watcher: one
12+
static binary, one YAML file.
1513

1614
```console
1715
$ ramify ./batch --config stocks.yml
@@ -27,24 +25,22 @@ $ ramify ./batch --config stocks.yml
2725

2826
- **One batch, many destinations.** Declare endpoints in YAML; add or remove
2927
one without touching code. Every endpoint uploads in its own goroutine.
30-
- **Three protocols, one model:** `ftp`, `ftps` (explicit AUTH TLS), and
31-
`sftp` (password or private-key auth, host keys checked against
32-
`~/.ssh/known_hosts`).
33-
- **Verification on by default.** Every upload's remote size is checked
34-
against the local file to catch truncation. Disable with `--no-verify`.
28+
- **Three protocols:** `ftp`, `ftps` (explicit AUTH TLS), and `sftp` (password
29+
or private-key auth, host keys from `~/.ssh/known_hosts`).
30+
- **Verification on by default.** After each upload ramify compares the remote
31+
size against the local file to catch truncation. Disable with `--no-verify`.
3532
- **Retries with reconnect.** A fixed number of attempts per file with a
3633
fixed delay, each attempt reconnecting from scratch. Configurable globally
3734
and per endpoint.
3835
- **Fail-fast preflight** (`--dry-run`): connect, authenticate, and prove the
3936
target directory is writable with a self-deleting probe file, without
40-
touching the files you are actually sending.
37+
touching the files you are sending.
4138
- **Automation-friendly output:** quiet / default / verbose text, or
42-
newline-delimited JSON (`--json`), with meaningful exit codes and errors
43-
always on stderr even under `--quiet`.
39+
newline-delimited JSON (`--json`), with meaningful exit codes and errors on
40+
stderr even under `--quiet`.
4441
- **Secrets via env vars:** `${ENV_VAR}` interpolation in any config field,
4542
resolved at load time. Literal values still allowed.
46-
- **Zero runtime dependencies:** one static binary, no daemon, no config
47-
server.
43+
- **Zero runtime dependencies:** one static binary, no daemon.
4844

4945
## Install
5046

@@ -90,14 +86,14 @@ amd64 and arm64 are built for every release; each archive is listed in
9086
```
9187

9288
Positional arguments may be files or directories, mixed freely. Directories
93-
expand **non-recursively**: every regular file is included; subdirectories and
94-
dotfiles are skipped. Two inputs that would land under the same remote name
95-
(same basename) are rejected up front.
89+
expand **non-recursively**: ramify takes every regular file and skips
90+
subdirectories and dotfiles. Two inputs sharing a basename would land under the
91+
same remote name, so ramify rejects them up front.
9692

9793
## Configuration
9894

9995
Config is a YAML file with a list of `endpoints` plus optional global policy
100-
defaults. Every policy key can be overridden per endpoint.
96+
defaults. Any endpoint can override any policy key.
10197

10298
```yaml
10399
endpoints:
@@ -125,7 +121,7 @@ endpoints:
125121
# Global policy defaults (each overridable per endpoint):
126122
attempts: 3 # total tries per file
127123
retry_delay: 2s
128-
connect_timeout: 30s # bounds the whole connect + login, not just the TCP dial
124+
connect_timeout: 30s # bounds the whole connect + login, TCP dial included
129125
stall_timeout: 5m # fail a transfer idle this long; 0 disables
130126
max_consecutive_connect_failures: 3 # write an endpoint off after this many connect failures in a row
131127
```
@@ -140,29 +136,29 @@ max_consecutive_connect_failures: 3 # write an endpoint off after this many co
140136
| `port` | all | Optional; defaults to the protocol's standard port. |
141137
| `username` | all | Required. |
142138
| `password` | ftp, ftps, sftp | Required for ftp/ftps. For sftp, an alternative or complement to `private_key`. |
143-
| `private_key` | sftp | Path to an SSH private key (`~` expanded). If both this and `password` are set, the key wins and `password` is used only as its passphrase when the key is encrypted. |
139+
| `private_key` | sftp | Path to an SSH private key (`~` expanded). Set both this and `password` and the key wins; `password` then serves as the passphrase for an encrypted key. |
144140
| `overwrite` | all | `delete-first` (default) deletes any existing remote file first; `direct` uploads straight over it. |
145141
| `insecure_skip_verify` | ftps | Disables TLS certificate verification. For self-signed / test servers only; never use against a production endpoint. |
146142

147143
### Policy fields (global or per-endpoint)
148144

149145
| Field | Default | Meaning |
150146
|---|---|---|
151-
| `attempts` | `3` | Total tries per file before it is counted as failed. |
147+
| `attempts` | `3` | Total tries per file before ramify counts it as failed. |
152148
| `retry_delay` | `2s` | Fixed wait between attempts (Go duration string). |
153149
| `connect_timeout` | `30s` | Bounds the entire connect + authenticate sequence. |
154150
| `stall_timeout` | `5m` | Fails a transfer that makes no forward progress for this long. `0` disables. |
155-
| `max_consecutive_connect_failures` | `3` | After this many connect failures in a row, the endpoint's remaining files are skipped as unreachable instead of each retrying. |
151+
| `max_consecutive_connect_failures` | `3` | After this many connect failures in a row, ramify skips the endpoint's remaining files as unreachable instead of retrying each one. |
156152

157153
### Secrets
158154

159-
`${ENV_VAR}` interpolation works in any string field and is resolved when the
160-
config loads. A referenced variable that is unset is a load-time error, so a
161-
typo surfaces immediately instead of as a confusing auth failure mid-run.
155+
`${ENV_VAR}` interpolation works in any string field, resolved when the config
156+
loads. An unset variable is a load-time error, so you catch a typo before the
157+
run starts rather than as a confusing auth failure mid-run.
162158

163-
Literal secrets are allowed but discouraged. Because a config file can hold
164-
plaintext credentials, `ramify` prints a warning if the file is readable by
165-
group or others; `chmod 600 stocks.yml` and keep it out of version control.
159+
Literal secrets work but invite leaks. `ramify` warns when the config file is
160+
readable by group or others; `chmod 600 stocks.yml` and keep it out of version
161+
control.
166162

167163
## Usage
168164

@@ -182,7 +178,12 @@ ramify <path>... --config <file> [flags]
182178
| `--help` | Print usage and exit. |
183179
184180
`--quiet` and `--verbose` are mutually exclusive. Flags and paths may appear in
185-
any order.
181+
any order. ramify treats everything after a bare `--` as a path, so you can
182+
upload a file whose name starts with `-`:
183+
184+
```console
185+
$ ramify --config stocks.yml -- -weird-name.jpg
186+
```
186187

187188
### Output modes
188189

@@ -192,9 +193,6 @@ any order.
192193
| default | per-file success + per-endpoint summary | errors |
193194
| `--verbose` | full event stream incl. byte-level progress | errors |
194195

195-
Failure events always go to stderr, even under `--quiet`: an unattended job
196-
still needs to see what broke.
197-
198196
### JSON output
199197

200198
`--json` reformats whatever the current verbosity level would print as one JSON
@@ -219,25 +217,25 @@ $ ramify ./batch --config stocks.yml --json
219217

220218
## How it works
221219

222-
- **Concurrency.** One goroutine per endpoint, fully parallel. Within an
223-
endpoint, files upload sequentially over a single reused connection (connect
224-
once, upload all, disconnect), which amortizes login cost and respects
225-
servers that cap concurrent connections.
220+
- **Concurrency.** One goroutine per endpoint. Within an endpoint, files upload
221+
sequentially over a single reused connection (connect once, upload all,
222+
disconnect), which amortizes the login cost and keeps ramify inside server
223+
limits on concurrent connections.
226224
- **Retries.** A fixed attempt count with a fixed delay between tries. Each
227-
retry reconnects. Upload and verification are retried together: a
228-
verification failure is treated the same as an upload failure.
229-
- **Verification.** After each upload the remote size is compared against the
230-
local file (`SIZE` / `Stat`), which catches truncation cheaply and works on
231-
all three protocols. Hash-based verification is planned but not in v1; SFTP
232-
has no standard equivalent. `--no-verify` turns it off.
233-
- **SSH host keys.** SFTP endpoints are verified against `~/.ssh/known_hosts`.
234-
There is no insecure-by-default fallback: a host you have not trusted yet
235-
fails with an error, the same trust-on-first-use flow as OpenSSH. Connect
236-
once with `ssh` (or `ssh-keyscan`) to record the key first.
225+
retry reconnects from scratch and re-runs verification, so a failed
226+
verification triggers a retry the same way a failed upload does.
227+
- **Verification.** After each upload ramify compares the remote size against
228+
the local file (`SIZE` / `Stat`), which catches truncation cheaply on all
229+
three protocols. Hash verification waits for a later release, since SFTP has
230+
no standard equivalent. `--no-verify` turns it off.
231+
- **SSH host keys.** ramify checks SFTP hosts against `~/.ssh/known_hosts` and
232+
errors out on an untrusted host, with no insecure fallback, the same
233+
trust-on-first-use flow as OpenSSH. Record the key first by connecting once
234+
with `ssh` or `ssh-keyscan`.
237235
- **Cancellation.** Ctrl-C / SIGTERM cancels cleanly: workers stop starting new
238-
transfers and retries promptly, though an in-flight blocking transfer runs to
236+
transfers and retries, though an in-flight blocking transfer runs to
239237
completion first. Each endpoint still reports its results. A second Ctrl-C
240-
exits immediately (code 130) rather than waiting out a stuck transfer.
238+
exits at once with code 130 rather than waiting out a stuck transfer.
241239

242240
## Use as a Go library
243241

@@ -293,13 +291,12 @@ make lint # golangci-lint
293291
```
294292

295293
Integration tests spin up real `pure-ftpd` and `atmoz/sftp` containers via the
296-
`docker` CLI and are gated behind the `integration` build tag, so a plain
297-
`go test ./...` never needs Docker. They skip themselves with a message if
298-
Docker is not reachable.
294+
`docker` CLI, behind the `integration` build tag, so a plain `go test ./...`
295+
needs no Docker. Without a reachable Docker they skip with a message.
299296

300297
## Releasing (maintainers)
301298

302-
Releases are cut by [GoReleaser](https://goreleaser.com) from a tag:
299+
[GoReleaser](https://goreleaser.com) cuts releases from a tag:
303300

304301
```console
305302
git tag v0.1.0
@@ -309,15 +306,15 @@ git push origin v0.1.0
309306
The `Release` workflow then cross-compiles every target, attaches
310307
checksummed archives to the GitHub Release, and updates the Homebrew tap.
311308
Publishing the Homebrew cask requires a `HOMEBREW_TAP_GITHUB_TOKEN` repository
312-
secret: a token with write access to `alexeyu/homebrew-tap`. If that secret is
313-
absent the release still succeeds and only the cask push is skipped. Validate
309+
secret: a token with write access to `alexeyu/homebrew-tap`. Without that
310+
secret the release still succeeds and skips only the cask push. Validate
314311
config changes locally with `goreleaser check` and dry-run a full build with
315312
`goreleaser release --snapshot --clean`.
316313

317314
## Stability
318315

319-
Pre-1.0. The tool works and is tested, but the config schema and the Go library
320-
API may still change between minor versions. `v1.0.0` will mark both as stable.
316+
Pre-1.0. The config schema and the Go library API may change between minor
317+
versions; `v1.0.0` will freeze both.
321318

322319
## License
323320

0 commit comments

Comments
 (0)