Skip to content

Commit ba4f4e1

Browse files
pengpengeball
andauthored
feat(cli): olares-cli profile + files (ls/upload/download/cat/rm) + skills (#2949)
* feat(cli): add olares-cli profile/auth/credential subsystem (Phase 1) Introduce the multi-profile credential subsystem for olares-cli, covering the "operate on behalf of a user" scenario via password login and refresh token import, while keeping the existing kubeconfig-based commands (osinfo/os/node/gpu/amdgpu/disk) untouched and orthogonal. New packages: - pkg/cliconfig: ~/.olares-cli/{config.json,tokens.json} layout, file perms (0700/0600), atomic write, MultiProfileConfig with current / previous profile pointers, FindByOlaresID, etc. - pkg/olares: olaresId -> auth/vault/desktop URL derivation (single source of truth for all per-profile URLs). - pkg/auth: stateless protocol layer. * login.go (mode A): /api/firstfactor with optional TOTP, password salting, cookie-jar HTTP client. * refresh.go (mode B): one-shot /api/refresh exchange used to bootstrap an access token from a user-supplied refresh token. * token_store.go: plaintext-JSON TokenStore (Phase 1) with Get / Set / Delete / List / MarkInvalidated. StoredToken includes InvalidatedAt (Phase 1 defines, Phase 2 writes on /api/refresh 401/403); Set() defensively zeroes it on every fresh grant. * jwt.go: ExpiresAt(token) only - no signature verification, no other claims exposed (username/groups/mfa/jid are explicitly untrusted). - pkg/credential: Provider chain + ResolvedProfile. * DefaultProvider resolves Profile + token, returning typed errors with invalidated > expired > ok priority (ErrTokenInvalidated / ErrTokenExpired / ErrNotLoggedIn), each with a "run profile login" CTA. Profile config is never silently mutated. * EnvProvider stub for future Phase 3 in-cluster scenario. - pkg/cmdutil/Factory: lazy DI for Credential / ResolvedProfile / HTTP client (auto-injects Authorization: Bearer; no auto-refresh in Phase 1). New commands (cmd/ctl/profile, no separate `auth` namespace): - profile list / use / remove / login / import. - login & import auto-create the profile on first use, reuse-and-overwrite when the existing token is missing/expired/invalidated, and reject only when a still-valid token is present (with a profile-remove hint). - list shows STATUS: logged-in (Xh) / expired / invalidated / never. Also: - Register profile command in cmd/ctl/root.go. - Ignore cli/docs/ (design notes are local-only, not part of the shipping repo). Phase 1 trade-offs (deferred to Phase 2): OS keychain backend, automatic refresh-with-lock (sync.Map + flock + double-check), LarePass OAuth device-flow login, wizard activation -> profile bridge. Made-with: Cursor * feat(cli): add `olares-cli files ls` and Phase 1 cleanup Closes the Phase 1 loop with a first authenticated command and rolls in the audit fixes from the cleanup pass. `files ls <fileType>/<extend>[/<subPath>]`: - Parse + validate the 3-segment front-end path used by files-backend (drive/Home, drive/Data, sync/<repo>, awss3/<account>/<bucket>, ...); unknown fileType / bad drive extend are rejected client-side before any HTTP. - Percent-encode each segment via url.PathEscape (mirrors the web app's encodeUrl) so filenames with `#`, `?`, `+`, spaces, `%`, non-ASCII survive the trip to /api/resources. - Render a one-line header (`<path> (N dirs, M files, modified ...)`) followed by a MODE / SIZE / TYPE / MODIFIED / NAME table; MODE decodes os.FileMode (drwxr-xr-x / -rw-r--r-- / Lrwxr-xr-x) and TYPE surfaces the backend's class (video / image / audio / pdf / text / blob / ...). `--json` passes the raw response through verbatim. - 401/403 reuses DefaultProvider's "run profile login" CTA; other non-2xx surface the backend's error/code+message JSON verbatim. Supporting plumbing: - olares.ID.FilesURL derives `https://files.<terminusName>` (matches the web app's getModuleSever('files')). - ResolvedProfile + DefaultProvider expose FilesURL so commands don't have to redo the derivation. - Factory's HTTP client now injects the access token via the custom `X-Authorization` header (was `Authorization: Bearer`). Confirmed via l4-bfl-proxy + BFL filters that the standard Authorization header is filtered at the edge and never reaches per-user services; the web app uses the same X-Authorization path. - Root command grows a persistent `--profile` flag bound straight onto the shared Factory.ProfileOverride, so any subcommand that calls factory.ResolveProfile honors it without re-declaring the flag. - Drop the unused Factory.Stderr field. .gitignore: exclude the local `/files/` and `/market/` checkouts that are kept beside the repo for cross-reference but must never be committed. Tests: ParseFrontendPath (15 cases incl. URL-escape table covering `#?+%` / spaces / non-ASCII), formatSize (10 cases incl. 1023/1024/1MB boundary), formatMode (incl. the live-observed dir mode 2147484141 → drwxr-xr-x), formatType, formatHTTPError (401/403, error/code+message, raw fallback), renderListing (header + counts + dirs-first sort). Made-with: Cursor * feat(cli): move profile tokens into the OS keychain (Phase 2) Replaces the Phase 1 plaintext ~/.olares-cli/tokens.json store with a per-OS keychain backend so refresh tokens no longer sit on disk in clear text. The auth.TokenStore surface is unchanged; production code now constructs a keychainStore via auth.NewTokenStore, while tests inject an in-memory fake through auth.NewTokenStoreWith. Backends (cli/internal/keychain): - darwin: system Keychain for the master key, AES-256-GCM file blobs under StorageDir; falls back to a file-only master key if the system keychain is unavailable (sandbox / CI). - linux/other: file-based master key + AES-256-GCM blobs under StorageDir, all 0600/0700. - windows: DPAPI-encrypted values under HKCU\Software\OlaresCli\keychain. Hardening / UX in the same change: - keychainStore.List tolerates a single corrupted entry (warn to stderr, skip) instead of aborting the whole `profile list`. - StorageDir falls back to os.TempDir with a stderr warning when UserHomeDir is unresolvable, so we never silently write to '/'. - keychain.Backend(service) reports the active backend label (system-keychain / file-fallback / file / registry+dpapi); printed after every successful login/import so users notice when they land on the file fallback. - keychain.PurgeService is invoked when the last profile is removed, cleaning up the master key + storage dir / registry subkey so we don't leave orphan secrets behind. - wrapError stays terse by default and only attaches the verbose troubleshooting hint when OLARES_CLI_DEBUG is set. Refactor: - AES-GCM constants and the safeFileName helper live in a single aesgcm.go (//go:build !windows) so darwin and linux can't drift on the on-disk envelope. - The in-memory keychain fake is promoted to its own keychainfake subpackage and shared by pkg/auth and cmd/ctl/profile tests. Docs in cli/internal/keychain/doc.go explain the deltas vs. lark-cli's upstream copy, including why olares-cli keeps the KeychainAccess interface + keychainfake (keychainStore has MarkInvalidated / List / InvalidatedAt semantics that need unit-test coverage). Made-with: Cursor * refactor(cli): align auth flow with TS onFirstFactor/loginTerminus shape Restructure cli/pkg/auth so it mirrors the two-tiered authentication pattern in apps/packages/app/src/utils/{account.ts,BindTerminusBusiness.ts} 1:1, and migrate the wizard package off its own duplicated copies of the salt math, cookie jar, and 2FA wiring. - pkg/auth.LoginRequest: drop the hallucinated TargetURL/SkipSecondFactor fields; add NeedTwoFactor (controls vault vs desktop targetURL only) and AcceptCookie (1:1 with TS `acceptCookie` arg). Docstrings cite the TS file + line numbers so the next change does not have to reverse- engineer ground truth. - pkg/auth.FirstFactor: new exported low-level primitive that POSTs /api/firstfactor and returns the raw token without inspecting `fa2`, matching TS onFirstFactor (account.ts L7-71). - pkg/auth.Login: rewritten as a thin wrapper around the shared firstFactorWithClient + optional /api/secondfactor/totp escalation, matching TS loginTerminus (BindTerminusBusiness.ts L353-446). Gate uses tok.FA2 only, deliberately diverging from TS's `tok.FA2 || needTwoFactor` because the CLI has no caller-side knowledge to defensively force 2FA and OR'ing would surface a spurious ErrTOTPRequired the moment a caller probes with the desktop targetURL. - pkg/wizard.UserBindTerminus: switch from auth.Login to auth.FirstFactor with NeedTwoFactor=false / AcceptCookie=false, mirroring TS L58-66. This restores `olares-cli wizard activate`, which broke when the unified auth.Login enforced a 2FA gate the original wizard flow intentionally bypassed (no MFA seed exists at signup time). - pkg/wizard.LoginTerminus: pass NeedTwoFactor through and set AcceptCookie=true to match TS L364-372; keep the eager-TOTP and ErrTOTPRequired retry paths that bridge the CLI to the MFA-store TOTP source. - profile login: pass NeedTwoFactor=true so /api/firstfactor uses the desktop targetURL and Authelia honestly reports fa2=true on 2FA-enabled accounts. With the previous vault targetURL the server silently downgraded to fa2=false and the TOTP prompt never fired. Made-with: Cursor * feat(cli): add files upload/download/cat/rm commands Adds four new verbs to `olares-cli files`, mirroring the LarePass web app's Drive UX over the per-user files-backend on `files.<terminusName>`: - `files upload` — resumable chunked upload (Drive v2 protocol: upload-link + file-uploaded-bytes + Content-Range POSTs). Recurses into local directories; the destination directory must pre-exist on the server because POST /api/resources/<dir>/ auto-renames an existing dir to "Dir (1)" instead of returning 409. - `files download` — single-file with server-driven `Range:` resume + atomic `.tmp`+rename overwrite, or recursive directory mirroring with `--parallel N` errgroup-bounded concurrency. - `files cat` — streams `/api/raw/<path>?inline=true` to stdout, refusing directories up-front for a friendlier message than the backend's terse 400. - `files rm` — Unix-like `-r/-R` and `-f/--force`; groups targets by parent dir and issues one `DELETE /api/resources/<parent>/` per group with `{"dirents":[...]}` in the body, matching the frontend's batchDelete wire. `Stat` uses the parent-listing strategy (list parent dir, find leaf in items) rather than `GET /api/resources/<file>` directly, because the backend's single-file List handler hard-codes `Content: true` and 500s on most real files. This matches what the LarePass web app does. All four commands reuse the existing 3-segment `FrontendPath` parser and the `X-Authorization` access-token transport from `pkg/cmdutil/factory.go`. httptest-driven unit tests cover resume / overwrite / retries / range headers (download), plan grouping + dir-without-r refusal + wire shape (rm), encodeURIComponent parity + chunked upload protocol (upload). Made-with: Cursor * docs(cli): add olares-shared and olares-files agent skills Two new SKILL.md files under cli/skills/, modeled after larksuite-cli's lark-shared / lark-drive style, so AI coding assistants get reliable guidance on: - olares-shared: profile model, password+TOTP login (mode A), refresh-token import (mode B), profile use/list/remove + global --profile, OS keychain storage layout, the re-authentication state machine, and a recovery table for HTTP 401/403 / already-authenticated / 2FA-required errors. - olares-files: the 3-segment frontend path schema, trailing-slash conventions, two server-side quirks the agent MUST respect (POST mkdir auto-renames existing dirs to "Foo (1)"; GET single-file resource returns HTTP 500 — Stat must list parents instead), X-Authorization transport, and per-verb cheatsheets for ls / upload / download / cat / rm with wire shapes and key flags. The frontmatter follows the lark-cli schema (name / version / description with trigger keywords / metadata.requires.bins / metadata.cliHelp), so Cursor's skill loader can pick them up from cli/skills/<name>/SKILL.md. Made-with: Cursor * fix(cli): resume download retries + dedupe byte format helpers - Re-stat the local file before each download attempt in --resume mode so Range: bytes=N- tracks partial progress after a failed append; accumulate per-attempt bytes for the success return and align error returns with that accounting. - Add TestDownloadFile_ResumeRetryRefreshesRange (partial 206 + cut, then second GET with bytes=6-). - Drop redundant atomic adds under mutex in files upload/download commands; use formatBytes from ls.go and remove duplicate humanBytes. Made-with: Cursor * fix(cli): align files ls URLPath with EncodeURL (JS encodeUrl) Bugbot: FrontendPath.URLPath used url.PathEscape per segment while download/cat/rm/upload use upload.EncodeURL, diverging for '+', '!*()', etc. Delegate URLPath to EncodeURL(p.String()) and extend path tests (report (1).txt, x+y → x%2By). Update ls command comment. Made-with: Cursor * refactor(cli): extract JS-shaped path encoding to pkg/files/encodepath Move EncodeURL / EncodeURIComponent out of package upload into a small shared package so download, rm, path, and upload all depend on the same wire encoder without cmd/pkg layers importing upload only for URLs. - Add cli/pkg/files/encodepath with tests (former upload/encode*.go). - Update upload api/uploader, download client, rm, FrontendPath.URLPath. - Clarify download/list.go is for the download walker, not `files ls`. Made-with: Cursor * refactor(cli): move Drive files-backend clients under internal/files Relocate encodepath, download, rm, and upload from cli/pkg/files to cli/internal/files so they are clearly olares-cli implementation details (not a stable public library surface). The legacy root package cli/pkg/files (installer URLs, rate limiter, etc.) stays in pkg for existing importers (storage, bootstrap, terminus, ...). Update `files` Cobra imports and olares-files SKILL links. Made-with: Cursor * feat(cli): add olares-cli market commands with --watch + skill Add a profile-authenticated `market` command tree that talks to the per-user Market app-store v2 API (`<MarketURL>/app-store/api/v2`) using the same `cmdutil.Factory` / `X-Authorization` transport as `olares-cli files`. The existing `cli/cmd/ctl/app/` tree is kept intact so reviewers can diff the two side by side. Verbs: - Catalog (read-only): `list`, `categories`, `get`. - Runtime: `status` with the "not installed" UX fix, the source-fallback hint, and `--watch` for op-agnostic recovery (e.g. confirming a previously fire-and-forget install reaches `running`). - Lifecycle (mutating, all support `--watch`): `install`, `upgrade`, `uninstall`, `clone`, `stop`, `resume`, `cancel`. - Local sources: `upload` / `delete`, restricted to `-s upload|studio|cli`. The `--watch` machinery (`cli/cmd/ctl/market/watch.go`) blocks until the backend reaches a terminal state. Per-op success/failure sets are derived from the backend's `ApplicationManagerState` enum; `matchOpType` gates "success" against the in-flight `OpType` so an `upgrade` issued on an already-`running` app cannot return success on tick zero. `cancel` and `status --watch` are deliberately op-agnostic. Output: TTY emits one info line per state transition; `-o json` emits a single final `OperationResult` with the new `finalState` / `finalOpType` fields (`omitempty`, so non-watch JSON output is byte-identical). Ctrl-C exits cleanly via `signal.NotifyContext` with `watch canceled by user`; the underlying mutation is not stopped — re-attach via `status --watch`. Plumbing: - `cli/pkg/olares.ID.MarketURL(localPrefix)` derives the market origin in the same shape as the existing `FilesURL` / `DesktopURL` / `VaultURL`. - `credential.ResolvedProfile.MarketURL` and `buildResolved` propagate it through the factory, so commands never touch kubeconfig. - `cmd/ctl/root.go` registers `market.NewMarketCommand(factory)` next to the existing `app` / `profile` / `files` commands. Skill: - `cli/skills/olares-market/SKILL.md` mirrors the `olares-files` / `olares-shared` shape (frontmatter, top callout, core concepts → auth transport → command cheatsheet → `--watch` section → errors → workflows → security rules). Defers profile / login / 401-403 recovery to `olares-shared` instead of duplicating it. Tests: `cli/cmd/ctl/market/watch_test.go` covers the classifier (per-op terminal sets, OpType gating, `cancel` op-agnostic path, op-agnostic `status --watch`) and end-to-end `waitForTerminal` against an `httptest.Server` (state transitions, timeout surfacing last-seen state, JSON output stability). Made-with: Cursor * fix: update go module --------- Co-authored-by: eball <[email protected]>
1 parent 4cdf097 commit ba4f4e1

91 files changed

Lines changed: 15946 additions & 247 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ olares-cli-*.tar.gz
3030
.vscode
3131
.DS_Store
3232
cli/output
33+
cli/docs/
3334
daemon/output
3435
daemon/bin
3536

@@ -43,3 +44,6 @@ node_modules
4344
cli/olares-cli*
4445

4546
framework/app-service/bin
47+
48+
/files/
49+
/market/

cli/cmd/ctl/files/cat.go

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
package files
2+
3+
import (
4+
"context"
5+
"fmt"
6+
"io"
7+
"strings"
8+
9+
"github.com/spf13/cobra"
10+
11+
"github.com/beclab/Olares/cli/pkg/cmdutil"
12+
"github.com/beclab/Olares/cli/internal/files/download"
13+
)
14+
15+
// NewCatCommand: `olares-cli files cat <remote-path>`
16+
//
17+
// Streams the raw bytes of a single remote file to stdout. The wire
18+
// call is GET /api/raw/<encPath>?inline=true (same path the LarePass
19+
// web app uses for text-content previews — `inline=true` only
20+
// affects Content-Disposition, the body is identical).
21+
//
22+
// Like `cat` itself, this is binary-safe: we don't sniff or
23+
// interpret the body, we just copy it through. That means cat-ing a
24+
// huge image will dump the bytes — the user is expected to pipe to
25+
// `less`, `head`, or a similar tool when they care about safety.
26+
//
27+
// We Stat the path before fetching so a directory target produces a
28+
// clear "is a directory" error rather than the server's terser
29+
// "not a file, path: ..." 400.
30+
func NewCatCommand(f *cmdutil.Factory) *cobra.Command {
31+
cmd := &cobra.Command{
32+
Use: "cat <remote-path>",
33+
Short: "stream a remote file's contents to stdout",
34+
Long: `Stream the raw bytes of a single file on the per-user files-backend to stdout.
35+
36+
Equivalent to ` + "`olares-cli files download <remote> -`" + ` if a future
37+
` + "`-`" + ` -means-stdout convention is added — for now ` + "`cat`" + ` is the explicit
38+
verb. The transfer is binary-safe (no buffering, no transformation),
39+
so piping into ` + "`less`" + ` / ` + "`hexdump`" + ` / ` + "`head -c`" + ` works as expected.
40+
41+
Directories produce an error rather than a recursive concatenation
42+
(use ` + "`files download <remote>/`" + ` if you want the contents on disk
43+
first).
44+
45+
Examples:
46+
47+
olares-cli files cat drive/Home/Documents/notes.md
48+
olares-cli files cat drive/Home/Logs/today.log | tail -n 50
49+
`,
50+
Args: cobra.ExactArgs(1),
51+
RunE: func(cmd *cobra.Command, args []string) error {
52+
return runCat(cmd.Context(), f, cmd.OutOrStdout(), args[0])
53+
},
54+
}
55+
return cmd
56+
}
57+
58+
func runCat(ctx context.Context, f *cmdutil.Factory, out io.Writer, remoteArg string) error {
59+
if ctx == nil {
60+
ctx = context.Background()
61+
}
62+
63+
fp, err := ParseFrontendPath(remoteArg)
64+
if err != nil {
65+
return err
66+
}
67+
68+
rp, err := f.ResolveProfile(ctx)
69+
if err != nil {
70+
return err
71+
}
72+
73+
httpClient := newUploadHTTPClient(rp.InsecureSkipVerify)
74+
client := &download.Client{
75+
HTTPClient: httpClient,
76+
BaseURL: rp.FilesURL,
77+
AccessToken: rp.AccessToken,
78+
}
79+
80+
plain := strings.TrimSuffix(fp.String(), "/")
81+
82+
// Probe before streaming. Two cheap wins:
83+
// - friendly "is a directory" message for `cat drive/Home/`
84+
// instead of the server's terse 400;
85+
// - 401/403/404 reformatted with the standard CTA before we
86+
// start writing partial data to stdout.
87+
st, err := client.Stat(ctx, plain)
88+
if err != nil {
89+
return reformatHTTPErr(err, rp.OlaresID, "stat", plain)
90+
}
91+
if st.IsDir {
92+
return fmt.Errorf("%s is a directory: cat only works on files (use `olares-cli files ls %s` to list it)",
93+
fp.String(), fp.String())
94+
}
95+
96+
if _, err := client.StreamRaw(ctx, plain, out); err != nil {
97+
return reformatHTTPErr(err, rp.OlaresID, "cat", plain)
98+
}
99+
return nil
100+
}

0 commit comments

Comments
 (0)