Reverse proxy drops com.atproto.sync.getRepo for any User-Agent containing tap/ (breaks indigo Tap backfill)

[matteomarjanovic]

Describe the bug

Requests to com.atproto.sync.getRepo on blacksky.app are rejected at the edge (connection reset, no HTTP response) whenever the User-Agent header contains the case-sensitive substring tap/. Bluesky's indigo Tap sync service sends User-Agent: tap/<version> for its historical backfill, so every attempt to backfill a blacksky-hosted repo fails. Live firehose events are unaffected (they don't call getRepo), so the symptom is subtle: a blacksky user's new activity indexes fine on a third-party appview, but their existing records never do.

The working responses carry via: 1.1 Caddy, consistent with a Caddy header_regexp User-Agent "tap/" → abort rule (abort closes the connection with no response, exactly matching the observed behavior).

To Reproduce

Steps to reproduce the behavior (only the User-Agent varies):

1. Pick any public account hosted on blacksky.app, e.g. DID=did:plc:dnb6tvh7ncewggcbsgaivunj
2. Request its repo with a tap/ User-Agent — connection is reset, no HTTP response:

curl -sS -A "tap/1.2.3" -o /dev/null "https://blacksky.app/xrpc/com.atproto.sync.getRepo?did=$DID"

curl: (92) ... HTTP/2 stream ... INTERNAL_ERROR

3. Repeat with any other User-Agent — succeeds with HTTP 200 and the full CAR:

curl -sS -A "indigo/1.2.3" -o /dev/null -w '%{http_code} %{size_download}\n' "https://blacksky.app/xrpc/com.atproto.sync.getRepo?did=$DID"

200 7052224

Expected behavior

getRepo returns the repo CAR regardless of User-Agent (or at least for the tap/<version> UA used by indigo Tap), so third-party appviews/relays can backfill blacksky-hosted repos.

Details

• Operating system: reproduced from macOS and from Linux (Docker) — it's server-side, so client OS is irrelevant
• Node version: N/A (reproduced with curl 8.x; no client SDK involved)

Additional context

Observed matrix (same URL, User-Agent is the only variable):

User-Agent	Result
tap/1.0, tap/anything, xtap/1.0	connection reset (curl exit 92)
tap (no slash), TAP/1.0, curl/8, indigo/devel, foo/1.0	HTTP 200, full CAR

The match is the literal substring tap/, case-sensitive — xtap/1.0 is caught too, while uppercase TAP/ and tap without a slash pass.

Impact: indigo Tap hardcodes its UA to tap/<version> with no override, so no blacksky-hosted repo can be backfilled by any Tap-based appview/relay/indexer; affected accounts appear partially or empty on those services.

Is this block intentional (e.g. a deliberate stance on third-party repo crawling), or an over-broad anti-scraper rule? If unintended, tightening the matcher so it doesn't catch tap/<version> (or allowlisting it) would restore backfill. Happy to provide more detail.