Skip to content

fix(client): discard expired cert on startup - #95

Merged
lidel merged 2 commits into
mainfrom
fix/renew-expired-cert-ari
Jul 28, 2026
Merged

fix(client): discard expired cert on startup#95
lidel merged 2 commits into
mainfrom
fix/renew-expired-cert-ari

Conversation

@lidel

@lidel lidel commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Problem

A node that comes back online after its TLS cert expired can never renew it. certmagic references the old cert in the ACME ARI replaces field, Let's Encrypt rejects the order with HTTP 404 urn:ietf:params:acme:error:malformed because it no longer knows that cert, and certmagic retries the same rejected order forever.1 The node runs without a usable AutoTLS cert and logs ERRORs every 10 minutes until someone deletes the cert directory by hand. Seen on a kubo 0.43.0-rc1 node that was offline for a year.

Fix

  • on startup, when the stored cert is past NotAfter, discard it (cert, key, metadata) through the certmagic Storage interface
  • issuance then starts from scratch: a fresh order carries no replaces field, and the node re-checks broker health and libp2p reachability first

A daemon that stays up while its cert expires in place can still wedge until restart. That half needs a certmagic fix (drop replaces when the ARI refresh 404s, like the 409 handling in caddyserver/certmagic#361) and is out of scope here. I left it out of the PR because the exposure is narrow: it takes ~45 days of continuously failing renewals with zero restarts, and any restart (kubo upgrade, reboot) heals it once this patch is deployed.

Footnotes

  1. RFC 9773, section 5: servers SHOULD validate the certificate referenced by replaces and SHOULD reject the newOrder request when validation fails.

lidel added 2 commits July 29, 2026 01:29
An expired certificate cannot be renewed: certmagic references it via
the ACME ARI 'replaces' field, and the CA rejects orders that reference
a certificate it no longer considers current (RFC 9773, section 5;
Let's Encrypt returns HTTP 404 urn:ietf:params:acme:error:malformed),
so renewal retries loop forever and the node never gets a usable cert.
Happens when a node comes back online after its cert already expired.

Discarding the expired cert on startup routes the client through fresh
issuance, which sends no 'replaces' field and re-checks broker health
and libp2p reachability first.
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 62.50000% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.03%. Comparing base (bf900a8) to head (babc5fd).

Files with missing lines Patch % Lines
client/acme.go 62.50% 8 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #95      +/-   ##
==========================================
- Coverage   69.17%   69.03%   -0.15%     
==========================================
  Files          22       22              
  Lines        1765     1805      +40     
==========================================
+ Hits         1221     1246      +25     
- Misses        425      433       +8     
- Partials      119      126       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

Suggested tag: v0.10.1

Comparing to: v0.10.0 (diff)

gorelease says:

# summary
v0.10.1 is a valid semantic version for this release.

gocompat says:

HEAD is now at a804ecb feat(client): check broker health before issuance (#91)
Previous HEAD position was a804ecb feat(client): check broker health before issuance (#91)
Switched to branch 'main'
Your branch is up to date with 'origin/main'.

Cutting a Release (and modifying code files)

This PR is modifying both version.json and code files (not markdown, YAML, TOML or lock files).
The Release Checker is not able to analyse files that are not checked in to main. This might cause the above analysis to be inaccurate.
Please consider performing all the code changes in a separate PR before cutting the release.

Automatically created GitHub Release

A draft GitHub Release has been created.
It is going to be published when this PR is merged.
You can modify its' body to include any release notes you wish to include with the release.

@lidel
lidel marked this pull request as ready for review July 28, 2026 23:37
@lidel
lidel merged commit 1743bb9 into main Jul 28, 2026
8 of 10 checks passed
@lidel
lidel deleted the fix/renew-expired-cert-ari branch July 28, 2026 23:38
lidel added a commit that referenced this pull request Jul 28, 2026
Merging main brought #95, which found that an expired certificate cannot
be renewed at all: the authority rejects an order whose ARI 'replaces'
names a certificate it no longer considers current, so certmagic retries
a doomed renewal forever. The address path had exactly that bug, reaching
for a renewal when the certificate in storage had lapsed. It discards and
issues fresh now, through the helper #95 added.
lidel added a commit to ipfs/kubo that referenced this pull request Jul 28, 2026
Picks up the review round on ipshipyard/p2p-forge#94: an address the
address factory holds back is still offered for certification, the error
for a node that cannot be validated says whether the port is unreachable
or merely the wrong one, and a named TLS handshake is left to certmagic
as it was before this feature existed.

Also brings ipshipyard/p2p-forge#95 through that branch, so an expired
certificate is discarded rather than renewed, on the address path as well
as the brokered one.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant