ACME: option to fail http-01/tls-alpn-01 challenges on connection errors

Hello!

• Vote on this issue by adding a reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

When step-ca validates an http-01 or tls-alpn-01 challenge and cannot reach the target (name does not resolve, connection refused, TLS dial fails, or the HTTP response has status >= 400), it stores the error on the challenge but leaves it pending (storeError(..., markInvalid=false, ...)). The idea is that the client can fix the problem and retry.

Proposal

Add an opt-in ACME provisioner option. When it is set, connection errors during http-01 and tls-alpn-01 validation mark the challenge invalid immediately. Combined with #2802, the authorization and the order then become invalid too, and the client stops within seconds with the connection error as the reason:

{
  "type": "ACME",
  "name": "acme",
  "challengeFailFast": true
}

I run this as a patch on v0.30.2. With the step CLI, an unresolvable name now ends after about one second with The server could not connect to validation target instead of polling.

I have a draft implementation with tests and will open a PR that references this issue. It changes markInvalid from false to the provisioner setting at the three call sites (http-01 GET error, http-01 status >= 400, tls-alpn-01 dial error), and does not change the acme.Provisioner interface.

Open design questions

1. Name of the option. challengeFailFast is a working title. Alternatives: failChallengeOnConnectionError, or a more general challengeRetry: false.
2. Default. The draft keeps the current behavior (off). Since RFC 8555, section 8.2, lets the server retry validation but does not require it, failing fast could arguably be the default. That would change behavior for existing installations, though.
3. dns-01. The draft leaves dns-01 alone, because a failed or empty TXT lookup is often just propagation delay and clients do retry there. Should the option cover dns-01 too, maybe as a separate setting?
4. Scope of errors. Should HTTP status >= 400 count as a connection error? A 404 usually means the client is not serving the token, but a 503 could be temporary.
5. Remotely managed provisioners. The option is a new field in provisioner.ACME, so it works in ca.json. Provisioners stored in the database via the admin API are converted to and from linkedca.ACMEProvisioner in authority/provisioners.go, which has no such field, so the setting would be lost there. Supporting that needs a field in smallstep/linkedca and probably a flag in step ca provisioner add/update. Is it fine to start with ca.json only?

Why is this needed?

In practice, most clients (certbot, Caddy/acmez, step CLI) do not retry the challenge. They poll the authorization until it is no longer pending. If the target will never be reachable, for example a typo in the hostname or a name that only resolves on another network, the client keeps polling until the authorization expires (24 hours by default). The user gets no useful error, and the CA log fills up with poll requests.

This is a separate problem from the authorization staying pending after a challenge became invalid, which is a bug (RFC 8555, section 7.1.6) and is fixed in #2802. That fix only helps once the challenge is actually invalid, which connection errors never cause today.