[Bug] Credential Proxy injects Authorization too late for streamed uploads, causing authentication failures

Summary

Credential Proxy header injection appears to occur too late when an upload is streamed by the egress proxy.

Small uploads succeed with a Vault-managed Authorization header. A 64 MiB upload to the same endpoint fails because the downstream gateway receives the request without that header.

A limited experiment that moved header-only credential injection to requestheaders() made the same 64 MiB upload succeed, without changing the client, gateway authentication requirements, or network permissions.

Environment

• OpenSandbox Server: 0.2.2
• OpenSandbox egress: 1.1.5
• OpenSandbox Python SDK: 0.1.15
• Python inside the sandbox: 3.14.7
• Sandbox platform: Linux amd64
• Deployment: Kubernetes
• Credential Proxy enabled
• HTTPS with certificate verification enabled
• Synthetic data and test-only credentials

The upload client uses Python's standard library, sends a known Content-Length, and produces the body in 64 KiB chunks. It does not set an Authorization header itself; Credential Proxy is responsible for injecting it.

Steps to reproduce

1. Prepare an HTTPS endpoint behind a gateway that:

   • Requires an Authorization header.
   • Accepts a POST upload.
   • Returns the received byte count and SHA-256.
   • Records whether the upstream upload handler was invoked.

2. Create an OpenSandbox sandbox with Credential Proxy enabled and allow egress to the test hostname.

3. Create a Credential Vault credential and a header-only binding:

   • Match the test HTTPS hostname, port, upload path, and POST method.
   • Inject Authorization using auth.type = "customHeaders".
   • Do not configure body, path, or query substitutions.

4. From inside the sandbox, upload synthetic payloads of:

   • 1 KiB
   • 1 MiB
   • 64 MiB

5. Keep client retries disabled and correlate the proxy, gateway, and upload-handler observations.

Expected behavior

Credential matching and Authorization header injection should finish before request headers are forwarded upstream, including when the request body is streamed.

Large uploads should retain streaming behavior without requiring the client to receive or manually attach the credential.

Actual behavior

The upload behavior reproduced in two separate sandboxes.

For the failed 64 MiB uploads:

• The egress log indicated that the request was being streamed.
• The downstream gateway returned HTTP 401 because the Bearer credential was missing.
• Correlated upstream records showed that the upload handler was not invoked.

The client may report a transport error when the gateway rejects the request while the body is still being sent; the missing-header 401 was confirmed from the gateway logs.

Root-cause analysis

The egress system addon at /var/egress/mitmscripts/system.py performs credential matching and header injection in:

def request(flow: http.HTTPFlow) -> None:
    ...

The mitmproxy event documentation states that, when request streaming is active, the request event fires after the entire body has been streamed:

https://docs.mitmproxy.org/stable/api/events.html#HTTPEvents.request

At that point, modifying request headers is too late for the already-forwarded request.

This is consistent with the small-upload success, large-upload failure, and missing-header observations.

Limited validation of a possible fix

In one disposable sandbox, we experimentally invoked the existing matching, validation, and injection logic from requestheaders() for header-only bindings.

The same 64 MiB upload then:

• Returned HTTP 200.
• Produced the expected byte count and SHA-256.
• Executed the upstream upload handler exactly once.

Authentication remained enabled, and network permissions were unchanged.

This experiment only validates the injection-timing hypothesis. It is not a proposed production-ready patch, and it did not validate all substitution modes or credential-update races.

Suggested fix considerations

Could header-only credential handling be completed before headers are forwarded, while preserving the existing security checks?

A complete fix should account for:

• Host, scheme, port, method, and path matching.
• Ambiguous-path rejection and binding conflicts.
• Avoiding duplicate header injection.
• Credential revision consistency during a request.
• Response credential redaction.
• Separate handling of bindings that require body, path, or query substitutions.
• Preserving bounded memory usage for large uploads.

Suggested regression coverage includes buffered and streamed uploads, unknown-length/chunked requests, non-matching bindings, rejected paths, and early upstream rejection.

If there is already a supported configuration or a released fix for this behavior, please point us to it.