[ext_proc] add a control message directs modes overrides before request header response
Title: [ext_proc] add a control message directs modes overrides before request header response
Description: In MT cloud proxy, oftentimes proxy itself doesn't want to buffer the whole body payload due to RAM pressure. Esp so when AI traffic could easily be up to MBs in size. Modes allows in ext_proc are usually streamed or FULL_DUPLEX_STREAMED.
A feature frequently asked by customer: I want to see the headers, based on it I can make decision to either reject it or I will want body for further inspection, but DONOT send headers to upstream yet.
STREAMED mode would cover the part of the story: the NO path
- send headers to ext_proc server;
- ext_proc server sends back headers response, and mode override set to None; with a immediate response.
- immediate response goes back to sender.
The YES path is a little bit trickier tho: (STREAMED)
- send headers to ext_proc server;
- ext_proc server sends back headers response, and mode override set to FULL_DUPLEX_STREAMED
- Data chunks sent to ext_proc for further inspect, ext_proc server sends immdiate_response.
Streamed mode break the expectation at step 3.: Headers response when received, will be forwarded to upstream already.
FULL_DUPLEX_STREAMED mode doesn't work neither: body chunk streaming starts not waiting for headers responses.
Let's add a control-message, very similar to how override_message_timeout is done today, to FULL_DUPLEX_STREAMED mode (or more modes), that can apply mode-overides/timeout/other config application from ext_proc server, before any header response begins.
this way we can answer what's asked by the customers:
- in streamed mode, ext_proc sends back the control message: mode switch to full_duplex_streamed;
- no header response received yet, body sent to ext_proc server;
- ext_proc server sends back immediate response, or normal header responses, after check the headers AND the body.
Source: envoyproxy/envoy