Scale to Zero

Is this a BUG REPORT or FEATURE REQUEST?:

Feature Request

What happened:

Current autoscaling option uses an HPA, but we have many low-traffic Functions. I'd trade off cold start latency for 0 resource usage under 0 load. I'd propose we turn this off by default.

Knative Serving addresses this but has a hard dependency on Istio which is not an option for my cluster.

What you expected to happen:

When Functions receive 0 traffic for some threshold (e.g. 180 seconds), the HPA pod count scales to 0. When a function has traffic, it defaults to the HPA or static RS behavior.

I'd propose this with a mechanism similar to knative's activator, where all 0-pod Function traffic is routed to an operator which receives the request, scales the Function to a non-0 number of pods, then forwards the original request accordingly. Obviously there will be increased latency on cold starts, and we can mitigate by respecting client timeouts and responding with QoS response codes if requests become a thundering herd.