[Serve] Pass replica_id through load balancing and key request accounting by replica identity

Problem

SkyServe currently uses the replica endpoint URL as both:

• the network routing address; and
• the replica identity for load accounting.

The load balancer receives URL-keyed replica information, selects a URL, and later calls pre_execute_hook() / post_execute_hook() with that URL. This is unsafe because URLs are not stable replica identities.

Current Flow

The controller currently returns URL-keyed information from /controller/load_balancer_sync:

{
  "replica_info": {
    "http://replica-a": {"gpu_type": "A100"}
  }
}

The load balancer then uses the URL for all three purposes:

controller sync -> ready URLs -> policy selects URL
                                  |
                                  +-> client pool lookup by URL
                                  +-> pre_execute_hook(URL)
                                  +-> post_execute_hook(URL)
                                  +-> load_map[URL]

The URL is a useful routing address, but it is not a stable replica identity. ReplicaInfo.replica_id already provides that identity in the controller.

Proposed Change

Use replica_id as the identity for load accounting while continuing to use the endpoint URL for network routing.

This prevents the load balancer from confusing:

• an old replica whose URL is later reused by a new replica; and
• the same logical replica whose endpoint changes after a relaunch or recovery.

Immutable Descriptor

This can be done by Introducing an immutable descriptor such as:

@dataclasses.dataclass(frozen=True)
class ReadyReplica:
    replica_id: int
    url: str
    gpu_type: str = 'unknown'

The controller should return a list of replica descriptors rather than a URL-keyed map:

{
  "replica_info": [
    {
      "replica_id": 1,
      "url": "http://replica-a",
      "gpu_type": "A100"
    }
  ]
}

Using a list also avoids making URL uniqueness part of the protocol.

Policy interface changes

The base policy interface changes from URL values to ReadyReplica values:

def set_ready_replicas(
    self, ready_replicas: List[ReadyReplica]
) -> None:

def select_replica(
    self, request: fastapi.Request
) -> Optional[ReadyReplica]:

def begin_request(
        self, replica: ReadyReplica,  request: 'fastapi.Request'
) -> Callable[[], None]:

def pre_execute_hook(
    self, replica: ReadyReplica, request: fastapi.Request
) -> None:

def post_execute_hook(
    self, replica: ReadyReplica, request: fastapi.Request
) -> None:

The policy-specific changes would be:

• RoundRobinPolicy rotates over descriptors and returns the selected descriptor.
• LeastLoadPolicy stores counts by replica_id and retains retired IDs until their in-flight requests finish.
• InstanceAwareLeastLoadPolicy reads gpu_type from the descriptor and can drop its separate URL-to-information map.