Expose explicit lifecycle API for gRPC storage proxy server
Motivation
Optuna's gRPC storage proxy is useful for distributed optimization where one process hosts a proxy in front of an RDBStorage backend and other processes connect with GrpcStorageProxy.
Today the public server API is:
optuna.storages.run_grpc_proxy_server(storage, host="...", port=13000)That function starts the server and then blocks on server.wait_for_termination(). For users who want to embed the gRPC proxy inside a larger application/process, the practical pattern is to run run_grpc_proxy_server(...) in a background thread/process and let process shutdown clean up the server. This works, but it makes the server lifecycle implicit and makes graceful shutdown harder to reason about.
The client side has GrpcStorageProxy.close(), but the server side does not appear to expose an equivalent public start/stop/context-manager API.
Proposal
Expose a public server lifecycle API, for example one of:
server = optuna.storages.create_grpc_proxy_server(storage, host="0.0.0.0", port=13000)
server.start()
try:
...
finally:
server.stop(grace=30)or:
with optuna.storages.grpc_proxy_server(storage, host="0.0.0.0", port=13000):
...This could wrap the existing internal make_server(...) behavior while keeping the public API stable and documented.
Documentation request
It would also be useful for the gRPC storage docs to explicitly call out lifecycle semantics for embedded/distributed use cases:
run_grpc_proxy_server(...)is a blocking helper intended for standalone server processes.- If the proxy is hosted inside one worker/rank of a distributed job, callers must ensure that worker/rank does not shut down the server until other participants are done using the storage.
- A graceful server stop can drain active RPCs, but it cannot know whether an idle remote worker intends to issue another storage request later; that still requires an application/framework-level barrier or external storage whose lifetime is independent of any one worker.
Why this matters
A common distributed pattern is:
- Rank 0 starts an Optuna gRPC storage proxy backed by local or remote
RDBStorage. - Other ranks connect through
GrpcStorageProxyand run trials. - Rank 0 does post-optimization work and eventually exits.
If rank 0 owns the proxy lifecycle, explicit lifecycle APIs and docs make it easier to avoid accidentally terminating the proxy while other ranks may still read final study state.
Source: optuna/optuna