cluster_link: node shutdown can hang on unabortable SR sync destination writes

Symptom

Node shutdown hangs until the ducktape 30s node-stop timeout kills the process, reported as TimeoutError('Redpanda node docker-rp-N failed to stop in 30 seconds') at test teardown, with Service cluster_link::service is taking more than 15 seconds to shut down in the node log. Observed in ConfluentSchemaRegistrySyncE2ETest.test_schema_registry_api_sync_memory_backpressure in release-mode CI while a shadow-link Schema Registry sync was mid-flight at teardown.

Root cause

Application shutdown stops partitions (including kafka/_schemas/0) and raft before cluster_link::service. An SR sync run fiber that is inside a destination write when raft dies parks forever:

• reconciler import / mode-config apply -> schema::registry (src/v/schema/registry.cc:120 area) -> pandaproxy seq_writer -> produce to the local _schemas partition
• that write path takes no abort source and no deadline, so a produce parked on a stopping/stopped raft never resolves
• the task-stop join (from the link/leadership teardown path) waits for the run fiber, manager::stop()'s queue drain waits for the join, node stop times out

The trigger observed in CI: _schemas/0 leadership lost at the instant of shutdown (Handling leadership change for NTP={kafka/_schemas/0}, is_ntp_leader=false 21ms before service stop) while the sync's mode/config phase was completing, whose apply step always performs a destination write against a Confluent source (the destination global-compat rewrite). Release-only in practice: timing shifts which sync phase teardown lands in.

Note task::stop() moves _task_runner out before joining, so whichever caller reaches it first owns the join; a later link::stop() completes vacuously while the first caller stays wedged. Any fix should keep that in mind when reasoning about who is blocked.

This predates the SR sync rate limiting (#31101); the rate cap stretches how long a sync (and its read->apply pipelines) stays in flight, which widened the race window enough for CI to hit it. #31101 fixed two adjacent orderings (manager stop draining before aborting tasks; mirroring_task joining before stopping its reader) — this remaining wedge is the destination write itself.

Candidate fixes

1. Make the SR write path abortable/bounded: thread an abort source or deadline through schema::registry writes -> seq_writer -> produce, so task aborts reach a parked destination write.
2. Alternatively (or additionally) reorder application shutdown so cluster_link::service — a consumer of partitions — stops before the partition manager rather than after; today its syncs outlive the raft they write to.

Workaround in tree

The affected ducktape test pins max_source_requests_per_second high so syncs complete quickly and teardown lands idle, restoring the pre-rate-limiting probability profile; remove that once this is fixed.

Evidence

CI artifacts: production-devprod/redpanda/87110/019f6462-2c39-4b8e-81f6-b1bcd2f03b61/vbuild/ducktape/results/2026-07-15--001/ConfluentSchemaRegistrySyncE2ETest/test_schema_registry_api_sync_memory_backpressure/147/ (docker-rp-27 log: link/tasks stop cleanly in 2ms, manager stop: links stopped logged, work queue drained never logged; leadership-change handler owns the wedged join).