[Bug Report] K8s cube-node Pod recreate kills sandboxes and leaves CubeVS port mappings that block new creates

Summary

On Helm/Kubernetes deployments (default cubeNode.hostNetwork: false), deleting/recreating the cube-node Big Pod interrupts all sandboxes on that node and can leave orphan CubeVS eBPF port mappings on the host. After recreate, new template/sandbox creates that reuse the same TAP ifindex fail in EnsureNetwork with a port-mapping conflict. hostNetwork: true avoids the netns teardown path but introduces its own CIDR / CNI operational hazards, so it is not a clean fix by itself.

Environment

• CubeSandbox version / commit: v0.7.0-class Helm deploy (cube-shim v0.7.0)
• Host OS and kernel version: Linux compute node (single-node K8s)
• Deployment mode: Kubernetes / Helm (cube-system)
• Relevant component: Cubelet network runtime / CubeVS (local_port_mapping / remote_port_mapping), cube-node DaemonSet

Steps to Reproduce

1. Deploy CubeSandbox via Helm with default Pod networking (hostNetwork: false).
2. Create a sandbox (or build a template) that exposes guest port 49983 (e.g. envd). Confirm a live mapping, for example:
   • cubevsmapdump --map local_port_mapping,remote_port_mapping → ifindex=4, guest_port=49983, host_port=20003
   • cubecli cubebox ls shows the sandbox Up
   • pgrep -af containerd-shim-cube-rs shows the corresponding shim
3. Delete the Big Pod: kubectl -n cube-system delete pod cubesandbox-node-<suffix>
4. Wait until the replacement cube-node Pod is Ready (new UID / PodIP).
5. Inspect again, then create another template/sandbox with the same exposed port.

Expected Behavior

• Recreating cube-node should either:
  • not destroy in-use sandbox networking without an explicit drain/isolate path, or
  • fully tear down sandbox dataplane state (including CubeVS port maps) so the node can create new sandboxes immediately after recover.
• New EnsureNetwork for the same guest port on a reused TAP ifindex should succeed (or get a clear, recoverable cleanup path).

Actual Behavior

A. Existing business sandboxes are killed / become unusable

• Guest session dies (rpc error: Unavailable ... EOF).
• containerd-shim-cube-rs for the old sandbox disappears.
• cubecli cubebox ls is empty on the new Pod.
• This matches the documented Big Pod recreate / netns teardown behavior, but the operational impact is severe for any accidental kubectl delete pod / DaemonSet roll.

B. After recreate, new creates fail on stale port maps

Port maps remain on host bpffs even though no sandbox owns them:

{
  "maps": {
    "local_port_mapping": [
      { "ifindex": 4, "guest_port": 49983, "host_port": 20003 }
    ],
    "remote_port_mapping": [
      { "host_port": 20003, "ifindex": 4, "guest_port": 49983 }
    ]
  }
}

New create allocates a fresh host port (e.g. 20001) for the same ifindex/listen_port and fails:

network runtime EnsureNetwork failed: ifindex/listen_port 4/49983 already maps to host port 20003, cannot map to 20001

So the node is left in a state where old sandboxes are gone and new sandboxes cannot start until the orphan map entries are deleted manually.

Comparison (before / after one controlled recreate)

Why hostNetwork: true is not a complete answer

Docs/chart already note that Big Pod recreate with Pod networking tears down sandbox netns. Enabling hostNetwork keeps TAPs on the host netns and can avoid (A), but:

1. CIDR blackhole risk (called out in TKE values): if cubevs/sandbox CIDR overlaps cluster Service/VPC CIDR, cube-node installs a host route into the cubevs bridge and can swallow ClusterIP / DNS (node registration + in-cluster DNS break). Example: Chart default cubevs 172.16.0.0/18 vs TKE Service CIDR 172.16.0.0/16.
2. CNI / node-agent load: the TAP pool (hundreds of z172.* links) becomes visible on the host; Flannel / similar netlink watchers may burn CPU on link churn.
3. Even with hostNetwork, orphan port-map cleanup is still a runtime correctness gap if teardown is incomplete.

So hostNetwork trades one failure mode for other deployment constraints; it does not remove the need for reliable port-map cleanup on recreate/recover.

Additional Context

• Related (different path): #1589 (legacy recovery leaving port mappings on an old TAP ifindex). This report is specifically the K8s Big Pod recreate path: sandboxes disappear from cubelet, but host-pinned CubeVS maps remain and block reuse of the same ifindex + guest port.
• Suspected gap: create path (applyPortMappings) assumes a clean ifindex, while recover uses DeletePortMappingsByIfindex / reconcile. After Pod recreate there may be no durable Active owner left to drive cleanup, leaving orphans until manual bpf map delete / cubevsmapdump-guided cleanup.
• Binary/systemctl restart cubelet is harder to hit for this exact combo (no Pod netns teardown); the easy, reproducible trigger is Helm cube-node Pod delete/recreate.

Ask

1. Treat accidental/kubectl delete Big Pod recreate as a first-class failure mode: either require isolate+drain, or make recreate safe.
2. On startup / EnsureNetwork, sweep or reconcile orphan CubeVS port mappings for reusable TAP ifindexes so a recreated node can create again without manual bpf surgery.
3. Keep documenting hostNetwork trade-offs (CIDR vs Service/VPC, CNI CPU) so it is not presented as a zero-cost workaround for (A)+(B).