[Bug] Plugin availability API returns only self=true and all remote nodes=false in Nacos 3.2.4 cluster
Bug Report
Describe the bug
In a 3-node Nacos 3.2.4 cluster, the new Console Plugin Management page
reports all enabled plugins as available on only 1 / 3 nodes.
This is not limited to the MySQL datasource plugin. Multiple enabled plugins show the same result, for example:
datasource-dialect:mysql 1 / 3
ai-storage:nacos_config 1 / 3
auth:nacos 1 / 3
control:nacos 1 / 3
visibility:nacos 1 / 3
ai-pipeline:skill-scanner 1 / 3The three Nacos nodes themselves are running normally.
More importantly, querying /v3/console/plugin/availability directly against
each individual Console node shows that each node reports only itself as
available, while the other two nodes are reported as unavailable.
Environment
- Nacos Server:
3.2.4 - Docker image:
nacos/nacos-server:v3.2.4 - Java:
17 - Deployment mode:
cluster - Kubernetes: Tencent Cloud TKE
- Replicas:
3 - Deployment type: StatefulSet
- External datasource: MySQL
- Host mode:
hostname
Cluster members:
nacos-engine-0.nacos-hs.engine.svc.cluster.local:8848
nacos-engine-1.nacos-hs.engine.svc.cluster.local:8848
nacos-engine-2.nacos-hs.engine.svc.cluster.local:8848The three pods use the same datasource configuration:
SPRING_DATASOURCE_PLATFORM=mysql
MYSQL_SERVICE_HOST=<same MySQL host>
MYSQL_SERVICE_PORT=<same MySQL port>
MYSQL_SERVICE_DB_NAME=nacos_engine
MYSQL_SERVICE_USER=<same user>
PREFER_HOST_MODE=hostnameThe Nacos server identity key/value are also configured consistently across all three cluster members.
What has already been verified
All three Nacos pods are Running and stable.
Configuration persistence works normally. A test configuration created from the Nacos Console:
Data ID: test-nacos.yaml
Group: DEFAULT_GROUPis successfully persisted into the external MySQL config_info table.
Inter-node TCP connectivity has also been verified successfully between the Nacos pods:
8848 OK
9848 OK
9849 OKAfter the cluster became stable, nacos.log did not show continuing errors
matching:
Server check fail
Fail to connect server
UNAVAILABLE
grpc fail
cluster failSteps to reproduce
- Deploy a 3-node Nacos 3.2.4 cluster.
- Use hostname mode with a Kubernetes Headless Service.
- Configure all three nodes to use the same MySQL datasource.
- Wait until all three nodes are Running and the cluster is stable.
- Log in to the Nacos Console.
- Open
Plugin Management. - Observe that enabled plugins show
1 / 3available nodes. - Query the plugin availability API directly on each Nacos Console node.
Example:
curl -sS \
-H "Authorization: Bearer $TOKEN" \
'http://nacos-engine-0.nacos-hs.engine.svc.cluster.local:8080/v3/console/plugin/availability?pluginType=datasource-dialect&pluginName=mysql'Result from nacos-engine-0:
{
"code": 0,
"message": "success",
"data": {
"nacos-engine-2.nacos-hs.engine.svc.cluster.local:8848": false,
"nacos-engine-1.nacos-hs.engine.svc.cluster.local:8848": false,
"nacos-engine-0.nacos-hs.engine.svc.cluster.local:8848": true
}
}Querying nacos-engine-1:
{
"code": 0,
"message": "success",
"data": {
"nacos-engine-2.nacos-hs.engine.svc.cluster.local:8848": false,
"nacos-engine-1.nacos-hs.engine.svc.cluster.local:8848": true,
"nacos-engine-0.nacos-hs.engine.svc.cluster.local:8848": false
}
}Querying nacos-engine-2:
{
"code": 0,
"message": "success",
"data": {
"nacos-engine-2.nacos-hs.engine.svc.cluster.local:8848": true,
"nacos-engine-1.nacos-hs.engine.svc.cluster.local:8848": false,
"nacos-engine-0.nacos-hs.engine.svc.cluster.local:8848": false
}
}Therefore the behavior is deterministic:
request node-0 -> node-0=true, node-1=false, node-2=false
request node-1 -> node-0=false, node-1=true, node-2=false
request node-2 -> node-0=false, node-1=false, node-2=trueEach node appears to recognize its local plugin correctly, but remote-node
plugin availability is always reported as false.
Expected behavior
According to the Nacos Console API documentation,
GET /v3/console/plugin/availability returns:
Map<node address, availability>When the same enabled plugin is available on all three healthy Nacos nodes, the expected result should be similar to:
{
"nacos-engine-0.nacos-hs.engine.svc.cluster.local:8848": true,
"nacos-engine-1.nacos-hs.engine.svc.cluster.local:8848": true,
"nacos-engine-2.nacos-hs.engine.svc.cluster.local:8848": true
}and the Console should display:
3 / 3Actual behavior
Each Console node reports only its own plugin as available:
local node = true
remote node = false
remote node = falseThe Console therefore always displays:
1 / 3for all enabled plugins.
Additional observations
This does not currently appear to affect basic Nacos Config functionality.
- All three Nacos server pods remain healthy.
- MySQL persistence works.
- Configurations created through the Console are written successfully to MySQL.
- Inter-node 8848/9848/9849 TCP connectivity is working.
Because the same symptom occurs for multiple unrelated plugins, this appears more likely to be related to the cluster-level plugin availability query mechanism rather than the MySQL datasource plugin itself.
The current Nacos internal RPC specification documents:
PluginAvailabilityRequest
PluginAvailabilityResponseas the internal RPC used to query plugin availability on a node.
Could you please confirm whether there is a known issue in Nacos 3.2.4 with
cross-node PluginAvailability queries, RPC dispatch, payload registration,
or plugin availability aggregation?
Official documentation references
Console Plugin Availability API:
https://nacos.io/docs/latest/manual/admin/console-api/
Nacos internal RPC specification:
https://github.com/alibaba/nacos/blob/develop/specs/en/design/foundation-internal-rpc-spec.md
Nacos 3.2.4 release:
https://github.com/alibaba/nacos/releases/tag/3.2.4
Source: alibaba/nacos