[Bug]: Disabled 'All Nodes' policy is bypassed when the host is an egress gateway in another network (blanket net-to-net ACCEPT in NETMAKER-ACL-IN)
What happened?
On a CE server that hosts several networks, disabling the default All Nodes policy in one network and adding explicit per-device policies stops working as soon as the server node is an egress gateway (with at least one range) in any other network on the same host. netclient then installs a blanket -s <netA> -d <netA> -j ACCEPT rule into NETMAKER-ACL-IN, and since it is evaluated before the per-device DROP rules, every client in netA can reach every other client in netA. The policies look correct in the UI and via /api/v1/acls, only the rendered firewall differs.
Observed on v1.6.0 (server and netclient). The same code is present in v1.5.1 and in the v1.7.0 tag, so I expect it to reproduce there as well.
Steps to reproduce
Single server host, community edition, docker compose, netclient on the host.
- Create network
netA(e.g.10.101.0.0/24) with the server node as ingress gateway. Add a few external clients, e.g.test1,test2,test3. - In
netA, disable the defaultAll Nodespolicy, keepAll Gateways, add a policytest1 -> test2. - Verify with
iptables -S NETMAKER-ACL-IN: per-deviceACCEPTfor the pair, per-deviceDROPfor every client, nothing else.test1reachestest2,test1cannot reachtest3,test2cannot reachtest3. So far correct. - Create a second network
netB(e.g.10.102.0.0/24) on the same host, leave its defaults, and add an egress resource to the server node innetB(POST /api/v1/egress, any range, e.g.203.0.113.10/32). - Look at
NETMAKER-ACL-INagain. A new rule appears fornetA:
-A NETMAKER-ACL-IN -s 10.101.0.0/24 -d 10.101.0.0/24 -m comment --comment NETMAKER -j ACCEPTIt sits above all the per-device DROP rules. test1 -> test3 and test2 -> test3 now succeed although no policy allows them. Deleting the egress in netB makes the rule disappear and isolation in netA works again.
Nothing in netA itself was changed between steps 3 and 5.
Root cause (from reading the code)
logic/peers.go, GetPeerUpdateForHost. In the per-node loop, a network gets an entry in FwUpdate.AllowedNetworks when
if (defaultDevicePolicy.Enabled && defaultUserPolicy.Enabled) ||
(!CheckIfAnyPolicyisUniDirectional(node, acls) &&
!(node.EgressDetails.IsEgressGateway && len(node.EgressDetails.EgressGatewayRanges) > 0)) {
aclRule := models.AclRule{ ..., IPList: []net.IPNet{node.NetworkRange}, ... }
if !(defaultDevicePolicy.Enabled && defaultUserPolicy.Enabled) {
aclRule.Dst = []net.IPNet{node.NetworkRange}
}
hostPeerUpdate.FwUpdate.AllowedNetworks = append(hostPeerUpdate.FwUpdate.AllowedNetworks, aclRule)
} else {
hostPeerUpdate.FwUpdate.AllowAll = false
...
}So a default-deny network (All Nodes disabled, no uni-directional policies, node not an egress gateway in that network) still gets an AllowedNetworks entry, and for the default-deny case it gets Dst = NetworkRange, i.e. an allow-all-within-the-network rule.
That entry is harmless as long as AllowAll stays true, because netclient then never installs AllowedNetworks. But AllowAll is a single flag per host. Once any other network on the host takes the else branch (the node is an egress gateway there), the deferred block at the top of the function copies every AllowedNetworks entry into AclRules:
defer func() {
if !hostPeerUpdate.FwUpdate.AllowAll {
...
for _, aclRule := range hostPeerUpdate.FwUpdate.AllowedNetworks {
hostPeerUpdate.FwUpdate.AclRules[aclRule.ID] = aclRule
hostPeerUpdate.FwUpdate.EgressInfo["allowed-network-rules"].EgressFwRules[aclRule.ID] = aclRule
}
}
}()netclient (firewall/acl.go, ProcessAclRules / AddAclRules) inserts each AclRules entry at the top of NETMAKER-ACL-IN, so the netA -> netA ACCEPT shadows the per-device DROPs that netclient also installs for the same network.
The per-device rules for the default-deny network are generated independently and are already present in the chain, so the copied network-wide rule is purely additive and, for a network with All Nodes disabled, contradicts the configured policies.
Expected behaviour
Disabling All Nodes in a network should produce the same firewall regardless of whether the same host is an egress gateway in some other network. Concretely, the deferred block should only copy AllowedNetworks entries for networks whose default device policy is enabled (the ones without Dst set). For a default-deny network the network-wide rule should not be emitted into AclRules at all.
Workaround
Adding an egress resource with an unused range (for example 192.0.2.1/32) to the server node in the default-deny network itself flips that network into the else branch, the blanket rule is no longer generated for it and the per-device rules take effect. It works but it is clearly not how egress is meant to be used.
Version
v1.6.0 (server and netclient). Same code in v1.5.1 and v1.7.0.
What OS are you using?
Linux, Ubuntu, docker compose deployment of the CE server, netclient v1.6.0 installed on the host.
Relevant log output
No errors are logged. The only visible symptom is the extra rule in iptables -S NETMAKER-ACL-IN shown above.
Source: gravitl/netmaker