Network security testing for Kubernetes DevSecOps workflows
Network security testing for Kubernetes DevSecOps workflows
[![Testing Workflow][testing_workflow_badge]][testing_workflow_badge] [![Release Workflow][release_workflow_badge]][release_workflow_badge]
NetAssert is a command line tool that enables you to check the network connectivity between Kubernetes objects such as Pods, Deployments, DaemonSets, and StatefulSets, as well as test their connectivity to remote hosts or IP addresses. NetAssert v2 is a rewrite of original NetAssert tool in Go that utilises the ephemeral container support in Kubernetes to verify network connectivity. NetAssert test(s) are defined in YAML format. NetAssert currently supports TCP and UDP protocols:
To perform a TCP test, only a scanner container is used. This container requires no privileges nor any Linux capabilities.
To run a UDP test, a sniffer ephemeral container is injected into the target Pod which requires cap_raw capabilities to read data from the network interface. During UDP testing, NetAssert runs both container scanner and sniffer container images which are injected as ephemeral containers into running Pods.
The sniffer and scanner container images can be downloaded from:
docker.io/controlplane/netassertv2-l4-client:latestdocker.io/controlplane/netassertv2-packet-sniffer:latestcap_raw capabilities to read data from the network interfaceNetAssert utilises the above containers during test and configures them using environment variables. The list of environment variables that are used can be found here and here. It is possible to override the sniffer and scanner images from command line during a run, so one can also bring their own container image(s) as long as they support the same environment variables.
Please download the latest stable version of NetAssert from releases page. The binary is available for Linux, MacOS and Windows platforms.
If you are on Unix/Linux, you can also use the download.sh script to download the latest version of NetAssert into the current path:
curl -sL https://raw.githubusercontent.com/controlplaneio/netassert/master/download.sh | bashNetAssert v2 tests are written in YAML format. Each test is a YAML document which supports the following mappings:
NetAssert test. Each test has the following keys:deployment, statefulset, daemonset or podk8sResource and host are not supported at the same time :deployment, statefulset, daemonset or podThis is an example of a test that can be consumed by NetAssert utility
…NetAssert has three main components:
Netassert-Engine or simply the EngineAll the tests are read from an YAML file or a directory (step 1) and the results are written following the TAP format (step 5 for UDP and step 4 for TCP). The tests are performed in two different manners depending on whether a TCP or UDP connection is used
src and dst fields are correct: for udp tests both of them must be of type k8sResourcedstPod in the object defined by the dst.k8sResource field. Ensure that the Pod is in running state and has an IP address allocated by the CNIsrcPod in the object defined by the src.k8sResource field. Ensure that the Pod is in running state and has an IP address allocated by the CNInetassert-l4-client as an ephemeral container in the srcPod (step 2) and set the port and protocol according to the test specifications. Provide also the target host equal to the previously found dstPod IP address, and the random UUID that was generated in the previous step as the message to be sent over the udp connection. At the same time, inject the netassertv2-packet-sniffer (step 3) as an ephemeral container in the dstPod using the protocol, search string, number of matches and timeout defined in the test specifications. The search_string environment variable is equal to the UUID that was generated in the previous step which is expected to be found in the data sent by the scanner when the connections are successful.netassertv2-packet-sniffer ephemeral sniffer container’s exit status matches the one defined in the test specificationnetassert-l4-client, exits with exit status of zero. This should always be the case as UDP is not a connection oriented protocol.src field is of type k8sResourcesrcPod in the object defined by the src.k8sResource field. Ensure that the Pod is in running state and has an IPAddressdst has k8sResource defined as a child object. If so then find a running Pod defined by the dst.K8sResourcenetassert-l4-client as an ephemeral container in the srcPod (step 2). Configure the netassert-l4-client similarly to the udp case. If the dst field is set to host then use the host name field as the scanner target hostexitCode field defined in the test specificationjust build. The resulting binary will be in cmd/netassert/cli/netassert. To run unit tests you can use just test. There is a separate README.md that details end-to-end testing.kind and its prerequisitesjustNetAssert binary from the release page:❯ VERSION="v2.1.3" # change it to the version you want to install
❯ OS_DISTRO=linux_amd64 # change it to your OS_DISTRO (for reference check the NetAssert release page)
❯ curl -L -o netassert.tar.gz https://github.com/controlplaneio/netassert/releases/download/${VERSION}/netassert_${VERSION}_${OS_ARCH}.tar.gz
❯ tar -xzf netassert.tar.gz -C bin/netassertNetAssert from source:❯ just buildjustfile included in the repo:❯ just kind-down ; just kind-up
❯ just calico-apply❯ kubectl get nodes -w❯ just k8s-apply
kubectl apply -f ./e2e/manifests/workload.yaml
namespace/fluentd created
daemonset.apps/fluentd created
namespace/echoserver created
namespace/busybox created
deployment.apps/echoserver created
deployment.apps/busybox created
namespace/pod1 created
namespace/pod2 created
pod/pod2 created
pod/pod1 created
namespace/web created
statefulset.apps/web created❯ just netpol-apply
kubectl apply -f ./e2e/manifests/networkpolicies.yaml
networkpolicy.networking.k8s.io/web createdjust k8s-apply in a previous step):…❯ bin/netassert run --input-file ./e2e/manifests/test-cases.yaml
❯ cat results.tap
TAP version 14
1..9
ok 1 - busybox-deploy-to-echoserver-deploy
ok 2 - busybox-deploy-to-echoserver-deploy-2
ok 3 - fluentd-deamonset-to-echoserver-deploy
ok 4 - busybox-deploy-to-web-statefulset
ok 5 - web-statefulset-to-busybox-deploy
ok 6 - fluentd-daemonset-to-web-statefulset
ok 7 - busybox-deploy-to-control-plane-dot-io
ok 8 - test-from-pod1-to-pod2
ok 9 - busybox-deploy-to-fake-host…NetAssert is architected for compatibility with Kubernetes versions that offer support for ephemeral containers. We have thoroughly tested NetAssert with Kubernetes versions 1.25 to 1.35, confirming compatibility and performance stability.
For broader validation, our team has also executed comprehensive [end-to-end tests](./e2e/
No open issues yet, or sync has not completed.