cpu command in Go, inspired by the Plan 9 cpu command
This repo is an implementation the Plan 9 cpu command, both client and server, for Linux. More detail is available in the CPU chapter of the LinuxBoot book. Unlike the Plan 9 command, this version uses the ssh protocol for the underlying transport. It includes features familiar to ssh users, such as support for the ssh config file.
The cpu command lets you log in from a local system to a remote system and see some or all of the files (how much is up to you) from the local system.
This is wonderfully convenient for embedded systems programmers. Because some or all the files can come from your local machine, including binaries, the only thing you need installed on the remote machine is the cpu daemon itself.
Consider the case of running a complex Python program on an embedded system. We will need to either do a full install of some distro on that system, meaning we need USB ports and local storage; or we will need to run the program over the network.
Installing distros can turn into a mess. Some programs only work under specific distros. In some cases, when two programs are needed in a pipeline, it can happen that they only work under different distros! Users are left juggling USB sticks and NVME cards, and this fails the first time there are two programs which need two different distros.
Running over a network is usually done with ssh, but ssh can not supply the programs and files. We would need to either set up a network file system, meaning finding a sysadmin willing to set it up, and keep it working; or, trying to figure out which files the program needs, and using rsync or scp to get them there. In some cases, the target system might not have enough memory to hold those files!
Cpu looks like ssh, but with an important difference: it also provides a file transport so that the files your program needs are available via a 9p mount. For example, if I have an embedded system named camera, and I need to read the flash with the flashrom command, I simply type:
cpu camera flashrom -r rom.imgBreaking this down: cpu is the cpu command; camera is the host name; flashrom is the command to run; the options are to do a read (-r) into a file called rom.img.
Where does that file end up? In whatever of my home directories I ran the cpu command from. I need not worry about scp'ing it back, or any such thing; it's just there.
You can easily build your own docker container to try things out.
docker build -t "${USER}/cpu:latest" .or if you have installed a version of docker buildx, you can build a multi-arch manifest container and push it to docker hub:
% docker login
% docker buildx build --platform "linux/amd64,linux/arm64,linux/arm/v7" --progress plain --pull -t "${USER}/cpu:latest" .We have created a docker container so you can try cpu client and server:
ghcr.io/u-root/cpu:mainIt includes both the cpud (server) and cpu (client) commands. In the container, you only have access to date and cat commands, but that is enough to get the idea.
You will need keys. You can either use your own SSH keys that you use for other things, for example:
export KEY=~/.ssh/id_rsa
export KEY=~/.ssh/a_special_key_for_this_dockeror generate one and use it.
ssh-keygen -f key -t rsa -P ""
export KEY=`pwd`/keyNOTE! The name KEY is not required. Instead of KEY, you can use any name you want, as long as you use it in the docker command below.
To start the cpud, you need docker installed. Once that is done, you need to create a docker network and start the daemon, with public and private keys. The --mount option allows docker to provide the keys, using a bind mount for both the private and public key. That is how we avoid storing keys in the container itself.
docker network create cpud
# If you ran docker before and it failed in some way, you may need to remove the
# old identity (e.g. docker rm cpud_test)
docker run --rm -v $KEY.pub:/key.pub -v $KEY:/key -v /tmp --name cpud_test --privileged=true -t -i -p 17010:17010 ghcr.io/u-root/cpu:mainThen you can try running a command or two by using the embedded cpu client in the docker container.
NOTE: when you run cpu in this way, it does not immediately have access to your host's file system,
which means you won't be able to really leverage the back-mount and you'll have to artificially set
environment variables (like PWD) so that the remote task can execute.
docker exec -it -e PWD=/root cpud_test /bin/cpu -key /key localhost /bin/dateRemember, this cpu command is running in the container. You need to use the name /key in the container, not $KEY and you'll only be able to run binaries that have been pre-loaded into the container (which in the public container is /bin/cat and /bin/date)
To see the mounts:
docker exec -it -e PWD=/root cpud_test /bin/cpu -key /key localhost /bin/cat /proc/mountsYou might want to just get a cpu command to let you talk to the docker cpud directly:
go install github.com/u-root/cpu/cmds/cpu@latestAnd now you can run
cpu -key $KEY localhost dateNOTE: if you are running on OSX, remember that your cpud docker is linux, so if you try the above command you'll see:
$ cpu -key $KEY localhost date
2022/10/13 18:52:17 CPUD(as remote):fork/exec /bin/date: exec format errorTo deal with that we'll have to play games with the namespace, which we'll cover next.
The cpu command sets up the various 9p mounts with a default namespace. Users can override this default with the -namespace switch. The argument to the switch looks like PATH variables, with comma-separated values, but with one extra option: users can, optionally, specify the local path and the remote path. This is useful when running ARM binaries hosted from an x86 system.
In the example below, we show starting up a bash on an ARM system (solidrun honeycomb) using a cpu command running on an x86 system.
cpu -namespace /home:/bin=`pwd`/bin:/lib=`pwd`/lib:/usr=`pwd`/usr honeycomb /bin/bashBreaking this down, we set up the namespace so that:
pwd/bin -- which, in this case, was an unpacked arm64 file system imagepwd/libpwd/usrWe can use the path /bin/bash, because /bin/bash on the remote points to pwd/bin/bash on the local
machine.
We can use the same trick to cpu to Linux from OSX, but instead of having an Arm tree under pwd
we'll need a Linux binary tree to pick binaries from.
There are many IoT like devices that do not have an ethernet port. Fear not though: The Linux USB gadget drivers offer ethernet via USB!
There are tutorials out there, and here is the gist:
CONFIG_USB_GADGETCONFIG_USB_ETHCONFIG_INETCMDLINE, e.g., g_ether.dev_addr=12:34:56:78:9a:bc g_ether.host_addr=12:34:56:78:9a:bdg_cdc.dev_addr/g_cdc.host_addrYou will also need to configure the network. You can either go with DHCP or
fixed addresses. Since cpud spawns a shell, if your shell allows for it, you
can use a shell rc file.
As mentioned, cpu looks and feels a lot like ssh, to the point of honoring ssh config files. For the honeycomb, for example, the ssh config entry looks like this (we shorten the name to 'h' for convenience):
Host h
HostName honeycomb
Port 17010
User root
IdentityFile ~/.ssh/apu2_rsaNote that the cpu command is itself a 9p server; i.e., your instance of cpu runs your server. The remote cpu server may run as root, but all file accesses happen locally as you. Hence, the cpu command does not grant greater access to the local machine than you already possess. I.e., there is no privilege escalation.
Maintaining file system images is inconvenient. We can use Docker containers on remote hosts instead. We can take a standard Docker container and, with suitable options, use docker to start the container with cpu as the first program it runs.
That means we can use any Docker image, on any architecture, at any time; and we can even run more than one at a time, since the namespaces are private.
In this example, we are starting a standard Ubuntu image:
…Note that the image was updated and then started. The /lib64 mount fails, because there is no /lib64 directory in the image, but that is harmless.
On the local host, on which we ran docker, this image will show up in docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b92a3576229b ubuntu "/home/rminnich/go/b…" 9 seconds ago Up 9 seconds inspiring_mcnultyEven though the binaries themselves are running on the remote ARM system.
Vsock is a useful transport layer available in Linux, and support by at least QEMU.
We use the mdlayher/vsock package.
In the cpu and cpud, the switch
-net vsockwill enable vsock.
In the host kernel, you need vhost_vsock module:
sudo modprobe vhost_vsock.
When starting qemu, add
-device vhost-vsock-pci,id=vhost-vsock-pci0,guest-cid=3to the command line. The '3' is arbitrary; it just needs to be agreed upon on both sides.
When running a cpu command, the host name is the vsock guest-cid you specified in qemu:
cpu -net vsock 3 dateIf you want a different port, you can use the same -sp switch you use for other network types.
A variation of the cpu and cpud commands now exist which use dns-sd to autodiscover and register cpu resources. DNS-SD is a multi-cast DNS protocol that will multi-cast out requests for resources and get responses from participating cpud nodes. Meta-data provides information such as architecture, OS, number of cores, free memory, load average, and number of existing cpu clients. When you start a decpud you can specify additional meta-data which might be harder to auto-disocver such as near=storage, near=gateway, near=gpu, or secure=true (if running in a confidential computing domain).
In order to use this functionality, you can use decpu and decpud just as you would cpu and cpud. decpud will enable dns-sd registration by default, and multicast its information in response to requests. In order to use this from decpu, you can just specify decpu without a hostname and, by default, it will find the lowest loaded decpud with the same architecture and OS as the host you run the decpu command from.
You can specify additional constraints by using a dns-sd URI formulation:
decpu dnssd://?requirement=value\&otherrequirement=\) Finally, you can specify any number of numeric keys to sort based on. If you want to ignore one of the default requirements (e.g. arch, os) then you can set to a \*:decpu dnssd://?arch=*&os=*
…% docker create network cpud % go install github.com/decent-e/dnssd/cmd/relay@latest % export PATH=$GOBIN:$PATH % relay & % docker run -d --network cpud ghcr.io/decent-e/dnssd:main % docker run -d --network cpud -v $KEY:/key -v $KEY.pub:/key.pub -v /tmp:/tmp --privileged --rm --name decpud ghcr.io/decent-e/cpu:decent-e /bin/decpud % docker exec -i -t -e PWD=/ decpud /b
No open issues yet, or sync has not completed.