File ownership checks might break K8s installs once they will be enforced

Hi,

there is a possible problem with the file ownership checks in mosquitto when files are mounted from K8S secrets. Mosquitto checks file ownerships on startup and if these are not owned by user 1883 and group 1883 (or whatever was specified in PUID and PGID), a warning will be displayed, saying that these files will be rejected in future versions (v3?). Although this warning can be ignored for now, it would break installs once the ownership is enforced in the future.

The entrypoint script tries to set UID/GID ownership on all files via chown. However, this does not work in K8s as files mounted through secrects are read-only. While you can configure K8S to use a specific GID for those files using the fsgroup option in securityContext, you cannot set the UID (because there is no fsuser option). Conequently, all mounted files are onwed by root.

As soon as a future mosquitto version enforces UIDs to on files, this will break K8s installs with secret mounts. There are two possible workarounds:

1. Set PUID environment variable to 0. Then the owner of the files matches. However mosquitto would then run as root, which is not desired.
2. Make a copy of the mounted files in an init container, change their ownership and store them elsewhere. This is cumbersome.

It would be nice, if the ownership enforcement could be relaxed for K8S installs to only check for group ownership and allow files owned by root as well so that K8S installs would continue to work.

This problem is related #3461, there mosquitto v2.1 does not allow to follow links for better security. This makes v2.1 currently unusable on K8S if you mount files via secrets as the mount makes use of symbolic links. The proposed solution is to introduce a config option to allow links. The same approach could be used here aws well to optionally allow files onwed by root.