nginx.conf.sigil pre-validation chowns proxy_cache_path directories to nobody, breaking the running nginx for the whole build
Description
fn-nginx-vhosts-pre-validate-custom-template runs sudo nginx -t -c <wrapper> on a
rendered app template during core-post-extract. The wrapper comes from
plugins/nginx-vhosts/templates/validate.conf.sigil, which is:
events { worker_connections 768; }
http {
access_log off;
error_log /dev/null;
include {{ $.NGINX_CONF }};
}It carries no user directive. Running as root, nginx therefore falls back to the
build-time default user, which is nobody on the Debian and Ubuntu packages.
ngx_create_paths() chowns every proxy_cache_path directory to that user, and it does so
for directories that already exist, not only for ones it creates. An app template that
declares a proxy_cache_path thus has its live cache directory taken away from the user
the running nginx uses, for the entire duration of the build. Every request the cache
would serve or store then fails:
[crit] open() "/var/cache/nginx/app/1/bc/daf1f8b5..." failed (13: Permission denied)which nginx turns into HTTP 500. The deploy's own reload at the end restores the ownership, so the incident heals by itself and leaves no trace in the directory metadata.
For us that was six minutes of HTTP 500 on a production site, with a healthy app container and a healthy nginx, triggered by a routine dependency-bump deploy.
Steps to reproduce
- Give an app an
nginx.conf.sigilcontaining aproxy_cache_pathand aproxy_cache. - Deploy once so the cache directory exists and is owned by the nginx user.
- Deploy again and request a cacheable URL while the build runs.
Minimal demonstration of the underlying behaviour, no Dokku involved:
# mkdir -p /tmp/probe/cache && chown www-data:www-data /tmp/probe/cache
# printf '%s\n' \
'events { worker_connections 16; }' \
'http {' \
' proxy_cache_path /tmp/probe/cache levels=1:2 keys_zone=z:1m max_size=10m inactive=1m use_temp_path=off;' \
' server { listen 127.0.0.1:18999; location / { proxy_cache z; proxy_pass http://127.0.0.1:18998; } }' \
'}' > /tmp/probe/w.conf
# ls -ldn /tmp/probe/cache
drwxr-xr-x 2 33 33 4096 ... /tmp/probe/cache
# nginx -t -c /tmp/probe/w.conf
nginx: configuration file /tmp/probe/w.conf test is successful
# ls -ldn /tmp/probe/cache
drwxr-xr-x 2 65534 33 4096 ... /tmp/probe/cacheuid 33 is www-data, uid 65534 is nobody. A plain syntax check changed the owner.
Expected
Pre-validation checks syntax without side effects on the running server's state.
Suggested fix
Emit a user directive into the validation wrapper that matches the one the main
configuration uses, so the chown becomes a no-op:
user {{ $.NGINX_USER }};
events { worker_connections 768; }
...sourced from the running nginx.conf or from the existing nginx user detection.
Running the test under a temporary prefix with rewritten cache paths would also work, but that changes what is being validated. Suppressing the chown from the configuration side is not possible.
Environment
- dokku 0.38.27 on Ubuntu 24.04
- nginx 1.24.0 (Ubuntu), built without
--user=, soNGX_USERisnobody plugins/nginx-vhosts/templates/validate.conf.sigilon master is unchanged as of today
Source: dokku/dokku