OSError: [Errno 88] Socket operation on non-socket when running luigid --background on Python 3.13
Problem
When attempting to run the Luigi daemon (luigid) in the background using the --background flag, it crashes with an OSError: [Errno 88] Socket operation on non-socket. This appears to be an issue with how the underlying python-daemon library interacts with sockets in Python 3.13.
The command runs perfectly fine without the --background parameter, and the exact same --background command works without any issues on Python 3.12.
Steps to Reproduce
Set up an environment using Python 3.13.12.
Install Luigi.
Run the following command:
Bash
python --version && luigid --background --logdir ./Expected behavior
The Luigi daemon should start in the background without raising an exception, matching the behavior seen in Python 3.12.
Actual behavior
The process crashes during daemonization with the following traceback: Python
Python 3.13.12 Traceback (most recent call last): File "/mnt/c/pdrive/repos/luigiprobe/.venv/bin/luigid", line 10, in sys.exit(luigid()) ~~~~~~^^ File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/cmdline.py", line 34, in luigid luigi.process.daemonize( ~~~~~~~~~~~~~~~~~~~~~~~^ luigi.server.run, api_port=opts.port, address=opts.address, pidfile=opts.pidfile, logdir=opts.logdir, unix_socket=opts.unix_socket ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ) ^ File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/luigi/process.py", line 94, in daemonize ctx = daemon.DaemonContext( stdout=stdout_proxy, ...<2 lines>... initgroups=False, ) File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 280, in init detach_process = is_detach_process_context_required() File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 801, in is_detach_process_context_required if is_process_started_by_init() or is_process_started_by_superserver(): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 778, in is_process_started_by_superserver if is_socket(stdin_fd): ~~~~~~~~~^^^^^^^^^^ File "/mnt/c/pdrive/repos/luigiprobe/.venv/lib/python3.13/site-packages/daemon/daemon.py", line 743, in is_socket file_socket = socket.fromfd(fd, socket.AF_INET, socket.SOCK_RAW) File "/home/py/conda/lib/python3.13/socket.py", line 549, in fromfd return socket(family, type, proto, nfd) File "/home/py/conda/lib/python3.13/socket.py", line 233, in init _socket.socket.init(self, family, type, proto, fileno) ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [Errno 88] Socket operation on non-socket
Environment details:
Python version: 3.13.12 (Fails) / 3.12 (Works)
Luigi version: [Insert your luigi version here, e.g., 3.6.0]
OS: Linux / WSL (Debian/Ubuntu based on paths)Additional Context
Because the traceback points heavily to daemon/daemon.py, it seems likely this is a compatibility issue between python-daemon (which Luigi uses for the --background flag) and the newly updated socket module in Python 3.13. For now, running luigid in the foreground (without --background) acts as a temporary workaround.
Source: spotify/luigi