Scalene fails profiling when using joblib for multiprocessing
Author: legendof-seldaCreated Oct 26, 2021Updated Feb 21, 2025
Describe the bug
When trying to profile with joblib multiprocessing using loky or multiprocessing backend, scalene gives out an error.
To Reproduce Steps to reproduce the behavior:
- Create the following script
scalene_test.py
from joblib import Parallel, delayed
from math import sqrt
from time import sleep
def func(i):
sleep(0.5)
return sqrt(i)
def main():
Parallel(n_jobs=2)(delayed(func)(i**2) for i in range(2))
return
main()- Run
python -m scalene scalene_test.py - See error
Expected behavior
Expected behavior is that the profiling complete without any failure. It seems like the semaphore lock is released too many times that it fails. But this works for for from multiprocessing import Pool but not for from joblib import Parallel
Screenshots If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
- OS: Ubuntu 18.04.6 LTS
- Browser: None
- Version: scalene==1.3.15, joblib==1.1.0, Python 3.8.11
Additional context Add any other context about the problem here.
Terminal output
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: could not find input file scalene_test.py
Error in program being profiled:
[Errno 9] Bad file descriptor
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
Error in program being profiled:
semaphore or lock released too many times
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
[Errno 9] Bad file descriptor
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 368, in spawnv_passfds
return fork_exec(args, _pass)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/fork_exec.py", line 46, in fork_exec
os.execve(sys.executable, cmd, child_env)
FileNotFoundError: [Errno 2] No such file or directory: 'python'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 162, in ensure_running
pid = spawnv_passfds(exe, args, fds_to_pass)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 370, in spawnv_passfds
os.close(errpipe_read)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/scalene/scalene_profiler.py", line 1326, in profile_code
exec(code, the_globals, the_locals)
File "scalene_test.py", line 12, in <module>
main()
File "scalene_test.py", line 9, in main
Parallel(n_jobs=2)(delayed(func)(i**2) for i in range(2))
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/parallel.py", line 968, in __call__
n_jobs = self._initialize_backend()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/parallel.py", line 735, in _initialize_backend
n_jobs = self._backend.configure(n_jobs=self.n_jobs, parallel=self,
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 494, in configure
self._workers = get_memmapping_executor(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/executor.py", line 20, in get_memmapping_executor
return MemmappingExecutor.get_memmapping_executor(n_jobs, **kwargs)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/executor.py", line 52, in get_memmapping_executor
_executor, executor_is_reused = super().get_reusable_executor(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 142, in get_reusable_executor
_executor = executor = cls(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 98, in __init__
super(_ReusablePoolExecutor, self).__init__(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1024, in __init__
self._setup_queues(job_reducers, result_reducers)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 231, in _setup_queues
super(_ReusablePoolExecutor, self)._setup_queues(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1034, in _setup_queues
self._call_queue = _SafeQueue(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 302, in __init__
super(_SafeQueue, self).__init__(max_size, reducers=reducers, ctx=ctx)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/queues.py", line 37, in __init__
super().__init__(maxsize=maxsize, ctx=ctx)
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/queues.py", line 48, in __init__
self._sem = ctx.BoundedSemaphore(maxsize)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/context.py", line 322, in BoundedSemaphore
return BoundedSemaphore(value)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, value, value)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/synchronize.py", line 96, in __init__
resource_tracker.register(self._semlock.name, "semlock")
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 190, in register
self.ensure_running()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 168, in ensure_running
os.close(w)
OSError: [Errno 9] Bad file descriptor
Scalene: could not find input file scalene_test.py
/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/_memmapping_reducer.py:607: UserWarning: Failed to delete temporary folder: /dev/shm/joblib_memmapping_folder_4522_c1440b6b43ec48deac3181cceea5e4ba_a8dbe3816c7847f981370e49be6541ee
warnings.warn("Failed to delete temporary folder: {}"
Error in atexit._run_exitfuncs:
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
Error in program being profiled:
[Errno 9] Bad file descriptor
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 368, in spawnv_passfds
return fork_exec(args, _pass)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/fork_exec.py", line 46, in fork_exec
os.execve(sys.executable, cmd, child_env)
FileNotFoundError: [Errno 2] No such file or directory: 'python'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 162, in ensure_running
pid = spawnv_passfds(exe, args, fds_to_pass)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 370, in spawnv_passfds
os.close(errpipe_read)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/scalene/scalene_profiler.py", line 1326, in profile_code
exec(code, the_globals, the_locals)
File "scalene_test.py", line 12, in <module>
main()
File "scalene_test.py", line 9, in main
Parallel(n_jobs=2)(delayed(func)(i**2) for i in range(2))
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/parallel.py", line 968, in __call__
n_jobs = self._initialize_backend()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/parallel.py", line 735, in _initialize_backend
n_jobs = self._backend.configure(n_jobs=self.n_jobs, parallel=self,
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 494, in configure
self._workers = get_memmapping_executor(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/executor.py", line 20, in get_memmapping_executor
return MemmappingExecutor.get_memmapping_executor(n_jobs, **kwargs)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/executor.py", line 52, in get_memmapping_executor
_executor, executor_is_reused = super().get_reusable_executor(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 142, in get_reusable_executor
_executor = executor = cls(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 98, in __init__
super(_ReusablePoolExecutor, self).__init__(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1024, in __init__
self._setup_queues(job_reducers, result_reducers)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/reusable_executor.py", line 231, in _setup_queues
super(_ReusablePoolExecutor, self)._setup_queues(
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 1044, in _setup_queues
self._result_queue = SimpleQueue(reducers=result_reducers,
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/queues.py", line 199, in __init__
super().__init__(ctx=ctx)
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/queues.py", line 336, in __init__
self._rlock = ctx.Lock()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/context.py", line 327, in Lock
return Lock()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/synchronize.py", line 180, in __init__
super(Lock, self).__init__(SEMAPHORE, 1, 1)
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/synchronize.py", line 96, in __init__
resource_tracker.register(self._semlock.name, "semlock")
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 190, in register
self.ensure_running()
File "/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/externals/loky/backend/resource_tracker.py", line 168, in ensure_running
os.close(w)
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Traceback (most recent call last):
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 132, in __del__
self._close()
File "/anaconda/envs/pymmm/lib/python3.8/multiprocessing/connection.py", line 361, in _close
_close(self._handle)
OSError: [Errno 9] Bad file descriptor
Scalene: Program did not run for long enough to profile.
Scalene: could not find input file scalene_test.py
/anaconda/envs/pymmm/lib/python3.8/site-packages/joblib/_memmapping_reducer.py:607: UserWarning: Failed to delete temporary folder: /dev/shm/joblib_memmapping_folder_4522_c1440b6b43ec48deac3181cceea5e4ba_a8dbe3816c7847f981370e49be6541ee
warnings.warn("Failed to delete temporary folder: {}"
Error in atexit._run_exitfuncs:
OSError: [Errno 9] Bad file descriptor
Exception ignored in: <function _ConnectionBase.__del__ at 0x7f3488062050>
Exception ignored in: <function _ConnectionBase.__del__ at 0x7fSource: plasma-umass/scalene