Bug: Crawls fails to resume and stop
Author: v4u6h4nCreated Jun 24, 2026Updated Jun 24, 2026
Provide a screenshot and describe the bug
Sorry for all the bug reports! :-)
I'm not sure if this is a bug, or just hardware? I'm running a pretty beefy system though, so assumed it was a bug. In logs attached I tried to resume a paused crawl, and then to pause a running crawl, and both seem to have failed. Both the UI and CLI reported the crawls to be paused. The log is truncated because it was quite log so I just added some of the end of the log.
Steps to reproduce
1. Pause or resume a crawl.Logs or errors
daphne.cli Starting server at tcp:port=8000:interface=0.0.0.0
daphne.server HTTP/2 support not enabled (install the http2 and tls Twisted extras)
daphne.server Configuring endpoint tcp:port=8000:interface=0.0.0.0
daphne.server Listening on TCP address 0.0.0.0:8000
> /venv/bin/archivebox version; TS=2026-06-23__05:57:29 VERSION=0.9.35rc88 IN_DOCKER=True IS_TTY=True
archivebox.workers.models Crawl.save() outside runner process: id=06a3a20c4c8e76888000379dc82ad512 status=queued retry_at=2026-06-23 05:59:32.784723+00:00 process=cli root=server caller=/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/concurrent/futures/thread.py:59; queue/status writes outside the runner should usually use safe_update()
> /venv/lib/python3.13/site-packages/archivebox/__main__.py run --daemon; TS=2026-06-23__05:59:38 VERSION=0.9.35rc88 IN_DOCKER=True IS_TTY=False
django.request Internal Server Error: /admin/personas/persona/add/
Traceback (most recent call last):
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 254, in _get_response_async
response = await wrapped_callback(
^^^^^^^^^^^^^^^^^^^^^^^
request, *callback_args, **callback_kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 506, in __call__
ret = await asyncio.shield(exec_coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", line 40, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 561, in thread_handler
return func(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 711, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/views/decorators/cache.py", line 80, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/sites.py", line 247, in inner
return view(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1943, in add_view
return self.changeform_view(request, None, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 48, in _wrapper
return bound_method(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1799, in changeform_view
return self._changeform_view(request, object_id, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1869, in _changeform_view
form = ModelForm(initial=initial)
File "/venv/lib/python3.13/site-packages/archivebox/personas/forms.py", line 107, in __init__
self.discovered_profiles = discover_local_browser_profiles()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 370, in discover_local_browser_profiles
discovered.extend(discover_persona_template_profiles())
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 406, in discover_persona_template_profiles
for profile_dir in _list_profile_names(user_data_dir):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 723, in _list_profile_names
if _looks_like_profile_dir(child):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in _looks_like_profile_dir
if any(marker.exists() for marker in marker_paths):
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in <genexpr>
if any(marker.exists() for marker in marker_paths):
~~~~~~~~~~~~~^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_abc.py", line 450, in exists
self.stat(follow_symlinks=follow_symlinks)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_local.py", line 515, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/data/personas/Default/chrome_profile/ActorSafetyLists/Preferences'
django.request Internal Server Error: /admin/personas/persona/06a38dda557b7a4b8000de3461080589/change/
Traceback (most recent call last):
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 254, in _get_response_async
response = await wrapped_callback(
^^^^^^^^^^^^^^^^^^^^^^^
request, *callback_args, **callback_kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 506, in __call__
ret = await asyncio.shield(exec_coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", line 40, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 561, in thread_handler
return func(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 711, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/views/decorators/cache.py", line 80, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/sites.py", line 247, in inner
return view(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django_object_actions/utils.py", line 100, in change_view
return super().change_view(request, object_id, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1946, in change_view
return self.changeform_view(request, object_id, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 48, in _wrapper
return bound_method(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1799, in changeform_view
return self._changeform_view(request, object_id, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1874, in _changeform_view
form = ModelForm(instance=obj)
File "/venv/lib/python3.13/site-packages/archivebox/personas/forms.py", line 107, in __init__
self.discovered_profiles = discover_local_browser_profiles()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 370, in discover_local_browser_profiles
discovered.extend(discover_persona_template_profiles())
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 406, in discover_persona_template_profiles
for profile_dir in _list_profile_names(user_data_dir):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 723, in _list_profile_names
if _looks_like_profile_dir(child):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in _looks_like_profile_dir
if any(marker.exists() for marker in marker_paths):
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in <genexpr>
if any(marker.exists() for marker in marker_paths):
~~~~~~~~~~~~~^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_abc.py", line 450, in exists
self.stat(follow_symlinks=follow_symlinks)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_local.py", line 515, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/data/personas/Default/chrome_profile/ActorSafetyLists/Preferences'
archivebox.workers.models Crawl.save() outside runner process: id=06a3a275df4e736180005f2cc4973041 status=queued retry_at=2026-06-23 06:27:41.956562+00:00 process=cli root=server caller=/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/concurrent/futures/thread.py:59; queue/status writes outside the runner should usually use safe_update()
> /venv/lib/python3.13/site-packages/archivebox/__main__.py run --daemon; TS=2026-06-23__06:27:51 VERSION=0.9.35rc88 IN_DOCKER=True IS_TTY=False
archivebox.workers.models Crawl.save() outside runner process: id=06a3a27c98397d4c80008005319c7129 status=queued retry_at=2026-06-23 06:29:29.514078+00:00 process=cli root=server caller=/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/concurrent/futures/thread.py:59; queue/status writes outside the runner should usually use safe_update()
> /venv/lib/python3.13/site-packages/archivebox/__main__.py run --daemon; TS=2026-06-23__06:29:35 VERSION=0.9.35rc88 IN_DOCKER=True IS_TTY=False
archivebox.workers.models Crawl.save() outside runner process: id=06a3a2a86cb97a7280008847f6830ca6 status=queued retry_at=2026-06-23 06:41:10.795298+00:00 process=cli root=server caller=/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/concurrent/futures/thread.py:59; queue/status writes outside the runner should usually use safe_update()
> /venv/lib/python3.13/site-packages/archivebox/__main__.py run --daemon; TS=2026-06-23__06:41:15 VERSION=0.9.35rc88 IN_DOCKER=True IS_TTY=False
django.request Internal Server Error: /admin/personas/persona/add/
Traceback (most recent call last):
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 254, in _get_response_async
response = await wrapped_callback(
^^^^^^^^^^^^^^^^^^^^^^^
request, *callback_args, **callback_kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 506, in __call__
ret = await asyncio.shield(exec_coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", line 40, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 561, in thread_handler
return func(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 711, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/views/decorators/cache.py", line 80, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/sites.py", line 247, in inner
return view(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1943, in add_view
return self.changeform_view(request, None, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 48, in _wrapper
return bound_method(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1799, in changeform_view
return self._changeform_view(request, object_id, form_url, extra_context)
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 1869, in _changeform_view
form = ModelForm(initial=initial)
File "/venv/lib/python3.13/site-packages/archivebox/personas/forms.py", line 107, in __init__
self.discovered_profiles = discover_local_browser_profiles()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 370, in discover_local_browser_profiles
discovered.extend(discover_persona_template_profiles())
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 406, in discover_persona_template_profiles
for profile_dir in _list_profile_names(user_data_dir):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 723, in _list_profile_names
if _looks_like_profile_dir(child):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in _looks_like_profile_dir
if any(marker.exists() for marker in marker_paths):
~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/archivebox/personas/importers.py", line 741, in <genexpr>
if any(marker.exists() for marker in marker_paths):
~~~~~~~~~~~~~^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_abc.py", line 450, in exists
self.stat(follow_symlinks=follow_symlinks)
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/uv/python/cpython-3.13.14-linux-x86_64-gnu/lib/python3.13/pathlib/_local.py", line 515, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/data/personas/Default/chrome_profile/ActorSafetyLists/Preferences'
django.request Internal Server Error: /admin/personas/persona/06a38dda557b7a4b8000de3461080589/change/
Traceback (most recent call last):
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 557, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.13/site-packages/django/core/handlers/base.py", line 254, in _get_response_async
response = await wrapped_callback(
^^^^^^^^^^^^^^^^^^^^^^^
request, *callback_args, **callback_kwargs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 506, in __call__
ret = await asyncio.shield(exec_coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/asgiref/current_thread_executor.py", line 40, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.13/site-packages/asgiref/sync.py", line 561, in thread_handler
return func(*args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/contrib/admin/options.py", line 711, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 192, in _view_wrapper
result = _process_exception(request, e)
File "/venv/lib/python3.13/site-packages/django/utils/decorators.py", line 190, in _view_wrapper
response = view_func(request, *args, **kwargs)
File "/venv/lib/python3.13/site-packages/django/views/decSource: ArchiveBox/ArchiveBox