[Bug]: Manual file-manager delete of a permanently failing secondary-storage file retries forever with no backoff (unlike the retention sweep fixed in #15608)
What happened?
When a file's secondary storage source (e.g. a Code Interpreter / execute_code backing session) refuses to delete an object with something other than a 404/ENOENT-shaped error, isMissingStorageError does not forgive it, so processDeleteRequest (api/server/services/Files/process.js) leaves the file's DB record in place and reports the delete as failed. That's correct for a case that might be a real permission problem — but there is then no way for the user to ever successfully delete that file again: every subsequent click of "Delete" in the file manager (DELETE /api/files → api/server/routes/files/files.js → processDeleteRequest directly) reissues the exact same call, gets the exact same permanent failure, and nothing is ever recorded against the file to back off or give up.
This is the same underlying failure mode reported in #15511 and fixed for the automated retention sweep in #15608 (deletionRetryAt/deletionAttempts/FILE_RETENTION_SWEEP_MAX_ATTEMPTS, in packages/api/src/files/sweep.ts). That fix is scoped entirely to the sweep's own wrapper around processDeleteRequest (deferExpiredFile) — it does not touch the manual/file-manager delete route at all, which calls processDeleteRequest directly with no backoff, no attempt cap, and no path to ever stop retrying. A file a user is actively trying to delete (not yet retention-expired) that hits this condition is stuck forever, with the client simply retrying indefinitely in the background.
Confirmed via reading current main/rc3 source: deletionRetryAt/deletionAttempts bookkeeping exists only in packages/api/src/files/sweep.ts; api/server/routes/files/files.js's call to processDeleteRequest (line ~277) has no equivalent.
Version Information
- LibreChat version: v0.8.8-rc3 (also confirmed present, unchanged, on upstream
devas of today — nothing touchingapi/server/routes/files/files.js,api/server/services/Files/process.js, orpackages/api/src/files/sweep.tshas landed since rc3) - OS: N/A (server-side logic)
- Node: N/A
- Browser: N/A (server-side logic; reproduces via any client)
Steps to Reproduce
- Have a file whose
metadata.codeEnvRef(or other secondary-storage reference) points at a Code Interpreter/execute_codesession (or any other secondary storage source) that will return a non-404/ENOENT error — e.g. 401/403/500 — for a delete call, for a reason that is permanent from the caller's point of view (expired/rotated credentials, a session predating some ownership-tracking feature, a backend-side policy change, etc). - As the file's owner, delete it from the file manager UI (not via the retention sweep — the file must not yet be retention-expired).
- Observe: the local copy deletes fine (or 404s harmlessly), but the secondary-storage delete fails with the non-404 error. The overall delete request fails, and the file's DB record remains.
- Repeat step 2 (or just wait for the client to retry automatically). Observe the exact same failure recurring indefinitely, with no backoff and no indication anywhere that this file can never be deleted this way.
What browsers are you seeing the problem on?
No response
Relevant log output
{"code":"ENOENT","errno":-2,"level":"error","message":"ENOENT: no such file or directory, unlink '/app/uploads/<userId>/<fileId><filename>'","name":"Error","path":"/app/uploads/<userId>/<fileId><filename>","requestId":"<redacted>","request_id":"<redacted>","request_method":"DELETE","request_path":"/api/files","stack":"Error: ENOENT: no such file or directory, unlink '/app/uploads/<userId>/<fileId>__<filename>'\n at async Object.unlink (node:internal/fs/promises:1070:10)\n at async unlinkFile (/app/api/server/services/Files/Local/crud.js:212:5)\n at async deleteLocalFile (/app/api/server/services/Files/Local/crud.js:254:5)\n at async /app/api/server/services/Files/process.js:206:7\n at async Promise.allSettled (index 0)\n at async processDeleteRequest (/app/api/server/services/Files/process.js:326:3)\n at async /app/api/server/routes/files/files.js:277:22","syscall":"unlink","timestamp":"<redacted>","userId":"<redacted>"}
{"code":"ERR_BAD_REQUEST","isAxiosError":true,"level":"error","message":"Request failed with status code 403","name":"AxiosError","requestId":"<redacted>","requestInfo":{"method":"delete","url":"http://<code-interpreter-backend>/v1/files/<sessionId>/<fileId>"},"timestamp":"<redacted>"}
The above two log lines repeat identically, minutes to hours apart, for the same file, indefinitely — with no growing interval and no eventual cutoff.Screenshots
No response
Code of Conduct
- I agree to follow this project's Code of Conduct
Source: danny-avila/LibreChat