UmbrelOS 1.7.4 — umbreld filesystem watcher exhausts inotify resources

UmbrelOS 1.7.4 – umbreld filesystem watcher failures, Immich interaction, and unexpected password timestamp

Summary

I am experiencing intermittent system freezes on UmbrelOS 1.7.4.

During a freeze, basic network connectivity remains available, but the Umbrel management layer becomes effectively unusable:

• ICMP/ping continues to respond.
• TCP port 22 accepts connections, but SSH does not complete normally.
• HTTP connections may be accepted but do not return the Umbrel web interface.
• The system requires a physical/power-button reboot.
• After reboot, the system becomes operational again.

Persistent journal logs show repeated inotify_add_watch() failures with:

No space left on device

and repeated umbreld filesystem-watcher health-check failures.

Further investigation shows that umbreld recursively watches a very large Immich directory tree.

Environment

• UmbrelOS: 1.7.4
• Hardware: Shuttle DS10U
• CPU: Intel Core i5-8265U
• RAM: 16 GB
• Storage: approximately 240 GB SSD
• Immich installed on Umbrel
• Other applications include Bitcoin Knots, Jellyfin, Portainer, qBittorrent, Fulcrum, Stalwart and DATUM

Freeze behavior

During the affected state:

• The machine remains reachable by ping.
• TCP port 22 accepts a connection.
• SSH can establish the TCP connection but hangs before normal authentication/session startup.
• HTTP connections can be accepted but the web interface does not respond.
• Port 443 may refuse connections.
• A physical reboot is required to restore normal operation.

After reboot, SSH and the Umbrel web interface operate normally again.

Relevant umbreld log errors

The persistent journal contains errors including:

inotify_add_watch ... No space left on device

Examples include failures to watch:

/Home

/Trash

/Apps

There are also repeated umbreld watcher health-check errors such as:

Health check failed: watcher did not deliver sentinel event within timeout. Recovering...

and:

Failed to watch directory ...

This appears to indicate that the umbreld filesystem watcher is failing to create or maintain filesystem watches.

Disk space and inode investigation

The filesystem itself is not out of space.

At the time of investigation:

• Approximately 29 GB of disk space was available.
• Filesystem inode usage was approximately 3%.

df -i / reported:

• Inodes: 14,237,696
• Used: 410,201
• Free: 13,827,495
• Usage: 3%

Therefore normal filesystem inode exhaustion does not explain the ENOSPC errors.

Inotify configuration

The configured inotify limits are:

fs.inotify.max_user_watches = 122404

fs.inotify.max_user_instances = 256

A system-wide inspection showed approximately 58 open inotify file descriptors.

Therefore the max_user_instances limit also does not appear to be exhausted.

umbreld watcher usage

The main process involved is:

umbreld PID 10860

It is a Node.js process running Umbrel's management layer.

Inspection of its /proc/10860/fdinfo/* entries showed four file descriptors containing the same large inotify watch set.

The watch list contains approximately 21,199 entries.

The multiple file descriptors appear to reference the same watcher state, so simply summing all four file descriptors would incorrectly count the same watches multiple times.

Immich correlation

The most significant finding is that almost the entire umbreld watch tree corresponds to the Immich application data directory.

Immich contains:

Approximately 20,742 directories under:

/home/umbrel/umbrel/app-data/immich/data

The major directory counts are:

• upload/upload: 10,316 directories
• upload/thumbs: 5,990 directories
• upload/encoded-video: 4,372 directories
• profile: 1
• library: 1
• backups: 1

Total:

Approximately 20,681 directories in these major branches, with the remaining directories accounting for the total of approximately 20,742.

The number of Immich directories (~20,742) is remarkably close to the number of umbreld watches (~21,199).

This strongly suggests that umbreld is recursively watching essentially the entire Immich directory tree.

Interpretation

I am not claiming that Immich itself is defective.

Rather, the evidence suggests a possible interaction between:

1. Umbrel's umbreld recursive filesystem watcher;
2. large application directory trees;
3. Immich's storage structure containing tens of thousands of directories;
4. Linux inotify resource handling.

The configured max_user_watches limit is not exhausted, so the exact reason for:

inotify_add_watch() -> ENOSPC

remains unclear.

Possible causes include:

• an umbreld filesystem-watcher bug;
• duplicated or leaked watches;
• an interaction between recursive application-data monitoring and large directory trees;
• another kernel/inotify resource constraint;
• a watcher recovery/retry loop;
• failure to correctly release or manage watches after filesystem changes.

Why this may be related to the system freeze

The journal shows umbreld repeatedly failing its filesystem watcher health checks.

At the same time, the machine can reach a state where:

• networking still works;
• TCP connections can be established;
• SSH does not complete normally;
• the Umbrel web UI stops responding;
• a reboot is required.

This suggests that failure of the Umbrel management/file-watching subsystem may contribute to the overall system instability.

However, the exact causal relationship between the watcher failures and the complete system freeze has not yet been proven.

Additional security/account observation

There is a separate unexpected observation concerning the Linux umbrel account password timestamp.

After an affected reboot, the following command was used:

passwd -S umbrel

It reported a password last-change date of:

2026-07-10

This was unexpected because the password was believed to have been configured later.

The password was subsequently changed through the Umbrel web interface on:

2026-09-15

After that change, passwd -S umbrel reported:

2026-09-15

Security interpretation

The earlier 2026-07-10 timestamp does not by itself prove unauthorized access or that someone changed the password.

It is nevertheless unexpected and should be explained.

Please clarify whether UmbrelOS installation, provisioning, initialization, upgrades, recovery mechanisms, or other system processes can modify the Linux umbrel account password or its password-last-change timestamp.

Requested investigation

Please investigate the following:

1. umbreld filesystem watcher

• Why does umbreld recursively watch the complete application data tree?
• Is this intended behavior?
• Should large application directories such as Immich be excluded from recursive watching?

2. Inotify ENOSPC

Why does:

inotify_add_watch()

return:

ENOSPC

when:

• max_user_watches = 122404
• filesystem inode usage is only ~3%
• max_user_instances = 256
• only ~58 inotify file descriptors are observed?

3. Watch lifecycle

Please check whether umbreld:

• leaks watches;
• duplicates watches;
• fails to release watches;
• creates multiple references to the same watcher;
• repeatedly retries failed watches;
• accumulates watches after filesystem changes.

4. Large applications

Please test UmbrelOS with applications containing large directory trees, particularly Immich.

A test installation containing approximately 20,000 directories appears sufficient to expose the problem.

5. Failure handling

If a filesystem watcher cannot create a watch, Umbrel should ideally degrade gracefully rather than allowing the management layer to become unstable or unresponsive.

6. Password timestamp

Please clarify whether UmbrelOS 1.7.4 installation/provisioning/recovery can cause the Linux umbrel password last-change timestamp to become the OS installation/release date.

Privacy

This report intentionally excludes:

• public IP addresses;
• private IP addresses;
• hostnames;
• email addresses;
• passwords;
• SSH keys;
• personal files;
• personal photographs;
• other personally identifying information.

The directory names and UUIDs shown in the investigation have also been omitted where they are not necessary to reproduce the issue.

Conclusion

The strongest current finding is that umbreld is recursively monitoring approximately 21,000 filesystem directories, with approximately 20,700 of those directories belonging to Immich.

UmbrelOS subsequently reports repeated inotify_add_watch() failures with ENOSPC and filesystem-watcher health-check failures.

The system can then enter a state where the Umbrel web interface and SSH become unusable until reboot.

The configured inotify watch limit is not exhausted, so this appears to warrant investigation as a potential umbreld filesystem-watcher/resource-management bug rather than simply an insufficient max_user_watches configuration.