#7504·juicefs

Hadoop SDK: positioned reads on one stream are serialized

Author: mwkangCreated Sep 3, 2026Updated Sep 3, 2026
Labelskind/bug

What happened:

read(long pos, byte[] b, int off, int len) on a JuiceFS Hadoop stream runs one call at a time per stream. Two locks cause it:

  1. Java: both positioned read methods of FileInputStream are synchronized (sdk/java/src/main/java/io/juicefs/JuiceFileSystemImpl.java:1132). They only read fd and path; the buffered-read state (buf, position) is untouched, so the lock protects nothing but the close() ordering.
  2. Go: File.Pread holds the fs.File mutex across rdata.Read, i.e. across the object storage wait (pkg/fs/fs.go:1334-1343). vfs.FileReader.Read is already safe for concurrent callers and the FUSE path calls it concurrently for one handle (pkg/vfs/vfs.go:783-791).

Random preads do not trigger readahead, so an application that shares one FSDataInputStream across threads (HBase does this per HFile) gets at most one in-flight object storage request per open file when the block cache misses.

What you expected to happen:

Positioned reads on one stream overlap, as PositionedReadable intends and as the FUSE path already does. HDFS DFSInputStream preads are lock-free.

How to reproduce it (as minimally and precisely as possible):

Go test in pkg/fs, no external services:

  1. Build a FileSystem on an object.ObjectStorage wrapper whose Get counts in-flight calls and blocks until 4 have arrived or 2 s pass. CacheSize: 0, Readahead: 0.
  2. Write a 6-block file, reopen with vfs.MODE_MASK_R, call f.Pread from 4 goroutines at 4 different block offsets (not block 0, not the last 32 KiB).
  3. Expected max in-flight Get: 4. Actual: 1.

Java: hold synchronized (in.getWrappedStream()) in one thread and call in.read(pos, buf, 0, len) from another. The read blocks until the monitor is released.

Anything else we need to know?

Environment:

  • JuiceFS version (use juicefs --version) or Hadoop Java SDK version:
  • Cloud provider or hardware configuration running JuiceFS:
  • OS (e.g cat /etc/os-release):
  • Kernel (e.g. uname -a):
  • Object storage (cloud provider and region, or self maintained):
  • Metadata engine info (version, cloud provider managed or self maintained):
  • Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
  • Others: