Baike.dev
Log in
Back to tool/Back to issues
#4936·antlr4

[Java] Data races in ANTLR runtime

Author: hcantuncCreated May 11, 2026Updated Aug 19, 2026

Using the Java runtime, ThreadSanitizer flags many data races in the ANTLR runtime when each thread creates its own lexer + parser from the same generated grammar and parses concurrently.

Environment

  • ANTLR 4.13.2
  • JDK: OpenJDK 21 run with -XX:+ThreadSanitizer
  • Linux x86_64

Reproduce

Filter.g4:

grammar Filter;
prog       : clause EOF ;
clause     : or_clause ;
or_clause  : and_clause (OR and_clause)* ;
and_clause : atom (AND atom)* ;
atom       : ID OP value | LPAREN clause RPAREN ;
value      : ID | INT | STRING ;
AND : 'AND' ; OR : 'OR' ;
OP  : '=' | '!=' | '<' | '>' | '<=' | '>=' ;
LPAREN : '(' ; RPAREN : ')' ;
ID : [a-zA-Z_][a-zA-Z_0-9]* ;
INT : [0-9]+ ;
STRING : '"' (~["\r\n])* '"' ;
WS : [ \t\r\n]+ -> skip ;

RaceMain.java:

import org.antlr.v4.runtime.*;
public class RaceMain {
    private static final String INPUT = "a = 1 AND b > 2 OR (c = 3 AND d != \"x\")";
    public static void main(String[] args) throws Exception {
        Thread[] ts = new Thread[8];
        for (int i = 0; i < ts.length; i++) {
            ts[i] = new Thread(() -> {
                FilterLexer lexer = new FilterLexer(CharStreams.fromString(INPUT));
                FilterParser parser = new FilterParser(new CommonTokenStream(lexer));
                parser.prog();
            });
            ts[i].start();
        }
        for (Thread t : ts) t.join();
    }
}

Run:

java -jar antlr-4.13.2-complete.jar -no-listener -no-visitor -o gen Filter.g4
javac -d build -cp antlr4-runtime-4.13.2.jar gen/*.java RaceMain.java
java -XX:+ThreadSanitizer -cp build:antlr4-runtime-4.13.2.jar RaceMain

Example race

Many races are reported. One example: a write in addDFAEdge racing against a read in getExistingTargetState on the same DFAState.edges slot.

WARNING: ThreadSanitizer: data race (pid=1900922)
   Write of size 4 at 0x00021fa3f124 by thread T20 (mutexes: write M0):
     #0 org.antlr.v4.runtime.atn.LexerATNSimulator.addDFAEdge(...)V                LexerATNSimulator.java:656
     #1 org.antlr.v4.runtime.atn.LexerATNSimulator.computeTargetState(...)         LexerATNSimulator.java:283
     #2 org.antlr.v4.runtime.atn.LexerATNSimulator.execATN(...)I                   LexerATNSimulator.java:204
     #3 org.antlr.v4.runtime.atn.LexerATNSimulator.matchATN(...)I                  LexerATNSimulator.java:156
     #4 org.antlr.v4.runtime.atn.LexerATNSimulator.match(...)I                     LexerATNSimulator.java:111
     #5 org.antlr.v4.runtime.Lexer.nextToken()                                     Lexer.java:141
     #6 org.antlr.v4.runtime.BufferedTokenStream.fetch(I)I                         BufferedTokenStream.java:169
     #7 org.antlr.v4.runtime.BufferedTokenStream.sync(I)Z                          BufferedTokenStream.java:152
     #8 org.antlr.v4.runtime.BufferedTokenStream.setup()V                          BufferedTokenStream.java:254
     #9 org.antlr.v4.runtime.BufferedTokenStream.lazyInit()V                       BufferedTokenStream.java:249
     #10 org.antlr.v4.runtime.CommonTokenStream.LT(I)                              CommonTokenStream.java:92
     #11 org.antlr.v4.runtime.Parser.enterRule(...)V                               Parser.java:628
     #12 FilterParser.prog()                                                       FilterParser.java:106
     #13 RaceMain.lambda$main$0()V                                                 RaceMain.java:12
     #14 java.lang.Thread.runWith(...)                                             Thread.java:1596
     #15 java.lang.Thread.run()V                                                   Thread.java:1583

   Previous read of size 4 at 0x00021fa3f124 by thread T18:
     #0 org.antlr.v4.runtime.atn.LexerATNSimulator.getExistingTargetState(...)     LexerATNSimulator.java:246
     #1 org.antlr.v4.runtime.atn.LexerATNSimulator.execATN(...)I                   LexerATNSimulator.java:202
     #2 org.antlr.v4.runtime.atn.LexerATNSimulator.matchATN(...)I                  LexerATNSimulator.java:156
     #3 org.antlr.v4.runtime.atn.LexerATNSimulator.match(...)I                     LexerATNSimulator.java:111
     #4 org.antlr.v4.runtime.Lexer.nextToken()                                     Lexer.java:141
     #5 org.antlr.v4.runtime.BufferedTokenStream.fetch(I)I                         BufferedTokenStream.java:169
     #6 org.antlr.v4.runtime.BufferedTokenStream.sync(I)Z                          BufferedTokenStream.java:152
     #7 org.antlr.v4.runtime.BufferedTokenStream.setup()V                          BufferedTokenStream.java:254
     #8 org.antlr.v4.runtime.BufferedTokenStream.lazyInit()V                       BufferedTokenStream.java:249
     #9 org.antlr.v4.runtime.CommonTokenStream.LT(I)                               CommonTokenStream.java:92
     #10 org.antlr.v4.runtime.Parser.enterRule(...)V                               Parser.java:628
     #11 FilterParser.prog()                                                       FilterParser.java:106
     #12 RaceMain.lambda$main$0()V                                                 RaceMain.java:12
     #13 java.lang.Thread.runWith(...)                                             Thread.java:1596
     #14 java.lang.Thread.run()V                                                   Thread.java:1583

Many similar reports come from addDFAState, computeTargetState, Interval.of, IntervalSet, and DefaultErrorStrategy.sync. Full ThreadSanitizer log attached.

tsan.log

Question

Is using a separate lexer + parser instance per thread supposed to be thread-safe?

Source: antlr/antlr4

View original on GitHubView discussion on GitHub