[C++] Option to write generated header files to a separate directory

Hi all,

When generating C++ code, ANTLR writes the headers and the .cpp files into the same directory, whatever -o points at. I'd like a way to send the headers somewhere else.

This came up for me while packaging a grammar as a C++ library. The usual layout is a public include/ directory for consumers and a private src/ for the implementation. Today the only way to get there is to put the whole generated directory on the include path, .cpp files and all, or to add a build step that moves the headers after ANTLR runs. Neither is terrible, but both feel like working around the tool.

The CMake helpers in this repo run into the same thing. ANTLR_TARGET in FindANTLR.cmake takes a single OUTPUT_DIRECTORY, and its README suggests include_directories on that same folder. antlr4_generate in antlr4-generator.cmake.in also uses its generated source directory as the include directory.

#2439 asked for this back in 2018 but never got a proposal or any discussion, so I thought it was worth picking up again with a more specific suggestion.

Reproducing

Tried with 4.13.2 and with current dev:

grammar T;
s : ID ;
ID : [a-z]+ ;
WS : [ \t\r\n]+ -> skip ;

antlr4 -Dlanguage=Cpp -o gen/src -visitor T.g4
ls gen/src

Everything ends up in gen/src: TLexer.h, TLexer.cpp, TParser.h, TParser.cpp, the listener and visitor pairs, plus the .tokens and .interp files. What I'm after is being able to put the .h files in, say, gen/include instead.

What I have in mind

A new option, -header-dir <dir>:

antlr4 -Dlanguage=Cpp -o gen/src -header-dir gen/include -visitor T.g4

Only the headers would move. The .cpp, .tokens, .interp and .dot files would stay under -o. The new directory would be resolved the same way -o is, so grammar subdirectories and -Xexact-output-dir behave as people already expect. If you don't pass it, nothing changes, and targets that don't generate headers would simply ignore it.

The generated #include lines would stay as they are. You'd add the header directory to your include path, as with any other C++ library.

To make it usable from build tools, -depend would report the new header paths, ANTLR_TARGET in FindANTLR.cmake would get an optional HEADER_OUTPUT_DIRECTORY argument, antlr4_generate an optional header directory argument, and the Maven plugin an optional headerOutputDirectory parameter. Leaving those out gives exactly the command line you get today.

Why not a -source-dir too, or changing -o?

I went back and forth on this. A matching -source-dir sounds tidy, but -o already controls every file for every target, and the Maven plugin, both CMake helpers and plenty of existing build scripts rely on it. A -source-dir would do the same job as -o for every target except C++, and we'd need rules for what wins when both are given. Keeping -o as the base directory and adding one optional option for headers avoids all of that, and there's nothing to deprecate. It also doesn't rule out adding more options later if someone needs them.

On the name, I stuck with a single dash like every other ANTLR option, and kept it short enough to line up with the rest of the -help output.

Where it would go in the code

CodeGenerator.writeRecognizer and its siblings already know whether they're writing a header, via SourceType. That information gets dropped in Target.genFile before Tool.getOutputFileWriter picks the directory. Passing it through, and only using a different directory when the option is set and the file is a header, keeps every other code path as it is.

Related: #2001, about inaccurate -depend output for C++, and #4102, the proposed CMake overhaul. I've kept the CMake change small so it shouldn't get in the way of that work.

I'm happy to put together a few PRs (tool, Maven plugin and a C++ runtime) if this sounds like a reasonable direction. Thanks for taking a look!