automake < 1.16 doesn't properly generate Makefiles for tests

automake < 1.16 does not handle variable expansion in _SOURCES properly causing make test to fail. I successfully built curl from master in a ten year old Ubuntu 16 LTS VM that has automake 1.15, but make test was failing due to bad variable expansion. I think earlier this year it worked, I'm not sure, so the change was probably introduced recently.

git clean -fdx
autoreconf -fi
./configure ...
make
make test   # this step fails

make[2]: *** No rule to make target '.deps/perf.Po'. Stop.

I checked the .deps directory and found a $(BUNDLE).Po file (actual name) and it was also in other test directories:

owner@ubuntu1604-x64-vm:~/curl$ find -name '$(BUNDLE).Po'
./tests/tunit/.deps/$(BUNDLE).Po
./tests/libtest/.deps/$(BUNDLE).Po
./tests/server/.deps/$(BUNDLE).Po
./tests/unit/.deps/$(BUNDLE).Po
./tests/perf/.deps/$(BUNDLE).Po

This appears to be because of changes to curl to make some test directories do a unity build by default. They now have a BUNDLE name that is specified in Makefile.inc which is the name of the unity file where all the c files and header files etc are #included into one bundle file. That variable name is then used in Makefile.am like nodist_[dirname]_SOURCES = $(BUNDLE).c

https://github.com/curl/curl/blob/01346829096c61b372692f6dc43ffa778c6caccd/tests/perf/Makefile.am#L68-L71

automake 1.15 does not handle this properly specifically in _SOURCES, causing the depfiles to be incorrectly generated as a literal $(BUNDLE).Po which is then later correctly expanded by make as the actual name (eg perf.Po as seen in the output) which does not exist. I can fix it by changing the SOURCES to use the actual name instead of a variable:

diff --git a/tests/perf/Makefile.am b/tests/perf/Makefile.am
index 783cee1..07c6609 100644
--- a/tests/perf/Makefile.am
+++ b/tests/perf/Makefile.am
@@ -62,7 +62,7 @@ $(BUNDLE).c: $(top_srcdir)/scripts/mk-unity.pl Makefile.inc $(FIRST_C) $(UTILS_C
        @PERL@ $(top_srcdir)/scripts/mk-unity.pl --include $(UTILS_C) $(curlx_c_lib) $(TOOLX_C) --test $(TESTS_C) > $(BUNDLE).c
 
 noinst_PROGRAMS = $(BUNDLE)
-nodist_perf_SOURCES = $(BUNDLE).c
+nodist_perf_SOURCES = perf.c
 LDADD = $(top_builddir)/lib/libcurl.la @LIBCURL_PC_LIBS_PRIVATE@
 CLEANFILES = $(BUNDLE).c

I made several attempts at this to preserve $(BUNDLE) using the methods described here, but none of them worked. I asked ChatGPT specifically why the literal name worked when my attempts to preserve $(BUNDLE) didn't, and it said that the behavior is not a bug in curl but in automake handling prior to 1.16, likely fixed by this.

I confirmed that by trying automake-1.16 and it worked.

export PATH="$HOME/opt/automake-1.16/bin:$PATH"
export ACLOCAL_PATH="/usr/share/aclocal:${ACLOCAL_PATH:-}"
git clean -fdx
autoreconf -fi
./configure ...   # depfiles generated properly (server.Po, perf.Po, etc)
make
make test   # ok

It looks like if we want to continue supporting automake < 1.16 we would have to stop using the $(BUNDLE) variable in the SOURCES.