Standard library tests that move large amounts of data are very slow

Correction to the first version of this issue

This issue first said that the problem was specific to macOS, because "the same packages are much faster on Linux". That was wrong. The packages are slow on every platform. The macos-15-intel runner only made the problem easier to see, because it is about 3 times slower than the other runners.

Measurement

Run time of each package in make tinygo-test, from macOS run 34445433537 and Linux run 34446941721:

package	macos-15-intel	macos-14	linux
archive/zip	808s	174s	245s
index/suffixarray	251s	60s	87s
encoding/xml	111s	115s	107s

The cost is in a small number of tests

Measured on Linux with a local build of dev. archive/zip takes 70.87s in total:

test	time
TestZip64LargeDirectory	35.14s
TestZip64WriterCDGoldens	28.24s
the other 52 tests together	~7.5s

Two tests give 89% of the time. Both write more than 4GB through a writer.

Comparison with Go

The same test sources, on the same machine, with go1.27.0 linux/amd64:

package	Go	TinyGo	ratio
archive/zip	13.60s	70.87s	5.2
encoding/xml	3.85s	50.45s	13.1
index/suffixarray	29.89s	116.78s	3.9

The two slow archive/zip tests give a much larger difference than the package as a whole:

test	Go	TinyGo
TestZip64LargeDirectory	0.00s	35.14s
TestZip64WriterCDGoldens	0.00s	28.24s

Go reports these two as 0.00s. TinyGo takes 63s for the same code. A difference of that size is not the usual constant factor between the two compilers, thus something in these paths is probably much worse than linear. It is worth a profile.

index/suffixarray shows the shape of the problem more clearly

With -short, index/suffixarray omits no test at all. Its tests only make their input smaller. The time still goes from 116.78s to 0.829s, which is 141 times faster.

Thus the cost is proportional to the quantity of data, not to the number of tests. This is a throughput problem, not a problem of one package.

What -short gives

package	full	-short	tests, full	-short passed	-short skipped
archive/zip	70.87s	0.110s	54	47	7
encoding/xml	50.45s	0.070s	83	81	2
index/suffixarray	116.78s	0.829s	6	6	0

#5663 runs these three packages with -short. It omits 9 tests of 143 and removes almost all of the time.

That pull request hides the cost, it does not correct it. Keep this issue open. The omitted tests are the ones that move the most data, thus they are the ones that show the problem best.

What to look at

The slow tests write and copy large byte slices. Look at the throughput of the copy, append, and io.Writer paths, and at the garbage collector, because these tests allocate a lot.