RTX 3060 fork — 27 experiments, 25% improvement (1.604 → 1.201 val_bpb)

Hi! I ran autoresearch on an RTX 3060 12GB and documented 27 experiments.

Repo: https://github.com/zzoccultzz/autoresearch

Key findings:

• Step count is the dominant factor: TOTAL_BATCH_SIZE 2^19 → 2^14 was the biggest win
• DEPTH=6 over DEPTH=8: halving depth doubles steps in 5-min budget, net improvement
• Global attention (L in WINDOW_PATTERN=SSSL) is necessary — SSSS hurts
• SCALAR_LR 0.3 better than default 0.5 on this platform
• ADAM_BETAS beta2=0.999 (AdamW default) slightly better than 0.95

Best val_bpb: 1.201005 (baseline 1.604912) Full experiment log: results.tsv in the repo

Would love to be added to Notable Forks if appropriate!