`fab-include` silently ignores a nested `pyproject.toml`, and rejects a nested `LICENSE`
Environment
flwr==1.37.0, Python 3.12, macOS 15. Re-verified against 1.37.0 on 17 September 2026.
What happens
An app that ships a second app bundle inside itself (ours starts it on SuperGrid for a model call) cannot package that bundle, and the two failure modes contradict each other:
sub/pyproject.tomllisted infab-include→ build succeeds, the file is missing from the FAB, nothing is reported. The failure only appears at runtime, inside the installed app.sub/LICENSElisted infab-include, file present and tracked → hard error:Pattern in "fab-include" did not match any files: "sub/LICENSE".
So one nested file that cannot be included is silently dropped, and another fails the build loudly.
Minimal reproduction
mkdir -p demo/demo demo/sub/pkg && cd demo
printf '' > demo/__init__.py
printf 'from flwr.serverapp import ServerApp\napp = ServerApp()\n' > demo/server.py
printf 'from flwr.clientapp import ClientApp\napp = ClientApp()\n' > demo/client.py
printf '' > sub/pkg/__init__.py
printf '[project]\nname = "sub"\nversion = "0.1.0"\n' > sub/pyproject.toml
echo "Apache License placeholder" > LICENSE
cat > pyproject.toml <<'TOML'
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "demo"
version = "0.1.0"
description = "FAB nesting repro"
license = { file = "LICENSE" }
requires-python = ">=3.11,<4.0"
dependencies = ["flwr>=1.35.0,<2.0"]
[tool.hatch.build.targets.wheel]
packages = ["demo"]
[tool.flwr.app]
publisher = "test"
fab-format-version = 1
flwr-version-target = "1.35.0"
fab-include = ["sub/**/*.py", "sub/pyproject.toml", "LICENSE"]
[tool.flwr.app.components]
serverapp = "demo.server:app"
clientapp = "demo.client:app"
TOML
flwr build
unzip -l *.fab | grep -E 'sub/|LICENSE|pyproject'Observed:
Successfully built test.demo.0-1-0.640bc7e4.fab
587 pyproject.toml
27 LICENSE
0 sub/pkg/__init__.pysub/pyproject.toml is absent and nothing was reported. Now add a nested licence:
cp LICENSE sub/LICENSE
# fab-include = ["sub/**/*.py", "sub/pyproject.toml", "LICENSE", "sub/LICENSE"]
flwr build╭─ Error ──────────────────────────────────────────────────────────────────────╮
│ Pattern in "fab-include" did not match any files: "sub/LICENSE". Correct the │
│ pattern or remove it. │
╰──────────────────────────────────────────────────────────────────────────────╯No FAB is produced.
Why it behaves this way
common/constant.py:85—FAB_EXCLUDE_PATTERNScontainsFAB_CONFIG_FILE("pyproject.toml") as a bare gitignore-style pattern, so it matches at any depth.cli/build.py:417-419— files removed by built-in excludes are explicitly treated as "expected removals and should not be flagged", which is why the user's include is ignored in silence.common/constant.py:71-80—FAB_INCLUDE_PATTERNSanchors the licence as/LICENSE, so a nestedLICENSEis never a candidate file and the user pattern "matches nothing".
Expected
Either behaviour would be defensible on its own; the combination is what surprises. At minimum, a user fab-include pattern that is nullified by a built-in exclude should produce a warning at build time rather than a missing file at runtime.
Suggestions
- Warn (or fail) when a
fab-includepattern matches files that the built-in excludes then remove. - Anchor the built-in
pyproject.tomlexclude as/pyproject.toml, matching the/LICENSEconvention, so nested manifests can be shipped deliberately. - Longer term: a supported "sub-app" concept for an app that ships another app it may launch.
Impact and workaround
This cost us a demo cycle during a hackathon: our ServerApp ran from the installed FAB, found no bundle, and fell back to a rule-based path that looked identical from the outside.
Workaround: ship the nested manifest as agent/pyproject.fab.toml, copy the tree to a temp dir at runtime, rename it back, and copy the app-root LICENSE in beside it — soteria/llm.py.
Source: flwrlabs/flower