Go: table datasets on Elasticsearch/OpenSearch have no field_map, so table SQL retrieval (NL2SQL) is unavailable

Summary

In the Go runtime a table dataset served by Elasticsearch / OpenSearch cannot use table SQL retrieval (NL2SQL / "ask the table"): no field_map is published for it, so the SQL path never activates and chat silently falls back to vector search.

Why it currently cannot work

The SQL retrieval path is driven by the dataset's field_map (internal/service/chat_pipeline.go, Phase 6): it is non-empty → the engine-specific prompt is built from the map keys and SQL runs; empty → the phase is skipped.

Python's table chunker stores table metadata differently per engine (rag/app/table.py):

Engine family	Storage
Infinity / OceanBase / GaussDB / SereneDB	chunk_data JSON keyed by the raw column name, field_map = raw name → display name (rag/app/table.py:558)
Elasticsearch / OpenSearch / default	one typed field per cell — pinyin key + type suffix, e.g. country_kwd, amount_flt, title_tks (fields_map at rag/app/table.py:527, built at :556), plus _raw variants (:645-648); field_map maps those typed keys → display names

The Go port only produces the first representation: the table row renderer emits chunk_data keyed by the raw column name (internal/parser/parser/table_row_render.go), and indexdoc.BuildFieldMap keys the dataset map by raw column name too. Publishing that map on ES/OS would be wrong — the ES SQL prompt treats field_map keys as direct fields (esSQLUserPromptTemplate, internal/service/chat_pipeline.go:3324), so the model would generate SQL against columns that do not exist (or only inside the chunk_data object).

That is why the write is deliberately gated today:

• internal/engine/engine.go:122 — StoresTableChunkData() is true only for Infinity / OceanBase / SeekDB / SereneDB;
• internal/ingestion/task/pipeline_executor.go:300-305 — a run on any other engine publishes the discovered column names only and leaves field_map untouched (so chat does not even try SQL).

Impact

	Infinity / OceanBase / SereneDB	Elasticsearch / OpenSearch
Table chunks (text + chunk_data)	✅	✅
Document metadata aggregation, dataset table_column_names	✅	✅
Table SQL retrieval (NL2SQL)	✅	❌ (vector search only)

Existing ES datasets written by the Python runtime do have the typed fields and a working field_map; the gap is specific to documents ingested by the Go runtime (and to datasets whose field_map Go never publishes).

Proposal

Port the typed-field representation to the Go ingestion writer for non-chunk_data engines:

1. Infer per-column type on ingest (text/int/float/bool/datetime — Python's column_data_type, rag/app/table.py:360) and write one top-level field per stored cell using the same key shape Python produces (pinyin base + _tks/_long/_flt/_kwd/_dt, with _raw for text), so both runtimes stay index-compatible.
2. Build the dataset field_map from those typed keys → display names (BuildFieldMap would take the engine into account or gain a typed variant).
3. Lift the StoresTableChunkData gate in pipeline_executor.go once the map is truthful for ES/OS, and cover it with a test that runs the table template end-to-end against a stubbed ES writer.

Until then the behaviour is documented as a known limitation of #19347 (Go table column mode), and Go-written ES table datasets deliberately stay vector-only.

References

• PR #19347 (Go table column mode; the gate and the limitation were introduced/documented there)
• Python: rag/app/table.py:527, :556-558, :645-648, :360
• Go: internal/engine/engine.go:122, internal/ingestion/task/pipeline_executor.go:300-305, internal/parser/parser/table_row_render.go, internal/service/chat_pipeline.go:3324 (esSQLUserPromptTemplate)