[Compression] wcdb_decompress rewrite skips joinClause; JOIN queries return compressed BLOB
Version: 2.1.13 (also reproduced by source inspection on 2.1.16) Platform: Android (Java/Kotlin WINQ API), applies to all platforms sharing common/core
When a compressed column is read through a SELECT that uses the JOIN form of FROM (i.e. StatementSelect.from(Join)), the automatic decompression rewrite does not apply. The query returns the raw stored zstd BLOB instead of the decompressed value. There is no error — the failure is silent, so callers get corrupted/undecodable data (e.g. a TEXT column read back as binary garbage).
Single-table queries on the same column decompress correctly, so the problem only surfaces once a compressed column is read via a JOIN.
Root cause
A compressed column read via a JOIN is never rewritten to wcdb_decompress(...), because
the rewrite fails to register the JOIN's tables. There are two ways a JOIN reaches SelectCore,
and the rewrite misses both:
1. Java/Kotlin/Swift bindings (what most users hit). There is no dedicated from(Join) on
the binding side — Join is a TableOrSubqueryConvertible, so from(join) goes through
from(TableOrSubqueryConvertible...), and the bridge wraps the Join into a TableOrSubquery
with switcher == Switch::JoinClause, pushed into SelectCore.tableOrSubqueries:
// StatementSelectBridge.cpp
case WCDBBridgedType_JoinClause:
cppTableOrSubqueries.emplace_back(WCDBGetMultiTypeArrayObject(WCDB::Join, ...));
...
cppSelect->from(cppTableOrSubqueries); // ends up calling from(TablesOrSubqueries)
// TableOrSubquery.cpp
TableOrSubquery::TableOrSubquery(const Join& join) {
syntax().switcher = Switch::JoinClause;
syntax().joinClause = join;
} 2. Pure C++ select.from(join). Stored in the SelectCore.joinClause field
(from(Join) clears tableOrSubqueries).
But parseTable — which registers "which in-scope tables are compressed" — only handles
Switch::Table / Switch::TableOrSubqueries / aliased subqueries:
for (const auto& table : tables) {
if (table.switcher == Switch::Table) { ...; return true; }
else if (table.switcher == Switch::TableOrSubqueries) { parseTable(table.tableOrSubqueries, ...); }
else if (!table.alias.empty()) { tableInfos.insert(table.alias, nullptr); }
// no Switch::JoinClause branch
} So the Switch::JoinClause element (path 1) falls through every branch and is skipped, and the
SelectCore.joinClause field (path 2) is never read by adaptCompressingColumn at all. Either
way no table is registered → tableInfo == nullptr for the projected compressed column → the
rewrite is skipped → the raw zstd BLOB is returned silently.
Note SelectCore::iterate traverses both forms correctly, so the AST walk does reach the
columns; the gap is purely in parseTable's table registration.
Suggested fix
- In
parseTable, add aSwitch::JoinClausebranch that recurses into the element'sjoinClause().tableOrSubqueries. - In
adaptCompressingColumn, also parse theselectCore.joinClausefield. Both are needed to cover the binding path and the pure-C++ path respectively.
Reproduction
Configure compression on a table:
db.setCompression(info -> { if (info.getTable().equals("conversation")) info.addZSTDNormalCompress(DBConversation.content); });
Insert rows so content gets compressed on disk.
Read it back with a JOIN:
StatementSelect stmt = new StatementSelect() .select(DBConversation.content.table("conversation")) .from(new Join("conversation") .leftJoin("conversation_meta") .on(DBConversation.fk.table("conversation") .eq(DBConversationMeta.id.table("conversation_meta")))); // rows.get(0).get(0) => raw zstd BLOB, NOT the original text
Expected: content is transparently decompressed (as it is for single-table selects). Actual: raw compressed BLOB is returned, silently, with no error.
Impact
- Any read of a compressed column through a JOIN is affected — projection list, WHERE, and ON alike, because the table is never registered.
- Failure is silent (no exception, no notification), so it easily reaches production as "corrupted" text.
Source: Tencent/wcdb