persist_empty=True clears existing table data on Table construction
Description
Table.__init__ runs table.clear() whenever persist_empty=True:
if persist_empty:
self._update_table(lambda table: table.clear())(tinydb/table.py)
The parameter docs say this is for storing a new empty table even with no operations (see #513 / the empty-table persistence feature). They do not say that opening an existing non-empty table with persist_empty=True should wipe it.
On current master (4aa5311, TinyDB 4.9.0), first construction of a Table for a name that already has documents deletes those documents.
Minimal reproduction
import tempfile, os
from tinydb import TinyDB
fd, path = tempfile.mkstemp(suffix='.json')
os.close(fd)
db = TinyDB(path)
db.table('t').insert({'a': 1})
db.close()
db = TinyDB(path)
t = db.table('t', persist_empty=True) # first access with the flag
print(len(t), t.all())
# 0 []
print(open(path).read())
# {"t": {}}
db.close()
os.unlink(path)Note: a second db.table('t', persist_empty=True) on the same TinyDB instance returns the cached table and ignores new kwargs (so the wipe only happens on first construction).
Expected behavior (for discussion)
Possible directions:
- Only create/persist an empty table when the name is missing / empty in storage — do not
clear()existing documents - Keep current behavior but document it as destructive and warn
- Reject
persist_empty=Truewhen the table already has data
Silent data loss on a flag meant for empty-table persistence feels wrong.
Suggested tests
- Insert into
t, reopen DB,table('t', persist_empty=True)→ documents must still be present (if option 1) - Empty / missing table +
persist_empty=True→ table name still appears in storage (existing #513 intent)
Happy to open a PR once the preferred behavior is clear.
Environment
- TinyDB 4.9.0 /
master@4aa5311 - Related: #513 (why
persist_emptyexists)
Source: msiemens/tinydb