.grist file grows unreasonably large

Describe the current behavior

I have a relatively simple .grist file that I am using with Grist Desktop. It contains five data tables plus four pages of graphs. Grist's "Raw Data" panel reports that my document's data size is just under 6MB. The .grist file, however, is currently over 310 MB and is constantly growing larger.

Nothing in the Grist interface gave any clue about what is consuming all that space or what I can do to fix it. I dug into the underlying sqlite database and discovered that the _gristsys_ActionHistory table is the culprit. It appears to contain undo/redo information, but many of those records contain the complete (uncompressed) contents of an entire data table. Some even contain multiple copies. For large tables, that's a tremendous amount of data to hold onto. The large table entries seem to all contain references to "GristHidden_import" within the "userActions" section. One particularly egregious entry looked like this (when exported as JSON):

"stored": [
  [
    0,
    [
      "BulkAddRecord",
      "GristHidden_import",
      [
        <a list of values 1-31000 (the number of rows in the imported data>
      ]
      {
         <a complete copy of the imported data>
      }
    ]
  ]
],
"undo": [
  [
    "BulkRemoveRecord",
    "GristHidden_import",
    [
      <long list of line numbers again>
    ]
  ]
],
"userActions": [
  [
    "BulkAddRecord",
    "GristHidden_import",
    [
      <yet another copy of the list of line numbers>
    ],
    {
       <another complete copy of the imported data>
    }
  ]
]

The JSON-exported version of this history entry is 30MB by itself. Of the 1247 entries in my history table, the vast majority are under 10KB. There are only a few super-large ones, and from what I can tell they all appear to be related to import operations.

Steps to reproduce

1. Create a new Grist document
2. Record the file size of the .grist file
3. Import a large .csv file (10k+ rows with 20+ columns)
4. On the final screen of the import process, click "Cancel" to abort the import
5. Close Grist and check the .grist file size again

The file will now be considerably larger than in step 2, even though the import was aborted and the document was not changed.

Examine the entries in the _gristsys_ActionHistory table corresponding to this session. There should be several very large entries, each containing a complete copy (or several) of the data from the aborted import operation.

Describe the expected behavior

1. Aborted import operations should not create history entries. This is not something a user would expect to be in a history buffer, and it's not clear what "undoing" a cancelled operation would even mean. The history list and the document state should be the same before and after the operation.
2. The history entry representing an import operation probably doesn't need to include a complete copy of the data.
   • Undoing only involves deleting the data, which you should be able to do with just the table name (if importing into a new table) or the table name and column identifiers (if merging into an existing table). You'd lose the ability to re-do that action after an undo, but that doesn't seem too terrible. Import -> Undo -> Redo shouldn't happen so often that doing the import again manually is an unreasonable burden.
3. The UI needs to display how much space the history table is using so that the user can know why their .grist file is unreasonably large
4. The user needs a UI for configuring how the history feature works. For example:
   • "Clear history" button
   • Control how many history entries are saved
   • Option to set a max size for individual history entries or for the history table as a whole
   • [for Grist Desktop] Option to clear undo history when program closes (like most desktop programs do)

In addition, consider making your history table more compact by de-duplicating large items. The example above had multiple copies of the same data in it, and other entries had their own copies of that same data. Storing this data once and then recording a reference to it would drastically reduce the amount of storage space that's wasted. Certain items like the array of line numbers could be stored as a compact object akin to a range(1, 31000, 1) instead of wasting the space storing a list of sequential or repeated values.

Where have you encountered this bug?

• On docs.getgrist.com
• On a self-hosted instance

Instance information (when self-hosting only)

• Grist instance:
  • Version: Grist Desktop 0.3.14 (uses Grist Core 1.7.17)
  • URL (if it's OK for you to share it): N/A
  • Installation mode: Portable version from Github releases