Imagine you have a cheap virtual machine with 2 GB of RAM, absolutely no Swap space, and an ambitious goal: to run a distributed AI search engine capable of processing and vectorizing thousands of incoming documents (the "Harvest" pipeline).
Most developers, upon hearing the words "vector search," immediately rush to deploy heavy enterprise solutions like pgvector, Pinecone, or Milvus.
However, on a 2GB RAM machine, these memory-hungry monsters will crash from an Out-of-Memory (OOM) error before they even finish initializing.
For NGP 4.5 (NetGlyph Knowledge Protocol), we decided to embrace extreme minimalism and chose the battle-tested, time-proven SQLite.
In this article, we'll show you how we tuned our embedded database to handle hundreds of transactions per second, completely eliminated file descriptor leaks, and kept memory consumption flat within a negligible margin.
1.
Anatomy of a Disaster: How to Kill a Server in One Minute During the development of our vector engine () and document vectorizer, we encountered a classic architectural friction point.
One of our AI agents ("Hermes"), responsible for auto-importing data, stored vectors like this: What's wrong with this code?
Phantom Connections: To simply fetch the current formatted time, the engine took a wild detour: it opened a completely new, independent connection via directly inside the argument list, ran a query to the SQL function , and... left that connection open.
File Descriptor Leak: Every single one of these hanging connections held a file descriptor open.
On our tiny 2GB VPS, after processing a stream of 2,258 documents, the operating system ran out of file descriptors and memory.
OOM Crashes: The OS kernel's would ruthlessly terminate our process before we could even process the first hundred documents.
2.
The Patch: Native Calls and Context Managers The first step in saving the system was a complete refactoring of how we manage database connections.
We replaced manual SQL-based time requests with lightweight, native Python system calls and migrated to safe, idiomatic context managers.
The Optimal Solution: What changed: guarantees that even if a crash, error, or database corruption occurs during the transaction, Python will automatically commit (or rollback) and close the file descriptor.
Native Timestamp: Invoking is an incredibly fast, nanosecond-level OS kernel system call.
We cut out SQL query parsing and saved precious CPU cycles for actual vectorization.
3.
Tuning the "Light-Weight" SQLite Configuration To make SQLite perform as a high-speed, concurrent embedded engine on ultra-constrained hardware, the default out-of-the-box settings simply won't cut it.
Here is our optimal "Light-Weight" configuration that squeezed maximum performance on our 2GB RAM server: Explaining the PRAGMA Magic: : Write-Ahead Logging allows reader threads to query the database concurrently even while a writer thread is executing.
This is absolutely critical for multi-threaded vector search. : On a cheap server, you cannot let the process map several gigabytes of raw database files into memory.
A 256MB limit keeps the hottest indexes and tables mapped directly in the process's address space, giving you sub-millisecond access times without redundant I/O operations. : A hidden SQLite syntax hack.
Standard positive values set the cache in number of pages, but negative values strictly enforce a limit in Kibibytes ().
This is our armor against memory leaks. : Combined with WAL, is fully durable and secure.
The database remains consistent in the event of an application crash, but the VPS disk is spared from constant block-level system calls.
4.
Multi-threading and Race Conditions In a distributed agentic system, multiple workers write to the database concurrently.
To avoid the dread , we implemented a two-level defense: : If another thread locks the database, SQLite won't crash instantly.
Instead, it waits up to 5 seconds for the lock to clear. : We isolate all write operations insi