LMDB is an embedded, memory-mapped, B+tree key-value store written in C by Howard Chu of OpenLDAP. It is one of the fastest embedded databases ever built — reads come straight out of the OS page cache via mmap with zero copying, and writes are committed via a single fsync per transaction. LMDB underpins OpenLDAP’s back-mdb backend, the Bitcoin Core block index, monero, several Python ML data loaders, and many other systems where read latency matters more than feature richness.
LMDB is a library linked into your process, not a server. A database environment is a directory containing one data file and one lock file. On open, LMDB maps the data file into memory with mmap. A read walks the B+tree by following page offsets in that mapping and returns a pointer to the value bytes in place, which is why reads are so cheap: there is no buffer pool, no serialisation layer and no system call per lookup once pages are cached. The operating system decides which pages stay in RAM, and several processes opening the same environment share those pages.
LMDB never overwrites live pages. A write transaction copies each page it modifies, along with the path up to a new root, and commits by writing one of two meta pages that point at the current root. A crash before that write leaves the previous meta page intact, so there is no write-ahead log and no recovery step. Readers take a snapshot by recording the current root; they are lock-free and see a fixed version for the duration of the transaction. Only one write transaction can be active at a time per environment, which keeps the design simple and avoids deadlocks, but it means write throughput is limited to what a single writer can achieve.
import lmdb # pip install lmdb
# map_size is the maximum database size; it reserves address space, not disk
env = lmdb.open("students.lmdb", map_size=1 << 30, max_dbs=4)
students = env.open_db(b"students")
with env.begin(write=True, db=students) as txn: # single writer
txn.put(b"1001", b'{"name": "Alice", "state": "WA"}')
txn.put(b"1002", b'{"name": "Bob", "state": "OR"}')
txn.put(b"1003", b'{"name": "Charlie", "state": "CA"}')
with env.begin(db=students) as txn: # snapshot read
print(txn.get(b"1002"))
cur = txn.cursor()
cur.set_range(b"1002") # range scan from a key
for key, value in cur:
print(key, value)
print(env.stat()["entries"], "entries")
Keys are sorted bytewise, so integer keys should be stored big-endian (for example struct.pack(">Q", n)) if numeric order matters. Batching many puts in one write transaction is much faster than committing each put, because each commit flushes to disk by default.
| Setting | Effect |
|---|---|
map_size | Upper bound on database size. Writes fail with MapFullError when it is reached; it can be increased later with env.set_mapsize(). |
sync=False / metasync=False | Faster commits at the cost of durability of the most recent transactions after an OS crash (still consistent). |
readonly=True, lock=False | For read-only datasets such as ML training data on shared storage. |
max_readers | Size of the reader lock table; raise it for many threads or processes. |
env.copy(path, compact=True)) to reclaim space.Environment object.