bbolt

bbolt is an embedded, pure-Go B+tree key-value store, originally written by Ben Johnson as BoltDB and now maintained by the etcd team as bbolt. It is to Go what LMDB is to C: a single-writer, mmap-based, B+tree key-value store with strict ACID semantics and remarkable simplicity. bbolt is the storage engine inside etcd, Consul, InfluxDB’s metadata, Open Policy Agent, and many other Go infrastructure tools.

The rest of this page explains the on-disk layout, how copy-on-write commits keep the file crash-safe, a complete Go example, and the pitfalls that matter in production.

1. Key Features

• B+tree on a single file. The whole database is one mmap-backed file; no compaction, no SSTables, no background threads.
• Single-Writer ACID. One writer transaction at a time (serializable); concurrent readers are wait-free.
• Buckets. Hierarchical namespaces inside the file — nested key-value collections.
• Pure Go, No CGO. Builds and deploys as part of any Go binary.
• Crash-Safe. Copy-on-write pages and a meta-page double-buffer guarantee atomic commits even on power loss.
• Tiny Footprint. ~3,000 lines of Go; the whole API surface fits in your head.

2. bbolt vs. BadgerDB

• bbolt. B+tree, single-writer, simpler, lower write throughput. Best for control-plane / metadata workloads.
• BadgerDB. LSM, concurrent writers, higher write throughput. Better for application data.

3. bbolt vs. LMDB

• Conceptually nearly identical — both are mmap-backed B+tree single-writer KV stores.
• LMDB in C, used outside Go ecosystems; bbolt in Go, used inside Go ecosystems.

4. Use Cases

• etcd uses bbolt as the underlying storage for the Raft state machine.
• Consul, InfluxDB metadata, OPA, Tendermint, NATS Jetstream — bbolt is the de-facto Go embedded KV for control-plane data.
• CLI tools that need a small persistent store (e.g. credentials, history, indexes).
• Workloads where strict serializable transactions matter more than write throughput.

5. Architecture and file layout

A bbolt database is a single file divided into fixed-size pages, normally the operating system page size. The first two pages are meta pages, each holding a transaction ID, the root page of the bucket tree, the freelist location and a checksum. Data lives in B+tree pages: branch pages hold keys and child pointers, and leaf pages hold keys and values sorted byte-wise. Each bucket is its own B+tree, and buckets can nest. Readers access the file through a read-only memory map, so a read transaction is essentially a consistent pointer to one meta page and costs no locks or copies.

6. How a commit works

A write transaction never modifies pages in place. It copies every page it touches, plus the path up to the root, into memory, then writes those pages to free locations in the file. After syncing data, it writes the new meta page into whichever meta slot holds the older transaction ID and syncs again. On open, bbolt picks the meta page with the highest valid transaction ID, so a crash at any point leaves either the old or the new state, never a mix. Pages freed by a transaction cannot be reused while any read transaction that might still see them is open.

7. Worked example

package main

import (
    "bytes"
    "fmt"
    "log"
    "time"

    bolt "go.etcd.io/bbolt"
)

func main() {
    db, err := bolt.Open("app.db", 0600, &bolt.Options{Timeout: time.Second})
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // read-write transaction: create bucket and store keys
    err = db.Update(func(tx *bolt.Tx) error {
        b, err := tx.CreateBucketIfNotExists([]byte("users"))
        if err != nil {
            return err
        }
        if err := b.Put([]byte("u:1001"), []byte(`{"name":"alice"}`)); err != nil {
            return err
        }
        return b.Put([]byte("u:1002"), []byte(`{"name":"bob"}`))
    })
    if err != nil {
        log.Fatal(err)
    }

    // read-only transaction: prefix scan with a cursor
    db.View(func(tx *bolt.Tx) error {
        c := tx.Bucket([]byte("users")).Cursor()
        prefix := []byte("u:")
        for k, v := c.Seek(prefix); k != nil && bytes.HasPrefix(k, prefix); k, v = c.Next() {
            fmt.Printf("%s = %s\n", k, v)
        }
        return nil
    })
}

Update commits if the function returns nil and rolls back otherwise; View is read-only. Keys are sorted as bytes, so a cursor Seek followed by Next gives efficient range and prefix scans. The Timeout option makes Open fail instead of blocking forever if another process holds the file lock.

8. When to use it

9. Pitfalls

• The file does not shrink. Deleted pages go on the freelist and are reused, but the file size stays; use bbolt compact to rewrite it.
• Long read transactions pin old pages and make the file grow under write load. Keep View functions short.
• Values are only valid inside the transaction. Byte slices point into the mmap; copy them before the transaction ends.
• One process at a time. The file is locked by the opening process; it is not a shared server database.
• Small random writes are slow because each commit fsyncs twice; batch them.

10. Related

See the Key-Value Stores overview, LevelDB for the LSM design that RocksDB extends, and FoundationDB for a distributed transactional key-value store.

Featured Videos