go-memdb and go memdb: in-memory database for Golang

Tech reviewed: Deepak Prasad
go-memdb and go memdb: in-memory database for Golang

People searching for go-memdb, go memdb, memdb, golang in memory database, golang in memory db, go in memory database, in memory database golang, or mem db are usually looking for HashiCorp’s github.com/hashicorp/go-memdb: an embedded, in-process indexed store—not a network server and not AWS DynamoDB (unrelated “dynamdob” typos in search logs). It gives you tables, multiple indexes, MVCC-style readers, and read/write transactions (*memdb.Txn) on top of immutable radix trees. You model rows as Go structs (see structs in Go). Official API reference: pkg.go.dev/github.com/hashicorp/go-memdb.

Tested with Go 1.24 on Linux and github.com/hashicorp/go-memdb v1.3.5.


Install github.com/hashicorp/go-memdb

From a module:

bash
go get github.com/hashicorp/go-memdb
text
go: added github.com/hashicorp/go-immutable-radix v1.3.1
go: added github.com/hashicorp/go-memdb v1.3.5
go: added github.com/hashicorp/golang-lru v0.5.4

Import the package as memdb (the module path ends in go-memdb).


Schema, primary index id, and transactions

You declare a memdb.DBSchema with tables and memdb.IndexSchema entries. The primary index must be named id (the implementation looks up that key internally). Other indexes can use any names (name, score, …).

  • memdb.NewMemDB(schema) builds the database.
  • db.Txn(true) starts a writable transaction; db.Txn(false) is read-only.
  • Always defer txn.Abort() on read transactions (it is safe after Commit on writers too; readers rely on it for cleanup).
  • txn.Insert(table, obj) adds or replaces a row; txn.Commit() publishes writes atomically.

Reads: First, Get, LowerBound, and *memdb.Txn

Typical read helpers on memdb.Txn:

  • First(table, index, args...) — first row matching exact index constraints.
  • Get(table, index, args...) — iterator over all rows for that index lookup.
  • LowerBound(table, index, args...) — range scan from a lower bound (walk with Next() until your upper bound).

Prefix scans are available when the indexer supports them: append _prefix to the index name as described in the package docs.

go
package main

import (
	"fmt"

	"github.com/hashicorp/go-memdb"
)

type Student struct {
	Id    int
	Name  string
	Score int
}

func main() {
	schema := &memdb.DBSchema{
		Tables: map[string]*memdb.TableSchema{
			"student": {
				Name: "student",
				Indexes: map[string]*memdb.IndexSchema{
					"id": {
						Name:    "id",
						Unique:  true,
						Indexer: &memdb.IntFieldIndex{Field: "Id"},
					},
					"name": {
						Name:    "name",
						Unique:  false,
						Indexer: &memdb.StringFieldIndex{Field: "Name"},
					},
					"score": {
						Name:    "score",
						Unique:  false,
						Indexer: &memdb.IntFieldIndex{Field: "Score"},
					},
				},
			},
		},
	}

	db, err := memdb.NewMemDB(schema)
	if err != nil {
		panic(err)
	}

	txn := db.Txn(true)
	for _, p := range []*Student{
		{1, "Anna", 95},
		{2, "Bob", 75},
		{3, "Dorothy", 80},
		{4, "Daniel", 90},
	} {
		if err := txn.Insert("student", p); err != nil {
			panic(err)
		}
	}
	txn.Commit()

	txn = db.Txn(false)
	defer txn.Abort()

	raw, err := txn.First("student", "name", "Anna")
	if err != nil {
		panic(err)
	}
	fmt.Printf("Hello student %s!\n", raw.(*Student).Name)

	it, err := txn.Get("student", "name")
	if err != nil {
		panic(err)
	}
	fmt.Println("All the students:")
	for obj := it.Next(); obj != nil; obj = it.Next() {
		fmt.Printf("  %s\n", obj.(*Student).Name)
	}

	it, err = txn.LowerBound("student", "score", 75)
	if err != nil {
		panic(err)
	}
	fmt.Println("Student with score 75 - 90:")
	for obj := it.Next(); obj != nil; obj = it.Next() {
		p := obj.(*Student)
		if p.Score > 90 {
			break
		}
		fmt.Printf("  %s with score %d\n", p.Name, p.Score)
	}
}
text
Hello student Anna!
All the students:
  Anna
  Bob
  Daniel
  Dorothy
Student with score 75 - 90:
  Bob with score 75
  Dorothy with score 80
  Daniel with score 90

Iteration order for Get follows the chosen index (here name), not insertion order.


Deletes: Delete and DeleteAll

  • Delete(table, obj) removes one object; the object must exist (primary fields are read from obj via the id indexer).
  • DeleteAll(table, index, args...) removes every row matching the index constraint.
go
package main

import (
	"fmt"

	"github.com/hashicorp/go-memdb"
)

type Student struct {
	Id    int
	Name  string
	Score int
}

func main() {
	schema := &memdb.DBSchema{
		Tables: map[string]*memdb.TableSchema{
			"student": {
				Name: "student",
				Indexes: map[string]*memdb.IndexSchema{
					"id":    {Name: "id", Unique: true, Indexer: &memdb.IntFieldIndex{Field: "Id"}},
					"name":  {Name: "name", Unique: true, Indexer: &memdb.StringFieldIndex{Field: "Name"}},
					"score": {Name: "score", Unique: false, Indexer: &memdb.IntFieldIndex{Field: "Score"}},
				},
			},
		},
	}
	db, err := memdb.NewMemDB(schema)
	if err != nil {
		panic(err)
	}
	txn := db.Txn(true)
	for _, p := range []*Student{
		{1, "Anna", 80},
		{2, "Bob", 85},
		{3, "Clair", 90},
		{4, "Dorothy", 95},
		{5, "Caitlyn", 90},
	} {
		if err := txn.Insert("student", p); err != nil {
			panic(err)
		}
	}
	txn.Commit()

	printAll(db.Txn(false))
	fmt.Println("deleting one")
	txn = db.Txn(true)
	if err := txn.Delete("student", &Student{Id: 1}); err != nil {
		panic(err)
	}
	txn.Commit()
	printAll(db.Txn(false))

	fmt.Println("deleting many")
	txn = db.Txn(true)
	if _, err := txn.DeleteAll("student", "score", 90); err != nil {
		panic(err)
	}
	txn.Commit()
	printAll(db.Txn(false))
}

func printAll(txn *memdb.Txn) {
	defer txn.Abort()
	it, err := txn.Get("student", "id")
	if err != nil {
		panic(err)
	}
	fmt.Println("All the students:")
	for obj := it.Next(); obj != nil; obj = it.Next() {
		fmt.Printf("  %s\n", obj.(*Student).Name)
	}
}
text
All the students:
  Anna
  Bob
  Clair
  Dorothy
  Caitlyn
deleting one
All the students:
  Bob
  Clair
  Dorothy
  Caitlyn
deleting many
All the students:
  Bob
  Dorothy

What go-memdb gives you (and what it does not)

Go-memdb is ideal when a single Go process needs a fast, transactional, indexed working set (similar intent to “golang in memory database” tutorials) with many concurrent readers and serialized writers via MVCC on immutable radix trees. It provides atomicity and isolation for transactions; because everything lives in memory, it does not offer durability—restart the process and the data is gone unless you snapshot or persist elsewhere. For cross-process or replicated data, use a real database or service; for cryptographic randomness, pair with crypto/rand instead of math/rand patterns.


Summary

Go memdb (github.com/hashicorp/go-memdb) is the usual answer to go-memdb and memdb searches: define a schema whose primary index is literally named id, open read or write transactions, insert structs with Insert, query with First, Get, or LowerBound, and mutate data with Delete / DeleteAll. It fits the “golang in memory db” niche inside one application binary, not cloud DynamoDB workloads. If you see signatures like getPolicy(txn *memdb.Txn) in the wild, read that as “policy code receives a read txn over memdb-backed state,” not a built-in memdb method.


References


Frequently Asked Questions

1. Why must my primary index be named id in go-memdb?

The library hardcodes the primary key index under the name id (see txn.go). You can add other named indexes, but the unique primary indexer must live at id.

2. What does getpolicy(txn *memdb.Txn) mean in search results?

That is usually application code taking a read transaction to evaluate policy against data stored in memdb; memdb itself does not define a GetPolicy API.

3. Is go-memdb a full SQL or DynamoDB-style database?

No. It is an in-process indexed store backed by immutable radix trees with ACID-style atomicity, consistency, and isolation but no durability. Use real databases for persistent SQL or cloud key-value services.

4. Can I update a row by mutating a struct I already inserted?

No. Insert expects you to pass a new or copied object for updates; mutating objects already stored in memdb is unsupported and can break indexes.
Tuan Nguyen

Data Scientist

Proficient in Golang, Python, Java, MongoDB, Selenium, Spring Boot, Kubernetes, Scrapy, API development, Docker, Data Scraping, PrimeFaces, Linux, Data Structures, and Data Mining. With expertise spanning these technologies, he develops …

  • Deep Learning with TensorFlow
  • Machine Learning with Python
  • Go (programming language)
  • Python (programming language)
  • Java (programming language)
  • MongoDB