In this tutorial, we will build a simple in-memory key-value store example in Go with FlashDB. FlashDB is a simple in-memory key/value store written entirely in Go. It persists data on disk, is ACID compliant and employs locking for multiple readers and a single writer. It allows for redis-like operations, SET, SORTED SET, HASH, and STRING are examples of data structures.
FlashDB architecture
FlashDB is made up of easily understandable composable libraries. The goal is to connect the learning gap for anyone who is learning how to build a simple ACID database. FlashDB supports a variety of Redis commands. Redis uses the following data structures to implement various types:
- Set: Redis Sets are unsorted string collections. It is also possible to perform a variety of other operations on sets, such as determining whether a given element already exists, performing the intersection, union, or difference of multiple sets, and so on. This is also done with a simple HashMap data structure.
- ZSet: Sorted sets are a data type that resembles a cross between a Set and a Hash. Sorted sets, like sets, are composed of unique, non-repeating string elements, so they are also sets in some ways. While elements within sets are not ordered, each element in a sorted set is assigned a floating point value known as the score (this is why the type is also similar to a hash, since every element is mapped to a value).
- String: The Redis String value type is the most basic type of value that can be associated with a Redis key. Because Redis keys are strings, using the string type as a value also maps a string to another string.. A simple implementation of Adaptive Radix Tree (ART) in Go, that scans could be done easily.
- Hash: While hashes are useful for representing objects, the number of fields you can store in a hash has no practical limit (other than available memory), so you can use hashes in a variety of ways within your application. This is done with a very simple HashMap data structure.
- Append Only Log: A simple immutable append only log in Go. aol is a rewrite of wal to allow only appends to a log.
Install GO FlashDB
Run the below command to install FlashDB and its dependencies:
$ go get github.com/arriqaaq/flashdb
go: finding module for package github.com/arriqaaq/flashdb
go: downloading github.com/arriqaaq/flashdb v0.1.6
go: found github.com/arriqaaq/flashdb in github.com/arriqaaq/flashdb v0.1.6
go: downloading github.com/arriqaaq/aol v0.1.2
go: downloading github.com/arriqaaq/art v0.1.1
go: downloading github.com/arriqaaq/hash v0.1.2
go: downloading github.com/arriqaaq/set v0.1.2
go: downloading github.com/arriqaaq/zset v0.1.2
go: downloading github.com/stretchr/testify v1.7.1
go: downloading github.com/davecgh/go-spew v1.1.0
go: downloading gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
go: downloading github.com/armon/go-radix v1.0.0
go: downloading github.com/arriqaaq/skiplist v0.1.6
Create and Open a database
The primary object in FlashDB is a DB
. To open or create your database, use the flashdb.New()
 function:
package main
import (
"log"
"github.com/arriqaaq/flashdb"
)
func main() {
config := &flashdb.Config{Path: "/path", EvictionInterval: 10}
db, err := flashdb.New(config)
if err != nil {
log.Fatal(err)
}
defer db.Close()
}
EvictionInterval
(in seconds): NoSync disables fsync after writes. This is less durable and puts the log at risk of data loss when there's a server crash.
Verify the path after executing the code:
$ ls -l /path/
total 8
drwxr-x--- 2 root root 4096 Oct 4 00:42 ./
drwxr-xr-x 21 root root 4096 Oct 4 00:42 ../
-rw-r----- 1 root root 0 Oct 4 00:42 00000000000000000001
To open a database that does not persist to disk, leave the path empty.
config := &flashdb.Config{Path:"", EvictionInterval: 10}
flashdb.New(config)
Performing Different Operations on FlashDB
When you need to make changes to your data, you use a read/write transaction. At any given time, only one read/write transaction can be active. As a result, make sure to close it as soon as you are finished with it.
All reads and writes must take place within a transaction. FlashDB can only have one write transaction open at a time, but many concurrent read transactions are possible. Each transaction keeps a consistent view of the database. In other words, once a transaction has begun, other transactions cannot change the data for that transaction.
When a transaction fails, it rolls back and undoes all changes to the database that occurred during that transaction. When a read/write transaction completes successfully, all changes are saved to disk.
Create a database and add key value pair based data
When you need to make changes to your data, you use a read/write transaction. At any given time, only one read/write transaction can be active. As a result, make sure to close it as soon as you are finished with it. To set /get key-value pair, open a read/write transaction and use Update()
function.
Here is an example of set multiple key-value records with FlashDB:
package main
import (
"fmt"
"log"
"github.com/arriqaaq/flashdb"
)
func main() {
config := &flashdb.Config{Path: "", EvictionInterval: 10}
db, err := flashdb.New(config)
if err != nil {
log.Fatal(err)
}
defer db.Close()
// define a map to contains key-value pair
studentMap := map[string]string{"Anna": "anna@gmail.com", "Bob": "bob@gmail.com", "Cloe": "cloe@gmail.com", "Daniel": "deniel@gmail.com"}
err = db.Update(func(tx *flashdb.Tx) error {
for key, value := range studentMap {
// set key-value record
err := tx.Set(key, value)
return err
}
return nil
})
if err != nil {
log.Fatal(err)
}
}
Read the content from FlashDB Database
We can query key-value pair by using View()
function:
db.View(func(tx *flashdb.Tx) error {
val, _ := tx.Get("Anna")
fmt.Printf("value is %s\n", val)
return nil
})
Output:
value is anna@gmail.com
Delete a record from FlashDB Database
If you want to delete a record in FlashDB, you can use Delete()
function:
db.Update(func(tx *flashdb.Tx) error {
var err error
for key, value := range studentMap {
err = tx.Set(key, value)
if err != nil {
panic(err)
}
}
err = tx.Delete("Anna")
return err
})
db.View(func(tx *flashdb.Tx) error {
val, err := tx.Get("Anna")
val2, err := tx.Get("Bob")
if err != nil {
return err
}
fmt.Printf("After delete : value is %s\n", val)
fmt.Printf("Another value is %s\n", val2)
return nil
})
Output:
After delete : value is
Another value is bob@gmail.com
Summary
In this article, I have given some examples of performing operations in flashdb.
The functions of different data types are listed below:
String | Hash | Set | ZSet |
---|---|---|---|
SET | HSET | SADD | ZADD |
GET | HGET | SISMEMBER | ZSCORE |
DELETE | HGETALL | SRANDMEMBER | ZCARD |
EXPIRE | HDEL | SREM | ZRANK |
TTL | HEXISTS | SMOVE | ZREVRANK |
HLEN | SCARD | ZRANGE | |
HKEYS | SMEMBERS | ZREVRANGE | |
HVALS | SUNION | ZREM | |
HCLEAR | SDIFF | ZGETBYRANK | |
SCLEAR | ZREVGETBYRANK | ||
ZSCORERANGE | |||
ZREVSCORERANGE | |||
ZCLEAR |
We have a topic about memdb (a in-memory database) in Golang. Depending on the application, you may want to consider using the databases listed above.
References
https://en.wikipedia.org/wiki/In-memory_database
https://github.com/arriqaaq/flashdb