44Computationally focused database using pluggable stores
55
66## Provided Stores
7- | Store | Type | Embedded | Persistence | Read Perf | Write Perf | Concurrency | Transactions | Full-Text Search | Notes |
8- | ------------------------------------------------------------------------| --------------------| ----------| -------------| -----------| ------------| -------------| ------------------| ------------------| ---------------------------------------|
9- | [ HaloDB] ( https://github.com/yahoo/HaloDB ) | KV Store | ✅ | ✅ | ✅ | ✅✅ | 🟡 (Single-threaded write) | 🟡 (Basic durability) | ❌ | Fast, simple write-optimized store |
10- | [ ChronicleMap] ( https://github.com/OpenHFT/Chronicle-Map ) | Off-Heap Map | ✅ | ✅ (Memory-mapped) | ✅✅ | ✅✅ | ✅✅ | ❌ | ❌ | Ultra low-latency, off-heap storage |
11- | [ LMDB] ( https://www.symas.com/mdb ) | KV Store (B+Tree) | ✅ | ✅ | ✅✅✅ | ✅ | 🟡 (Single write txn) | ✅✅ (ACID) | ❌ | Read-optimized, mature B+Tree engine |
12- | [ MapDB] ( https://mapdb.org ) | Java Collections | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | Easy Java-native persistence |
13- | [ RocksDB] ( https://rocksdb.org ) | LSM KV Store | ✅ | ✅ | ✅✅ | ✅✅✅ | ✅ | ✅ | ❌ | High-performance LSM tree |
14- | [ Redis] ( https://redis.io ) | In-Memory KV Store | 🟡 (Optional) | ✅ (RDB/AOF) | ✅✅✅ | ✅✅ | ✅ | ✅ | ❌ | Popular in-memory data structure store|
15- | [ Lucene] ( https://lucene.apache.org ) | Full-Text Search | ✅ | ✅ | ✅✅ | ✅ | ✅ | ❌ | ✅✅✅ | Best-in-class full-text search engine |
16- | [ SQLite] ( https://www.sqlite.org ) | Relational DB | ✅ | ✅ | ✅ | ✅ | 🟡 (Write lock) | ✅✅ (ACID) | ✅ (FTS5) | Lightweight embedded SQL |
17- | [ H2] ( https://h2database.com ) | Relational DB | ✅ | ✅ | ✅ | ✅ | ✅ | ✅✅ (ACID) | ❌ (Basic LIKE) | Java-native SQL engine |
18- | [ DuckDB] ( https://duckdb.org ) | Analytical SQL | ✅ | ✅ | ✅✅✅ | ✅ | ✅ | ✅ | ❌ | Columnar, ideal for analytics |
19- | [ PostgreSQL] ( https://www.postgresql.org ) | Relational DB | ❌ (Server-based) | ✅ | ✅✅✅ | ✅✅ | ✅✅ | ✅✅✅ (ACID, MVCC) | ✅✅ (TSVector) | Full-featured RDBMS |
7+ | Store | Type | Embedded | Persistence | Read Perf | Write Perf | Concurrency | Transactions | Full-Text Search | Prefix Scan | Notes |
8+ | ------------------------------------------------------------------------| --------------------| ----------| -------------| -----------| ------------| -------------| ------------------| ------------------| -----------| ----------- ----------------------------|
9+ | [ HaloDB] ( https://github.com/yahoo/HaloDB ) | KV Store | ✅ | ✅ | ✅ | ✅✅ | 🟡 (Single-threaded write) | 🟡 (Basic durability) | ❌ | ❌ | Fast, simple write-optimized store |
10+ | [ ChronicleMap] ( https://github.com/OpenHFT/Chronicle-Map ) | Off-Heap Map | ✅ | ✅ (Memory-mapped) | ✅✅ | ✅✅ | ✅✅ | ❌ | ❌ | ❌ | Ultra low-latency, off-heap storage |
11+ | [ LMDB] ( https://www.symas.com/mdb ) | KV Store (B+Tree) | ✅ | ✅ | ✅✅✅ | ✅ | 🟡 (Single write txn) | ✅✅ (ACID) | ❌ | ✅ | Read-optimized, mature B+Tree engine |
12+ | [ MapDB (B-Tree) ] ( https://mapdb.org ) | Java Collections | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | Uses BTreeMap for ordered/prefix scans |
13+ | [ RocksDB] ( https://rocksdb.org ) | LSM KV Store | ✅ | ✅ | ✅✅ | ✅✅✅ | ✅ | ✅ | ❌ | ✅ | High-performance LSM tree |
14+ | [ Redis] ( https://redis.io ) | In-Memory KV Store | 🟡 (Optional) | ✅ (RDB/AOF) | ✅✅✅ | ✅✅ | ✅ | ✅ | ❌ | ❌ | Popular in-memory data structure store|
15+ | [ Lucene] ( https://lucene.apache.org ) | Full-Text Search | ✅ | ✅ | ✅✅ | ✅ | ✅ | ❌ | ✅✅✅ | ❌ | Best-in-class full-text search engine |
16+ | [ SQLite] ( https://www.sqlite.org ) | Relational DB | ✅ | ✅ | ✅ | ✅ | 🟡 (Write lock) | ✅✅ (ACID) | ✅ (FTS5) | 🟡 | Lightweight embedded SQL |
17+ | [ H2] ( https://h2database.com ) | Relational DB | ✅ | ✅ | ✅ | ✅ | ✅ | ✅✅ (ACID) | ❌ (Basic LIKE) | 🟡 | Java-native SQL engine |
18+ | [ DuckDB] ( https://duckdb.org ) | Analytical SQL | ✅ | ✅ | ✅✅✅ | ✅ | ✅ | ✅ | ❌ | 🟡 | Columnar, ideal for analytics |
19+ | [ PostgreSQL] ( https://www.postgresql.org ) | Relational DB | ❌ (Server-based) | ✅ | ✅✅✅ | ✅✅ | ✅✅ | ✅✅✅ (ACID, MVCC) | ✅✅ (TSVector) | 🟡 | Full-featured RDBMS |
2020
2121### Legend
2222- ✅: Supported / Good
@@ -32,12 +32,12 @@ Computationally focused database using pluggable stores
3232
3333To add all modules:
3434``` scala
35- libraryDependencies += " com.outr" %% " lightdb-all" % " 4.12.0-SNAPSHOT "
35+ libraryDependencies += " com.outr" %% " lightdb-all" % " 4.12.0"
3636```
3737
3838For a specific implementation like Lucene:
3939``` scala
40- libraryDependencies += " com.outr" %% " lightdb-lucene" % " 4.12.0-SNAPSHOT "
40+ libraryDependencies += " com.outr" %% " lightdb-lucene" % " 4.12.0"
4141```
4242
4343## Videos
@@ -69,7 +69,7 @@ Ensure you have the following:
6969Add the following dependency to your ` build.sbt ` file:
7070
7171``` scala
72- libraryDependencies += " com.outr" %% " lightdb-all" % " 4.12.0-SNAPSHOT "
72+ libraryDependencies += " com.outr" %% " lightdb-all" % " 4.12.0"
7373```
7474
7575---
@@ -161,7 +161,7 @@ val adam = Person(name = "Adam", age = 21)
161161// city = None,
162162// nicknames = Set(),
163163// friends = List(),
164- // _id = StringId("zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK ")
164+ // _id = StringId("rdsWgq0lgl2jDHbivox9Vfz20zQ9Oe9L ")
165165// )
166166db.people.transaction { implicit txn =>
167167 txn.insert(adam)
@@ -172,7 +172,7 @@ db.people.transaction { implicit txn =>
172172// city = None,
173173// nicknames = Set(),
174174// friends = List(),
175- // _id = StringId("zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK ")
175+ // _id = StringId("rdsWgq0lgl2jDHbivox9Vfz20zQ9Oe9L ")
176176// )
177177```
178178
@@ -186,7 +186,7 @@ db.people.transaction { txn =>
186186 println(s " People in their 20s: $peopleIn20s" )
187187 }
188188}.sync()
189- // People in their 20s: List(Person(Adam,21,None,Set(),List(),StringId(IDmTU51mzoBQCEyaxBuHrwtLEcmHTags)), Person(Adam,21,None,Set(),List(),StringId(KGrBn5aofL4Nr9U3rhfv3dFHFiZQLBBp)), Person(Adam,21,None,Set(),List(),StringId(zKsjLb0Oh67NU7cXuCqefzuYqEkLNYou)), Person(Adam,21,None,Set(),List(),StringId(YtDDj7Lf0ys2sVAl5KbaGwYX1cRJdV41)), Person(Adam,21,None,Set(),List(),StringId(JzoJoBINhzejipsrAYzdaUGVvlxEFW5g)), Person(Adam,21,None,Set(),List(),StringId(5o9UsGhDtjTKVOLvHZCg0Y9CYjoh5g7C)), Person(Adam,21,None,Set(),List(),StringId(SpOTvdzPy3w302cWeQXRvtuVrJFDm13Z)), Person(Adam,21,None,Set(),List(),StringId(9WD5mBb0Y5IXtF2vuDa7fi8Y0pSw0Da0)), Person(Adam,21,None,Set(),List(),StringId(1gh7JtBVdDNqjihBDogvU4NNRGPJsXkb)), Person(Adam,21,None,Set(),List(),StringId(Xa6wUoSrdhjLP2vkbKyiyUjlyWBAz4kD)), Person(Adam,21,None,Set(),List(),StringId(iT74rK8QvkrRf6DrevkvgwQcHRgFuoUE)), Person(Adam,21,None,Set(),List(),StringId(QLvnBifleraDeNmCHkKeIPqyzhnib2Eg)), Person(Adam,21,None,Set(),List(),StringId(SJAjOvPNYLRg5wQ00zxEZUOUES7tCxcP)), Person(Adam,21,None,Set(),List(),StringId(zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK)))
189+ // People in their 20s: List(Person(Adam,21,None,Set(),List(),StringId(IDmTU51mzoBQCEyaxBuHrwtLEcmHTags)), Person(Adam,21,None,Set(),List(),StringId(KGrBn5aofL4Nr9U3rhfv3dFHFiZQLBBp)), Person(Adam,21,None,Set(),List(),StringId(zKsjLb0Oh67NU7cXuCqefzuYqEkLNYou)), Person(Adam,21,None,Set(),List(),StringId(YtDDj7Lf0ys2sVAl5KbaGwYX1cRJdV41)), Person(Adam,21,None,Set(),List(),StringId(JzoJoBINhzejipsrAYzdaUGVvlxEFW5g)), Person(Adam,21,None,Set(),List(),StringId(5o9UsGhDtjTKVOLvHZCg0Y9CYjoh5g7C)), Person(Adam,21,None,Set(),List(),StringId(SpOTvdzPy3w302cWeQXRvtuVrJFDm13Z)), Person(Adam,21,None,Set(),List(),StringId(9WD5mBb0Y5IXtF2vuDa7fi8Y0pSw0Da0)), Person(Adam,21,None,Set(),List(),StringId(1gh7JtBVdDNqjihBDogvU4NNRGPJsXkb)), Person(Adam,21,None,Set(),List(),StringId(Xa6wUoSrdhjLP2vkbKyiyUjlyWBAz4kD)), Person(Adam,21,None,Set(),List(),StringId(iT74rK8QvkrRf6DrevkvgwQcHRgFuoUE)), Person(Adam,21,None,Set(),List(),StringId(QLvnBifleraDeNmCHkKeIPqyzhnib2Eg)), Person(Adam,21,None,Set(),List(),StringId(SJAjOvPNYLRg5wQ00zxEZUOUES7tCxcP)), Person(Adam,21,None,Set(),List(),StringId(zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK)), Person(Adam,21,None,Set(),List(),StringId(rdsWgq0lgl2jDHbivox9Vfz20zQ9Oe9L)))
190190```
191191
192192---
@@ -226,7 +226,7 @@ db.people.transaction { txn =>
226226 println(s " Results: $results" )
227227 }
228228}.sync()
229- // Results: List(MaterializedAggregate({"ageMin": 21, "ageMax": 21, "ageAvg": 21.0, "ageSum": 294 },repl.MdocSession$MdocApp$Person$@184126b2 ))
229+ // Results: List(MaterializedAggregate({"ageMin": 21, "ageMax": 21, "ageAvg": 21.0, "ageSum": 315 },repl.MdocSession$MdocApp$Person$@788212e3 ))
230230```
231231
232232### Grouping
@@ -237,7 +237,7 @@ db.people.transaction { txn =>
237237 println(s " Grouped: $grouped" )
238238 }
239239}.sync()
240- // Grouped: List(Grouped(21,List(Person(Adam,21,None,Set(),List(),StringId(IDmTU51mzoBQCEyaxBuHrwtLEcmHTags)), Person(Adam,21,None,Set(),List(),StringId(KGrBn5aofL4Nr9U3rhfv3dFHFiZQLBBp)), Person(Adam,21,None,Set(),List(),StringId(zKsjLb0Oh67NU7cXuCqefzuYqEkLNYou)), Person(Adam,21,None,Set(),List(),StringId(YtDDj7Lf0ys2sVAl5KbaGwYX1cRJdV41)), Person(Adam,21,None,Set(),List(),StringId(JzoJoBINhzejipsrAYzdaUGVvlxEFW5g)), Person(Adam,21,None,Set(),List(),StringId(5o9UsGhDtjTKVOLvHZCg0Y9CYjoh5g7C)), Person(Adam,21,None,Set(),List(),StringId(SpOTvdzPy3w302cWeQXRvtuVrJFDm13Z)), Person(Adam,21,None,Set(),List(),StringId(9WD5mBb0Y5IXtF2vuDa7fi8Y0pSw0Da0)), Person(Adam,21,None,Set(),List(),StringId(1gh7JtBVdDNqjihBDogvU4NNRGPJsXkb)), Person(Adam,21,None,Set(),List(),StringId(Xa6wUoSrdhjLP2vkbKyiyUjlyWBAz4kD)), Person(Adam,21,None,Set(),List(),StringId(iT74rK8QvkrRf6DrevkvgwQcHRgFuoUE)), Person(Adam,21,None,Set(),List(),StringId(QLvnBifleraDeNmCHkKeIPqyzhnib2Eg)), Person(Adam,21,None,Set(),List(),StringId(SJAjOvPNYLRg5wQ00zxEZUOUES7tCxcP)), Person(Adam,21,None,Set(),List(),StringId(zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK)))))
240+ // Grouped: List(Grouped(21,List(Person(Adam,21,None,Set(),List(),StringId(IDmTU51mzoBQCEyaxBuHrwtLEcmHTags)), Person(Adam,21,None,Set(),List(),StringId(KGrBn5aofL4Nr9U3rhfv3dFHFiZQLBBp)), Person(Adam,21,None,Set(),List(),StringId(zKsjLb0Oh67NU7cXuCqefzuYqEkLNYou)), Person(Adam,21,None,Set(),List(),StringId(YtDDj7Lf0ys2sVAl5KbaGwYX1cRJdV41)), Person(Adam,21,None,Set(),List(),StringId(JzoJoBINhzejipsrAYzdaUGVvlxEFW5g)), Person(Adam,21,None,Set(),List(),StringId(5o9UsGhDtjTKVOLvHZCg0Y9CYjoh5g7C)), Person(Adam,21,None,Set(),List(),StringId(SpOTvdzPy3w302cWeQXRvtuVrJFDm13Z)), Person(Adam,21,None,Set(),List(),StringId(9WD5mBb0Y5IXtF2vuDa7fi8Y0pSw0Da0)), Person(Adam,21,None,Set(),List(),StringId(1gh7JtBVdDNqjihBDogvU4NNRGPJsXkb)), Person(Adam,21,None,Set(),List(),StringId(Xa6wUoSrdhjLP2vkbKyiyUjlyWBAz4kD)), Person(Adam,21,None,Set(),List(),StringId(iT74rK8QvkrRf6DrevkvgwQcHRgFuoUE)), Person(Adam,21,None,Set(),List(),StringId(QLvnBifleraDeNmCHkKeIPqyzhnib2Eg)), Person(Adam,21,None,Set(),List(),StringId(SJAjOvPNYLRg5wQ00zxEZUOUES7tCxcP)), Person(Adam,21,None,Set(),List(),StringId(zdX8DTpGZyn3MkGJhHaKnUz1cu9ZUdrK)), Person(Adam,21,None,Set(),List(),StringId(rdsWgq0lgl2jDHbivox9Vfz20zQ9Oe9L)))))
241241```
242242
243243---
@@ -251,14 +251,14 @@ import lightdb.backup._
251251import java .io .File
252252
253253DatabaseBackup .archive(db.stores, new File (" backup.zip" )).sync()
254- // res5: Int = 15
254+ // res5: Int = 16
255255```
256256
257257Restore from a backup:
258258
259259``` scala
260260DatabaseRestore .archive(db, new File (" backup.zip" )).sync()
261- // res6: Int = 15
261+ // res6: Int = 16
262262```
263263
264264---
@@ -328,7 +328,7 @@ luceneDb.init.sync()
328328luceneDb.notes.transaction(_.insert(Note (" the quick brown fox" ))).sync()
329329// res8: Note = Note(
330330// text = "the quick brown fox",
331- // _id = StringId("jG4vbbyuCaioTXX3GpJV8pxfkcVMbxez ")
331+ // _id = StringId("cGw4Y83JUCTcaJHV7GWboFJ5CPGD9rpO ")
332332// )
333333val hits = luceneDb.notes.transaction { txn =>
334334 txn.query.search.flatMap(_.list)
@@ -345,6 +345,10 @@ val hits = luceneDb.notes.transaction { txn =>
345345// Note(
346346// text = "the quick brown fox",
347347// _id = StringId("jG4vbbyuCaioTXX3GpJV8pxfkcVMbxez")
348+ // ),
349+ // Note(
350+ // text = "the quick brown fox",
351+ // _id = StringId("cGw4Y83JUCTcaJHV7GWboFJ5CPGD9rpO")
348352// )
349353// )
350354```
@@ -381,7 +385,7 @@ spatialDb.places.transaction(_.insert(Place("NYC", Point(40.7142, -74.0119)))).s
381385// res10: Place = Place(
382386// name = "NYC",
383387// loc = Point(latitude = 40.7142, longitude = -74.0119),
384- // _id = StringId("vPweysyUCPYjwqC4ubdkaaqCHaYJFwl1 ")
388+ // _id = StringId("D3aaqfudDRAsx19qhl3AvNYCm89MSaHY ")
385389// )
386390// Distance filters are supported on spatial-capable backends; example filter:
387391val nycFilter = Place .loc.distance(Point (40.7 , - 74.0 ), 5_000 .meters)
@@ -477,11 +481,11 @@ object shardDb extends LightDB {
477481}
478482
479483val tenantA = shardDb.shards(" tenantA" )
480- // tenantA: HashMapStore[TenantDoc, TenantDoc] = lightdb.store.hashmap.HashMapStore@159cebb8
484+ // tenantA: HashMapStore[TenantDoc, TenantDoc] = lightdb.store.hashmap.HashMapStore@41c671b5
481485tenantA.transaction(_.insert(TenantDoc (" hello" ))).sync()
482486// res12: TenantDoc = TenantDoc(
483487// value = "hello",
484- // _id = StringId("WHldMxQ6dxcjgHfafu92XV4qGuGpWks8 ")
488+ // _id = StringId("gdepDOioIra2Y4EvCfsmhcRbOn5z8kzq ")
485489// )
486490```
487491
@@ -502,8 +506,8 @@ cfgDb.init.sync()
502506val featureFlag = cfgDb.stored[Boolean ](" featureX" , default = false )
503507// featureFlag: StoredValue[Boolean] = StoredValue(
504508// key = "featureX",
505- // store = lightdb.store.hashmap.HashMapStore@42502db4 ,
506- // default = repl.MdocSession$MdocApp$$Lambda/0x0000000053769240@4d171618 ,
509+ // store = lightdb.store.hashmap.HashMapStore@3798aff8 ,
510+ // default = repl.MdocSession$MdocApp$$Lambda/0x00000000277a0c48@4c6134c1 ,
507511// persistence = Stored
508512// )
509513featureFlag.set(true ).sync()
@@ -538,7 +542,7 @@ sqlDb.init.sync()
538542sqlDb.rows.transaction(_.insert(Row (" hi sql" ))).sync()
539543// res16: Row = Row(
540544// value = "hi sql",
541- // _id = StringId("RuOBLLb87vBewnsUDQ1nqE6Cq2sXzULT ")
545+ // _id = StringId("02dClngmqh8sn71tyKLSUD3U6udsfMtn ")
542546// )
543547```
544548
0 commit comments