You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(replication): remove all NATS/legacy clustering content
Strips the Legacy NATS Clustering section from clustering.md, removes
NATS-era callout blockquotes from overview.md, and drops version
annotations that only make sense in a migration-history context.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
<!-- Source: release-notes/v4-tucker/4.5.0.md (confirmed cluster status timing statistics in v4.5.0) -->
18
7
19
8
# Clustering
20
9
21
-
This page documents the Operations API for managing Harper's native replication system (available since v4.4.0). For an overview of how replication works, see [Replication Overview](./overview.md). For sharding configuration, see [Sharding](./sharding.md).
22
-
23
-
> **Legacy NATS Clustering**: The NATS-based clustering used in v4.0–4.3 is described in the [Legacy NATS Clustering](#legacy-nats-clustering) section below. The operations described here apply to the native replication system in v4.4.0+.
24
-
25
-
## Operations API
10
+
Operations API for managing Harper's replication system. For an overview of how replication works, see [Replication Overview](./overview.md). For sharding configuration, see [Sharding](./sharding.md).
26
11
27
12
All clustering operations require `super_user` role.
28
13
@@ -32,8 +17,6 @@ All clustering operations require `super_user` role.
32
17
33
18
Adds a new Harper instance to the cluster. If `subscriptions` are provided, it creates the specified replication relationships between the nodes. Without `subscriptions`, a fully replicating system is created (all data in all databases).
34
19
35
-
Added in: v4.4.0
36
-
37
20
**Parameters**:
38
21
39
22
-`operation`_(required)_ — must be `add_node`
@@ -80,8 +63,6 @@ Added in: v4.4.0
80
63
81
64
Modifies an existing Harper instance in the cluster. Will attempt to add the node if it does not exist.
82
65
83
-
Added in: v4.4.0
84
-
85
66
**Parameters**:
86
67
87
68
-`operation`_(required)_ — must be `update_node`
@@ -121,8 +102,6 @@ Added in: v4.4.0
121
102
122
103
Removes a Harper node from the cluster and stops all replication to and from that node.
Bulk creates or resets subscriptions for any number of remote nodes. **Resets and replaces any existing clustering setup.**
214
193
215
-
Added in: v4.4.0
216
-
217
194
**Parameters**:
218
195
219
196
-`operation`_(required)_ — must be `configure_cluster`
@@ -275,8 +252,6 @@ Added in: v4.4.0
275
252
276
253
Adds routes to the `replication.routes` configuration. Behaves as a PATCH/upsert — adds new routes while leaving existing routes untouched.
277
254
278
-
Added in: v4.4.0
279
-
280
255
**Parameters**:
281
256
282
257
-`operation`_(required)_ — must be `cluster_set_routes`
@@ -372,159 +347,3 @@ Removes routes from the Harper config file.
372
347
}
373
348
```
374
349
375
-
---
376
-
377
-
## Legacy NATS Clustering
378
-
379
-
> **Applies to v4.0–4.3 only.** Harper v4.4.0 replaced NATS clustering with the native replication system. The operations and configuration below are documented for legacy reference.
380
-
381
-
Harper 4.0–4.3 used a clustering system based on NATS. NATS clustering used a bi-directional pub/sub model on a per-table basis with eventual consistency. Individual transactions were sent in the order they were transacted, and once received by the destination instance, processed in an ACID-compliant manner. Conflict resolution used last-writer-wins based on recorded transaction time.
382
-
383
-
### Requirements
384
-
385
-
A cluster requires two or more Harper nodes (instances). A node is a single installation of Harper and can operate independently with clustering enabled or disabled.
386
-
387
-
### Enabling NATS Clustering
388
-
389
-
Clustering must be explicitly enabled. Set `clustering.enabled` to `true` in `harperdb-config.yaml`:
390
-
391
-
```yaml
392
-
clustering:
393
-
enabled: true
394
-
```
395
-
396
-
Or via the operations API (requires restart):
397
-
398
-
```json
399
-
{
400
-
"operation": "set_configuration",
401
-
"clustering_enabled": true
402
-
}
403
-
```
404
-
405
-
Or via CLI / environment variable:
406
-
407
-
```bash
408
-
harperdb --CLUSTERING_ENABLED true
409
-
# or
410
-
CLUSTERING_ENABLED=true
411
-
```
412
-
413
-
### Naming a Node
414
-
415
-
Each node must have a unique name in the cluster. The name cannot contain `.`, `,`, `*`, `>`, or whitespace.
416
-
417
-
```yaml
418
-
clustering:
419
-
nodeName: Node1
420
-
```
421
-
422
-
Or via the operations API:
423
-
424
-
```json
425
-
{
426
-
"operation": "set_configuration",
427
-
"clustering_nodeName": "Node1"
428
-
}
429
-
```
430
-
431
-
### Creating a Cluster User
432
-
433
-
Inter-node authentication in NATS clustering used a special `cluster_user` role. All clustered nodes must share the same cluster user credentials.
434
-
435
-
Create the user via the operations API:
436
-
437
-
```json
438
-
{
439
-
"operation": "add_user",
440
-
"role": "cluster_user",
441
-
"username": "cluster_account",
442
-
"password": "letsCluster123!",
443
-
"active": true
444
-
}
445
-
```
446
-
447
-
Then configure the cluster user in `harperdb-config.yaml`:
448
-
449
-
```yaml
450
-
clustering:
451
-
user: cluster_account
452
-
```
453
-
454
-
Or set both at install time:
455
-
456
-
```bash
457
-
harperdb install \
458
-
--CLUSTERING_ENABLED true \
459
-
--CLUSTERING_NODENAME Node1 \
460
-
--CLUSTERING_USER cluster_account \
461
-
--CLUSTERING_PASSWORD letsCluster123!
462
-
```
463
-
464
-
### Routes
465
-
466
-
A route is a connection between two nodes. Routes create a mesh network — you do not need to cross-connect all nodes. One route from any node to the cluster is sufficient for all nodes to reach each other.
467
-
468
-
Route configuration used the `clustering.hubServer.cluster.network.routes` key:
469
-
470
-
```yaml
471
-
clustering:
472
-
hubServer:
473
-
cluster:
474
-
network:
475
-
routes:
476
-
- host: 3.62.184.22
477
-
port: 9932
478
-
- host: 3.735.184.8
479
-
port: 9932
480
-
```
481
-
482
-
Routes could also be set via `cluster_set_routes`, `cluster_get_routes`, and `cluster_delete_routes` operations (same API as the native system, but with NATS-specific parameters).
483
-
484
-
### Subscriptions
485
-
486
-
In NATS clustering, subscriptions were the mechanism for controlling which tables replicated and in which direction. A subscription consisted of:
487
-
488
-
- `database`— the database the table belongs to
489
-
- `table`— the table name
490
-
- `publish`— if `true`, local transactions are replicated to the remote table
491
-
- `subscribe`— if `true`, remote transactions are replicated to the local table
492
-
493
-
Subscriptions were managed with the `set_node_replication` operation:
494
-
495
-
```json
496
-
{
497
-
"operation": "set_node_replication",
498
-
"node_name": "Node2",
499
-
"subscriptions": [
500
-
{
501
-
"database": "data",
502
-
"table": "dog",
503
-
"publish": true,
504
-
"subscribe": true
505
-
}
506
-
]
507
-
}
508
-
```
509
-
510
-
### Certificate Management (NATS)
511
-
512
-
Harper generated self-signed certificates for NATS cluster connections by default, stored in `<ROOTPATH>/keys/`. For development:
513
-
514
-
```yaml
515
-
clustering:
516
-
tls:
517
-
certificate: ~/hdb/keys/certificate.pem
518
-
certificateAuthority: ~/hdb/keys/ca.pem
519
-
privateKey: ~/hdb/keys/privateKey.pem
520
-
insecure: true
521
-
verify: true
522
-
```
523
-
524
-
For production, use certificates from your own CA or a public CA with matching CNs. Certificates must have `Extended Key Usage` including both `TLS Web Server Authentication` and `TLS Web Client Authentication`.
525
-
526
-
### What Replicated
527
-
528
-
The same data operations that replicate in the native system also replicated in NATS: insert, update, upsert, delete, and bulk loads. Destructive operations (`drop_database`, `drop_table`, `drop_attribute`) and users/roles did not replicate.
529
-
530
-
NATS clustering had built-in resiliency for network interruptions — when reconnected, a catchup routine replayed missed transactions.
<!-- Source: release-notes/v4-tucker/4.5.0.md (confirmed cluster status improvements, default port change, certificate revocation, expanded sharding) -->
8
7
9
8
# Replication Overview
10
9
11
-
Added in: v4.4.0
12
-
13
10
Harper's replication system is designed to make distributed data replication fast and reliable across multiple nodes. You can build a distributed database that ensures high availability, disaster recovery, and data localization — all without complex setup. Nodes can be added or removed dynamically, you can choose which data to replicate, and you can monitor cluster health without jumping through hoops.
14
11
15
-
> **Legacy NATS Clustering**: Harper 4.0–4.3 used a NATS-based clustering system. In v4.4.0, Harper replaced this with a new native replication system (codenamed Plexus) that provides better performance, reliability, and data consistency. The legacy NATS system is documented in [Clustering](./clustering.md).
16
-
17
12
## Peer-to-Peer Model
18
13
19
14
Harper replication uses a peer-to-peer model where every node in your cluster can send data to and receive data from other nodes. Nodes communicate over WebSockets, allowing data to flow in both directions. Harper automatically manages these connections and subscriptions, so you don't need to manually track data consistency. Connections between nodes are secured and reliable by default.
@@ -43,7 +38,7 @@ replication:
43
38
port: 9933
44
39
```
45
40
46
-
By default, replication connects on the secure port `9933`. Added in: v4.5.0 (changed from 9925).
41
+
By default, replication connects on the secure port `9933`.
47
42
48
43
```yaml
49
44
replication:
@@ -289,7 +284,7 @@ The following data operations are replicated across the cluster:
289
284
290
285
Users and roles are not replicated across the cluster.
291
286
292
-
Added in: v4.4.0 (replicated operations): certain management operations — including component deployment and rolling restarts — can also be replicated across the cluster.
287
+
Certain management operations — including component deployment and rolling restarts — can also be replicated across the cluster.
0 commit comments