Skip to content

Commit 9381fdc

Browse files
committed
Update docs with max retry option
1 parent 2b0a79f commit 9381fdc

3 files changed

Lines changed: 135 additions & 51 deletions

File tree

docs/diagram.md

Lines changed: 27 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,45 +3,63 @@
33
44
55
flowchart TD
6-
A[Service start] --> B[Connect to RabbitMQ]
6+
7+
A[Service start] --> A1[Validate options<br/>positive integers only]
8+
A1 --> B[Connect to RabbitMQ]
9+
710
B --> C[Register signal & process handlers]
8-
C --> D[Create consumer<br/>noAck=false<br/>requeue=false<br/>prefetch=2*concurrency]
9-
D --> E[Log initial queue depth]
11+
12+
C --> D[Create consumer<br/>noAck=false<br/>requeue=false<br/>prefetch=2*concurrency<br/>consumerTag=hostname.tag]
13+
14+
D --> E[Log queue depth if available]
1015
1116
E --> F[Message received]
1217
13-
F --> G{Valid JSON?}
18+
%% retry limit guard
19+
F --> R0{maxRetries configured?}
20+
R0 -- Yes --> R1[Read x-death count]
21+
R1 --> R2{Retries exceeded?}
22+
R2 -- Yes --> R3[ACK and DROP]
23+
R2 -- No --> G
24+
R0 -- No --> G
25+
26+
%% json parsing
27+
G{Valid JSON?}
1428
G -- No --> G1[NACK]
1529
G1 --> G2{Redelivered?}
1630
G2 -- No --> G3[REQUEUE]
1731
G2 -- Yes --> G4[DLQ / DROP]
1832
33+
%% schema validation
1934
G -- Yes --> H{Valid schema?}
2035
H -- No --> H1[NACK]
2136
H1 --> H2{Redelivered?}
2237
H2 -- No --> H3[REQUEUE]
2338
H2 -- Yes --> H4[DLQ / DROP]
2439
40+
%% normalization
2541
H -- Yes --> I{schema == action v2?}
26-
I -- Yes --> J[Normalize IDs<br/>Decrypt PII if enabled]
42+
I -- Yes --> J[Normalize IDs<br/>Decrypt PII if keystore]
2743
I -- No --> K[Skip normalization]
2844
29-
L[syncer call]
30-
J --> L
45+
%% business processing
46+
J --> L[syncer call]
3147
K --> L
3248
3349
L --> M{syncer result}
50+
3451
M -- true --> N[ACK]
52+
3553
M -- false --> O[NACK]
3654
O --> O1{Redelivered?}
3755
O1 -- No --> O2[REQUEUE]
3856
O1 -- Yes --> O3[DLQ / DROP]
3957
4058
M -- throws / non-boolean --> P[Fatal error]
4159
P --> Q[Log error]
42-
Q --> R[Process exit]
60+
Q --> S[Graceful shutdown]
4361
44-
R --> S[Shutdown handler]
62+
%% shutdown
4563
S --> T[Close consumer]
4664
T --> U[Close connection]
4765
U --> V[Exit]

docs/workflow.md

Lines changed: 108 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -3,88 +3,144 @@
33
## 1. Connection & Startup
44

55
1. The service connects to RabbitMQ using `connect(queueUrl)`
6-
2. Signal and process handlers are registered:
76

7+
2. Signal and process handlers are registered:
88
- `SIGINT`, `SIGTERM` → graceful shutdown
99
- `uncaughtException`, `unhandledRejection` → log and exit
1010

1111
3. A consumer is created with:
12-
1312
- `noAck: false`
1413
- `requeue: false`
1514
- bounded `concurrency`
1615
- `prefetch = 2 × concurrency`
16+
- consumer tag = `<hostname>.<tag || package name>`
1717

18-
On startup, the consumer logs the **current queue depth**.
18+
4. Consumer options are validated:
19+
- `concurrency`, `prefetch`, `maxRetries` must be **positive integers (> 0)**
20+
- invalid values throw at startup
21+
22+
On startup, the consumer logs the **queue depth if available** from consumer stats.
1923

2024
---
2125

2226
## 2. Message Intake
2327

2428
For each message received:
2529

26-
1. The message body is parsed as JSON
30+
### 2.1 Retry limit guard (`x-death` header)
31+
32+
If `maxRetries` is configured:
33+
34+
- retry count is read from `message.headers['x-death'][0].count`
35+
- if retry count **exceeds maxRetries**:
36+
37+
```
38+
ACK and drop (message considered permanently failed)
39+
```
40+
41+
No further processing occurs.
42+
43+
---
44+
45+
### 2.2 JSON parsing
2746

28-
- ❌ Invalid JSON → **NACK**
29-
- First failure → **requeue**
30-
- Redelivered → **dead-letter**
47+
The message body is parsed as JSON.
3148

32-
2. The message schema is validated
33-
Accepted schemas:
49+
- ❌ Invalid JSON → **NACK**
50+
- First failure → **requeue**
51+
- Redelivered → **dead-letter**
3452

35-
- `proca:action:2`
53+
---
3654

37-
- `proca:event:2`
55+
### 2.3 Schema validation
3856

39-
- ❌ Unknown schema → **NACK**
57+
Accepted schemas:
4058

41-
- First failure → **requeue**
59+
- `proca:action:2`
60+
- `proca:event:2`
4261

43-
- Redelivered → **dead-letter**
62+
If schema is unknown:
63+
64+
- ❌ Unknown schema → **NACK**
65+
- First failure → **requeue**
66+
- Redelivered → **dead-letter**
4467

4568
---
4669

4770
## 3. Message Normalization (Action v2 only)
4871

4972
If `schema === proca:action:2`:
5073

51-
- IDs are copied into their nested objects for convenience:
74+
### ID normalization
5275

53-
- `campaign.id`
54-
- `action.id`
55-
- `org.id`
56-
- `actionPage.id`
76+
IDs are copied into nested objects for convenience:
5777

58-
- If a keystore is configured:
78+
- `campaign.id`
79+
- `action.id`
80+
- `org.id`
81+
- `actionPage.id`
5982

60-
- `personalInfo` is decrypted
61-
- decrypted fields are merged into `contact`
83+
---
84+
85+
### Optional PII decryption
86+
87+
If:
88+
89+
- `personalInfo` exists
90+
- `keyStore` is configured
91+
92+
Then:
93+
94+
- `personalInfo` is decrypted
95+
- decrypted fields are merged into `contact`
96+
- decrypted values overwrite existing fields
6297

6398
No side effects occur at this stage.
6499

65100
---
66101

67102
## 4. Business Processing (`syncer`)
68103

69-
The normalized message is passed to the user-provided `syncer(msg)`.
104+
The normalized message is passed to the user-provided:
105+
106+
```
107+
await syncer(msg)
108+
```
70109

71110
### Expected contract
72111

73-
The `syncer` **must** return a boolean:
112+
The `syncer` **must return a boolean**.
74113

75-
| syncer result | Consumer behavior |
76-
| ----------------------------- | --------------------------------------- |
77-
| `true` | **ACK** (message removed from queue) |
78-
| `false` | **NACK** (retry once, then dead-letter) |
79-
| throws or returns non-boolean | **Fatal error → process exit** |
114+
| syncer result | Consumer behavior |
115+
| ------------- | ------------------------------------- |
116+
| `true` | **ACK** (message removed) |
117+
| `false` | **NACK → requeue once → dead-letter** |
118+
| throws | **Fatal error → process exit** |
119+
| non-boolean | **Fatal error → process exit** |
80120

81121
---
82122

83123
## 5. Retry & Dead-Letter Strategy
84124

85-
- Messages are retried **at most once**
86-
- Retry detection uses `message.redelivered`
87-
- Second failure → **DROP** (dead-letter exchange)
125+
Two independent retry controls exist.
126+
127+
### 5.1 Consumer redelivery guard
128+
129+
- First failure → requeue
130+
- If message is redelivered again → **DROP → dead-letter**
131+
132+
This prevents infinite retry loops.
133+
134+
---
135+
136+
### 5.2 Retry limit (`maxRetries`)
137+
138+
If configured:
139+
140+
- retry count is read from RabbitMQ `x-death` header
141+
- when exceeded → **ACK and drop**
142+
143+
This is used when queues already implement retry pipelines via DLX.
88144

89145
---
90146

@@ -97,19 +153,25 @@ If the `syncer`:
97153

98154
Then:
99155

100-
1. The error is logged
101-
2. The process exits immediately
156+
1. Error is logged
157+
2. Consumer closes
158+
3. Connection closes
159+
4. Process exits immediately
160+
161+
This is treated as a **permanent system failure**, not a message failure.
102162

103163
---
104164

105165
## 7. Shutdown Behavior
106166

107167
On shutdown (`SIGINT`, `SIGTERM`, fatal error):
108168

109-
1. The consumer is closed
110-
2. The RabbitMQ connection is closed
169+
1. Consumer is closed
170+
2. RabbitMQ connection is closed
111171
3. Final ACK/NACK counters are logged
112-
4. The process exits
172+
4. Process exits
173+
174+
Shutdown is graceful unless the process crashes unexpectedly.
113175

114176
---
115177

@@ -121,10 +183,16 @@ The following counters are tracked in memory:
121183
- `count.ack` – successfully processed messages
122184
- `count.nack` – rejected / retried messages
123185

186+
Counters reset on process restart.
187+
124188
---
125189

126190
## Summary (TL;DR)
127191

128-
- **Good messages** → ACK
129-
- **Recoverable failures** → retry once → DLQ
130-
- **Bad code or corrupted data** → crash fast
192+
- Valid message + handler success → ACK
193+
- Recoverable failure → requeue once → DLQ
194+
- Retry limit exceeded → ACK and drop
195+
- Invalid message → requeue once → DLQ
196+
- Handler crash or contract violation → process exits immediately
197+
198+
---

src/queue.ts

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -139,8 +139,6 @@ export const syncQueue = async (
139139
if (maxRetries) {
140140
const deaths = message.headers?.['x-death']?.[0]?.count ?? 0;
141141

142-
console.log('x-death header:', deaths, 'headers', message.headers);
143-
144142
if (deaths > maxRetries) {
145143
console.error(
146144
`retry limit exceeded (${deaths} > ${maxRetries}) — ACK and drop`

0 commit comments

Comments
 (0)