Proposal: Opt-in modular DynamoDB high-level client capabilities

[iulianbudau]

Describe the feature

Today, the aws-sdk-js-v3 DynamoDB experience at the low-level API layer (@aws-sdk/client-dynamodb) is complemented by the attribute marshalling/unmarshalling via @aws-sdk/lib-dynamodb. However, there is still a large gap in official, request-centric “high-level” helpers that many DynamoDB users repeatedly re-implement or adopt third-party libraries for, including:

• building DynamoDB expressions (Condition/Filter/Update/Projection) safely
• standardizing Query/Scan pagination and cursor encoding
• handling BatchGet/BatchWrite retries for UnprocessedKeys / UnprocessedItems
• composing TransactWrite/TransactGet
• implementing optimistic locking (version-based concurrency control)
• defining safe partial update behaviors (ignore nulls, write-if-not-exists, immutables)
• providing common atomic update recipes (counters, sets, list ops)
• optional attribute generators (ids/timestamps/version fields) via explicit hooks

Feature request

Introduce an opt-in, modular set of DynamoDB high-level helper packages, similar to the existing @aws-sdk/lib-* packages, but strictly request-centric (no ORM / no entity lifecycle / no table modeling opinions). Tentative packages:

1. Expressions (conditions, projections, updates, keys)
2. Pagination (Query/Scan pagination + async iteration)
3. Batch operations (BatchGet/BatchWrite retry semantics + partial failure handling)
4. Transactions (thin wrapper/composer over transact APIs)
5. Optimistic locking (version-based concurrency control built on expressions)
6. Update behavior (predictable partial update semantics: ignore-nulls, write-if-not-exists)
7. Atomic operations (counters, sets, atomic update patterns)
8. Attribute generators (IDs, timestamps, versioning via explicit hooks)

Rollout approach

If maintainers agree the direction is valid, one possible incremental path:

1. Start with Expressions (foundation)
2. Add Pagination / Batch / Transactions (operational helpers)
3. Add Optimistic locking + Update behaviors + Atomic ops (recipes)
4. Attribute generators last (most opinionated; keep hook-based)

Use Case

Many DynamoDB users frequently hit repetitive boilerplate and correctness pitfalls when using DynamoDB directly:

1. Expression boilerplate + correctness risks

Building Condition/Update expressions involves manual token management (ExpressionAttributeNames/Values), reserved-word handling, merging expressions across layers, and avoiding token collisions.

Example (today, simplified):

await client.send(new UpdateItemCommand({
  TableName,
  Key,
  UpdateExpression: "SET #name = :name ADD #version :one",
  ConditionExpression: "attribute_exists(pk) AND #version = :expected",
  ExpressionAttributeNames: { "#name": "name", "#version": "version" },
  ExpressionAttributeValues: { ":name": { S: "Alice" }, ":one": { N: "1" }, ":expected": { N: "7" } }
}));

This is verbose, easy to get wrong, and teams tend to build internal utilities or adopt third-party solutions mainly for this.

2. Pagination patterns are re-implemented everywhere

Query/Scan loops around LastEvaluatedKey and (optionally) cursor encoding/decoding are repeated across codebases, often inconsistently.

3. Batch operations require careful retry handling

Correct BatchGetItem/BatchWriteItem flows require explicit handling of Unprocessed* (including backoff strategy and partial failure reporting). Many teams reimplement these loops.

4. Common DynamoDB “higher-level” patterns are not standardized

Optimistic locking, patch semantics, atomic counters/sets, and timestamps/ID/version hooks are extremely common. Third-party libraries provide these (often bundled with modeling opinions), but there is demand for an official, minimal, composable approach that stays close to the underlying DynamoDB API.

The goal is to reduce custom boilerplate and reliance on third-party modeling frameworks when users only want the mechanics (expressions/pagination/batch/transact/concurrency), while keeping v3’s principles: modularity, explicitness, and composability.

Proposed Solution

Shape

Add a set of opt-in helper packages that compile to plain DynamoDB Command inputs, without hiding intent:

• @aws-sdk/lib-dynamodb-expressions
• @aws-sdk/lib-dynamodb-pagination
• @aws-sdk/lib-dynamodb-batch
• @aws-sdk/lib-dynamodb-transact
• @aws-sdk/lib-dynamodb-optimistic-locking
• @aws-sdk/lib-dynamodb-update-behaviors
• @aws-sdk/lib-dynamodb-atomic
• @aws-sdk/lib-dynamodb-attribute-generators

Design constraints / boundaries

• Everything returns plain objects compatible with @aws-sdk/client-dynamodb commands (and ideally supports DynamoDBDocumentClient inputs too).
• No background retries except where explicitly requested (e.g., batchWriteAll() with a max retry policy).
• Shared internal utility for expression tokenization + safe merging (so modules compose cleanly).

Illustrative API sketches (non-binding)

Expressions (similar to .NET SDK ConditionExpressionBuilder and Go SDK Expression package)

const cond = condition().attribute("pk").exists().and("version").eq(expected);
const upd  = update().set("name", "Alice").add("version", 1);
const expr = mergeExpressions(cond, upd); // produces ExpressionAttribute* + expressions

Pagination

for await (const page of paginateQuery(client, input, { pageSize: 25, maxItems: 500 })) {
  // ...
}

Batch

const result = await batchWriteAll(client, input, { maxRetries: 5, backoff: "exponential" });
// returns processed + remaining unprocessed + stats

Transactions

const txInput = transactWrite(
  put({ "tableName", item, condition: condition().attribute("pk").notExists() }),
  update({ "tableName", key, update: upd, condition: cond })
);
await client.send(new TransactWriteItemsCommand(txInput));

Optimistic locking

const { updateInput } = optimisticUpdate({
  "tableName", key,
  expectedVersion,
  set: { name: "Alice" },
  versionAttribute: "version"
});
await client.send(new UpdateItemCommand(updateInput));

Update behaviors / atomic ops / attribute generators
These should remain “compile-only” helpers or explicit pre-write hooks (no models, no decorators).

Other Information

Community feedback requested

• Would you use these as official packages?
• Which of the modules are most valuable?
• Are any of these too opinionated for the AWS SDK scope?
• Should some be merged to reduce package sprawl (e.g., atomic ops inside expressions update builder)?

AWS JS SDK v3 module versions used

• @aws-sdk/client-dynamodb: 3.975.0
• @aws-sdk/lib-dynamodb: 3.975.0
• Node runtime: v20.20.0

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

v3.975.0

Environment details (OS name and version, etc.)