Async Tasks - Serverless Background Task Execution

A serverless AWS infrastructure that allows you to submit long-running tasks (code generation, research, data processing) that execute asynchronously in the cloud, with email notifications upon completion.

Why Use This?

• Work offline: Submit tasks before bed, get results by morning
• No laptop required: Tasks run in AWS Lambda even when your computer is off
• Cost-effective: Pay only for execution time (~$0.02 per task)
• Email notifications: Receive results with inline content and file attachments
• Simple CLI: Submit tasks with a single command

Architecture

┌─────────────┐
│   CLI Tool  │ ──HTTP──> API Gateway ──> Lambda: Dispatcher
└─────────────┘                                  │
                                                 ↓
                                            SQS Queue
                                                 │
                                                 ↓
                          ┌─────────> Lambda: Executor ────> Claude API
                          │                     │
                          │                     ↓
                    DynamoDB Table         S3 Bucket
                          │               (Results)
                          ↓
                   Lambda: Notifier ──> Amazon SES
                          │              (Email)
                          ↓
                    Your Inbox 📧

Components

Infrastructure (AWS SAM)

• API Gateway: HTTPS endpoint for task submission
• Lambda Functions: Dispatcher, Executor, Notifier
• SQS Queue: Reliable task delivery with retries
• DynamoDB: Task state tracking
• S3: Results storage
• SES: Email notifications

CLI Tool

• Submit tasks from your terminal
• Check task status
• List recent tasks
• Cancel running tasks

Prerequisites

• AWS Account with CLI configured
• AWS SAM CLI installed
• Anthropic API key (Claude)
• Verified email address in Amazon SES
• Node.js 20.x or higher

Quick Start

1. Deploy Infrastructure

cd infrastructure

# Store your Anthropic API key in AWS Secrets Manager
aws secretsmanager create-secret \
  --name anthropic-api-key \
  --secret-string "your-api-key-here" \
  --region us-east-1

# Verify your email in SES for notifications
aws ses verify-email-identity \
  --email-address [email protected] \
  --region us-east-1

# Deploy the stack
sam build
sam deploy --guided

During the guided deployment, you'll be asked for:

• Stack name (e.g., async-tasks)
• AWS Region (e.g., us-east-1)
• Notification email address
• Anthropic API key secret name

2. Install CLI Tool

cd cli
npm install -g .

3. Configure CLI

Create ~/.async-task-config.json:

{
  "apiEndpoint": "https://YOUR-API-ID.execute-api.us-east-1.amazonaws.com/prod",
  "tasksTable": "AsyncTasks",
  "region": "us-east-1"
}

(Replace YOUR-API-ID with the API endpoint from the SAM deployment outputs)

4. Submit Your First Task

async-task submit --type code --prompt "Write a function to calculate Fibonacci numbers"

You'll receive an email when the task completes!

Usage

Submit a Task

# Code generation
async-task submit --type code --prompt "Build a REST API with Express"

# Research task
async-task submit --type research --prompt "Best practices for AWS Lambda optimization"

Check Task Status

async-task status <task-id>

List Recent Tasks

async-task list

Cancel a Task

async-task cancel <task-id>

Task Types

• code: Code generation and programming tasks
• research: Research, analysis, and information gathering
• pkb-maintenance: Personal knowledge base operations (custom)
• scheduled: Recurring automation tasks

Email Notifications

When a task completes, you'll receive an email with:

1. Inline results: Full output displayed in the email body
2. File attachment: Downloadable file (task-xxxxx-result.txt)
3. S3 link: Presigned URL (valid for 24 hours)

Cost Estimate

Based on 500 tasks/month:

Service	Monthly Cost
AWS Lambda	$0.20
DynamoDB	$0.50
S3	$0.50
SQS	Free tier
SES	Free tier
API Gateway	Free tier
AWS Total	~$1.20
Claude API (500 tasks)	~$7.50
Grand Total	~$8.70/month

Configuration

Infrastructure Parameters

Edit infrastructure/template.yaml:

Parameters:
  AnthropicApiKeySecretName:
    Default: anthropic-api-key

  NotificationEmail:
    Default: [email protected]

Lambda Settings

• Executor timeout: 900 seconds (15 minutes max)
• Executor memory: 512 MB
• SQS visibility timeout: 900 seconds
• S3 lifecycle: Results deleted after 30 days

Security

• ✅ API key stored in AWS Secrets Manager
• ✅ S3 server-side encryption (AES-256)
• ✅ DynamoDB encryption at rest
• ✅ SQS encryption in transit
• ✅ IAM roles with least privilege
• ✅ No public access (authenticated CLI only)

Troubleshooting

Task failed with "credit balance too low"

Add credits at https://console.anthropic.com/settings/billing

Email not received

1. Check spam folder
2. Verify email in SES: aws ses get-identity-verification-attributes --identities [email protected] --region us-east-1
3. Check Lambda logs: aws logs tail /aws/lambda/AsyncTasks-Notifier --region us-east-1

Task stuck in "pending"

Check executor Lambda logs:

aws logs tail /aws/lambda/AsyncTasks-Executor --region us-east-1 --follow

Development

Project Structure

.
├── infrastructure/          # AWS SAM template
│   ├── template.yaml       # CloudFormation template
│   └── lambdas/
│       ├── dispatcher/     # Task validation & queueing
│       ├── executor/       # Task execution (Claude API)
│       └── notifier/       # Email notifications
└── cli/                    # CLI tool
    ├── src/
    │   ├── index.js
    │   └── commands/
    └── package.json

Local Testing

# Invoke Lambda locally
sam local invoke ExecutorFunction -e events/test-task.json

# Start local API
sam local start-api

Contributing

Contributions welcome! Please:

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

License

MIT License - See LICENSE file for details

Credits

Built with:

• AWS SAM
• Anthropic Claude API
• Node.js

---

Note: This system requires active AWS and Anthropic accounts. Costs may vary based on usage.