feat: Decouple audnexus from Audible-specific implementation to support multiple providers

[djdembeck]

GitHub Issue Draft: Provider Decoupling Architecture

Title

feat: Decouple audnexus from Audible-specific implementation to support multiple providers

Labels

• enhancement
• refactor
• architecture
• breaking-change (if migration required)

---

Description

Problem Statement

Currently, audnexus is tightly coupled to Audible as the sole data source. All code assumes Audible's data model, API structure, and identifier formats (ASIN). This prevents integration with other audiobook data providers.

Goal

Implement a provider abstraction layer that allows audnexus to fetch data from multiple sources (Audible, AudiobookGuild, etc.) while maintaining backward compatibility with existing API consumers.

Use Cases

1. Support audiobook data from sources other than Audible
2. Enable providers with different data retrieval methods (API vs scraping)
3. Allow gradual migration without breaking existing integrations
4. Provide fallback data sources when Audible data is unavailable

---

Proposed Solution

Provider Selection Strategy

• Query Parameter: /books/{id}?provider=audible
• Default: Audible (for backward compatibility)
• Invalid Provider: Returns 400 with VALIDATION_ERROR

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                        API Layer                             │
│  GET /books/{id}?provider=audiobookguild                   │
└─────────────────────────┬───────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                   Provider Registry                          │
│  Map provider name → Provider instance                       │
└─────────────────────────┬───────────────────────────────────┘
                          │
          ┌───────────────┴───────────────┐
          ▼                               ▼
┌─────────────────────┐       ┌─────────────────────┐
│  AudibleProvider    │       │ AudiobookGuild      │
│  (existing code)    │       │ Provider (new)      │
└──────────┬──────────┘       └──────────┬──────────┘
           │                             │
           ▼                             ▼
┌─────────────────────┐       ┌─────────────────────┐
│  ApiHelper          │       │  ScrapeHelper       │
│  ScrapeHelper       │       │  (new)              │
│  StitchHelper       │       └─────────────────────┘
└─────────────────────┘

Database Changes

Add provider field to all models:

// Book.ts, Author.ts, Chapter.ts
provider: types.string({
  enum: ['audible', 'audiobookguild'],
  required: true,
  default: 'audible'
})

Migration Required: Backfill provider='audible' for all existing records.

Files to Create/Modify

New Files:

• src/providers/types.ts - Provider interface definitions
• src/providers/registry.ts - Provider registry
• src/providers/index.ts - Provider exports
• src/providers/AudibleProvider.ts - Audible adapter
• src/providers/AudiobookGuildProvider.ts - New provider
• src/helpers/books/audiobookguild/ScrapeHelper.ts - AudiobookGuild scraper
• tests/providers/audiobookguild/ - Test suite
• scripts/migrate-add-provider-field.ts - Migration script

Modified Files:

• src/config/models/Book.ts - Add provider field
• src/config/models/Author.ts - Add provider field
• src/config/models/Chapter.ts - Add provider field
• src/helpers/database/papr/audible/*.ts - Add provider to queries
• src/helpers/routes/GenericShowHelper.ts - Use provider registry
• src/helpers/routes/RouteCommonHelper.ts - Validate provider param
• src/helpers/database/redis/RedisHelper.ts - Provider in cache keys
• src/helpers/utils/UpdateScheduler.ts - Provider-aware updates

---

Implementation Plan

Phase 1: Foundation (Week 1)

1. Add provider field to Book, Author, Chapter models
2. Create database migration script
3. Define Provider interface
4. Create ProviderRegistry

Phase 2: Audible Abstraction (Week 1-2)

1. Create AudibleProvider adapter
2. Update Papr helpers to include provider
3. Update route handlers with provider parameter
4. Ensure all existing tests pass

Phase 3: AudiobookGuild Implementation (Week 2-3)

1. Implement AudiobookGuild ScrapeHelper
2. Create AudiobookGuildProvider
3. Write comprehensive tests
4. Register in ProviderRegistry

Phase 4: Integration & Testing (Week 3-4)

1. Update UpdateScheduler for provider awareness
2. Update Redis cache key structure
3. Integration testing
4. Documentation updates

---

Acceptance Criteria

Functional Requirements

• Existing API calls without ?provider= parameter continue to work (backward compatible)
• Explicit ?provider=audible returns same data as no parameter
• ?provider=audiobookguild fetches from AudiobookGuild
• Invalid provider returns 400 with VALIDATION_ERROR
• Database includes provider field on all records
• Migration script successfully backfills existing records

Technical Requirements

• All existing tests pass (no regressions)
• New provider tests achieve 85%+ coverage
• Cache keys include provider context
• UpdateScheduler handles multiple providers
• Database queries include provider filter
• Compound index on (provider, asin, region) for uniqueness

Quality Requirements

• TypeScript compilation succeeds
• ESLint passes
• Code follows existing project conventions
• Commit messages follow conventional commit format

---

API Changes

New Query Parameter

Parameter	Type	Required	Default	Description
provider	string	No	audible	Data provider to use

Valid Providers

• audible - Audible API + scraping
• audiobookguild - AudiobookGuild scraping (new)

Example Requests

# Existing behavior (backward compatible)
GET /books/B017V4U2VQ

# Explicit Audible provider
GET /books/B017V4U2VQ?provider=audible

# AudiobookGuild provider
GET /books/12345?provider=audiobookguild

# Invalid provider (400 error)
GET /books/B017V4U2VQ?provider=invalid

---

Migration Guide

Database Migration

# Run migration script
npx ts-node scripts/migrate-add-provider-field.ts

# Verify migration
mongosh audnexus --eval "db.books.countDocuments({provider: {$exists: false}})"
# Expected: 0

Client Impact

No breaking changes for existing clients.

Clients not using the ?provider= parameter will continue to receive Audible data.

Clients wanting to use new providers should add ?provider=audiobookguild to requests.

---

Testing Strategy

Unit Tests

• Provider interface implementation
• ProviderRegistry lookup
• Database helpers with provider filter
• AudiobookGuild scraping logic

Integration Tests

• End-to-end provider selection
• Database migration verification
• Cache behavior with provider context
• UpdateScheduler with multiple providers

Regression Tests

• All existing tests must pass
• Backward compatibility verified
• Performance impact measured

---

Breaking Changes

None - This is designed to be fully backward compatible.

However, this sets the stage for a future v2.0 where:

• Provider parameter may become required
• Response format may change
• Default provider may be removed

---

Documentation Updates

• API documentation (docs/index.html)
• README.md - Add provider section
• AGENTS.md - Update architecture docs
• DEPENDENCY_UPDATE_LOG.md - N/A for this feature

---

Related Issues

N/A - This is a foundational architecture change.

---

Checklist

• Issue title follows conventional commit format
• Description clearly explains the problem and solution
• Acceptance criteria are testable
• Breaking changes documented
• Migration guide provided
• Labels applied correctly
• Assignees selected
• Milestone set

[binyaminyblatt]

i already am scraping audiobookguild and graphicaudio with scripts that update the dataset every night see
https://github.com/binyaminyblatt/audiobookguild
https://github.com/binyaminyblatt/graphicaudio_scraper
both pull the data from the sites and store it so even if its removed from the site it should be useable both store the data in a json file that can be downloaded and used however you want