Skip to content

Latest commit

 

History

History
417 lines (282 loc) · 9.59 KB

File metadata and controls

417 lines (282 loc) · 9.59 KB

Language Choice & Project Control Documents


Programming Language Decision

Selected Language: Python

Python is the most appropriate and pragmatic choice for BlogTracker given the project’s constraints, ecosystem, and intended usage as an MCP server consumed by LLM agents.

Rationale

  • MCP ecosystem alignment

    • Most existing MCP servers and mcporter examples are Python-based
    • Python integrates cleanly with stdio-based MCP execution
  • Excellent content tooling

    • Mature libraries for RSS and Atom parsing
    • Reliable HTML extraction and readability-style cleaning
    • Strong text diffing and normalization capabilities
  • Stateful systems fit

    • SQLite integration is simple, stable, and well understood
    • Python favors explicit, readable state transitions
  • Agent-driven development

    • Python minimizes boilerplate, allowing LLMs to reason about code structure
    • Faster iteration cycles for spec-heavy, behavior-driven development
  • Performance characteristics

    • BlogTracker is I/O-bound, not CPU-bound
    • Deterministic correctness outweighs raw throughput

Explicit Non-Choices

  • Node.js: async complexity and weaker text diff tooling
  • Go: excessive verbosity for MCP iteration speed
  • Rust: unnecessary complexity for this problem domain
  • Java / .NET: high ceremony with no clear upside

Conclusion:
Python maximizes development velocity, correctness, and LLM comprehension while remaining production-stable.


ISSUES.md

MVP 1 — Core Tracking & State

Issue 1.1 — Repository Bootstrap

  • Initialize blogtracker repository
  • Define top-level structure
  • Add minimal README with scope and non-goals

Issue 1.2 — Fetch Layer

  • Implement HTTP fetch with ETag support
  • Implement Last-Modified handling
  • Add retry and backoff strategy
  • Normalize fetch metadata

Issue 1.3 — RSS Normalization

  • Parse RSS feeds into canonical items
  • Extract title, URL, timestamps, content
  • Normalize canonical URLs
  • Define item identity rules

Issue 1.4 — Atom Normalization

  • Parse Atom feeds
  • Map Atom entries to internal item format
  • Validate behavior across feed variants

Issue 1.5 — SQLite State Layer

  • Design SQLite schema
  • Implement migrations
  • Persist sources, items, revisions
  • Enforce revision chaining

MVP 2 — Diffing & Noise Control

Issue 2.1 — Content Hashing

  • Normalize content for hashing
  • Implement stable content hash generation
  • Detect unchanged revisions

Issue 2.2 — Revision Diffing

  • Compare current and previous revisions
  • Compute change ratio
  • Classify change type

Issue 2.3 — Noise Filtering

  • Define minor vs substantive thresholds
  • Collapse repeated updates
  • Implement noise_level parameter

Issue 2.4 — HTML Source Support

  • Fetch raw HTML pages
  • Extract readable content
  • Normalize HTML into item + revision

MVP 3 — MCP Server

Issue 3.1 — MCP Server Bootstrap

  • Create MCP entrypoint
  • Register tools
  • Add tool metadata schemas

Issue 3.2 — Source Management Tools

  • Implement sources.list
  • Implement sources.upsert

Issue 3.3 — Collection Tool

  • Implement collect with since/until
  • Support source filtering
  • Apply noise filtering

Issue 3.4 — Inspection Tools

  • Implement item.get
  • Implement item.diff
  • Limit payload size

MVP 4 — Digest & LLM Ergonomics

Issue 4.1 — Digest Tool

  • Implement digest by date or range
  • Support timezone handling
  • Cluster related events

Issue 4.2 — Output Stabilization

  • Finalize event schema
  • Ensure citation-safe outputs
  • Guarantee stable ordering

MVP 5 — OpenClaw Skill Packaging

Issue 5.1 — Skill Definition

  • Write SKILL.md
  • Document MCP usage patterns
  • Define expected prompts

Issue 5.2 — Examples & Distribution

  • Provide example source configs
  • Add mcporter setup examples
  • Validate end-to-end with OpenClaw

CLAUDE.md

Purpose

This document defines how LLM agents (Claude Code or similar) should work on the BlogTracker codebase.

The primary goals are:

  • Deterministic behavior
  • Low cognitive load
  • Minimal ambiguity for agents

Development Rules

  • Follow the roadmap and issues strictly
  • Do not introduce features outside the current issue
  • Avoid speculative or “nice to have” changes
  • Prefer simple, explicit logic over clever abstractions

Code Style

  • Write readable, linear code
  • Avoid excessive comments
  • Avoid deep inheritance
  • Prefer pure functions where possible
  • Keep modules small and focused

File Structure Expectations

  • fetch: HTTP fetching and caching only
  • normalize: RSS, Atom, HTML parsing only
  • store: SQLite schema and persistence logic
  • diff: content comparison and change classification
  • tools: MCP tool handlers only

Each directory must have a single responsibility.


MCP Tool Guidelines

  • Tools must be idempotent
  • Tools must not depend on implicit time
  • Tools must accept explicit inputs (since, until, date)
  • Outputs must be structured and bounded in size

State Handling

  • State is persisted in SQLite
  • No in-memory-only state assumptions
  • Revision history must never be overwritten

Error Handling

  • Fail explicitly and clearly
  • Do not hide errors
  • Do not retry silently inside MCP tools

Testing Expectations

  • Focus on behavior, not implementation
  • Validate diff correctness
  • Validate noise filtering
  • Validate MCP outputs for stability

ADDITIONS — Development Memory & Sprint Documentation Rules

This document extends the existing project control files (ROADMAP.md, ISSUES.md, CLAUDE.md) with execution-time memory and documentation rules.
These rules are mandatory for all LLM agents working on BlogTracker.


scratchpad.md — Persistent Working Notes

Purpose

scratchpad.md is a living memory file for BlogTracker.

It captures:

  • Important implementation decisions
  • Non-obvious constraints discovered during work
  • Trade-offs made and why
  • Edge cases that influenced design
  • Rejected approaches worth remembering

It is not a changelog and not a task list.


Rules for scratchpad.md

  • Append-only (do not rewrite history)
  • Write short, precise notes
  • Each entry must include:
    • Date
    • Context (issue or sprint)
    • Insight or decision
  • No verbose explanations
  • No speculative ideas

Example Entry Format

  • 2026-02-01 — Sprint 2.1
    Content hashing must normalize whitespace and HTML entities first; raw hashing caused false-positive updates on multiple RSS feeds.

When to Write to scratchpad.md

Write an entry when:

  • A bug reveals a hidden assumption
  • A design choice prevents future errors
  • A library behaves unexpectedly
  • A constraint is discovered that affects later work
  • A “this would have been painful later” realization happens

If the insight would save time for a future agent, it belongs here.


Granular Task Requirement (Enforced)

All tasks must be granular enough that a single LLM can complete them without ambiguity.

Task Granularity Rules

A task:

  • Must describe one clear outcome
  • Must not bundle multiple concerns
  • Must be verifiable by inspection or test
  • Must not require guessing intent

Bad:

  • “Implement diffing system”

Good:

  • “Normalize content text before hashing by collapsing whitespace”
  • “Compute change_ratio between current and previous revision”
  • “Classify revision as minor if change_ratio < threshold”

If a task feels “conceptual”, it must be split further.


Sprint Documentation Rules

Sprint Records Location

All sprint execution notes must be written to:

/docs/sprints/


File Naming Convention

Each sprint must have one primary document:

SPRINT__<SHORT_DESCRIPTION>.md

Examples:

  • SPRINT_1_Repository_Bootstrap.md
  • SPRINT_2_Diffing_and_Noise_Control.md
  • SPRINT_3_MCP_Tools.md

Sprint Document Purpose

Sprint documents record what actually happened, not plans.

They serve as:

  • Execution log
  • Context for future agents
  • Explanation of why the current state exists

Required Sprint Document Sections

Each sprint file must contain the following sections:

1. Sprint Scope

  • Issues addressed
  • Explicit non-goals

2. Completed Tasks

  • Bullet list of granular tasks completed
  • Reference issue numbers where applicable

3. Key Decisions

  • Short list of decisions made during the sprint
  • Link or reference to scratchpad entries if relevant

4. Deviations

  • Any deviation from ISSUES.md or ROADMAP.md
  • Reason for deviation

5. Follow-ups

  • Tasks intentionally deferred
  • Known limitations introduced

Sprint Writing Rules

  • Be factual, not narrative
  • No marketing language
  • No future speculation
  • No duplication of scratchpad content
  • If something is subtle or fragile, reference scratchpad.md

CLAUDE.md — Addendum

Add the following rules to CLAUDE.md:

Working Memory Rule

  • Use scratchpad.md for long-term insights
  • Use sprint documents for execution history
  • Do not overload commit messages with reasoning

Documentation Discipline

  • Every completed sprint must produce exactly one sprint document
  • Sprint documents are mandatory, not optional
  • Missing sprint documentation is considered incomplete work

Agent Behavior Constraints

  • Do not refactor outside the current issue
  • Do not collapse tasks for convenience
  • Do not invent abstractions without an issue reference
  • Prefer boring, explicit code over cleverness

Guiding Principle

LLMs should reason over changes.
Developers and agents should reason over documented decisions.

scratchpad.md remembers why.
Sprint docs remember what.
The code reflects how.