Skip to content

README.md

Latest commit

 

History

History
85 lines (64 loc) · 3.04 KB

README.md

File metadata and controls

85 lines (64 loc) · 3.04 KB
Raw

Wikipedia Stream

Real-time Wikipedia edit events via the Wikimedia EventStreams Server-Sent Events API.

No setup required — works out of the box.

Quick Start

uv run python examples/wikipedia_example.py

Expected output within a few seconds:

Connecting to Wikipedia edit stream…
Press Ctrl+C to stop.

08:55:00 | Green leafhopper                                   | +522 bytes | Uffda608
08:55:01 | Daniela Avanzini                                   | +102 bytes | ~2026-12581-95
08:55:01 | Dilane Bakwa                                       | -60 bytes  | ~2026-12355-66
08:55:04 | Daoist texts                                       | +1473 bytes | ~2026-12409-27
08:55:08 | Daniel K. Ludwig                                   | +0 bytes   | Barnej

Environment Variables

No API key is needed. The only optional variable is the User-Agent header, which Wikimedia's robot policy requires to be descriptive. A default is set automatically — override it if you want to identify your app:

# .env (optional)
WIKIPEDIA_USER_AGENT=my-hackathon-app/1.0 (contact@example.com)

Note: Omitting a User-Agent entirely results in a 403 with the message "Please set a user-agent and respect our robot policy." The client sets one automatically, so you only need this variable if you want a custom value.

Usage

from streams.wikipedia import WikipediaStream

stream = WikipediaStream()
await stream.start()

async for event in stream:
    edit = event.payload
    print(f"{edit.wiki}: {edit.title} ({edit.delta:+d} bytes) by {edit.user}")

Filter to English Wikipedia main articles only:

async for event in stream:
    edit = event.payload
    if edit.wiki == "enwiki" and edit.namespace == 0:
        print(edit.title)

WikiEdit Fields

Field Type Description
id int Revision ID
wiki str Wiki identifier (e.g. enwiki)
title str Page title
namespace int MediaWiki namespace (0 = articles)
user str Editor username or IP
is_bot bool Whether the editor is a bot
is_new_page bool Whether this created a new page
is_minor bool Whether marked as minor edit
comment str Edit summary
old_length int Page size before edit (bytes)
new_length int Page size after edit (bytes)
delta int Bytes added (negative = removed)
revision_url str Direct link to diff
timestamp datetime Edit time (UTC)
server_name str Hostname (e.g. en.wikipedia.org)
page_url str Canonical page URL

Notes

  • Events arrive within 1–2 seconds of the actual edit
  • Volume is roughly 3–10 edits/second across all Wikimedia wikis
  • Common wiki values: enwiki, dewiki, frwiki, commonswiki, wikidatawiki
  • Common namespace values: 0 = articles, 1 = Talk, 4 = Wikipedia, 10 = Template