Handwriting recognition

[DavidMStraub]

This is an AI-generated design doc. **I evaluated the options and discarded Kraken. Although it's a great tool, it needs too much manual intervention to fit our use case.

---

Two new optional endpoints complement the existing Tesseract OCR endpoint
(POST /media/<handle>/ocr) for handwritten and historical documents.

---

1. Kraken HTR (POST /media/<handle>/handwriting)

Kraken is an open-source (Apache 2.0) HTR engine optimised
for historical and non-Latin scripts. It runs fully self-hosted.

Configuration

# config.py or environment-provided config
KRAKEN_MODELS = {
    "kurrent":  "10.5281/zenodo.10066165",   # Zenodo DOI — downloaded on first use
    "catmus":   "/models/catmus-medieval.mlmodel",  # local path
}
KRAKEN_DEFAULT_MODEL = "kurrent"  # optional; falls back to first key if unset

KRAKEN_MODELS unset or empty → feature disabled; GET /metadata/ reports
"handwriting": false; endpoint returns 503.

Model loading

• DOI value → downloaded via htrmopo (bundled with kraken) on first request,
  stored in ~/.cache/kraken/, then the loaded model object is kept in a
  module-level dict so subsequent requests incur no I/O cost.
• Path value → loaded directly from disk.
• Each model name is cached independently.

Query parameters

Parameter	Type	Default	Notes
model	string	KRAKEN_DEFAULT_MODEL	Must be a key in KRAKEN_MODELS
format	string	"text"	text / alto / page

New files

File	Purpose
gramps_webapi/api/resources/handwriting.py	Flask-RESTX resource, MediaHandwritingResource
(additions to) gramps_webapi/api/tasks.py	media_handwriting Celery task
(additions to) gramps_webapi/api/file.py	FileHandler.get_handwriting(model_name, output_format)

Core logic sketch

# file.py
_kraken_models: dict[str, Any] = {}

def get_handwriting(self, model_name: str, output_format: str) -> str:
    from kraken.lib import models as kraken_models
    from kraken import blla, rpred, serialization
    if model_name not in _kraken_models:
        spec = current_app.config["KRAKEN_MODELS"][model_name]
        _kraken_models[model_name] = kraken_models.load_any(
            _resolve_model(spec)  # download DOI or use path
        )
    model = _kraken_models[model_name]
    im = Image.open(self.path)
    seg = blla.segment(im, model=_seg_model())  # shared default seg model
    records = list(rpred.rpred(model, im, seg))
    if output_format == "text":
        return "\n".join(r.prediction for r in records)
    return serialization.serialize(records, image_name=self.path,
                                   image_size=im.size, template=output_format)

Metadata exposure

"handwriting": {
  "models": ["kurrent", "catmus"],
  "default": "kurrent"
}

Docker note

Kraken pins torch<=2.10,>=2.4.0. The existing Dockerfile.base installs
torch unpinned from the CPU index, which currently resolves to 2.11.0+cpu —
outside kraken's upper bound. pip therefore fetches torch==2.10.0 from the
default PyPI index, which is the CUDA build.

Fix: pin the CPU torch to a version that satisfies kraken's constraint before
installing kraken:

RUN pip install --no-cache-dir --index-url https://download.pytorch.org/whl/cpu "torch==2.10.0+cpu"
RUN pip install --no-cache-dir kraken

pip install kraken then finds a satisfying torch already present and skips it.

---

2. Transkribus Metagrapho (POST /media/<handle>/transkribus)

Transkribus is a commercial cloud HTR service
with high accuracy on handwritten historical documents. GDPR-compliant (EU-hosted).
Credit-based pricing.

Configuration

TRANSKRIBUS_USER     = "user@example.com"   # Transkribus account email
TRANSKRIBUS_PASSWORD = "secret"             # stored server-side only
TRANSKRIBUS_DEFAULT_MODEL = 38230           # integer model ID; default: HTR+ generic

TRANSKRIBUS_USER unset → feature disabled; endpoint returns 503.

Credentials are per-server (not per-tree): they represent an organisation's
Transkribus account and must not be exposed via GET /trees/-/config.

Authentication

OIDC Resource Owner Password Credentials flow — no client ID/secret:

POST https://account.transkribus.eu/auth/realms/Transkribus/protocol/openid-connect/token
  grant_type=password & username=… & password=… & client_id=Transkribus
→ { access_token, expires_in, refresh_token }

Token cached in-process; refreshed proactively before expiry.

Query parameters

Parameter	Type	Default	Notes
model_id	integer	TRANSKRIBUS_DEFAULT_MODEL	Transkribus HTR model ID
format	string	"text"	text / page / alto

New files

File	Purpose
gramps_webapi/api/resources/transkribus.py	Flask-RESTX resource, MediaTranskribusResource
gramps_webapi/api/transkribus.py	TranskribusClient (auth + submit + poll)
(additions to) gramps_webapi/api/tasks.py	media_transkribus Celery task

API flow

1. POST /processing/v2/processes with image + modelId → { processId }
2. Poll GET /processing/v2/processes/{processId} until status == "FINISHED"
3. Extract text / PAGE XML / ALTO from response

Metadata exposure

"transkribus": {
  "enabled": true,
  "default_model_id": 38230
}

---

Comparison

	Tesseract OCR	Kraken HTR	Transkribus
Endpoint	/ocr	/handwriting	/transkribus
Hosting	local	local	cloud (EU)
Cost	free	free	credits
Config key	TESSERACT_DATA_DIR	KRAKEN_MODELS	TRANSKRIBUS_USER/PASSWORD
Model selection	?lang=deu	?model=kurrent	?model_id=38230
Best for	printed text	historical scripts	handwriting, high accuracy
Output formats	text/hOCR/TSV	text/ALTO/PAGE	text/ALTO/PAGE