Limit API request size via Flask

[juhoinkinen]

• Add Flask's setting MAX_CONTENT_LENGTH to limit request payloads.
  • Use 20 MB value in production config by default.
  • Use 2 kB value in testing config (to avoid unnecessary memory consumption and faster runs in tests).
• Allow setting the value via env ANNIF_MAX_CONTENT_LENGTH and the value for the existing Flask's MAX_FORM_MEMORY_SIZE via env ANNIF_MAX_FORM_MEMORY_SIZE.
• Add 413 responses to OpenAPI spec.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.63%. Comparing base (3fb328b) to head (756540d).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #948   +/-   ##
=======================================
  Coverage   99.63%   99.63%           
=======================================
  Files         103      103           
  Lines        8242     8271   +29     
=======================================
+ Hits         8212     8241   +29     
  Misses         30       30           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR introduces request-size limiting for the Annif Flask/Connexion API by configuring Flask’s built-in request size controls and adding REST/OpenAPI tests to validate 413 responses for oversized payloads.

Changes:

• Add MAX_CONTENT_LENGTH (default 20,000,000 bytes) and make both MAX_CONTENT_LENGTH and MAX_FORM_MEMORY_SIZE configurable via ANNIF_MAX_CONTENT_LENGTH / ANNIF_MAX_FORM_MEMORY_SIZE.
• Add REST API tests asserting that oversized payloads are rejected with HTTP 413 and that smaller payloads still succeed.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
annif/default_config.py	Adds env-configurable Flask size limits (MAX_CONTENT_LENGTH, MAX_FORM_MEMORY_SIZE).
tests/test_openapi.py	Adds tests for 413 behavior when request payload exceeds the configured max size.

Suppressed comments (2)

tests/test_openapi.py:183

• This test currently builds a 5MB payload to assert the "within limit" case. You can keep the test much faster by monkeypatching MAX_CONTENT_LENGTH to a small value and using a small payload that stays under that value.

def test_rest_suggest_payload_within_max_content_length(app_client):
    """Test that requests within MAX_CONTENT_LENGTH limit are accepted."""
    # Create a payload well within the 20 MB limit (5 MB)
    moderate_text = "A" * 5_000_000
    data = {"text": moderate_text}
    req = app_client.post(
        "http://localhost:8000/v1/projects/dummy-fi/suggest",
        data=data,
    )

tests/test_openapi.py:197

• These tests build very large request bodies (~21MB). To avoid making the test suite slow/heavy, set a small MAX_CONTENT_LENGTH via monkeypatch in the test and use small payloads that are just above/below the limit.

def test_rest_detect_language_payload_exceeds_max_content_length(app_client):
    """Test that detect-language requests exceeding MAX_CONTENT_LENGTH are rejected."""
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit (21 MB)
    large_text = "A" * 21_000_000
    data = {"text": large_text, "languages": ["en", "fi"]}
    req = app_client.post(
        "http://localhost:8000/v1/detect-language",
        json=data,
    )
    assert req.status_code == 413  # Request Entity Too Large

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (6)

tests/test_openapi.py:170

• This test constructs a very large JSON body (~21MB) to exceed MAX_CONTENT_LENGTH, which is unnecessarily expensive for unit tests. Consider lowering MAX_CONTENT_LENGTH within the test and using a small JSON body that still exceeds the configured limit.

def test_rest_suggest_batch_payload_exceeds_max_content_length(app_client):
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit (21 MB)
    large_text = "A" * 21_000_000
    data = {"documents": [{"text": large_text}]}
    req = app_client.post(
        "http://localhost:8000/v1/projects/dummy-fi/suggest-batch",
        json=data,
    )
    assert req.status_code == 413  # Request Entity Too Large

tests/test_openapi.py:182

• This test currently sends a ~5MB form payload, which is expensive and also tightly couples the test to the hard-coded 20,000,000-byte default. Lowering MAX_CONTENT_LENGTH inside the test lets you validate the same behavior with a much smaller payload.

def test_rest_suggest_payload_within_max_content_length(app_client):
    # Create a payload well within the 20 MB limit (5 MB)
    moderate_text = "A" * 5_000_000
    data = {"text": moderate_text}
    req = app_client.post(
        "http://localhost:8000/v1/projects/dummy-fi/suggest",
        data=data,
    )
    assert req.status_code == 200
    assert "results" in req.json()

tests/test_openapi.py:193

• Building a ~21MB request body here is unnecessarily heavy for a unit test; it can be replaced by temporarily lowering MAX_CONTENT_LENGTH and using a small payload over the limit to still assert a 413 response.

def test_rest_detect_language_payload_exceeds_max_content_length(app_client):
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit (21 MB)
    large_text = "A" * 21_000_000
    data = {"text": large_text, "languages": ["en", "fi"]}
    req = app_client.post(
        "http://localhost:8000/v1/detect-language",
        json=data,
    )
    assert req.status_code == 413  # Request Entity Too Large

tests/test_openapi.py:204

• This test’s behavior depends on the global MAX_CONTENT_LENGTH configuration, but it uses a fixed payload size. To keep the test fast and independent of the default limit, consider setting a small MAX_CONTENT_LENGTH within the test and using a correspondingly small payload under the limit.

def test_rest_detect_language_payload_within_max_content_length(app_client):
    small_text = "A" * 5_000
    data = {"text": small_text, "languages": ["en", "fi"]}
    req = app_client.post(
        "http://localhost:8000/v1/detect-language",
        json=data,
    )
    assert req.status_code == 200
    assert "results" in req.json()

tests/test_openapi.py:159

• These tests allocate multi-megabyte payloads (e.g., 21,000,000 characters) just to trigger MAX_CONTENT_LENGTH. This can make the test suite slow and memory-hungry; you can instead temporarily lower MAX_CONTENT_LENGTH for the duration of the test and use a small payload slightly above the limit.

This issue also appears in the following locations of the same file:

• line 162
• line 173
• line 185
• line 196

def test_rest_suggest_payload_exceeds_max_content_length(app_client):
    large_text = "A" * 21_000_000
    data = {"text": large_text}
    req = app_client.post(
        "http://localhost:8000/v1/projects/dummy-fi/suggest",
        data=data,
    )
    assert req.status_code == 413  # Request Entity Too Large

annif/default_config.py:22

• Casting ANNIF_MAX_FORM_MEMORY_SIZE / ANNIF_MAX_CONTENT_LENGTH directly with int() will raise a generic ValueError at import time if the env var is set but not a plain integer (e.g., empty string, '20MB', or '20_000_000'). Consider validating the env var and raising a clearer error message.

    MAX_FORM_MEMORY_SIZE = int(
        os.environ.get("ANNIF_MAX_FORM_MEMORY_SIZE", default=20_000_000)
    )
    MAX_CONTENT_LENGTH = int(
        os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=20_000_000)
    )

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (7)

tests/test_openapi.py:166

• This test's expected 413 depends on MAX_CONTENT_LENGTH being 1_000; an external ANNIF_MAX_CONTENT_LENGTH env var can change that and make the test fail. Set the limit explicitly here to avoid environment-dependent failures.

def test_rest_suggest_batch_payload_exceeds_max_content_length(app_client):
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit
    large_text = "A" * 2_000
    data = {"documents": [{"text": large_text}]}

tests/test_openapi.py:177

• This test assumes MAX_CONTENT_LENGTH is 1_000; if ANNIF_MAX_CONTENT_LENGTH is set in the environment, the request may be rejected (or not) unexpectedly. Set MAX_CONTENT_LENGTH explicitly to keep the assertion stable.

def test_rest_suggest_payload_within_max_content_length(app_client):
    # Create a payload well within the limit
    moderate_text = "A" * 500
    data = {"text": moderate_text}

tests/test_openapi.py:189

• This test's 413 expectation depends on MAX_CONTENT_LENGTH being 1_000; if ANNIF_MAX_CONTENT_LENGTH is set externally, it may no longer exceed the limit. Set MAX_CONTENT_LENGTH explicitly in the test to avoid environment coupling.

def test_rest_detect_language_payload_exceeds_max_content_length(app_client):
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit
    large_text = "A" * 2_000
    data = {"text": large_text, "languages": ["en", "fi"]}

tests/test_openapi.py:200

• This test expects the request to be accepted based on MAX_CONTENT_LENGTH=1_000, but ANNIF_MAX_CONTENT_LENGTH in the environment can make it fail unexpectedly. Set MAX_CONTENT_LENGTH explicitly to keep the test deterministic.

def test_rest_detect_language_payload_within_max_content_length(app_client):
    small_text = "A" * 500
    data = {"text": small_text, "languages": ["en", "fi"]}
    req = app_client.post(

annif/default_config.py:37

• Keep env var parsing consistent with the base Config and provide a clearer error for invalid ANNIF_MAX_CONTENT_LENGTH values by reusing the same parsing helper.

    MAX_CONTENT_LENGTH = int(os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=1_000))

tests/test_openapi.py:155

• These tests implicitly rely on TestingConfig's default MAX_CONTENT_LENGTH=1_000. If ANNIF_MAX_CONTENT_LENGTH is set in the environment, the limit will change and this test may no longer trigger 413. Set MAX_CONTENT_LENGTH explicitly in the test to keep it deterministic.

This issue also appears in the following locations of the same file:

• line 163
• line 174
• line 186
• line 197

def test_rest_suggest_payload_exceeds_max_content_length(app_client):
    # Create a payload that exceeds the MAX_CONTENT_LENGTH limit
    large_text = "A" * 2_000
    data = {"text": large_text}

annif/default_config.py:22

• Using int(os.environ.get(...)) here will raise a generic ValueError if the env var contains a non-integer (e.g. "20MB"), making startup failures harder to diagnose. Add a small parsing helper that raises a clearer error including the env var name/value.

This issue also appears on line 37 of the same file.

    MAX_FORM_MEMORY_SIZE = int(
        os.environ.get("ANNIF_MAX_FORM_MEMORY_SIZE", default=20_000_000)
    )
    MAX_CONTENT_LENGTH = int(
        os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=20_000_000)
    )

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

annif/openapi/annif.yaml:264

• In the /detect-language operation, the newly added 413 response is defined inline and uses an unquoted numeric status code key. Elsewhere in this spec, 4xx responses are typically referenced via components/responses and status code keys are strings; updating this block improves consistency and avoids YAML parsers treating codes as integers.

        413:
          description: Payload Too Large
          content:
            application/problem+json:
              schema:

annif/default_config.py:37

• TestingConfig reads ANNIF_MAX_CONTENT_LENGTH from the environment, which can make the test suite non-deterministic (tests in tests/test_openapi.py assume a 1kB limit). For stable tests, set a fixed MAX_CONTENT_LENGTH in TestingConfig and let other configs be env-driven.

    MAX_CONTENT_LENGTH = int(os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=1_000))

annif/default_config.py:22

• Parsing ANNIF_MAX_FORM_MEMORY_SIZE / ANNIF_MAX_CONTENT_LENGTH with int(...) will raise ValueError at import time if the env var contains a non-integer (e.g. "20MB"), preventing the app from starting. Consider a small helper that provides a clearer error message (or falls back to the default) when parsing fails.

    MAX_FORM_MEMORY_SIZE = int(
        os.environ.get("ANNIF_MAX_FORM_MEMORY_SIZE", default=20_000_000)
    )
    MAX_CONTENT_LENGTH = int(
        os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=20_000_000)
    )

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (3)

annif/default_config.py:22

• MAX_FORM_MEMORY_SIZE and MAX_CONTENT_LENGTH are parsed with int(...) at import time. If ANNIF_MAX_FORM_MEMORY_SIZE / ANNIF_MAX_CONTENT_LENGTH is set to a non-integer value, the app will crash with a generic ValueError during startup; consider raising a clearer error message that points to the misconfigured env var.

    MAX_FORM_MEMORY_SIZE = int(
        os.environ.get("ANNIF_MAX_FORM_MEMORY_SIZE", default=20_000_000)
    )
    MAX_CONTENT_LENGTH = int(
        os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=20_000_000)

annif/default_config.py:37

• TestingConfig still reads ANNIF_MAX_CONTENT_LENGTH from the external environment. If that env var is set (e.g. in a developer shell/CI), the new tests that expect 413 for ~3KB payloads may stop exercising the limit and become flaky. Consider hardcoding the test limit here and overriding it explicitly in any tests that need a different value.

    MAX_CONTENT_LENGTH = int(os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=2_000))

annif/openapi/annif.yaml:264

• In the /detect-language responses block, status code keys are currently unquoted integers and the new 413 response duplicates the Problem schema instead of reusing the new components/responses/PayloadTooLarge. This is inconsistent with the rest of the spec (which uses quoted status code strings and $ref responses) and can trip OpenAPI tooling that expects string keys.

        413:
          description: Payload Too Large
          content:
            application/problem+json:
              schema:

[osma]

LGTM, except for the one comment

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Suppressed comments (2)

annif/default_config.py:22

• Parsing ANNIF_MAX_FORM_MEMORY_SIZE / ANNIF_MAX_CONTENT_LENGTH with int(os.environ.get(...)) will raise a ValueError during import if the env var is set but empty or non-numeric, and the resulting error message is not very actionable. Consider a small helper that validates and raises a clearer message naming the offending variable.

    MAX_FORM_MEMORY_SIZE = int(
        os.environ.get("ANNIF_MAX_FORM_MEMORY_SIZE", default=20_000_000)
    )
    MAX_CONTENT_LENGTH = int(
        os.environ.get("ANNIF_MAX_CONTENT_LENGTH", default=20_000_000)
    )

annif/openapi/annif.yaml:265

• The 413 response for /detect-language is defined inline even though an equivalent reusable component response (components.responses.PayloadTooLarge) is also added below. Duplicating this payload schema/description in two places risks them drifting out of sync; consider referencing the component response here as well.

        413:
          description: Payload Too Large
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'