Skip to content

Add space-ratio option - #649

Merged
MaximPlusov merged 2 commits into
mainfrom
text-line-space-ratio
Jul 29, 2026
Merged

Add space-ratio option#649
MaximPlusov merged 2 commits into
mainfrom
text-line-space-ratio

Conversation

@LonelyMidoriya

@LonelyMidoriya LonelyMidoriya commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Summary by CodeRabbit

  • New Features
    • Added an optional space-ratio setting (CLI/JSON/Python: space_ratio) to control automatic whitespace insertion during PDF text parsing, using a threshold of space-ratio × font size (decimals supported). Default behavior is 0.17.
    • The configured ratio is applied consistently during text extraction, including parallel page processing.
  • CLI/Docs
    • Added --space-ratio with validation (must be finite and > 0).
    • Updated the documented default for --image-resolution from 288.0 to 144.0.

@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Adds an optional TextLine space ratio across Java configuration, CLI parsing, preprocessing, parallel processing, JSON option metadata, and Python conversion bindings. It also updates image-resolution documentation, dependency versioning, and CLI option coverage.

Changes

TextLine space ratio

Layer / File(s) Summary
Configuration and CLI parsing
java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java, java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java
Adds the ratio setting, CLI definition, numeric parsing, validation, and updated image-resolution description.
Python option propagation
options.json, python/opendataloader-pdf/src/opendataloader_pdf/cli_options_generated.py, python/opendataloader-pdf/src/opendataloader_pdf/convert_generated.py
Exports option metadata and allows Python convert() calls to pass the ratio to the CLI.
Processing and dynamic threshold
java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/DocumentProcessor.java, java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/TextLineProcessor.java
Applies the configured ratio during preprocessing, propagates it to worker threads, and uses it for text-line spacing thresholds.
Dependency and coverage alignment
java/pom.xml, verification/ci-verify.py
Updates the WCAG algorithms version and includes --space-ratio in CLI option coverage.
Estimated code review effort: 3 (Moderate) ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PythonConvert
  participant CLIOptions
  participant Config
  participant DocumentProcessor
  participant StaticContainers
  participant TextLineProcessor
  PythonConvert->>CLIOptions: Pass --space-ratio value
  CLIOptions->>Config: setSpaceRatio(double)
  DocumentProcessor->>Config: getSpaceRatio()
  DocumentProcessor->>StaticContainers: Set and propagate ratio
  TextLineProcessor->>StaticContainers: Read ratio
  TextLineProcessor->>TextLineProcessor: Compute spacing threshold
Loading

Possibly related PRs

Suggested reviewers: hnc-jglee

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change by introducing the new space-ratio option.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`:
- Around line 390-398: Update the NumberFormatException handler in CLIOptions to
pass the caught exception e as the cause when constructing
IllegalArgumentException, preserving the original parsing stack trace while
retaining the existing error message.

In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java`:
- Around line 394-396: Update Config.setTextLineSpaceRatio to reject null?
Preserve documented nullable behavior, but validate non-null values as finite
and within the documented ratio domain before storing them. In CLIOptions, route
parsed text-line-space ratios through this setter and convert validation
failures into a clear CLI parse error; apply the changes in Config.java:394-396
and CLIOptions.java:390-398.

In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/DocumentProcessor.java`:
- Around line 656-659: Update the per-page worker state propagation in
processDocument so propagateState.run() also copies the configured
textLineSpaceRatio from StaticContainers, alongside the other StaticContainers
and StaticLayoutContainers ThreadLocal values. Ensure workers use the ratio
captured from config rather than their own default or stale state.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: ec5c231c-9d41-4b7b-b413-6f3b45e87118

📥 Commits

Reviewing files that changed from the base of the PR and between bbe91a2 and dfba438.

⛔ Files ignored due to path filters (2)
  • node/opendataloader-pdf/src/cli-options.generated.ts is excluded by !**/*.generated.*
  • node/opendataloader-pdf/src/convert-options.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (6)
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/DocumentProcessor.java
  • options.json
  • python/opendataloader-pdf/src/opendataloader_pdf/cli_options_generated.py
  • python/opendataloader-pdf/src/opendataloader_pdf/convert_generated.py

Comment thread java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`:
- Around line 397-398: Update the IllegalArgumentException message in the CLI
option parsing logic to reference the registered --space-ratio flag instead of
--text-line-space-ratio, while preserving the existing validation and exception
cause.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0596e800-3a38-4ee1-91f3-37ec36a15925

📥 Commits

Reviewing files that changed from the base of the PR and between dfba438 and d64d3ae.

⛔ Files ignored due to path filters (2)
  • node/opendataloader-pdf/src/cli-options.generated.ts is excluded by !**/*.generated.*
  • node/opendataloader-pdf/src/convert-options.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (7)
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/DocumentProcessor.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/TextLineProcessor.java
  • options.json
  • python/opendataloader-pdf/src/opendataloader_pdf/cli_options_generated.py
  • python/opendataloader-pdf/src/opendataloader_pdf/convert_generated.py

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java (2)

284-284: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Align the exported default with the documented default.

This definition exports default: null, while SPACE_RATIO_DESC documents 0.17 as the default. Generated option metadata and bindings can therefore advertise an unset value. Either export "0.17" or explicitly document that null means “use the processor default of 0.17.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`
at line 284, The SPACE_RATIO_LONG_OPTION definition currently exports a null
default while SPACE_RATIO_DESC documents 0.17. Update this OptionDefinition to
export "0.17" as the default, keeping the generated metadata aligned with the
documented processor behavior.

392-400: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Reject non-finite and negative ratios.

Double.parseDouble accepts NaN, infinities, and negative values. These produce invalid spacing thresholds and can silently disable or over-insert spaces. Validate that the parsed value is finite and non-negative before storing it, and add coverage for these inputs.

Proposed validation
 Double spaceRatio = Double.parseDouble(commandLine.getOptionValue(SPACE_RATIO_LONG_OPTION));
+if (!Double.isFinite(spaceRatio) || spaceRatio < 0.0) {
+    throw new IllegalArgumentException(
+            "Option --space-ratio requires a finite, non-negative value.");
+}
 config.setSpaceRatio(spaceRatio);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`
around lines 392 - 400, Update the space-ratio parsing in CLIOptions to reject
non-finite or negative values after Double.parseDouble and before
config.setSpaceRatio. Preserve the existing invalid-number exception behavior,
and add coverage for NaN, positive/negative infinity, and negative ratios.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`:
- Line 284: The SPACE_RATIO_LONG_OPTION definition currently exports a null
default while SPACE_RATIO_DESC documents 0.17. Update this OptionDefinition to
export "0.17" as the default, keeping the generated metadata aligned with the
documented processor behavior.
- Around line 392-400: Update the space-ratio parsing in CLIOptions to reject
non-finite or negative values after Double.parseDouble and before
config.setSpaceRatio. Preserve the existing invalid-number exception behavior,
and add coverage for NaN, positive/negative infinity, and negative ratios.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c3bf4049-b981-4359-994b-f60cfb2f702a

📥 Commits

Reviewing files that changed from the base of the PR and between d64d3ae and 87ad805.

📒 Files selected for processing (1)
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java

@MaximPlusov MaximPlusov changed the title Add text-line-space-ratio option Add space-ratio option Jul 26, 2026
@LonelyMidoriya
LonelyMidoriya force-pushed the text-line-space-ratio branch from 87ad805 to 5063241 Compare July 29, 2026 08:51

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java`:
- Around line 399-411: Add focused tests for the SPACE_RATIO_LONG_OPTION parsing
in CLIOptions, covering positive decimal values and rejection of zero, negative,
NaN, infinity, and malformed input. Include an end-to-end test that configures a
ratio and verifies it changes text-line spacing, using the existing CLI parsing
and PDF/text layout test utilities.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 50917309-4a26-4a75-b95a-5e7fd591d101

📥 Commits

Reviewing files that changed from the base of the PR and between 87ad805 and 5063241.

⛔ Files ignored due to path filters (2)
  • node/opendataloader-pdf/src/cli-options.generated.ts is excluded by !**/*.generated.*
  • node/opendataloader-pdf/src/convert-options.generated.ts is excluded by !**/*.generated.*
📒 Files selected for processing (9)
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/Config.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/api/cli/CLIOptions.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/DocumentProcessor.java
  • java/opendataloader-pdf-core/src/main/java/org/opendataloader/pdf/processors/TextLineProcessor.java
  • java/pom.xml
  • options.json
  • python/opendataloader-pdf/src/opendataloader_pdf/cli_options_generated.py
  • python/opendataloader-pdf/src/opendataloader_pdf/convert_generated.py
  • verification/ci-verify.py

@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

@MaximPlusov
MaximPlusov merged commit e97bb90 into main Jul 29, 2026
9 of 10 checks passed
@MaximPlusov
MaximPlusov deleted the text-line-space-ratio branch July 29, 2026 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants