Skip to content

fix: lone quote in skipped line no longer consumes rest of file - #634

Closed
LeonidasZhak wants to merge 2 commits into
tidyverse:mainfrom
LeonidasZhak:fix/skip-lone-quote-readr-1577
Closed

LeonidasZhak wants to merge 2 commits into
tidyverse:mainfrom
LeonidasZhak:fix/skip-lone-quote-readr-1577

Conversation

@LeonidasZhak

Copy link
Copy Markdown

Fixes tidyverse/readr#1577

Problem

When skip is used and a skipped line contains a lone (unclosed) quote character, find_next_non_quoted_newline sets in_quote = true and never resets it. This causes all subsequent newlines to be treated as embedded within a quoted field, making the skip consume the entire file.

# Before fix: returns empty tibble
write('"\na,b\n1,2', "quote.csv")
read_csv("quote.csv", skip = 1)
#> # A tibble: 0 × 0

Fix

In find_next_non_quoted_newline (src/utils.h), when EOF is reached while in_quote is still true (indicating an unclosed quote), fall back to finding the first newline from the original start position. This ensures a lone quote in a skipped line doesn't consume the rest of the file.

# After fix: returns expected data
write('"\na,b\n1,2', "quote.csv")
read_csv("quote.csv", skip = 1)
#> # A tibble: 1 × 2
#>       a     b
#>   <dbl> <dbl>
#> 1     1     2

Changes

  • src/utils.h: Add fallback logic in find_next_non_quoted_newline for unclosed quotes
  • tests/testthat/test-vroom.R: Add regression test

Testing

  • New test passes
  • All existing tests pass (1163 pass, 5 pre-existing Unicode path encoding failures on macOS)
  • Properly quoted fields with embedded newlines still work correctly

Add a 'Multi-byte characters' section to vroom_fwf() documentation
explaining that field positions are byte offsets, not character positions.
For multi-byte encodings like UTF-8, these differ.

Add a test documenting the current byte-based behavior with the degree
symbol (°), which is 1 character but 2 bytes in UTF-8.

Closes #622.
When  is used and a skipped line contains a lone (unclosed) quote,
 would set  and never
reset it, causing all subsequent newlines to be treated as embedded
within a quoted field. This made the skip consume the entire file.

The fix detects when EOF is reached while  is still true
(i.e., the quote was never closed) and falls back to finding the first
newline from the original start position.

Regression test for tidyverse/readr#1577.
@LeonidasZhak

Copy link
Copy Markdown
Author

I am withdrawing this PR as part of a broader cleanup of an oversized automated contribution batch. I am sorry for the review noise and the pressure this may have put on maintainers. Going forward I will keep contributions to this project much more limited and higher-signal, ideally only one or two focused PRs at a time. Thank you for maintaining the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

read_csv() gives empty table with double quote in skipped line

1 participant