Skip to content

fread doesn't manage to read without additional pointers #7366

Closed

Description

@arunsrinivasan
require(data.table) # 1.17.8, loaded on a clean R session (v4.4.2)
# Macbook pro 14 inch (2023) running Sequoia 15.7

fread('"this_that"\n"2025-01-01 00:00:01"') 
# Empty data.table (0 rows and 2 cols): V1,"this_that"

# Warning messages:
# 1: In fread("\"this_that\"\n\"2025-01-01 00:00:01\"") :
#   Found and resolved improper quoting in first 100 rows. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning.
# 2: In fread("\"this_that\"\n\"2025-01-01 00:00:01\"") :
#   Detected 1 column names but the data has 2 columns (i.e. invalid file). Added an extra default column name for the first column which is guessed to be row names or an index. Use setnames() afterwards if this guess is not correct, or fix the file write command that created the file to create a valid file.
# 3: In fread("\"this_that\"\n\"2025-01-01 00:00:01\"") :
#   Discarded single-line footer: <<"2025-01-01 00:00:01">>

fread('"this_that"\n"2025-01-01 00:00:01"', nrow=1L)
#              this_that
#                 <POSc>
# 1: 2025-01-01 00:00:01

fread('"this_that"\n"2025-01-01 00:00:01"', sep=",")
#              this_that
#                 <POSc>
# 1: 2025-01-01 00:00:01

The entire file has 1 column header and a row. Both are quoted. I feel like this should be read in correctly. There are no mismatches in quotes here. But fread fails unless additional help is provided via nrow or sep for example. I guess it's attempting to use sep = ' ', but don't understand how it returns an empty table even then...

Seems like a bug to me. It's possible this was introduced somewhere before 1.12.2 (from my logs, I seem to have created a patch to workaround this around that time but forgot to file an issue). I can still reproduce in latest CRAN.

Activity

  1. ben-schwen commented on Oct 16, 2025

    @ben-schwen
    Member

    Ty for the report Arun! Out of curiosity do you also have a longer/broader example where this might fail?

    We have a current rule that more lines win, but this is currently bound to having at least 2 columns.

  2. arunsrinivasan commented on Nov 6, 2025

    @arunsrinivasan
    MemberAuthor

    @ben-schwen thanks for the super-fast fix. Missed your message. Unfortunately no. I just had a process that failed and while investigating it, the user had written a file with just those lines in it... If I find something similar / have more info, I'll write back here.

    Thank you,
    Arun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions