perldoc > Text::CSV_PP

man perldoc info pydoc3 ri search

📛 NAME

Text::CSV_PP — Text::CSV_XS compatible pure-Perl module

🚀 Quick Reference

Use Case	Command	Description
📥 Read CSV file as array of arrays	csv (in => "data.csv")	Read entire file into memory as AoA
📥 Read CSV as array of hashes	csv (in => "data.csv", headers => "auto")	First row becomes column names
📤 Write array of arrays to CSV	csv (in => $aoa, out => "file.csv")	Write with custom separator
🔍 Filter rows	csv (in => "data.csv", filter => { code => sub { $_ % 2 } })	Only rows where "code" is odd
📄 Object interface: parse line by line	$csv->getline ($fh)	Read one row into arrayref
📄 Object interface: write rows	$csv->say ($fh, \@row)	Write one row with eol

📋 SYNOPSIS

This section is taken from Text::CSV_XS.

# Functional interface
use Text::CSV_PP qw( csv );

# Read whole file in memory
my $aoa = csv (in => "data.csv");    # as array of array
my $aoh = csv (in => "data.csv",
               headers => "auto");   # as array of hash

# Write array of arrays as csv file
csv (in => $aoa, out => "file.csv", sep_char=> ";");

# Only show lines where "code" is odd
csv (in => "data.csv", filter => { code => sub { $_ % 2 }});

# Object interface
use Text::CSV_PP;

my @rows;
# Read/parse CSV
my $csv = Text::CSV_PP->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", "test.csv" or die "test.csv: $!";
while (my $row = $csv->getline ($fh)) {
    $row->[2] =~ m/pattern/ or next; # 3rd field should match
    push @rows, $row;
    }
close $fh;

# and write as CSV
open $fh, ">:encoding(utf8)", "new.csv" or die "new.csv: $!";
$csv->say ($fh, $_) for @rows;
close $fh or die "new.csv: $!";

📝 DESCRIPTION

Text::CSV_PP is a pure-perl module that provides facilities for the composition and decomposition of comma-separated values. This is (almost) compatible with much faster Text::CSV_XS, and mainly used as its fallback module when you use Text::CSV module without having installed Text::CSV_XS. If you don't have any reason to use this module directly, use Text::CSV for speed boost and portability (or maybe Text::CSV_XS when you write an one-off script and don't need to care about portability).

The following caveats are taken from the doc of Text::CSV_XS.

🔗 Embedded newlines

Important Note: The default behavior is to accept only ASCII characters in the range from 0x20 (space) to 0x7E (tilde). This means that the fields can not contain newlines. If your data contains newlines embedded in fields, or characters above 0x7E (tilde), or binary data, you must set "binary => 1" in the call to "new". To cover the widest range of parsing options, you will always want to set binary.

But you still have the problem that you have to pass a correct line to the "parse" method, which is more complicated from the usual point of usage:

my $csv = Text::CSV_PP->new ({ binary => 1, eol => $/ });
while (<>) {           #  WRONG!
    $csv->parse ($_);
    my @fields = $csv->fields ();
    }

this will break, as the "while" might read broken lines: it does not care about the quoting. If you need to support embedded newlines, the way to go is to not pass "eol" in the parser (it accepts "\n", "\r", and "\r\n" by default) and then

my $csv = Text::CSV_PP->new ({ binary => 1 });
open my $fh, "<", $file or die "$file: $!";
while (my $row = $csv->getline ($fh)) {
    my @fields = @$row;
    }

The old(er) way of using global file handles is still supported

while (my $row = $csv->getline (*ARGV)) { ... }

🌐 Unicode

Unicode is only tested to work with perl-5.8.2 and up.

See also "BOM".

The simplest way to ensure the correct encoding is used for in- and output is by either setting layers on the filehandles, or setting the "encoding" argument for "csv".

open my $fh, "<:encoding(UTF-8)", "in.csv"  or die "in.csv: $!";
or
my $aoa = csv (in => "in.csv",     encoding => "UTF-8");

open my $fh, ">:encoding(UTF-8)", "out.csv" or die "out.csv: $!";
or
csv (in => $aoa, out => "out.csv", encoding => "UTF-8");

On parsing (both for "getline" and "parse"), if the source is marked being UTF8, then all fields that are marked binary will also be marked UTF8.

On combining ("print" and "combine"): if any of the combining fields was marked UTF8, the resulting string will be marked as UTF8. Note however that all fields before the first field marked UTF8 and contained 8-bit characters that were not upgraded to UTF8, these will be "bytes" in the resulting string too, possibly causing unexpected errors. If you pass data of different encoding, or you don't know if there is different encoding, force it to be upgraded before you pass them on:

$csv->print ($fh, [ map { utf8::upgrade (my $x = $_); $x } @data ]);

For complete control over encoding, please use Text::CSV::Encoded:

use Text::CSV::Encoded;
my $csv = Text::CSV::Encoded->new ({
    encoding_in  => "iso-8859-1", # the encoding comes into   Perl
    encoding_out => "cp1252",     # the encoding comes out of Perl
    });

$csv = Text::CSV::Encoded->new ({ encoding  => "utf8" });
# combine () and print () accept *literally* utf8 encoded data
# parse () and getline () return *literally* utf8 encoded data

$csv = Text::CSV::Encoded->new ({ encoding  => undef }); # default
# combine () and print () accept UTF8 marked data
# parse () and getline () return UTF8 marked data

📖 BOM

BOM (or Byte Order Mark) handling is available only inside the "header" method. This method supports the following encodings: "utf-8", "utf-1", "utf-32be", "utf-32le", "utf-16be", "utf-16le", "utf-ebcdic", "scsu", "bocu-1", and "gb-18030". See Wikipedia https://en.wikipedia.org/wiki/Byte_order_mark.

If a file has a BOM, the easiest way to deal with that is

my $aoh = csv (in => $file, detect_bom => 1);

All records will be encoded based on the detected BOM.

This implies a call to the "header" method, which defaults to also set the "column_names". So this is not the same as

my $aoh = csv (in => $file, headers => "auto");

which only reads the first record to set "column_names" but ignores any meaning of possible present BOM.

⚙️ METHODS

This section is also taken from Text::CSV_XS.

📦 version

(Class method) Returns the current module version.

🆕 new

(Class method) Returns a new instance of class Text::CSV_PP. The attributes are described by the (optional) hash ref "\%attr".

my $csv = Text::CSV_PP->new ({ attributes ... });

The following attributes are available:

🔧 eol

my $csv = Text::CSV_PP->new ({ eol => $/ });
          $csv->eol (undef);
my $eol = $csv->eol;

The end-of-line string to add to rows for "print" or the record separator for "getline".

🔧 sep_char

my $csv = Text::CSV_PP->new ({ sep_char => ";" });
        $csv->sep_char (";");
my $c = $csv->sep_char;

The char used to separate fields, by default a comma (","). Limited to a single-byte character.

🔧 sep

my $csv = Text::CSV_PP->new ({ sep => "\N{FULLWIDTH COMMA}" });
          $csv->sep (";");
my $sep = $csv->sep;

The chars used to separate fields, by default undefined. Limited to 8 bytes.

🔧 quote_char

my $csv = Text::CSV_PP->new ({ quote_char => "'" });
        $csv->quote_char (undef);
my $c = $csv->quote_char;

The character to quote fields containing blanks or binary data, by default the double quote character (""").

🔧 quote

my $csv = Text::CSV_PP->new ({ quote => "\N{FULLWIDTH QUOTATION MARK}" });
            $csv->quote ("'");
my $quote = $csv->quote;

The chars used to quote fields, by default undefined. Limited to 8 bytes.

🔧 escape_char

my $csv = Text::CSV_PP->new ({ escape_char => "\\" });
        $csv->escape_char (":");
my $c = $csv->escape_char;

The character to escape certain characters inside quoted fields.

🔧 binary

my $csv = Text::CSV_PP->new ({ binary => 1 });
        $csv->binary (0);
my $f = $csv->binary;

If 1, allows binary characters in quoted fields including line feeds and NULL bytes.

🔧 strict

my $csv = Text::CSV_PP->new ({ strict => 1 });
        $csv->strict (0);
my $f = $csv->strict;

If 1, inconsistent number of fields causes error 2014.

🔧 skip_empty_rows

my $csv = Text::CSV_PP->new ({ skip_empty_rows => 1 });
        $csv->skip_empty_rows (0);
my $f = $csv->skip_empty_rows;

If 1, rows with only EOL are skipped (parsing only).

🔧 formula_handling

my $csv = Text::CSV_PP->new ({ formula => "none" });
        $csv->formula ("none");
my $f = $csv->formula;

Defines behavior for fields starting with '=' (formulas). Values: none, die, croak, diag, empty, undef, callback.

🔧 decode_utf8

my $csv = Text::CSV_PP->new ({ decode_utf8 => 1 });
        $csv->decode_utf8 (0);
my $f = $csv->decode_utf8;

If true, valid UTF-8 fields are automatically upgraded to UTF-8.

🔧 auto_diag

my $csv = Text::CSV_PP->new ({ auto_diag => 1 });
        $csv->auto_diag (2);
my $l = $csv->auto_diag;

Set to 1-9 to automatically call error_diag on errors: 1 = warn, >1 = die.

🔧 diag_verbose

my $csv = Text::CSV_PP->new ({ diag_verbose => 1 });
        $csv->diag_verbose (2);
my $l = $csv->diag_verbose;

Set verbosity of auto_diag output.

🔧 blank_is_undef

my $csv = Text::CSV_PP->new ({ blank_is_undef => 1 });
        $csv->blank_is_undef (0);
my $f = $csv->blank_is_undef;

Unquoted empty fields become undef.

🔧 empty_is_undef

my $csv = Text::CSV_PP->new ({ empty_is_undef => 1 });
        $csv->empty_is_undef (0);
my $f = $csv->empty_is_undef;

All empty fields become undef (including quoted empty).

🔧 allow_whitespace

my $csv = Text::CSV_PP->new ({ allow_whitespace => 1 });
        $csv->allow_whitespace (0);
my $f = $csv->allow_whitespace;

Whitespace around separator is removed when parsing.

🔧 allow_loose_quotes

my $csv = Text::CSV_PP->new ({ allow_loose_quotes => 1 });
        $csv->allow_loose_quotes (0);
my $f = $csv->allow_loose_quotes;

Allow unescaped quotes inside quoted fields.

🔧 allow_loose_escapes

my $csv = Text::CSV_PP->new ({ allow_loose_escapes => 1 });
        $csv->allow_loose_escapes (0);
my $f = $csv->allow_loose_escapes;

Allow escaping characters that don't need escaping.

🔧 allow_unquoted_escape

my $csv = Text::CSV_PP->new ({ allow_unquoted_escape => 1 });
        $csv->allow_unquoted_escape (0);
my $f = $csv->allow_unquoted_escape;

Allow escape_char in first position of an unquoted field.

🔧 always_quote

my $csv = Text::CSV_PP->new ({ always_quote => 1 });
        $csv->always_quote (0);
my $f = $csv->always_quote;

All defined fields are quoted.

🔧 quote_space

my $csv = Text::CSV_PP->new ({ quote_space => 1 });
        $csv->quote_space (0);
my $f = $csv->quote_space;

Trigger quotation on space in field (default true).

🔧 quote_empty

my $csv = Text::CSV_PP->new ({ quote_empty => 1 });
        $csv->quote_empty (0);
my $f = $csv->quote_empty;

Empty defined fields are quoted.

🔧 quote_binary

my $csv = Text::CSV_PP->new ({ quote_binary => 1 });
        $csv->quote_binary (0);
my $f = $csv->quote_binary;

Disable quotation trigger for bytes >= 0x7F.

🔧 escape_null

my $csv = Text::CSV_PP->new ({ escape_null => 1 });
        $csv->escape_null (0);
my $f = $csv->escape_null;

Escape NULL bytes (default true).

🔧 keep_meta_info

my $csv = Text::CSV_PP->new ({ keep_meta_info => 1 });
        $csv->keep_meta_info (0);
my $f = $csv->keep_meta_info;

Preserve quotation and binary flags for fields.

🔧 undef_str

my $csv = Text::CSV_PP->new ({ undef_str => "\\N" });
        $csv->undef_str (undef);
my $s = $csv->undef_str;

Output string for undefined fields (generating only).

🔧 comment_str

my $csv = Text::CSV_PP->new ({ comment_str => "#" });
        $csv->comment_str (undef);
my $s = $csv->comment_str;

String that marks comment lines (parsing only).

🔧 verbatim

my $csv = Text::CSV_PP->new ({ verbatim => 1 });
        $csv->verbatim (0);
my $f = $csv->verbatim;

Treat newline and CR as ordinary binary characters.

🔧 types

Set of column types; passed to types method.

🔧 callbacks

See Callbacks section below.

🔧 accessors

Default values:

$csv = Text::CSV_PP->new ({
    eol                   => undef, # \r, \n, or \r\n
    sep_char              => ',',
    sep                   => undef,
    quote_char            => '"',
    quote                 => undef,
    escape_char           => '"',
    binary                => 0,
    decode_utf8           => 1,
    auto_diag             => 0,
    diag_verbose          => 0,
    blank_is_undef        => 0,
    empty_is_undef        => 0,
    allow_whitespace      => 0,
    allow_loose_quotes    => 0,
    allow_loose_escapes   => 0,
    allow_unquoted_escape => 0,
    always_quote          => 0,
    quote_empty           => 0,
    quote_space           => 1,
    escape_null           => 1,
    quote_binary          => 1,
    keep_meta_info        => 0,
    strict                => 0,
    skip_empty_rows       => 0,
    formula               => 0,
    verbatim              => 0,
    undef_str             => undef,
    comment_str           => undef,
    types                 => undef,
    callbacks             => undef,
    });

For all flags, an accessor method is available.

🔍 known_attributes

@attr = Text::CSV_PP->known_attributes;
@attr = Text::CSV_PP::known_attributes;
@attr = $csv->known_attributes;

Returns ordered list of supported attributes.

🖨️ print

$status = $csv->print ($fh, $colref);

Efficiently writes arrayref to filehandle. Does not create result string.

💬 say

$status = $csv->say ($fh, $colref);

Like print, but eol defaults to $\.

🖨️ print_hr

$csv->print_hr ($fh, $ref);

Print a hashref using column_names.

🔗 combine

$status = $csv->combine (@fields);

Construct a CSV record from fields.

📜 string

$line = $csv->string ();

Returns last input to parse or output of combine.

📥 getline

$colref = $csv->getline ($fh);

Parse one row from filehandle, returns arrayref.

📥 getline_all

$arrayref = $csv->getline_all ($fh);
$arrayref = $csv->getline_all ($fh, $offset);
$arrayref = $csv->getline_all ($fh, $offset, $length);

Returns list of all rows, with optional offset and length.

📥 getline_hr

$hr = $csv->getline_hr ($fh);

Returns row as hashref (requires column_names).

📥 getline_hr_all

$arrayref = $csv->getline_hr_all ($fh);
$arrayref = $csv->getline_hr_all ($fh, $offset);
$arrayref = $csv->getline_hr_all ($fh, $offset, $length);

Returns list of hashref rows.

🔍 parse

$status = $csv->parse ($line);

Decompose a CSV string into fields.

🧩 fragment

my $AoA = $csv->fragment ($fh, $spec);

RFC7111 URI fragment selection (row, col, cell).

📛 column_names

$csv->column_names (qw( code name price description ));

Set keys for getline_hr.

📋 header

my @hdr = $csv->header ($fh);
$csv->header ($fh, { sep_set => [ ";", ",", "|", "\t" ] });
$csv->header ($fh, { detect_bom => 1, munge_column_names => "lc" });

Parse header, auto-detect separator, set column_names and encoding.

🔗 bind_columns

$csv->bind_columns (\$code, \$name, \$price, \$description);

Bind scalars to fields for getline or print.

🏁 eof

$eof = $csv->eof ();

Returns true if last getline/parse hit EOF.

🔢 types

$csv->types (\@tref);

Set column types for decoding: IV, NV, PV.

📋 fields

@columns = $csv->fields ();

Returns last parsed fields or combine input.

📋 meta_info

@flags = $csv->meta_info ();

Returns flags for each field (quoted/binary).

❓ is_quoted

my $quoted = $csv->is_quoted ($column_idx);

True if field was quoted.

❓ is_binary

my $binary = $csv->is_binary ($column_idx);

True if field contained binary bytes.

❓ is_missing

my $missing = $csv->is_missing ($column_idx);

True if field was missing (getline_hr).

✅ status

$status = $csv->status ();

Returns success/failure of last combine or parse.

⚠️ error_input

$bad_argument = $csv->error_input ();

Returns erroneous argument from last combine/parse.

⚠️ error_diag

Text::CSV_PP->error_diag ();
$csv->error_diag ();
$error_code               = 0  + $csv->error_diag ();
$error_str                = "" . $csv->error_diag ();
($cde, $str, $pos, $rec, $fld) = $csv->error_diag ();

Diagnostics for last error.

🔢 record_number

$recno = $csv->record_number ();

Returns number of records parsed.

🔄 SetDiag

$csv->SetDiag (0);

Reset diagnostics.

🔧 FUNCTIONS

This section is also taken from Text::CSV_XS.

📄 csv

This function is not exported by default and should be explicitly requested:

use Text::CSV_PP qw( csv );

High-level function for reading/writing CSV. Takes key-value pairs.

my $aoa = csv (in => "test.csv") or
    die Text::CSV_PP->error_diag;

Arguments: in, out, encoding, detect_bom, headers, munge_column_names, key, value, keep_headers, fragment, sep_set, set_column_names, callbacks, and CSV object attributes.

📥 in

Source: file name, filehandle, glob, scalar reference, or data structure.

📤 out

Destination: file name, filehandle, glob, scalar reference, or "skip".

🔤 encoding

Encoding for I/O, e.g. "UTF-8". Can be "auto" for BOM detection.

🔍 detect_bom

If true, invokes header method to detect BOM.

📋 headers

Accepts: "auto", "lc", "uc", "skip", arrayref, hashref, coderef.

⚙️ munge_column_names

Modify column names: "lc", "uc", "db", "none", hash, callback.

🔑 key

Field name to use as hash key; returns hash of hashes.

🔢 value

Field(s) to use as value when key is set.

📑 keep_headers

Keep column names in an arrayref.

🧩 fragment

Only output fragment per RFC7111.

🔧 sep_set

Set of possible separators for header detection.

📛 set_column_names

If false, first row is data, not header.

📞 Callbacks

Callbacks enable actions triggered from inside Text::CSV_PP.

$csv->callbacks (error => sub { $csv->SetDiag (0) });

• error — invoked on error when auto_diag is true.
• after_parse — after parsing with getline, can modify row or return \"skip\".
• before_print — before printing with print.
• filter — filter records by field conditions (hashref or coderef).
• after_in — after all records parsed, before returning.
• before_out — before printing each record.
• on_in — acts as after_in or before_out.

Additionally, the csv function supports callbacks: filter, after_in, before_out, on_in.

⚠️ DIAGNOSTICS

This section is also taken from Text::CSV_XS.

Still under construction ...

If an error occurs, $csv->error_diag can be used to get information. Error categories:

• INI — Initialization error or option conflict.
• ECR — Carriage-Return related parse error.
• EOF — End-Of-File related parse error.
• EIQ — Parse error inside quotation.
• EIF — Parse error inside field.
• ECB — Combine error.
• EHR — HashRef parse related error.

Complete list of error codes:

• 1001 — INI - sep_char is equal to quote_char or escape_char
• 1002 — INI - allow_whitespace with escape_char or quote_char SP or TAB
• 1003 — INI - \r or \n in main attr not allowed
• 1004 — INI - callbacks should be undef or a hashref
• 1005 — INI - EOL too long
• 1006 — INI - SEP too long
• 1007 — INI - QUOTE too long
• 1008 — INI - SEP undefined
• 1010 — INI - the header is empty
• 1011 — INI - the header contains more than one valid separator
• 1012 — INI - the header contains an empty field
• 1013 — INI - the header contains nun-unique fields
• 1014 — INI - header called on undefined stream
• 1500 — PRM - Invalid/unsupported argument(s)
• 1501 — PRM - The key attribute is passed as an unsupported type
• 1502 — PRM - The value attribute is passed without the key attribute
• 1503 — PRM - The value attribute is passed as an unsupported type
• 2010 — ECR - QUO char inside quotes followed by CR not part of EOL
• 2011 — ECR - Characters after end of quoted field
• 2012 — EOF - End of data in parsing input stream
• 2013 — INI - Specification error for fragments RFC7111
• 2014 — ENF - Inconsistent number of fields
• 2021 — EIQ - NL char inside quotes, binary off
• 2022 — EIQ - CR char inside quotes, binary off
• 2023 — EIQ - QUO character not allowed
• 2024 — EIQ - EOF cannot be escaped, not even inside quotes
• 2025 — EIQ - Loose unescaped escape
• 2026 — EIQ - Binary character inside quoted field, binary off
• 2027 — EIQ - Quoted field not terminated
• 2030 — EIF - NL char inside unquoted verbatim, binary off
• 2031 — EIF - CR char is first char of field, not part of EOL
• 2032 — EIF - CR char inside unquoted, not part of EOL
• 2034 — EIF - Loose unescaped quote
• 2035 — EIF - Escaped EOF in unquoted field
• 2036 — EIF - ESC error
• 2037 — EIF - Binary character in unquoted field, binary off
• 2110 — ECB - Binary character in Combine, binary off
• 2200 — EIO - print to IO failed. See errno
• 3001 — EHR - Unsupported syntax for column_names ()
• 3002 — EHR - getline_hr () called before column_names ()
• 3003 — EHR - bind_columns () and column_names () fields count mismatch
• 3004 — EHR - bind_columns () only accepts refs to scalars
• 3006 — EHR - bind_columns () did not pass enough refs for parsed fields
• 3007 — EHR - bind_columns needs refs to writable scalars
• 3008 — EHR - unexpected error in bound fields
• 3009 — EHR - print_hr () called before column_names ()
• 3010 — EHR - print_hr () called with invalid arguments

📚 SEE ALSO

Text::CSV_XS, Text::CSV

Older versions took many regexp from http://www.din.or.jp/~ohzaki/perl.htm

👤 AUTHOR

Kenichi Ishigaki, <ishigaki[at]cpan.org> Makamaka Hannyaharamitu, <makamaka[at]cpan.org>

Text::CSV_XS was written by <joe[at]ispsoft.de> and maintained by <h.m.brand[at]xs4all.nl>.

Text::CSV was written by <alan[at]mfgrtl.com>.

©️ COPYRIGHT AND LICENSE

Copyright 2017- by Kenichi Ishigaki, <ishigaki[at]cpan.org> Copyright 2005-2015 by Makamaka Hannyaharamitu, <makamaka[at]cpan.org>

Most of the code and doc is directly taken from the pure perl part of Text::CSV_XS.

Copyright (C) 2007-2016 H.Merijn Brand. All rights reserved. Copyright (C) 1998-2001 Jochen Wiedmann. All rights reserved. Copyright (C) 1997 Alan Citterman. All rights reserved.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

📛 NAME 🚀 Quick Reference 📋 SYNOPSIS 📝 DESCRIPTION ⚙️ METHODS 🔧 FUNCTIONS ⚠️ DIAGNOSTICS 📚 SEE ALSO 👤 AUTHOR ©️ COPYRIGHT AND LICENSE