std.csv

Implements functionality to read Comma Separated Values and its variants from an input range of dchar.

Comma Separated Values provide a simple means to transfer and store tabular data. It has been common for programs to use their own variant of the CSV format. This parser will loosely follow the

RFC-4180. CSV input should adhere

to the following criteria (differences from RFC-4180 in parentheses):

  • A record is separated by a new line (CRLF,LF,CR)
  • A final record may end with a new line
  • A header may be provided as the first record in input
  • A record has fields separated by a comma (customizable)
  • A field containing new lines, commas, or double quotes

    should be enclosed in double quotes (customizable)

  • Double quotes in a field are escaped with a double quote
  • Each record should contain the same number of fields

Example:

------- import std.algorithm; import std.array; import std.csv; import std.stdio; import std.typecons;

void main() { auto text = "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

foreach (record; csvReader!(Tuple!(string, string, int))(text)) { writefln("%s works as a %s and earns $%d per year", record[0], record[1], record[2]); }

// To read the same string from the file "filename.csv":

auto file = File("filename.csv", "r"); foreach (record; file.byLine.joiner("\n").csvReader!(Tuple!(string, string, int))) { writefln("%s works as a %s and earns $%d per year", record[0], record[1], record[2]); } } } -------

When an input contains a header the Contents can be specified as an associative array. Passing null to signify that a header is present.

------- auto text = "Name,Occupation,Salary\r" ~ "Joe,Carpenter,300000\nFred,Blacksmith,400000\r\n";

foreach (record; csvReader!(string[string]) (text, null)) { writefln("%s works as a %s and earns $%s per year.", record["Name"], record["Occupation"], record["Salary"]); }

// To read the same string from the file "filename.csv":

auto file = File("filename.csv", "r");

foreach (record; csvReader!(string[string]) (file.byLine.joiner("\n"), null)) { writefln("%s works as a %s and earns $%s per year.", record["Name"], record["Occupation"], record["Salary"]); } -------

This module allows content to be iterated by record stored in a struct, class, associative array, or as a range of fields. Upon detection of an error an CSVException is thrown (can be disabled). csvNextToken has been made public to allow for attempted recovery.

Disabling exceptions will lift many restrictions specified above. A quote can appear in a field if the field was not quoted. If in a quoted field any quote by itself, not at the end of a field, will end processing for that field. The field is ended when there is no input, even if the quote was not closed.

See Also

Wikipedia

Comma-separated values

Copyright

Copyright 2011

License

Boost License 1.0.

Authors

Jesse Phillips

Source: std/csv.d

enum Malformed
struct CsvReader CsvRecord Input
class CSVException HeaderMismatchException IncompleteCellException
fn csvNextToken csvReader

Types 7

classCSVException : Exception

Exception containing the row and column for when an exception was thrown.

Numbering of both row and col start at one and corresponds to the location in the file rather than any specified header. Special consideration should be made when there is failure to match the header see HeaderMismatchException for details.

When performing type conversions, ConvException is stored in the next field.

Fields
size_t row
Methods
string toString() @safe pure const
Constructors
this(string msg, string file = __FILE__, size_t line = __LINE__, Throwable next = null)
this(string msg, Throwable next, string file = __FILE__, size_t line = __LINE__)
this(string msg, size_t row, size_t col, Throwable next = null, string file = __FILE__, size_t line = __LINE__)
classIncompleteCellException : CSVException

Exception thrown when a Token is identified to not be completed: a quote is found in an unquoted field, data continues after a closing quote, or the quoted field was not closed before data was empty.

Fields
dstring partialDataData pulled from input before finding a problem
classHeaderMismatchException : CSVException

Exception thrown under different conditions based on the type of Contents.

Structure, Class, and Associative Array

  • When a header is provided but a matching column is not found

Other

  • When a header is provided but a matching column is not found
  • Order did not match that found in the input

Since a row and column is not meaningful when a column specified by the header is not found in the data, both row and col will be zero. Otherwise row is always one and col is the first instance found in header that occurred before the previous starting at one.

enumMalformed

Determines the behavior for when an error is detected.

Disabling exception will follow these rules:

  • A quote can appear in a field if the field was not quoted.
  • If in a quoted field any quote by itself, not at the end of a

    field, will end processing for that field.

  • The field is ended when there is no input, even if the quote was

    not closed.

  • If the given header does not match the order in the input, the

    content will return as it is found in the input.

  • If the given header contains columns not found in the input they

    will be ignored.

ignoreNo exceptions are thrown due to incorrect CSV.
throwExceptionUse exceptions when input has incorrect CSV.
private structInput(Range, Malformed ErrorLevel)
Fields
Range range
size_t row
private structCsvReader(Contents, Malformed ErrorLevel, Range, Separator, Header) if (isSomeChar!Separator && isInputRange!Range && is(immutable ElementType!Range == immutable dchar) && isForwardRange!Header && isSomeString!(ElementType!Header))
Fields
Input!(Range, ErrorLevel) * _input
Separator _separator
Separator _quote
size_t[] indices
bool _empty
bool _allowInconsistentDelimiterCount
string[] headerHeader from the input in array form.
Methods
@property auto front()Part of an input range as defined by isInputRange.
bool empty() @property @safe @nogc pure nothrow constPart of an input range as defined by isInputRange.
void popFront()Part of an input range as defined by isInputRange.
private void prime()
Constructors
this(Range input, Separator delimiter, Separator quote, bool allowInconsistentDelimiterCount)Constructor to initialize the input, delimiter and quote for input without a header.
this(Range input, Header colHeaders, Separator delimiter, Separator quote, bool allowInconsistentDelimiterCount)Constructor to initialize the input, delimiter and quote for input with a header.
private structCsvRecord(Contents, Malformed ErrorLevel, Range, Separator) if (!is(Contents == class) && !is(Contents == struct))
Fields
Input!(Range, ErrorLevel) * _input
Separator _separator
Separator _quote
Contents curContentsoken
typeof(appender!(dchar[])()) _front
bool _empty
bool _allowInconsistentDelimiterCount
size_t[] _popCount
Methods
Contents front() @property @safe purePart of an input range as defined by isInputRange.
bool empty() @property @safe pure nothrow @nogc constPart of an input range as defined by isInputRange.
private bool recordEnd()
void popFront()Part of an input range as defined by isInputRange.
private void prime(size_t skipNum)
private void prime()
Constructors
this(Input!(Range, ErrorLevel) * input, Separator delimiter, Separator quote, size_t[] indices, bool allowInconsistentDelimiterCount)

Functions 4

fnauto csvReader(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Separator = char)(Range input, Separator delimiter = ',', Separator quote = '"', bool allowInconsistentDelimiterCount = false) if (isInputRange!Range && is(immutable ElementType!Range == immutable dchar) && isSomeChar!(Separator) && !is(Contents T : T[U], U : string))Returns an input range for iterating over records found in `input`.
fnauto csvReader(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Header, Separator = char)(Range input, Header header, Separator delimiter = ',', Separator quote = '"', bool allowInconsistentDelimiterCount = false) if (isInputRange!Range && is(immutable ElementType!Range == immutable dchar) && isSomeChar!(Separator) && isForwardRange!Header && isSomeString!(ElementType!Header))ditto
fnauto csvReader(Contents = string, Malformed ErrorLevel = Malformed.throwException, Range, Header, Separator = char)(Range input, Header header, Separator delimiter = ',', Separator quote = '"', bool allowInconsistentDelimiterCount = false) if (isInputRange!Range && is(immutable ElementType!Range == immutable dchar) && isSomeChar!(Separator) && is(Header : typeof(null)))ditto
fnvoid csvNextToken(Range, Malformed ErrorLevel = Malformed.throwException, Separator, Output)(ref Range input, ref Output ans, Separator sep, Separator quote, bool startQuoted = false) if (isSomeChar!Separator && isInputRange!Range && is(immutable ElementType!Range == immutable dchar) && isOutputRange!(Output, dchar))Lower level control over parsing CSV