Options
All
  • Public
  • Public/Protected
  • All
Menu

Class DelimitedCollection

This class represents a collection in delimited format.

  • An instance can be created from different kinds of input using the static methods starting with from and converted to other kinds of output using the instance methods starting with to.
  • This class is designed to handle large files or directories by using iterators and reading files one line at a time.
  • The entries iterator will yield json objects for each line.

Hierarchy

Index

Properties

Protected _entries

_entries: function = [].values

The internal entries iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the entries iterator will be a function that reads and parses one line at a time.

Type declaration

    • (): IterableIterator<IAnyObject>
    • Returns IterableIterator<IAnyObject>

Protected _lines

_lines: function = [].values

The internal lines iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the lines iterator will be a function that reads one line at a time.

Type declaration

    • (): IterableIterator<string>
    • Returns IterableIterator<string>

Methods

entries

  • entries(): IterableIterator<IAnyObject>

lines

  • lines(): IterableIterator<string>
  • The lines iterator of the instance. Yields lines as strings.

    Returns IterableIterator<string>

setEntries

  • setEntries(entriesIterator: function): Collection
  • Sets the entries iterator of the instance. Useful while composing an instance from different sources. The "entries" might be JSON objects representing NDJSON lines or JS arrays of strings representing a line in CSV or TSV file.

    Parameters

    • entriesIterator: function
        • (): IterableIterator<IAnyObject>
        • Returns IterableIterator<IAnyObject>

    Returns Collection

    The instance to allow chaining

setLines

  • Sets the entries iterator of the instance. Useful while composing an instance from different sources

    Parameters

    • linesIterator: function

      The iterator to use

        • (): IterableIterator<string>
        • Returns IterableIterator<string>

    Returns Collection

    The instance to allow chaining

toArray

  • toArray(): IAnyObject[]
  • Converts the contents of the collection to array of "values". The values are json objects representing each line, excluding the header.

    NOTE: Don't use this for big objects/files because the output array is built into memory and then returned. For big files iterate over the entries instead, which will yield the same objects:

    for (const entry of collection.entries()) {
        // entry is an object representing a row
    }
    alias

    toJSON

    Returns IAnyObject[]

toFile

  • toFile(path: string, options?: IDelimitedFormatOptions): Collection

toJSON

  • toJSON(): IAnyObject[]
  • Converts the contents of the collection to array of "values". The values are json objects representing each line. The result does not include the header in case of delimited formats.

    NOTE: Don't use this for big objects/files because the output array is built into memory and then returned. For big files iterate over the entries instead, which will yield the same objects:

    for (const entry of collection.entries()) {
        // entry is an object representing a row
    }
    alias

    toArray This is just a "magic method" that will make it possible to call JSON.stringify() on an instance and get a valid JSON result.

    Returns IAnyObject[]

toNDJSON

  • toNDJSON(eol?: string): string
  • Converts the instance to NDJSON string.

    NOTE: Don't use this for big objects/files because the output string is built into memory and then returned. For big files iterate over the entries instead and serialize one line at a time:

    for (const entry of collection.entries()) {
        const line = JSON.stringify(entry);
    }

    Parameters

    • Default value eol: string = ""

      The new line character to use. Defaults to \r\n.

    Returns string

toString

  • toString(options?: IDelimitedFormatOptions): string
  • Serializes the contents of the collection to a string. The result can be a CSV, TSV or other delimited format depending on the options. The default options will output CSV string. For TSV use:

    instance.toString({ delimiter: "\t" });

    Parameters

    • Default value options: IDelimitedFormatOptions = {}

    Returns string

toStringArray

  • toStringArray(options?: IDelimitedFormatOptions): string[]
  • Converts the contents of the collection to an array of strings. The result strings can be in CSV, TSV or other delimited format depending on the options. The default options will output CSV strings. For TSV use:

    instance.toStringArray({ delimiter: "\t" });

    Parameters

    • Default value options: IDelimitedFormatOptions = {}

    Returns string[]

Static fromArray

  • If we happen to have the entire csv as array of objects, we can create an instance like so:

    const csv = CSV.fromArray([{ a: "1", b: "2" }, { a: "3", b: "4" }]);
    csv.lines();   // Lines iterator -> "1,2", "3,4"
    csv.entries(); // JSON iterator  -> { a: "1", b: "2" }, { a: "3", b: "4" }

    In this case we already have entries and we only need to handle their serialization in the lines iterator.

    Parameters

    • arr: IAnyObject[]

      An array of objects that can be serialized as JSON

    • Default value options: IDelimitedFormatOptions = {}

    Returns DelimitedCollection

Static fromDirectory

  • Creates and returns an instance from directory path. This will walk (recursively) through the directory and collect all the files having a .csv extension. The lines and entries iterators will yield results from all those files combined. Example:

    const csv = Delimited.fromDirectory("/path/to/directory/containing/csv/files");
    csv.lines();   // Lines iterator
    csv.entries(); // JSON iterator

    Parameters

    • path: string

      Absolute path to directory

    • Default value options: IDelimitedFormatOptions = {}

    Returns DelimitedCollection

Static fromFile

  • Creates and returns an instance from a file path. The lines and entries iterators will read the file one line at a time. Example:

    const csv = Delimited.fromFile("/path/to/file.csv");
    csv.lines();   // Lines iterator
    csv.entries(); // Entries iterator

    For TSV or another custom delimited format provide a delimiter option like so:

    const tsv = Delimited.fromFile("/path/to/file.tsv", { delimiter: "\t" });

    Parameters

    • path: string

      Absolute path to CSV or TSV file

    • Default value options: IDelimitedFormatOptions = {}

    Returns DelimitedCollection

Static fromString

  • If we have the entire csv or tsv as a string, we can create an instance like so:

    const csv = Delimited.fromString("a,b\n1,2\r\n3,4");
    csv.lines();   // Lines iterator -> "1,2", "3,4"
    csv.entries(); // JSON iterator  -> { a: "1", b: "2" }, { a: "3", b: "4" }

    Parameters

    • input: string

      The input string that can be parsed as CSV or TSV

    • Default value options: IDelimitedFormatOptions = {}

    Returns DelimitedCollection

Static fromStringArray

  • If we have the entire collection as an array of string lines, we can create an instance like so:

    const csv = Delimited.fromStringArray(["a,b", "1,2", "3,4"]);
    csv.lines();   // Lines iterator -> "1,2", "3,4"
    csv.entries(); // JSON iterator  -> { a: "1", b: "2" }, { a: "3", b: "4" }

    The fist line is used as header!

    Parameters

    • arr: string[]

      An array of strings that can be parsed as CSV or TSV

    • Default value options: IDelimitedFormatOptions = {}

    Returns DelimitedCollection

Generated using TypeDoc