Options
All
  • Public
  • Public/Protected
  • All
Menu

Class NDJSONCollection

This class represents an NDJSON object. An instance can be created from different kinds of input using the static methods starting with from and converted to other kinds of output using the instance methods starting with to. This class is designed to handle large files or directories by using iterators and reading files one line at a time.

Hierarchy

Index

Properties

Protected _entries

_entries: function = [].values

The internal entries iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the entries iterator will be a function that reads and parses one line at a time.

Type declaration

    • (): IterableIterator<IAnyObject>
    • Returns IterableIterator<IAnyObject>

Protected _lines

_lines: function = [].values

The internal lines iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the lines iterator will be a function that reads one line at a time.

Type declaration

    • (): IterableIterator<string>
    • Returns IterableIterator<string>

Methods

entries

  • entries(): IterableIterator<IAnyObject>

lines

  • lines(): IterableIterator<string>
  • The lines iterator of the instance. Yields lines as strings.

    Returns IterableIterator<string>

setEntries

  • setEntries(entriesIterator: function): Collection
  • Sets the entries iterator of the instance. Useful while composing an instance from different sources. The "entries" might be JSON objects representing NDJSON lines or JS arrays of strings representing a line in CSV or TSV file.

    Parameters

    • entriesIterator: function
        • (): IterableIterator<IAnyObject>
        • Returns IterableIterator<IAnyObject>

    Returns Collection

    The instance to allow chaining

setLines

  • Sets the entries iterator of the instance. Useful while composing an instance from different sources

    Parameters

    • linesIterator: function

      The iterator to use

        • (): IterableIterator<string>
        • Returns IterableIterator<string>

    Returns Collection

    The instance to allow chaining

toArray

  • toArray(): IAnyObject[]
  • Returns an array of JSON objects representing of the NDJSON object. NOTE: The array takes memory. Don't use this for big NDJSON objects. Use the lines or entries iterators instead to handle one entry at a time.

    Returns IAnyObject[]

toFile

  • Writes the NDJSON object to file. The file can be (re)created or appended to. Example:

    ```js const ndjson = NDJSON.fromStringArray(["{}", "{}" ]);

    // Create or overwrite the file with default settings ndjson.toFile("/my/output.ndjson");

    // Or use custom settings: ndjson.toFile("/my/output.ndjson", { eol: "\r\n", append: true });

    Parameters

    • path: string
    • Default value options: any = {}

    Returns NDJSONCollection

toJSON

  • toJSON(): IAnyObject[]
  • Converts the contents of the collection to array of "values". The values are json objects representing each line. The result does not include the header in case of delimited formats.

    NOTE: Don't use this for big objects/files because the output array is built into memory and then returned. For big files iterate over the entries instead, which will yield the same objects:

    for (const entry of collection.entries()) {
        // entry is an object representing a row
    }
    alias

    toArray This is just a "magic method" that will make it possible to call JSON.stringify() on an instance and get a valid JSON result.

    Returns IAnyObject[]

toString

  • toString(): string
  • Returns a string representation of the NDJSON object. NOTE: The string takes memory. Don't use this for big NDJSON objects. Use the lines or entries iterators instead to handle one entry at a time.

    Returns string

toStringArray

  • toStringArray(): string[]
  • Returns an array of JSON strings representing of the NDJSON object. NOTE: The array takes memory. Don't use this for big NDJSON objects. Use the lines or entries iterators instead to handle one entry at a time.

    Returns string[]

Static fromArray

  • If we happen to have the entire ndjson as array, we can create an NDJSON instance like so:

    const ndjson = NDJSON.fromArray([ {}, {} ]);
    ndjson.lines();   // Lines iterator
    ndjson.entries(); // JSON iterator

    Parameters

    • arr: IAnyObject[]

      An array of objects that can be serialized as JSON

    Returns NDJSONCollection

Static fromDirectory

  • Creates and returns an NDJSON instance from directory path. This will walk (recursively) through the directory and collect all the files having a .ndjson extension. The lines and entries iterators will yield results from all those files combined. Example:

    const ndjson = NDJSON.fromDirectory("/path/to/directory/containing/ndjson/files");
    ndjson.lines();   // Lines iterator
    ndjson.entries(); // JSON iterator

    Parameters

    • path: string

      Absolute path to directory

    Returns NDJSONCollection

Static fromFile

  • Creates and returns an NDJSON instance from a file path. Example:

    const ndjson = NDJSON.fromFile("/path/to/file.ndjson");
    ndjson.lines();   // Lines iterator
    ndjson.entries(); // JSON iterator

    Parameters

    • path: string

      Absolute path to NDJSON file

    Returns NDJSONCollection

Static fromString

  • If we have the entire ndjson as a string, we can create an instance of NDJSON which will provide the lines and entries iterators like so:

    const ndjson = NDJSON.fromString("{}\n{}");
    ndjson.lines();   // Lines iterator
    ndjson.entries(); // JSON iterator

    Parameters

    • input: string

      The input string that can be parsed as JSON

    Returns NDJSONCollection

Static fromStringArray

  • If we have the entire ndjson as array of strings, we can create an NDJSON instance like so:

    const ndjson = NDJSON.fromStringArray(["{}", "{}" ]);
    ndjson.lines();   // Lines iterator
    ndjson.entries(); // JSON iterator

    Parameters

    • arr: string[]

      An array of strings that can be parsed as JSON

    Returns NDJSONCollection

Generated using TypeDoc