Options
All
  • Public
  • Public/Protected
  • All
Menu

Class JSONCollection

This class represents a collection of JSON objects. An instance can be created from different kinds of input using the static methods starting with from and converted to other kinds of output using the instance methods starting with to. This class is designed to handle large directories by using iterators and reading one file at a time.

Hierarchy

Index

Properties

Protected _entries

_entries: function = [].values

The internal entries iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the entries iterator will be a function that reads and parses one line at a time.

Type declaration

    • (): IterableIterator<IAnyObject>
    • Returns IterableIterator<IAnyObject>

Protected _lines

_lines: function = [].values

The internal lines iterator. In some cases iterating over lines is simple. For example, if we create an instance from string or array we already have all the lines. However, if the input is a file, the lines iterator will be a function that reads one line at a time.

Type declaration

    • (): IterableIterator<string>
    • Returns IterableIterator<string>

Methods

entries

  • entries(): IterableIterator<IAnyObject>

lines

  • lines(): IterableIterator<string>
  • The lines iterator of the instance. Yields lines as strings.

    Returns IterableIterator<string>

setEntries

  • setEntries(entriesIterator: function): Collection
  • Sets the entries iterator of the instance. Useful while composing an instance from different sources. The "entries" might be JSON objects representing NDJSON lines or JS arrays of strings representing a line in CSV or TSV file.

    Parameters

    • entriesIterator: function
        • (): IterableIterator<IAnyObject>
        • Returns IterableIterator<IAnyObject>

    Returns Collection

    The instance to allow chaining

setLines

  • Sets the entries iterator of the instance. Useful while composing an instance from different sources

    Parameters

    • linesIterator: function

      The iterator to use

        • (): IterableIterator<string>
        • Returns IterableIterator<string>

    Returns Collection

    The instance to allow chaining

toArray

  • toArray(): IAnyObject[]
  • Returns an array of JSON objects representing of the JSON collection. NOTE: The array takes memory. Don't use this for big JSON collections (that represent a big file or directory). Use the lines or entries iterators instead to handle one entry at a time.

    Returns IAnyObject[]

toFile

  • Writes the JSON collection to file. The file can be (re)created or appended to. Example:

    ```js const ndjson = JSONCollection.fromStringArray(["{}", "{}" ]);

    // Create or overwrite the file with default settings ndjson.toFile("/my/output.ndjson");

    // Or use custom settings: ndjson.toFile("/my/output.ndjson", { eol: "\r\n", append: true });

    Parameters

    • path: string
    • Default value options: any = {}

    Returns JSONCollection

toJSON

  • toJSON(): IAnyObject[]
  • Converts the contents of the collection to array of "values". The values are json objects representing each line. The result does not include the header in case of delimited formats.

    NOTE: Don't use this for big objects/files because the output array is built into memory and then returned. For big files iterate over the entries instead, which will yield the same objects:

    for (const entry of collection.entries()) {
        // entry is an object representing a row
    }
    alias

    toArray This is just a "magic method" that will make it possible to call JSON.stringify() on an instance and get a valid JSON result.

    Returns IAnyObject[]

toNDJSON

  • toNDJSON(eol?: string): string
  • Converts the instance to NDJSON string.

    NOTE: Don't use this for big objects/directories because the output string is built into memory and then returned. For big collections iterate over the entries instead and serialize one line at a time:

    for (const entry of collection.entries()) {
        const line = JSON.stringify(entry);
    }

    Parameters

    • Default value eol: string = ""

      The new line character to use. Defaults to \r\n.

    Returns string

toString

  • toString(): string
  • Returns a string representation of the NDJSON object. NOTE: The string takes memory. Don't use this for big JSON (that represent a big file or directory). Use the lines or entries iterators instead to handle one entry at a time.

    Returns string

toStringArray

  • toStringArray(): string[]
  • Returns an array of JSON strings representing of the JSON collection. NOTE: The array takes memory. Don't use this for big JSON objects (that represent a big file or directory). Use the lines or entries iterators instead to handle one entry at a time.

    Returns string[]

Static fromArray

  • If we happen to have the entire json as array of objects, we can create a JSONCollection instance like so:

    const json = JSONCollection.fromArray([ {}, {} ]);
    json.lines();   // Lines iterator
    json.entries(); // JSON iterator

    Parameters

    • arr: IAnyObject[]

      An array of objects that can be serialized as JSON

    Returns JSONCollection

Static fromDirectory

  • Creates and returns a JSONCollection instance from directory path. This will walk (recursively) through the directory and collect all the files having a .json extension. The lines and entries iterators will yield results from all those files combined. Example:

    const json = JSONCollection.fromDirectory("/path/to/directory/containing/json/files");
    json.lines();   // Lines iterator
    json.entries(); // JSON iterator

    Parameters

    • path: string

      Absolute path to directory

    Returns JSONCollection

Static fromFile

  • Creates and returns an JSON collection instance from a file path. Example:

    const json = JSONCollection.fromFile("/path/to/file.json");
    json.lines();   // Lines iterator
    json.entries(); // JSON iterator

    Parameters

    • path: string

      Absolute path to NDJSON file

    Returns JSONCollection

Static fromString

  • If we have the entire json as a string, we can create an instance of JSONCollection which will provide the lines and entries iterators like so:

    const json = JSONCollection.fromString("[{},{}]");
    json.lines();   // Lines iterator
    json.entries(); // JSON iterator

    Parameters

    • input: string

      The input string that can be parsed as JSON

    Returns JSONCollection

Static fromStringArray

  • If we have the entire json as array of strings, we can create an instance like so:

    const json = JSONCollection.fromStringArray(["{}", "{}" ]);
    json.lines();   // Lines iterator
    json.entries(); // JSON iterator

    Parameters

    • arr: string[]

      An array of strings that can be parsed as JSON

    Returns JSONCollection

Generated using TypeDoc