608B71FC-006A-4934-A643-7D9BA9340450Blog

Scramjet v4.23

blog__author-img
Michał Czapracki
CEO at Scramjet, Data Streaming Expert.
23F2E8CD-3026-46A5-86CC-D13114F7176E425AE875-B1A1-4EA1-8529-075D08DA0BB1

11 April 2019

Scramjet v4.22 brings requested all and race methods, v4.23 fixes the TypeScript API for some optional parameters.

New methods are:

  • DataStream..all - processes a number of functions in parallel, returns a stream of arrays of results.
  • DataStream..race - processes a number of functions in parallel, returns the first resolved.

The new methods follow a similar logic to Promise.all and Promise.race.

A simple example of this is calling a couple URLs in parallel for each entry:


_10
const fetch = require("node-fetch");
_10
const { DataStream } = require("scramjet");
_10
const { reportLocally } = require("./lib");
_10
_10
DataStream.from(data)
_10
.all([entry => fetch(entry.reportURL), entry => fetch(entry.dataURL), reportLocally])
_10
.consume();

A similar effect can be achieved with promises:


_10
DataStream.from(data)
_10
.map(entry => Promise.all([fetch(entry.reportURL), fetch(entry.dataURL), reportLocally(entry)]))
_10
.consume();

race method works similarly to Promise.race so simply continues whenever the first of all transforms resolves. This could be useful when we just want to confirm one of some things happened, like this:


_10
DataStream.from(data)
_10
// we don't care which report is done first as long as at least one is.
_10
.race([reportLocally, reportRemotely])
_10
.consume();

Project co-financed by the European Union from the European Regional Development Fund under the Knowledge Education Development Program. The project is carried out as a part of the competition of the National for Research and Development: Szybka Ścieżka.