The time has come to start development of the next major version of the Scramjet framework. 4.x versions have brought stability and clear vision of the range of applications.
-compat module may be released if sufficient interest shows (it'll be v4 interface using v5 underneath).With the upcoming v5 we expect the following changes:
One of the issues in using current version is that usage of DataStream.pipeline or a number of transforms returns just the last of a chain of streams instead of a full stream.
In scramjet v5 it will be possible to write new chunks into the result of a stream, like this:
_14const stream = new DataStream()_14 .map(x => x + 1)_14 .each(console.log)_14 .run();_14_14stream.write(1);_14stream.write(2);_14stream.write(3);_14stream.end(4);_14_14// 2_14// 3_14// 4_14// 5
In most of the operations a stream that will be returned will be a Duplex stream that will allow writing data before any transforms and will allow reading data already transformed.
Multiple transforms are being merged in a single stream object. In v5, as long as the stream just gets piped to another stream that is able to receive an array of methods it will get that. Slicing of that array will happen at runtime so that you'll be able to pipe any point in the stream flow.
Streams will no longer accept any options, only those defined will be accepted.
Some methods, especially map allowed passing of a Clazz argument. This looks fairly nice on the face of it, but it does make the method quite unpredictable. This is also a feature that is inconsistently implemented in the framework.
New options destructure_args, restructure_args and structure_args will be provided to allow more complex operations to be simplified on sub-objects.
Usage of the WindowStream introduced in 4.14 series did not pick up too much, similarly NumberStream will be removed.
These streams will still be possible for use with the help of scramjet plugins as those will be introduced as separate modules.
Currently plugins inject their methods into prototypes of core streams. This causes that no two conflicting plugins may run in the same process. The refactor will address that issue.
Every method for each stream will be placed into a separate file - the resulting class will be constructed by
Core currently contains way too much methods. In scramjet-core v5.0.0 all main streams will be removed and only the PromiseTransformStream will be exposed.
All main methods will get moved to the main scramjet module.
All streams will now expose a stack property showing place of construction.
Array.prototype inspired methods like map/reduce/find etc. will now follow the ECMA interface. For example the MapCallback will now receive index and self arguments providing index in stream and the stream being mapped respectively.
Similarly the map method will accept thisObj argument, as well as any additional arguments that will be passed to the MapCallback.
Currently there's the referrer option and graph methods available. These work one way, so you can see the object that constructed the prior object, but finding where the stream is targeted (what other streams are consuming it) is rather hard.
Method graph will change it's operation returning a soup of all stream inter-connections - graph object will be shared between all streams.
sources and targets methods will be introduced (names to be clarified).