Blog
The Fastest Route to Hub API: Using the Hub Client in Application Context
18 March 2024
The Scramjet Transform Hub (STH) empowers developers to deploy and manage powerful data processing programs.
A key tool in this process is the Scramjet SDK. Scramjet SDK allows to easily control your Sequences and Instances,
but also in you are using Scramjet Cloud Platform, also your Space and Hubs. This provides powerful ways to implement code execution orchestration and monitoring.
Many existing serverless solutions provide SDK, but Scramjet goes one step further and provides also Scramjet SDK in your
application context!
This lets you send requests to your STH and Space API internally, without through the proxies, public API gateway or even API server reducing data transfer speed significantly.
Scramjet SDK available in the application context has two instances: Scramjet Hub Client and Scramjet Space Client.
Scramjet Hub Client lets developers send requests internally to STH API. It can be used to start your Sequences, terminate Instances,
collect logs and generally perform all STH operations that are exposed through STH API, but much faster and without the need to configure Scramjet SKD Hub Client.
This way to you can all possible API requests but only to the STH where the Instance that uses Scramjet Hub Client is deployed.
Another SDK available in the application context - Space Client is even more powerful. It lets developer to control all Space where the Instance that uses it is deployed.
From the convenience of your code, without the need to configure Scramjet SDK, you can not only list all Hubs connected to your Space, manage and monitor resources,
but also control every Hub in your Space! So you can start Sequences, terminate Instances, and so on, on any Hub that is connected to your Space.
You can for example from the cloud supervisor Instance, deploy and start your program on a Hub located on a different cloud, or on the edge environment.
In this blog post we will cover Hub Client, since its mechanisms or working are similar to Space Client but simpler. It will give us a good ground for learning about the powerful Space Client in the next blog post.
A Closer Look at the Implementation
In our example, a JavaScript Sequence is deployed on STH to initiate another Sequence within the same STH.
The asynchronous function, which is the main function of this Sequence, aims to start another Sequence identified by sequenceId, a parameter passed to it.
First the Sequence Client is retrieved using the provided sequenceId by utilizing the method this.hub.getSequenceClient.
Then, the Sequence start method is triggered, effectively launching its execution.
If successful, the function uses the this.logger method to log a confirmation message. In case of errors,
it logs the error message and provides detailed error information through instance log endpoint.
_29const { PassThrough } = require("stream");_29_29module.exports = async function(_input, sequenceId) {_29 this.logger.info("Working...");_29 const out = new PassThrough();_29_29 try {_29 const seqClient = await this.hub.getSequenceClient(sequenceId);_29_29 this.logger.info("Sequence client called:", seqClient);_29 const seqStart = await seqClient.start(sequenceId);_29 const seqStartStr = JSON.stringify(seqStart);_29_29 this.logger.info(`Sequence ${sequenceId} started: ${seqStartStr}`);_29 _29 const instClient = await this.hub.instClient(retInstObj._id);_29 const instHealth = await instClient.health();_29 const instinfo = await instClient.getInfo(); _29 _29 this.logger.info(`Instance health: ${instHealth}, Instance info: ${instinfo}`);_29 _29 out.write(seqStart._id); // write instance id to output stream_29 out.end();_29 } catch (e) {_29 // if error occurs it will be logged under /instance/:id/log endpoint (CLI: si inst log <instID>)_29 this.logger.error(e);_29 }_29 return out;_29};
Explanation:
1. Asynchronous Function: The function takes input and sequenceId as input and initiates the sequence based on the provided Sequence-id.
2. Retrieving Sequence Client: The hub.getSequenceClient method fetches the client object associated with the given Sequence-id.
3. Sequence Initiation: The seqClient.start method starts the desired Sequence.
4. Handling Results: Upon successful initiation, the Sequence-id is written to the stream and the stream is closed, indicating completion.
5. Error Handling: Catches and logs any errors encountered during the process.
Conclusion
In conclusion, Scramjet SDK offers the flexibility to create a powerful supervisory Sequence that not only monitor other Sequences but also possess the ability to restart them upon failure. This functionality, combined with the capability to initiate and manage other Sequences, makes such a Sequence ideal for automating long-running tasks like data synchronization and scheduled jobs. Furthermore, Scramjet's long-lived Instances ensure the supervisor Sequence remains operational throughout the process, providing a reliable and robust solution for managing complex workflows.
Register now for your free trial HERE.
Checkout Scramjet platform samples on GitHub.
