Sample Scramjet architecture for Artificial Intelligence and Machine Learning (AI/ML)
The diagram below shows collecting data from several different digital sources (social media, microblogging, news aggregation), preparing and finally feeding into AI/ML sentiment analysis engine both for training and inference.
Use Case: Pre-process data for AI/ML sentiment analysis
Companies offering advanced digital marketing analytics, need to collect and combine data from multiple digital channels. This goal brings complexity both in data collection and performance.
Scramjet Cloud Platform will be able to collect data from multiple sources and pre-process them on the fly, simplifying and reducing expenses of data preparation for Machine Learning algorithms training and verification.
Example:
An innovative tech company is a leader in Internet sentiment analysis. The company develops and deploys a set of interconnected data processing apps. These apps will collect data from multiple sources (blogs, social media, discussion boards), clean up data, extract content, and save high-quality datasets to data management platforms for usage in further AI/ML training and optimization.
Scramjet Cloud Platform can deploy and connect the apps in seconds, providing a serverless infrastructure for the data acquisition and wrangling tools. It will allow the company to leverage its machine learning infrastructure as a seamless part of the platform with self-hosted Scramjet Transform Hubs.
Use Case: Clean training data for ML/AI models
Companies that use AI/ML algorithms, need to ensure the quality of the algorithm training data by pre-processing them before the training occurs. It means cleaning, removing empty values and outliers, or filtering data that do not comply with quality standards.
Scramjet Cloud Platform Sequences can perform data cleaning needed for Machine Learning algorithms training and verification. The user's Sequences are simple programs, easy to write, with various data processing, transformation, cleaning, and standardization rules.
Example:
A big chain of retail stores has custom ML models predicting demand for various products and helping to replenish the warehouse in JIT mode. Data comes from multiple systems such as a retail chain consisting of several brands and two e-commerce stores; data formats also tend to change and there are data quality issues (empty data, data with errors).
Scramjet Cloud Platform can do the filtering, cleaning, completing, or removing data from various sources. Thanks to SCP datasets used for retraining ML models have high quality.
Use Case: Analyze stream of data on the fly with AI/ML models
Scramjet serverless sequences can embed trained ML models inside and perform ML inference (classification or regression) on the fly, as data flows through the Scramjet data engine. There is no need to issue expensive REST requests to ML models deployed as microservice, as data are available inside sequence along with ML model loaded to memory and ready to perform inference. The performance of this model will considerably surpass typical inference architectures with REST API deployed as another service.
Example:
An innovative startup created an anomaly detection engine that is used to spot anomalies in financial transactions, infrastructure logs, industry data, and more. The engine works perfectly, however, implementations for bigger customers face severe performance problems as each transaction is evaluated via an expensive REST API call and it becomes a bottleneck in case of bigger transaction volumes. Scramjet Cloud Platform can be used to create dedicated sequences with embedded ML models that will classify anomalies in the application, dramatically increasing speed.