By Al Basseri
Solar panel sensors continuously produce a great amount of data. Scalytics Density, based on infinimesh, collects this data and stores it in main memory using Redis. Scalytics Lumina then takes over and exploits the data stored in Redis to perform time-series forecasting. This allows us to predict the energy production from the solar panels and incorporate it into the grid more efficiently.
To perform time-series forecasting Scalytics Lumina first preprocesses the data to bring it into the right form and then runs an AI algorithm. Apache Wayang (incubating), the smart open-source technology behind Scalytics Lumina, is responsible for this step. Scalytics Lumina allows Redis to work in tandem with other platforms to achieve higher performance. Concretely, Scalytics Lumina reads the raw data from Redis and moves them to Spark for preprocessing.
Then, the preprocessed data is given as input in a Long-Short Term Memory (LSTM) Neural Network for building a model for energy production prediction. All this is done seamlessly; the developers do not need to worry about data migration or taking decisions on where to execute each task. Scalytics Lumina stores the model in Redis which is then used for predicting the energy production in the next time step. As the prediction phase needs to be rapid, Scalytics Lumina again synchronizes Redis with Tensorflow Serving.