Break the data matrix, and become a #RedisHero.

Win Prizes!

How to Build a Powerful E-Learning Platform Using Scala and Redis

Never before has online learning been so accessible. Whether you want to discover more about cryptocurrency, sharpen your programming skills or even just learn a new language, the digital age has gifted everyone access to a phenomenal amount of content. 

However, over time e-learning has been viewed as just another digital commodity, where users expect all online content to be instantaneous. Speed remains crucial to performance, where any lags or delays in page loading time kills the user’s experience. 

And so these high expectations require any competent e-learning platform to be powered by a database that’s capable of handline, processing and transmitting data with hyper-efficiency…which is exactly why this Launchpad App used Redis. 

From start to finish, this application was built to connect, educate and empower learners by connecting them with the most relevant courses based on their interests. 

Let’s investigate how this was achieved. But before we dive in, you may also want to check out all of the other amazing applications that we have on the Launchpad

  1. What will you build? 
  2. What will you need?
  3. Architecture
  4. Getting started

1. What will you build?

You’ll build a powerful e-learning platform that will connect students and teachers with one another along with a diverse library of online courses. With speed being the linchpin to performance, you’ll deploy a number of different Redis components to achieve this objective. 

Below we’ll reveal what components are required to make this application come to fruition along with the functionality of each item. 

2. What will you need?

  • Scala/Play Framework/Akka Streams: used as an open-sourced web application that makes building apps with Java and Scala easy.
  • React: used as a Javascript library to build user interfaces easily.
  • RedisGraph: used as a powerful graph database that translates Cypher queries to matrix operations executed over a GraphBLAS engine.
  • RediStreams: manages data consumption
  • RedisBloom: provides Redis with support for additional probabilistic data structures.
  • RedisGears: used as the engine for data processing in Redis
  • RediSearch: provides a powerful search engine for Redis.
  • RedisJSON: implements ECMA-404 The JSON Data Interchange Standard as a native data type
  • RedisTimeSeries: provides time series data
  • Keycloak: used as an open-sourced identity and access management solution for modern applications

3. Architecture

The data model is expressed through nodes and relations using RedisGraph. The model is very simple since it involves the Student, Course and Topic entities expressing the different kinds of relations between each other. 

X-Mentor follows an Event Driven Architecture approach in which the following Domain Events are considered:

  • student-enrolled
  • student-interested
  • student-interest-lost
  • course-created
  • course-rated
  • course-recommended
  • Student-progress-registered

4. Getting started

Prerequisites

  • Docker Engine and Docker Compose

Step 1: Clone the repository

Step 2: Start the docker

Step 3: Accessing the application

Wait until Keycloak and x-mentor-core are ready, then go to http://localhost:3000

You can access Keycloak via 8880 port as shown below:

Use admin/admin to login into keycloak.

Step 4: Logging in

This step starts the authentication process against Keycloak by:

  1. Verifying if the user’s username already exists in users bloom filter
  2. Providing an auth token

Use the following code to check and see whether the username already exists in users bloom filter:

Step 5: Signing up

Signing up involves 4 steps:

  1. Registering a user against Keycloak
  2. Adding a user’s username to users bloom filter
  3. Creating a user in RedisGraph
  4. Adding student’s time series key (needed for registering student progress)
  • To add a username to users bloom filter, insert the following code
  • To integrate students into the graph, use the below code
  • To create student progress timeseries key, use the below code

Step 6: Creating Courses

In this step we’re going to show you how to create courses to go on the e-learning platform. Each course is going to be stored as a JSON in RedisJSON. 

:

Follow the commands below:

  1. Obtain the last course id from the Redis key: course-last-index
  1. Increase course id key in 1
  1. Store course as JSON in redisJSON
  1. Add course id to courses bloom filter
  1. Create course in the graph
  1. Publish course-created event which sends notifications by Server Sent Event to the frontend

Step 7: Enrolling courses

Here we’ll uncover how you can enroll a student in a specific course. 

Below are the steps for you to follow:

  • To verify if a student exists in users bloom filter:
  • To get course as JSON from redisJSON
  • To highlight a relationship between the student and the course in redisGraph

Step 8: Course review

As part of any online resource, users generally are able to provide a review. To make this functionality happen, you need to carry out the following steps:

  1. First, verify if a studying relationship exists between the student and the course. 
  2. Next, verify whether a rates relation exists between the student and the course.
  3. Thirdly, create the rate relation in the graph (see diagram below).
  4. Publish event course-rated stream

The following diagram illustrates the interaction between Redis Graph and Redis Streams.

To bring make this functionality happen, follow the below commands:

  • Filter courses by student
  • Get courses rated by user
  • Create rates relation in the graph
  • Publish event to course-rated stream

Step 9: Course search

All

The following commands retrieves courses by query from redisJSON with rediSearch

FT.SEARCH courses-idx ${query}*

By ID

BF.EXISTS courses ${course.id}

JSON.GET course:${course.id}

By Student

GRAPH.QUERY xmentor “MATCH (student)-[:studying]->(course) where student.username = ‘$student’ RETURN course”

FT.SEARCH courses-idx ${course.title}

Step 10: Student’s interests

Now we’ll show you how to allow students to filter preferred courses based on their interests. Here’s how to do it:

  1. Get all of the interested relations from RedisGraph. 
  2. Distinguish the difference between already existing relations and new ones. This’ll allow you to separate new interests from existing ones. 
  3. Create new interested relations into RedisGraph
  4. Remove interested relations that don’t apply anymore
  5. Publish to student-interest-lost and student-interested stream

The following diagram shows the interaction between RedisGraph and Redis Streams.

Below are the commands for you to follow:

  • Capture all student interests
  • Create interest relation
  • Delete interest relation
  • Publishing to student-interested stream
  • Publishing to student-interest-lost stream

Step 11: Course recommendation system

Here we’re going to show you how to create a course recommendation system that connects users with courses that are most relevant to their interests. A lot of this comes down to the advanced capabilities of RedisGraph. 

Searching for relations between nodes in the graph database is the easiest way to implement the most effective recommendation strategies. Let’s have a look at how to do this.

In order for you to create a special recommendation system that matches users’ personal interests with the most relevant courses

How to carry out the enrolled recommendation strategy
  1. Randomly select a course the student is enrolled in 
  2. Get the topic of the course
  3. Look for students enrolled on the same course
  4. Search for courses on the same topic based on the students who are currently enrolled
  5. Recommend those courses
How to carry out the interest recommendation strategy
  1. Randomly select a student’s interest
  2. Look for students who are enrolled to the course of that topic
  3. Search for other courses of the same topic based on the students who are currently enrolled
  4. Provide the recommended courses 
How to carry out the discover recommendation strategy
  1. Get all topics
  2. Identify all of the topics that the students are interested in
  3. Gather all of the topics the user is enrolled in 
  4. Choose a topic where the user is neither interested in nor enrolled for
  5. Get the courses of that chosen topic and recommend them 
How the graph data is accessed
  • All student’s courses
  • Get all topics
  • Obtain topic by course
  • Identify students who are enrolled in (studying relation) a course
  • Generate courses by topic
  • Uncover student’s interests
  • Filter courses the student is enrolled in based on topic
  • Identify topics the user is enrolled in

Step 12: Student progress registration

This functionality will allow you to track the amount of time users spend watching courses on the platform. That information will then be used to implement the Leaderboard. 

Once x-mentor-core receives the request, it will then publish the Student Progress Registration Domain Event. This will end up as an element inside student-progress-registered stream (which is a Redis Stream) via the following command:

All of the data sent to RedisGears will be pushed to the stream and will then sink this data into Redis TimeSeries using the following command:

Step 13: Leaderboard 

The Leaderboard functionality enables you to have a board that displays the rankings of the top students that use X-Mentor. Students are ranked based on the amount of time they spend watching content on the platform – the more you watch, the higher you rank. 

To accomplish this, you need to separate two functionalities:

  • Register the student progress
  • Getting the board data

When the user request for the leaderboard data, first look at Redis for the time series keys

For each key, you need to use Redis TimeSeries to get the range of samples in a time window of three months performing sum aggregation. You can use the code below to make this happen:

Below are additional requisites that need to be implemented for this to happen.

  • student_key is the student’s time series key. For example: studentprogress:codi.sipes is the time series key for student codi.sipes.
  • three_months_back_timestamp is a Unix Timestamp that represents a point in time three months back than timestamp(in order to have a time window of three months).
  • timestamp the current timestamp (in Unix Timestamp format).
  • Next perform a sum aggregation of the sample values in those time windows using a Time Bucket of 1000 milliseconds. 

Carrying out these commands will provide you with the accumulated screen time of each student. Once you receive these rankings, you can create a ranking system based on those who have the most screen time. 

Conclusion: Empowering Learners With Redis

Being this far into the digital age, a simple prerequisite of any application is for it to operate at maximum speed. This is especially true for e-learning platforms where users are meant to be engaged with its course content for long periods of time. 

A mere lag will create friction between users and the application, inhibiting its ability to connect teachers with students as well as providing value through its courses. Having Redis as the application’s main database removed this threat and helped to create a fully optimal application that catered to the user’s demands with ease. 

To get a more visual insight into how this application was created, then you can watch this YouTube video here. We also have a diverse range of applications for you to check out on the Redis Launchpad that are having an impact on everyday life around the world.

So make sure to check them out!

Who built this application? 

Sergio Cano

Sergio is a full-stack engineer and in his own words ‘loves solving problems and learning new stuff.’ 

Being an enthusiastic learner, it’s not difficult to see where he got the inspiration to build this application. 

Make sure to check out his profile here and see what other projects he’s been involved in.