Sometimes your web application will initiate some server-side code that runs for an extended period. This can be a problem for a user left staring at a web page with no feedback – the temptation is to re-submit, go back or even close the application. I found this to be particularly relevant while working on network automation applications that involve a lot of relatively slow interactions with network devices.
What is needed is a way to continuously update the user with messages so that they can see that the application is functioning correctly.
This post will describe how to use WebSockets, Redis, and Webdis to send messages from the server-side of your web application to the client.
Websockets, Redis and Webdis
Websockets allow a permanent TCP connection to be established between a web server and a web client. This connection can then be used by a messaging application to send messages independent from the request/response cycle of a traditional web page. In other words, the receiver does not have to request messages or periodically poll for messages. Furthermore, the web client does not have to reconstruct the entire web page each time a message is received, the message is just added to the page.
Redis is an open-source, in-memory data structure store that can be used as a message broker using the publish/subscribe paradigm. Message receivers subscribe to a channel and message senders publish to the channel. Messages are sent to all receivers as soon as they are published by the sender.
Webdis is an open-source, lightweight web server that exposes Redis PUB/SUB channels to HTTP clients. Webdis is required because Redis does not have an HTTP interface.
Putting it Together
Client
When the client-side web page (that you need to receive messages) is loaded a JS script is run to establish a WebSocket connection to the Webdis/Redis server and to subscribe to a Redis channel.
<script type="text/javascript">
var socket = new WebSocket("ws://192.168.199.85:7379/.json");
var theplace = $("#results")
var ident = 666
socket.onopen = function() {
// subscribe to channel using identifier
socket.send(JSON.stringify(["SUBSCRIBE", ident]));
};
socket.onmessage = function(evt){
// Extract data string from MessageEvent array and convert to JSON
var data = JSON.parse(evt.data)
console.log(evt)
// Extract the actual message text from the JSON
var message = data.SUBSCRIBE[2]
// The server side sends a message of "1" to signal successful connection
// We do not want this to trigger a message on the client,
// so we catch it with an if statement and just print to console
if (message == 1){
console.log("Websocket connected.")
}
// Any other messages must be rendered to the client
else{
theplace.append("<div class=\"alert alert-info\" role=\"alert\"><p>" + message + "</p></div>");
$(".alert").show()
}
};
</script>Server
When the server-side code (Python in this example) is activated from the web page the Python Redis library is used to establish a TCP connection to the Redis server. With the connection established the Python function can run through its steps and publish messages at appropriate times to keep the user updated. The messages will appear on the client-side web page.
from flask import Flask, render_template, request
import time
import redis
app = Flask(__name__)
@app.route("/test_msg", methods=['GET', 'POST'])
def test_msg():
if request.method == 'POST':
# connect to redis server
r = redis.client.StrictRedis(host='192.168.199.85', port=6379, db=0, password=None)
time.sleep(2)
# publish message to redis using group name passed in via ident argument
r.publish(666, 'You have started a server side function')
time.sleep(2)
# publish message to redis using group name passed in via ident argument
r.publish(666, 'Function running, please wait......')
time.sleep(2)
# publish message to redis using group name passed in via ident argument
r.publish(666, 'Function still running, almost done.......')
time.sleep(2)
return render_template("func_finish.html")
elif request.method == 'GET':
return render_template("layout.html")
if __name__ == "__main__":
app.run(host='0.0.0.0', debug=True)Demo Application
In order to demonstrate this functionality, I created a simple web application using Flask running in a virtual environment on my Mac.
Redis and Webdis are installed and running on a VirtualBox Ubuntu 18.04 VM also running on my Mac. The VM is set up in bridged mode so that there is IP connectivity between the VM and the Flask app running in the virtual environment.
This Demo Application is for demonstration purposes only. It is not production-ready.
Set Up the Demo
- Set up Flask by following this guide. https://flask.palletsprojects.com/en/1.1.x/installation/#installation
- Clone the following repo into your venv. https://github.com/andsouth44/websocket_redis_webdis_demo
- On line 45 of layout.html, change the IP address to the IP address of your VM.
- On line 13 of routes.py, change the IP address to the IP address of your VM.
- Create a Linux VM on your PC. I used VirtualBox 6.0 and created a Ubuntu 18.04 VM.
- Install Redis on the VM by following this guide. https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-18-04
- In the redis.conf file, add the IP address of your VM to the bind statement.
- Install Webdis on the VM by following the Install instructions in this guide. http://webd.is/
- You may have to “sudo apt-get install build-essential” for Make to run.
- In the webdis.json file make sure that “websockets” is set to “true” and that the “redis server” is set to the IP of your VM.
- Start the Flask app inside the venv, “python routes.py”
Run the Demo
With the Flask application running in the virtual environment you should be able to connect to http://localhost:5000/test_msg
In the example below I am using Chrome and have the Developer Tools enabled to see the console. You can see that the websocket has connected successfully.
When the Run button is clicked the Server-side Python function should run and the 3 messages should be displayed.
You can see the 3 message events in the console.
When the Python function is finished, it returns the Flask render_template function and the function_finished.html page is displayed.
Channel Identifier
You may have noticed that in the demo we used a statically defined identifier of 666. The identifier determines the PUB/SUB channel and hence who gets to see the messages. I chose statically defined in the demo to keep things simple but this is not practical in most circumstances. If all the users of your web app used the same identifier they would all see each other’s messages.
It would be better to have a dynamically defined identifier that is unique to each user. This will be the subject of my next post.