Ever felt that in the so called two-way Web, updating Web pages in real-time is a pain? Well, the HTML5 specifications took care of that in a big way. Introducing WebSockets — the full power of a UNIX socket, within your Web server.
Some of the more low-key advances have been the product of what we call the “engineer’s itch”. We needed it, so we made it. Until now, if you wanted to talk to a Web page and continually update it with real-time data, you had two very inefficient solutions. You either had an HTTP Keep-Alive connection, where the server would keep pushing data to the page. This, depending on how you made it, could even mean that theoretically, the page would never finish loading. Or you could have a script on your page poll the Web server at regular intervals, using
yes, XmlHttpRequests to grab data off the server. Ouch!
WebSockets pretty much makes all of that obsolete.
The idea behind WebSockets
You all know what a socket is. If you don’t, go Google it.
All right, you didn’t go anywhere, so here’s what a socket is, for dummies — it’s basically an object where you attach a pipe. Now a pipe is another object; it behaves just like a pipe in your bathroom, except that what’s passing through it is not water but data. In the context of the Internet, the pipe is your Internet connection.
So the Web browser opens a socket and connects it to a pipe that goes to the Web server. The Web server has a socket open which is listening for connections — pipes — and once both the server and the client are connected through that pipe, they can talk together, and fetch and push data.
- You can only transmit one page of data per connection.
- You cannot transmit more than the content-length specified when the header is sent.
- HTTP connections are not conversational. You ask once, and you get once.
- HTTP connections can stay open for just as long as it takes to transmit the data.
Sure, there are ways around all these limitations. HTTP connections can be reused, Keep-Alives can be used to keep the connections open for the server to speak up later after the page has been loaded, and Web developers can choose what constitutes “one page” so that they can hack around the protocol to dynamically update pages. But these are all ugly hacks (which have a name — Comet; Google it sometime), and you really can’t have the conversational interface that a raw socket allows.
WebSockets aren’t raw system sockets. Rather, they’re implemented as a special extension on top of HTTP (think of extensions like WebDAV and the like). When you open a WebSocket, it goes through a HTTP handshake — so you can use normal HTTP connection-establishing methods such as authentication or SSL. But after the handshake is over, what you’re left with is a perpetually open connection, where you can pump in and receive any data you like. It doesn’t even have to have a schema.
In short, this means that you can use it just like a pure network socket.
WebSockets are actually the bleeding-edge part of the upcoming set of Web standards. As such, it was only finalised as RFC 6455 as recently as December 2011. It did exist as a number of drafts before. In fact, one such version, Draft 76, was the standard for quite some time, until a gaping security hole was discovered.
With the standard being just two months old, precious few browsers support it. Mozilla Firefox prior to version 11 required a “Moz” prefix. Chrome has been supporting it since 14.0, but surprisingly, Safari supports only the earlier plagued Draft 76 version of the protocol, as does Opera. And needless to say, Internet Explorer doesn’t support any version of WebSockets. This is slated to change with Windows 8, because the Internet Explorer 10 Technology Preview already includes support for WebSockets.
This is all very well, but how on earth am I supposed to use it? Well, I hope you know Python.
Playing with WebSockets
No, Python isn’t the only way to use WebSockets, but for the purposes of this article, it’s good for a demonstration. Also, it’ll let me introduce another piece of Python technology that’s used much less than it should be — Tornado.
Tornado is an application server written in Python. It includes its own HTTP server, a template language, and a Web framework that can be used like an MVC framework, but is meant to be used in a much easier fashion.
What makes Tornado unique is that it’s asynchronous and real-time. Everyone using FriendFeed retains one connection to them — and the updates are pushed out in real time. That’s easily more than 10,000 connections per server. Yes, Tornado is a FriendFeed technology that was open-sourced after the takeover by Facebook.
Let’s get started. Make sure you’re using Python 2.6 or higher. It’s highly unlikely that Tornado is in your distro’s package repository, though Tornado is included in the PyPI. So you can just use the following command:
sudo easy_install tornado
And the Tornado package will be installed, along with all dependencies. If you’re the more adventurous type, you could download the source archive from Tornado Web Server, extract it, and use Setuptools to install it the standard way, as follows:
tar -xvzf tornado-2.2.tar.gz cd tornado-2.2 ./setup.py build sudo ./setup.py install
If you’re even more adventurous, and want to keep Tornado isolated, you could just extract the archive, and copy the
tornado-2.2/tornado directory into your project. It’s all good.
Anyway, assuming that Tornado is installed (you can run the Python interpreter and
import tornado.web without any errors), you’re set to go.
First of all, let us build an application that simply echoes back what we say to it. It’s lifted straight off the Tornado examples, with a few minor edits. Here’s our Python application that’ll run on the server:
#!/usr/bin/python2 import tornado.web import tornado.websocket import tornado.ioloop class MainHandler(tornado.web.RequestHandler): def get(self): self.render("front.html") class WebSocketHandler(tornado.websocket.WebSocketHandler): def on_message(self, message): self.write_message(u"Server echoed: " + message) application = tornado.web.Application([ (r"/", MainHandler), (r"/websocket", WebSocketHandler), ]) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()
Tornado is still undergoing heavy development, but the developers say that the code has been cleaned up enough so that, theoretically, only the parts of Tornado that need to be used have to be imported. It’s worked pretty well for me so far, so in the first three lines, I’m importing the core web module, the websocket module and the IO loop module (which ties all the modules together and runs the application).
Next up is the main handler class. In tornado, you have a bunch of classes, which define the methods you want to support as a function. In this case, the
MainHandler class supports the HTTP GET method, and thus, I’ve sub-classed the
tornado.web.RequestHandler class (which is the root class you need to sub-class in order to create a serveable object) and overridden the
get() method, so that it delivers the
front.html page to the browser.
Then comes the WebSocket handler, which is the same deal as the
MainHandler, except that WebSockets is a message-based protocol, so the only method you have to override is the
on_message() function. The
on_message() function is an event handler, which fires every time a message is received. As such, in our little program, all it does is echo back (using the
write_message() method) whatever we’ve said, as a Unicode string.
Next up, I’ve instantiated the application, and mapped the
/ URL to the Main Handler, and the
/websocket path to the WebSocket handler. In the next if-clause (which, for Python novices, is the equivalent of C’s
main() function), I’ve instructed the app to listen on port 8888, and started the IO loop. Once you execute this file, our server is running.
Now, we’ve got to interface with this on the client side. Note that I’m using the standard interface, so the only browser this will work on now is Google Chrome/Chromium. It should work on Firefox 11 and above, or you can use a
moz prefix to the WebSocket object to make it work with versions of Firefox later than version 6 (use
MozWebSocket instead of
Without further ado, here’s the
front.html file, with the script embedded in it:
The file is standard HTML5 with a valid Doctype (you need to use HTML5 specifically, because WebSockets is an HTML5 feature). There are all sorts of in-line styling, because that’s one less file to send off for publication. Believe me, the number of mistakes in publishing is directly proportional to the number of files my article is split into ;-) The message is: in production code, keep your styling information separate. Use a CSS file. I’ve also used the Lekton font and large sizes — this keeps the screenshot legible. Lekton isn’t a Web-safe font. I have it installed, but most don’t. However, it’s available from Google Web Fonts if you want to use it, so it’s all good!
Coming down to the form, you’ll see that I’m hooking the
onsubmit event takes care of triggering the appropriate JS function when I submit the form.
Now, let’s come down to the script.
First of all, I’ve created a new WebSocket object. For versions of Firefox prior to version 11, you’ve got to use new MozWebSocket here. Anyway, the WebSocket points to the appropriate URL that we mapped our WebSocket handler to, in the Python server app. Notice the protocol:
ws. If you want to use WebSockets over SSL, the protocol is
Just like in the server app, I need to override the
onmessage() function, which again is an event handler that fires every time the server sends a message. What I’ve done here is a simple three-step procedure: I created a new HTML paragraph, put whatever the server sent into the paragraph, and appended it inside the big
Now let’s look at the more interesting
- Here, I first get the contents of the input field and store it in a variable, and immediately clear the field. After I’m done with the field, and provided that the function has been properly coded, you can now use the input field again.
- Again, I do exactly what I’d done in the
onmessage()function, displaying what I typed in the chatbox.
- And then, with the
ws.send()function, I sent what I typed to the server over the WebSocket.
That’s it. WebSockets is this easy to use.
Where to, from here?
Well, now that you know how to use a WebSocket, it’s time to leverage this thing.
Here are a few things to remember. A WebSocket is meant to be used with Web applications, and as such, is pretty difficult to use in standard websites with standard tools like PHP and Apache. Remember that applications like Tornado exist so that you can directly expose them to the world without a reverse-proxy in front. If you need to use authentication, you can do that through the WebSocket itself. You don’t need to use HTTP 1.1 authentication. You can, but again, it’s pretty darned difficult.
If you’re not going to use Socket.IO, you can check out CanIUse WebSockets, which is continually updated, to see a list of browsers that support WebSockets, along with the level of support they provide.
And finally, you may look at my JS code in the code listing above and laugh, cry or curse me to oblivion. Me, I’m simply waiting for browsers to start natively supporting CoffeeScript. Ha! Check out CoffeeScript if you don’t know what that is.
That’s all. Now go code.
Feature image courtesy: Phil Woodbridge. Reused under the terms of CC-BY-NC-SA 2.0 License.