Webapps are often built and managed by single person or small teams. For example, I run 11 sites and web apps, just in my spare time, on one virtual server. I need to know if that server is up. We need a lightweight, low/zero maintenance way to monitor applications and servers. Mainly we are concerned about presence of our application. Is it there ?

Where else does presence matter ? Why, in Instant Messaging applications of course. Currently we record presence of our friends in our messenger clients. Well, servers can be our friends too !

Jabber / XMPP

XMPP is the eXtensible Messaging and Presence Protocol. Originally know as Jabber (before it was standardised), it is an open, XML-based protocol for near-real-time, extensible instant messaging and presence information. It is currently used on thousands of Jabber servers across the Internet and is used by millions of people worldwide. The Internet Engineering Task Force (IETF) formed an XMPP Working Group in 2002 to make Jabber / XMPP an official standard.

When Google built Google Talk, they chose XMPP as the core protocol.

There are free (speech & beer) servers, server software and libraries for most languages. Sounds perfect.

Brief Jabber Glossary

The Jabber HOST is the domain name of your Jabber server. For example ‘jabber.org’, ‘gmail.com’ (if using Google Talk), etc. A JID (Jabber ID) (from Wikipedia):

Every user on the network has a unique Jabber ID (usually abbreviated as JID). To avoid the need for a central server with a list of IDs, the ID is structured like an e-mail address with a username and a DNS address for the server where that user resides separated by an at sign (@). For example, if the Montague family ran a Jabber server on the domain montague.net, then the user romeo might be assigned the address romeo@montague.net.

Since a user may wish to login from multiple locations, the server allows the client to specify a further string known as a resource, which identifies which of the user’s clients it is (for example home, work and mobile). This may then be included in the JID by adding a forward slash followed by the name of the resource. For example the full JID of Romeo’s mobile would be romeo@montague.net/mobile. Messages that are simply sent to romeo@montague.net will go to all clients that Romeo has logged in at the time, but those sent to romeo@montague.net/mobile will only go to his mobile.

The RESOURCE is the last part of the JID, usually ‘Home’.

A simple monitor in Python

Python’s main XMPP library is xmppy.sourceforge.net: Send a message.

    conn = xmpp.Client(HOST)
    conn.connect()
    conn.auth(USERNAME, PASSWORD, RESOURCE)
    conn.sendInitPresence()
    conn.sendMessage( xmpp.Message(' user@jabber.org', 'Hello User') )

Replace ‘user@jabber.org’ with your jabber id, and fill in HOST, USERNAME, PASSWORD and RESOURCE. USERNAME is the JID of the jabber account you created for your machine, for example ‘my_machine_xyz@jabber.org’. RESOURCE is usually ‘Home’.

If you are using Google, replace the ‘connect’ line with:

    conn.connect( server = ('talk.google.com',5223) )

If you just want to run the monitor, JIMBo (see later) is linked at the end of this article.

The connection will automatically use SSL / TLS if it is available. The Google Talk, for example, will use it.

Next you need to allow yourself to view the machine’s presence:

    roster = conn.getRoster()
    roster.Authorize(' user@jabber.org')

Add ‘USERNAME’ as a Buddy in your Jabber IM client and you’ll know when your machine is online or offline. Do the above as a thread in a Java server, and you know when the server is running. Set an alert when your buddy ‘USERNAME’ goes offline and you have server monitoring !

You don’t even need to run your own server: There are lots of public Jabber servers (such as jabber.org), Google Talk has some, and Google Apps For Your Domain will even run one for you on your domain !

A simple bot

But wait, there’s more. Now that our machine is our friend, we probably want to chat to it:

    def onMessageReceived(conn, mess):

       sender = mess.getFrom()
       text = mess.getBody()

       print str(sender) +" send: "+ str(text)

    conn.RegisterHandler('message', onMessageReceived)

We can do anything we want on our server, remotely. We have a bot.

An aside: Jabber for you

Jabber servers have gateways to other IM’s: AIM / Yahoo / MSN / ICQ / etc. Get a Jabber id and you don’t need a multi-protocol client anymore, the gateways do it for you. With your own server, or Google Apps For Your Domain, your Jabber id can be the same as your e-mail address !

Machines Jabber the world

If we can talk to our server, so could another server. And when two machines talk to each other, enterprise types call the bit in the middle a Messaging Solution ! Consultants will want you to buy an Enterprise Service Bus, to form the basis of your new Service Oriented Architecture. Ignore all that.

Jabber has Publish / Subscribe.

Jabber / XMPP makes it easy to wire machines together. The servers and protocols have millions of users. Find an ESB that can boast that. And you can have redundancy too: ejabberd has it built in, or get your machine to add all the redundant servers into a Buddy Group (named after the service they provide for example), then cycle through all of them until it finds one that is online !

How deep is this rabbit hole ?

The more you look into Jabber, the more interesting it gest. Probably the most widely deployed Jabber server, ejabberd, the one running at jabber.org, is written in Erlang.

Erlang, developed then open-sourced in 1998 by Ericsson Computer Science Laboratory, is a soft realtime, declarative, functional language for concurrent, distributed systems. In other words, it’s not like most languages you will have used (most likely procedural and object-oriented languages), except at college if you studied Computing.

But apparently Erlang has amazing light-weight thread support.

And there’s a web server written in Erlang: YAWS (Yet Another Web Server). And when you lookup YAWS, you find this graph:

Source: http://www.sics.se/~joe/apachevsyaws.html

On the left is web server throughput in KBytes/second. Along the bottom is number of concurrent requests on a web server. In green is Apache and in red is YAWS. What this is saying is that (on the hardware of the test) Apache dies where there are over 4,000 concurrent requests. YAWS is still going at 85,000. All the detail here: Apache vs. Yaws.

So that’s pretty interesting too.

Download JIMBo

Download the Jabber Instant Monitoring BOt / JIMBo. Linux / Unix only. Installation instructions are in the zip. GPL license. v0.2.2 now includes auto-reconnect, proper daemon behaviour, and Gentoo and RedHat startup scripts

I will be presenting this at BarCamp2London this weekend