UP | HOME

XMPP

XMPP is almost 20 years old now, and I've been using it (on a private server for the past few years, and additionally a local instance working over Tor for the past month) while looking for ways to get distributed messaging. New protocols and technologies are fun, and I don't quite like XML, but XMPP has good documentation, works, can be used in either federated or distributed fashion (as most of the other federated systems, coupled with Tor or a similar network), and has plenty of software – so now I'm thinking that it may be sufficient, at least for messaging.

Its specifications (see RFC 6120, RFC 6121, RFC 7622, and extensions) are well-written and apparently thought through, a small core with optional extensions is a nice approach, there is usable software, there are users, and even XML is not that bad (perhaps even quite suitable for a protocol like that, especially while not being abused).

Though the situation with software could have been better.

1 Message handling acknowledgements

Message delivery is pretty unreliable with some of the major clients and servers, since acknowledgements are not always supported – making it worse than email and approximately as IRC, with respect to reliability of delivery. I'd think XEP-0198 should be among the first extensions to implement (actually XEP-0387: XMPP Compliance Suites 2018 includes it, but just for advanced implementations – along with things such as user avatars), and usually assumed that it's there (without bothering to check, since didn't care much about XMPP, and assumed that it'll be dealt with anyway, because the protocol and the software are popular), but among the three XMPP implementations which I use:

  • libpurple (Pidgin, bitlbee, etc) didn't have it (though had a ticket open for 7 years). Submitted a PR a few days ago.
  • Prosody only has a community module.
  • jabberd2 implements a very dated version of it (about 10 years old; it didn't even have a "draft" status back then). Reported now, and switched to Prosody completely, since there were other issues with jabberd2 which Prosody doesn't have.

2 Client certificates (and a bit more)

Certificate-based client authentication is not an essential feature, but I wanted to get rid of passwords as I did with email, so investigated that too.

Prosody has a a bunch of useful modules for authentication, including dovecot SASL and client certificate-based authentication (via SASL EXTERNAL authentication mechanism), but client support is rather bad:

  • libpurple doesn't do it, it's not clear whether it'd be welcome there, and apparently it won't be quick or easy to add (not just technically, but because of a relatively slow/unresponsive process1).
  • bitlbee's own XMPP implementation seems to have even more limited SASL support, and also has issues similar to those of libpurple, caused by its API (which bitlbee reuses for its implementation as well): XMPP streams aren't getting closed properly there either, potentially leading to message loss in conditions where it shouldn't happen.
  • weechat's jabber.py uses an old and abandoned Python library, which implements a now-obsolete RFC even for the core; there are just PLAIN and DIGEST-MD5 authentication mechanisms (SCRAM is required to be available for authentication by the current specification).
  • mcabber's loudmouth library also doesn't seem to support client certificates (or XEP-0198, or SCRAM).
  • Emacs's jabber.el uses sasl.el, but that doesn't include EXTERNAL. Also no XEP-0198 (though I'm checking that and stream closing just occasionally, probably none of the others had it either). And it looks abandoned, too.
  • Conversations, an Android client in Java which I've checked out of curiosity, does support both client certificates and XEP-0198, and many other things. It also has documented observations on implementing XMPP. Though has some warts, such as long if-else-if chains, disconnecting after Integer.MAX_VALUE is reached, and apparently more.
  • ChatSecure's XMPPFramework doesn't seem to support it, but supports XEP-0198. And it's also for mobile devices anyway.
  • Profanity uses libstrophe, and both look nice at a glance, but no SASL EXTERNAL mechanism in libstrophe, and XEP-0198 implementation got stuck between libstrophe and Profanity for more than 2 years now.
  • Psi is a C++ monster, it may or may not be there; quick search reveals very few traces of SASL. Apparently XEP-0198 at least was there at some point.
  • Vacuum-IM is a smaller C++ monster, with SASLFeatureFactory and such; seems to only support ANONYMOUS, PLAIN, and DIGEST-MD5 (roughly as with weechat), so not even quite a valid XMPP implementation. No XEP-0198 in sight.
  • pontarius-xmpp is a Haskell client library, but checked it at once: there are just PLAIN, DIGEST-MD5, and SCRAM-SHA-1. No XEP-0198, and apparently it's tricky to add there.
  • Gajim/nbxmpp supports plenty of XEPs, including 0198, and various SASL mechanisms, apparently including EXTERNAL. OTR and OMEMO are available via plugins, and OpenPGP – by default. Pretty good overall, apart from UI (which is actually fine, just not Emacs), and no apparent way to hook it into Emacs. There is a D-Bus interface, though it doesn't include events such as new message receiving.
  • Dino I didn't check closely, but it seems to be similar to Gajim in supported features (claims to support XEP-0198, doesn't seem to support OTR, but supports OMEMO, and no SASL EXTERNAL in sight).
  • gloox, a C++ library without C API (hence not very reusable directly): supports a bunch of XEPs (including 0198), as well as SASL EXTERNAL, but no SCRAM (which is required by the standard).
  • Swiften, also a C++ library without C API, claims to be pretty good, seems to support XEP-0198, as well as SASL EXTERNAL and SCRAM-*. Much more of C++ than I'd be comfortable with, and apparently not much of documentation (had to check sources to find out what it supports – as with many client programs, but libraries are expected to provide better docs), but looks good functionality-wise.
  • SleekXMPP and slixmpp, a Python library and its fork: also not reusable as a library from arbitrary programs, but supports EXTERNAL and SCRAM methods, and many pluggable XEPs.

3 Encryption

The primary options are:

  • OpenPGP (obsolete XEP-0027, deferred XEP-0374), which is also commonly used for other purposes, has nice software such as gpg-agent, supported by smart cards, etc. Though it doesn't provide forward secrecy, has a relatively high overhead, and generally not that fancy.
  • OTR (deferred XEP-0364), which is generic, IM-oriented, and seemingly nice, but has (and leads to) annoying issues, partially caused by nuances of XMPP resource handling by clients. Leads to message loss all too often, with different client combinations.
  • OMEMO (experimental XEP-0384), which provides forward secrecy like OTR, offline and multi-user messages like OpenPGP, using a relatively new double ratchet algorithm developed for Signal, with some vendor-specific bits in the XEP (and its homepage is just a single page with little information on the Conversations website). The properties are nice, but no standalone specification or white paper in sight, the cryptographic bits are included right into XEP, and apparently there's no reusable library, increasing the risk of screw-ups in client-specific implementations.

All those have drawbacks (with no corresponding XEP being recommended for use by XMPP Standards Foundation), though any one would have been useful if it was implemented consistently and used commonly. But in my experience end-to-end encryption is mostly used by those who use private (somewhat trusted) servers and TLS anyway, while users of public servers rarely bother to enable those even when they are available in their clients.

4 Summary

Here's a table summarising support of some features in XMPP implementations (libraries, clients, servers; feature support via external plugins is counted; "C API" only applies to libraries):

Implementation SCRAM EXTERNAL XEP-0198 C API Language
Pidgin/libpurple + - - + C
Prosody + + +   Lua
jabberd2 + - -   C
bitlbee + - -   C
weechat/jabber.py - - ?   C/Python
mcabber/loudmouth - - - + C
Conversations + + +   Java
ChatSecure/XMPPFramework + - +   Java
Profanity/libstrophe + - - + C
Psi ? ? ?   C++
Vacuum-IM - - -   C++
pontarius-xmpp + - - - Haskell
Gajim/nbxmpp + + + - Python
Dino + ? +   Vala
gloox - + + - C++
Swift/Swiften + + + - C++
SleekXMPP/slixmpp + + + - Python

5 Improving the software

While patching libpurple's jabber plugin, some minor libpurple-specific issues surfaced (related to both this XEP and XMPP in general), as it often happens with generic interfaces: they rarely fit well. That's one of the reasons why it's more fun to work on new implementations than to improve existing software: the latter is harder, takes more time for the same features (both implementation and organisational chores), and even then the features barely (if at all) fit into existing APIs.2 But in order to get somewhere, there has to be incremental improvement (which may be long and boring sometimes), and it looks like there's not enough of it.

On the other hand, some features are impossible to cram into libpurple without changing the APIs, and even libpurple, while being reused by multiple clients and for multiple protocols, implements XMPP from scratch instead of reusing a library; though there seem to be just libstrophe and loudmouth out of actually reusable (from any common language via C FFI) ones, but they seem to have more issues than features. So here's that situation once again, in which there's many implementations already, but it feels like the right thing to do may be just to start over, and add one more – because none are perfect.

A compromise may be to try to get at least important and basic things to work first, to get as far as practical with existing software, and only proceed to new implementations and experimentation then.

If one was to start over, apparently the proper way (leading to reusable and portable code, relying on mature and complete libraries which wouldn't introduce unnecessary limitations) would also be a boring one, along the lines of:

Though GHC's foreign export may also be suitable to get a reusable library, and to avoid C.

Slightly related: Software extensibility.

Footnotes:

1

Not to blame the maintainers, of course, who seem to continuously spend a lot of time on it, and it doesn't seem to be among the slowest or least responsive ones. But still, it's hard to start working on such a feature right away, and to be sure that time and effort won't be wasted.

2

It's similar with protocols, too, what's probably a part of the reason why there's so many of them (without even counting closed ones). And formats, and all kinds of things; sometimes it even seems to be easier (and again, more fun) to write a program than to learn how to use an existing one. Or to try a new one, at least, even if it's less advanced.