Listening to Adam Bosworth's talk on scaling database stuff from the MySQL Users Conference. Adam is, as usual, very interesting to listen to, and quite inspiring.
It got me thinking about a side-project I have been pinging on for a while, setting up a solid cross-platform (which to me means Ruby -- Java -- Perl) messaging (as in MQ) system. I've been working with ActiveMQ because it is open, fast, and reliable + James has been very willing to help with cross-platform stuff. I think we have just been taking the wrong approach to cross platform.
Right now ActiveMQ has its openwire protocol. This protocol is derived from a bunch of constants defined in the Java sources, and ActiveMQ uses it to build a bunch of different marshalling/unmarshalling code in a variety of languages. In effect, the protocol is dereived from the capabilities of the source. This works if the goal is to provide the full capability of ActiveMQ on any platform, but... that is not what I want. I want basic, reliable, messaging between arbitrary platforms based on an easy wire protocol.
There is a SOAP spec for this. Okay, yeah, let's not go there.
So, what do we actually need? Well, a simple and effective protocol with a reasonable lowest common denominator -- text. We need to be able to subscribe, by event or by poll. Let's start with what a message looks like:
<message>
<headers>
<destination>http://messaging.example.org/queue/SAMPLE</destination>
<reply-to>auto</reply-to>
<x-header id="something">something else></x-header>
</headers>
<body>
Message Body Goes Here
</body>
</message>
That's it. Let's look at some details.
For the body element, we'll say that it must be text,
must. It may not be xml. I think sending xml may be great, but CDATA
it if you do. Encoding for XML in the body is CDATA, period. Clients
can run it through a parser themselves.
For the headers, I'll start right with the standard JMS headers. JMS works really well, good starting point.
So far, nothing fancy. It's worth dealing with JMS semantics as this is just a wire format (and I'll go ahead and spec a couple transports as well), ideally one which can have implementations against any messaging system. Universal type 4 jms driver ;-)
Okay, so we have a "send message" format. What about receiving? Let's define a message to say we are interested in receiving messages. As JMS has done a nice job of the heavy lifting, we'll go ahead and stick with semantics from that and just define a wire format for messages to subscribe:
<subscribe>
<destination>http://example.com/queue/SAMPLE</destination>
</subscribe>
So, that says we are interested, the subscribe operation may return
a result, depending on the transport. So far I am worried about this
transport-dependent stuff. We'll have to work that out. The
durable element is optional and says the subscription
is durable in jms terms. There is an unsubscribe
message as well to remove interest in a destination.
So, looking at the format, it is verbose and limited. I love it.
A REST style transport should be dirt simple, and may be the first go.
I think *too* complicated already. I want to drop a bunch of the JMS stuff to simplify further, but need to think about the right way to do that.
A TCP based transport is also dirt easy to envision. It is actually easier. May do this first, dunno. Need to think about it. For that I'd do an awfully thin wrapper around ActiveMQ's JMS API as a first implementation =)
Both will be slower than an optimized binary protocol, but WOMP (Wide Open Messaging protocol) is about interop, not about performance. ASN1 is more efficient than XML, after all...