View on GitHub

Spectrum backend protocol

Spectrum 2 communicates with backend using protocol based on Google Protocol Buffers. This protocol has libraries for many languages, so backend can be written practically in any language.

This document describes how to implement new backend in different language than C++.

Backend execution

When XMPP user wants to login to legacy network, Spectrum 2 executes particular backend according to configuration. Typically it executes backend with following arguments:

/usr/bin/backend --host localhost --port 31640 --service.backend_id=1 "/etc/spectrum2/transports/sample_client.cfg"

As the first thing after start, backed should connect Spectrum 2 main instance according to --host and --port argument. --service.backend_id is ID of backend and the last argument is always path to config file. Note that Spectrum 2 can pass more arguments to backend and backend should ignore them.

When Spectrum 2 starts the backend, you should see this in spectrum.login

INFO  NetworkPluginServer: Backend should now connect to Spectrum2 instance. Spectrum2 won't accept any connection before backend connects

When you establish the connection between your backend and Spectrum 2, it shows this in log:

NetworkPluginServer: New backend 0x84ad60 connected. Current backend count=1

Example code

Checkout the existing template code for creating new generic spectrum2 backends at backends/template/. These examples are just templates to get you started with your own backend.

Python-based backends can use the pyspectrum2 package for communicating with Spectrum.

The protocol

When connection betwen backend and Spectrum 2 is establish, Spectrum 2 starts communicating with the backend.

The protocol is defined in in include/transport/protocol.proto file. Usually there is tool for your language to compile this .proto file into source module which can be later used to serialize/deserialize structures in your language and make packets from them.

Once you serialize particular Protocol Buffer structure, you can send it in following format:

| size | serialized Protocol Buffer structure |

Receiving PING message

First message sent by Spectrum 2 is always PING. Backend has to response this message with PONG.

You will see this event also in Spectrum 2 log:

NetworkPluginServer: PING to 0x84ad60 (ID=)

Your backend will receive 4 bytes in network encoding. You should parse those 4 bytes to number representation and store it into variable N. Then you should read N bytes from the socket to receive serialized Protobuf message called WrapperMessage. Use the module generated from protocol.proto to parse it.

WrapperMessage has two fields, type and payload:

In our case, the type is TYPE_PING and the payload is not defined. You should now answer the PING message

Answering PING message

PING message has to be answered with PONG. To Answer this message, just generate WrapperMessage using the module generated from protocol.proto and set the type to TYPE_PONG. Now serialie the message, prepend it with its size as defined above and send it to Spectrum 2.

Once you answer the PING request properly, Spectrum 2 will show following message in log:

Component: Connecting XMPP server 127.0.0.1 port 5347

Packets

Spectrum 2 main instance and its backends communicate using TCP socket. Packets sent over this socket are serialized using Google Protobuf library. This library has bindings (unofficial) for many programming languages.

Packets look like this:

Byte Description —— ————————————————————————————————————————– 0-4 Length of payload sent in this packet in network byte order as generated by uint32_t htonl(uint32_t hostlong); method. 4-x WrapperMessage payload serialized using Google Protobuf.

WrapperMessage payload

This Protobuf object is used to contain payloads of different types. Every payload type has its own type and Protobuf object representing it. All payloads are explained later in this document.

All objects which can be used as payload in WrapperMessage object are declared in include/transport/protocol.proto.

Spawning the backend

1. Spectrum 2 main instance spawns the backend when it’s needed. Backend is executed with following arguments: "--host localhost --port 32453".\

  1. Backend has to initalize itself and connect the Spectrum 2 main instance using host and port.\
  2. Spectrum 2 main instance sends packet with payload of TYPE_PING (stored in WrapperMessage object).\
  3. Backend answers the TYPE_PING payload with TYPE_PONG payload and since the requests are forwarded to it.

User wants to login the legacy network

1. Type: TYPE_LOGIN, Payload: Login packet is sent to backend, backend starts connecting the user to legacy network.\

  1. When the user is connected, it populates his roster using Type: TYPE_BUDDY_CHANGED, Payload: Buddy packets.\
  2. It sends Type: TYPE_CONNECTED, Payload: Connected packet to Spectrum 2 main instance to inform it that user is connected to legacy network.

If something goes bad during the login process or later, backend sends Type: TYPE_DISCONNECTED, Payload: Disconnected packet at any time.

User wants to join the room

1. Type: TYPE_JOIN_ROOM, Payload: Room packet is sent to backend, backend starts joining the user to the room.\

  1. Backend populates room’s participant list using Type: TYPE_PARTICIPANT_CHANGED, Payload: Participant packets. As last Participant it has to send the user itself.\
  2. Backend can change room subject using Type: TYPE_ROOM_SUBJECT_CHANGED, Payload: ConversationMessage.

WrapperMessage payloads sent by Spectrum 2 main instance

This chapter describes all possible payloads which can be sent by Spectrum 2 main instance to backend and therefore received by backend. It also describes what should backend do when it receives payload of that type and what should sends back to Spectrum 2 main instance.

Type: TYPE_PING, Payload: Nothing

Backend has to responds with Type: TYPE_PONG, Payload: Nothing payload. If it does not responds in 20 seconds, it’s terminated.

Type: TYPE_LOGIN, Payload: Login

Backend has to login this user to legacy network. When it logins the user, it has to send Type: TYPE_CONNECTED, Payload: Connected back to transport. If the backend is not able to login the user, it has to send Type: TYPE_DISCONNECTED, Payload: Disconnected

Variable Description ———— ————————————————————— user JID of XMPP user who wants to login legacyName Legacy network user name (for example ICQ number) of the user password Legacy network password

Type: TYPE_LOGOUT, Payload: Logout

Backend has to logout user from legacy network.

Variable Description ———— ————————————————————— user JID of XMPP user legacyName Legacy network user name (for example ICQ number) of the user

Type: TYPE_STATUS_CHANGED, Payload: Status

Backend has to change the legacy network status of this user.

Variable Description ————— ——————————————————– userName JID of XMPP user status Status as defined by StatusType enum in protocol.proto statusMessage Status message

Type: TYPE_JOIN_ROOM, Payload: Room

Backend has to join this user to the room. Backend has to send list of participants back to Spectrum 2 using Type: TYPE_PARTICIPANT_CHANGED, Payload: Participant. As a last participant, the user who wanted to join the room has to be sent.

Variable Description ———- ——————————– userName JID of XMPP user nickname Nickname used to join the room room Name of the room password password

Type: TYPE_LEAVE_ROOM, Payload: Room

Backend has to disconnect this user from the room.

Variable Description ———- ——————————– userName JID of XMPP user nickname Nickname used to join the room room Name of the room

Type: TYPE_BUDDY_TYPING, Payload: Buddy

Backend should forward to buddy represented by buddyName information that XMPP user is typing now.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_TYPED, Payload: Buddy

Backend should forward to buddy represented by buddyName information that XMPP user paused typing now.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_STOPPED_TYPED, Payload: Buddy

Backend should forward to buddy represented by buddyName information that XMPP user stopped tying and is not typing for some time.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_REMOVED, Payload: Buddy

Backend should remove buddy from the legacy network contact list.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_CHANGED, Payload: Buddy

Backend should change the buddy alias or group according to Buddy payload. If the buddy is not stored in legacy network contact list yet, it should add it there. Backend has to send Type: TYPE_BUDDY_CHANGED, Payload: Buddy back.

Variable Description ———– ————————————– userName JID of XMPP user buddyName Name of the buddy in legacy network alias Alias group Group blocked True if this buddy should be blocked

Type: TYPE_ATTENTION, Payload: ConversationMessage

Backend should forward to buddy represented by buddyName information that XMPP user wants his attention.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network message Message

Type: TYPE_CONV_MESSAGE, Payload: ConversationMessage

Backend should forward to buddy represented by buddyName message received from user.

Variable Description ———– ————————————————— userName JID of XMPP user buddyName Name of the buddy in legacy network message Plain text message xhtml Message formatted using XHTML-IM XEP if available

Type: TYPE_VCARD, Payload: VCard

If buddyName is empty, backend should update XMPP user’s VCard according to VCard payload. If buddyName is not empty, backend has to fetch VCard of buddyName buddy including photo send it back using Type: TYPE_VCard, Payload: Buddy

Variable Description ———– ——————————————————————————————————————————– userName JID of XMPP user buddyName Name of the buddy in legacy network id Id used when sending the response with buddy’s photo back. You have to use the same id in response as you received in request. photo Binary photo nickname Nickname

WrapperMessage payloads sent by backend

This chapter describes all possible payloads which can be sent by backend to Spectrum 2 main instance.

Type: TYPE_CONV_MESSAGE, Payload: ConversationMessage

Backend sends this payload when it receives new message from legacy network which should be forwarded to XMPP user.

Variable Description ———– ——————————————————————————————————————————————————— userName JID of XMPP user buddyName Name of the buddy in legacy network who sent the message. If the conversation is room, buddyName is name of the room. message Plain text message xhtml Message formatted using XHTML-IM XEP if available nickname If the conversation is room, this is the nickname of user who sent the original message carbon If set, the message is a carbon copy of our own message sent in a different legacy network client. It should be treated as a message FROM us, not TO us

Type: TYPE_ATTENTION, Payload: ConversationMessage

Backend sends this payload when it receives attention request from legacy network which should be forwarded to XMPP user.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network message Message

Type: TYPE_VCARD, Payload: VCard

Backend sends this payload as a response to Type: TYPE_VCARD, Payload: VCard received from main Spectrum 2 instance.

Variable Description ———– ——————————————————————— userName JID of XMPP user buddyName Name of the buddy in legacy network id You have to use the same id in response as you received in request. photo Binary photo nickname Nickname

Type: TYPE_ROOM_SUBJECT_CHANGED, Payload: ConversationMessage

Backend sends this payload when it receives room subject from legacy network which should be forwarded to XMPP user.

Variable Description ———– ————————————— userName JID of XMPP user buddyName Name of the room. message Plain text subject. nickname Nickname of user who set the subject.

Type: TYPE_BUDDY_TYPING, Payload: Buddy

Backend sends this payload when buddy starts typing.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_TYPED, Payload: Buddy

Backend sends this payload when buddy paused typing.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_STOPPED_TYPED, Payload: Buddy

Backend sends this payload when buddy is not typing anymore.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_BUDDY_CHANGED, Payload: Buddy

Backend sends this payload when some information about Buddy changed.

Variable Description ———– ————————————– userName JID of XMPP user buddyName Name of the buddy in legacy network alias Alias group Group blocked True if this buddy should be blocked

Type: TYPE_BUDDY_REMOVED, Payload: Buddy

Backend sends this payload when it removes buddy from legacy network contact list or the buddy gets removed from the contact lists somehow.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_AUTH_REQUEST, Payload: Buddy

Backend sends this payload when it receives authorization request from legacy network. If XMPP user accepts the authorization request, Spectrum 2 main instances sends Type: TYPE_BUDDY_CHANGED, Payload: Buddy to backend, otherwise it will send Type: TYPE_BUDDY_REMOVED, Payload: Buddy.

Variable Description ———– ————————————- userName JID of XMPP user buddyName Name of the buddy in legacy network

Type: TYPE_CONNECTED, Payload: Connected

Backend sends this payload when it connects the user to legacy network.

Type: TYPE_DISCONNECTED, Payload: Disconnected

Backend sends this payload when it disconnects the user from legacy network.