Message framing for zcm? #502

TL;DR- Given a serial array containing 10 message-types (e.g. a Lat/Lon message, a compass message, and an IMU message), how do you know which bytes correlate with which zcm-defined message types?

@jbendes I love zcm (and the Grand Challenge heritage), thank you for the fantastic documentation. My questions relate to transport on embedded systems. The documentation has made it clear that zcm can effectively serialize/deserialize. My confusion is around how zcm knows which defined message to use when deserializing. Let's say I have a file stored on a hard drive saving 10 different zcm message types in serial: how can I load those messages? Or I have a radio link transmitting/receiving 10 different zcm message types in serial: how do I know which messages to parse, or how to reject noise? I'm aware I may be asking the wrong questions for my problems, that's why I'm reaching out. I feel like there are well-established methods to solve these problems that I'm missing.

I've made home-brewed solutions, but I'm interested in learning the right way, rather than my way. As zcm is built with embedded systems in mind, I was wondering if you've ever done any kind of message framing for transmitting over serial connections.

I can give one example that solves this problem. In Mavlink for instance when the serialized message is sent over a Serial or UDP connection it frames all packets with this kind of structure:

Using Mavlink1 messages for a simple example of message framing.

Let's pretend I've integrated zcm with mavlink, and I'm using Mavlink's framing structure to send a latitude/longitude I've saved in latlon_t.zcm:

struct latlon_t {
    int32_t lat;
    int32_t lon;
}

Here's an example packet, assuming I created a mavlink message with MSG_ID 0xFE=254 with the above latlon_t:

Here's how a device receiving this message might process it.

• STX- "This byte matches the Start-of-text: "0xFE", this must be a mavlink message."
• LEN- (assuming 8) "This payload will be 8 bytes long."
• SEQ- "Hmm, the last sequence number was 5, but this sequence number is 7, I must have missed a packet."
• SYS ID- "This message is coming from vehicle 3."
• COMP ID- "This message is coming from component 2 on vehicle 3."
• MSG ID- "Hey this message ID 0xFE matches the one I use for a zcm latlon_t message"
• PAYLOAD- "Let's unpack the payload to a latlon_t. 8 bytes total, 4 for the lat, and 4 for the lon."
• CHECKSUM- "The CRC bytes match so I'm 99.998% sure the message is as-sent."

Message framing is valuable when transmitting data over a serial link, as well as when saving/loading data to/from a serial flash memory.

• Has zcm ever been integrated with any libraries that perform serialization?
• Am I asking the wrong questions, and do you know of other ways to solve the problem of identifying messages and ensuring accuracy in serial?

Thank you for reading.