Jul 302009
 

[I posted this to the Arduino developer's mailing list, but figured others might find it useful too]

When I first started with Arduino, I thought Serial.available() was a very loose wrapping of the RXC bit in the USCRA register, i.e. if I didn’t get data out of there fast, it’d be gone. That led to convoluted code like:

 if( Serial.available() ) {
   val1 = Serial.read();
   while( !Serial.available() );
   val2 = Serial.read();
   // and so on
 }

Yuck. So you end up designing protocols that are too terse. Or maybe you think you need to buffer so you don’t lose it:

 while( Serial.available() ) {
   commandbuffer[i++] = Serial.read();
 }

Then parsing becomes a two step process: read serial, parse buffer. Confusing to newbies perhaps, but at least it allows for a better protocol down the line. (And descend into madness as you gaze into the maw of strtok())

Because Serial contains these big comfy buffers, we often don’t need a second buffer to let us easily implement good protocols. My current favorite is to do something like this:

 // protocol is "CCaaaa", two bytes of command, four bytes of args
 if( Serial.available() >= 6 ) {  // command length is 6 bytes
   cmd0 = Serial.read();
   cmd1 = Serial.read();
   arg0 = Serial.read();
   arg1 = Serial.read();
  // ...etc...
 }

I don’t think I’ve seen any Serial examples that check for a specific number of bytes available. It’s really handy.

Implementing a human-friendly protocol like “command arg0 arg1 arg2″, where command and args are space-separated strings like “servo 12 0xff”, is currently hard with Serial. I do this right now with a 2nd buffer and lots of C hackery:

 char* cmdbuf; char c; int i;
 while( Serial.available() && c!= '\n' ) {  // buffer up a line
   c = Serial.read();
   cmdbuf[i++] = c;
 }

 int i = 0;
 while( cmdbuf[++i] != ' ' ) ; // find first space
 cmdbuf[i] = 0;          // null terminate command
 char* cmd = cmdbuf;     //
 int cmdlen = i;         // length of cmd

 int args[5], a;         // five args max, 'a' is arg counter
 char* s; char* argbuf = cmdbuf+cmdlen+1;
 while( (s = strtok(argbuf, " ")) != NULL && a < 5 ) {
   argbuf = NULL;
   args[a++] = (byte)strtol(s,NULL,0); // parse hex or decimal arg
 }
 int argcnt = a;         // number of args read

This sort of functionality would be great in a library I think. Maybe not in Serial, but a core class.

Any other protocols people like to use and the Arduino code they use to do it?

 Posted by at 8:50 pm

  34 Responses to “Arduino Serial protocol design patterns”

  1. Hi todbot & Rob -

    *Loved* the code for serial communication- many many thanks…

    In using it I noticed it was getting so far and it was freezing.. going through the code (Rob re-wrote) I noticed that the malloc was been run and using up all the memory.. so put

    free(cmdbuf);

    somewhere when you are clearing variables and you shouldn’t have the same problem :D

  2. Yes, it’s easy to run into many traps when using malloc(). Since the memory space of the chip in Arduino is so small and the tasks being performed are usually fairly simple, I much prefer to use statically-allocated buffers. You never run the risk of memory leaks then and you can know at compile-time exactly how much RAM you have left. (Arduino doesn’t yet expose this however, but the “avr-size” command can give you static RAM usage)

  3. but
    while( Serial.available() && c!= ‘\n’ ) {
    requires that Arduino while is waiting for the endline, cannot do anything other.

    I think it is a big problem to solve.

    r.

  4. no, sorry… I mistaked!!! :)

  5. Hi ryuujin,
    There is the possibility for the lockup you fear when doing something like:

    while( Serial.available() < 5 ) {  // wait for 5 bytes
      // do nothing
    }
    

    If the host never sends 5 characters, it'll just hang. One wait to get around this is to have a timeout:

    long start_time = millis();
    while( Serial.available() < 5 ) {       // wait for 5 bytes
      if( (millis() - start_time) < 3000 )  // wait at most 3 seconds
        break;
    }
    
  6. [...] The best way (I think) is to structure everything as fixed-length messages along the lines of this example. Before you try to talk with EMC, write a Python program that sends and receives messages from the [...]

  7. Hi, very intresting post. I making Rc plane controle by Arduino and Xbee i try to make a light protocol. How could i do to send over serial floar data like lator lng gps data. Could you help me. Protocol seems like this
    Start frame 1 byte
    Command 1 byte
    Data 10 bytes
    Checksum 1 byte
    End frame 1 byte

    But 10 bytes to pass latitudelike -2.153627728 is a good idea?

    Thanks

  8. Hi, very good post. I try to make an RC plane where serial communication is very important.
    I wrote this

     long start_time = millis();
     
     while(Serial.available()){
    
       // Permet de rendre la main au reste du programme
         if((millis() - start_time) > 10 )  { // wait at most 10 ms
              break;
            }
          
          // Lecture du byte arrivé  
          byte theByte = Serial.read();
          //Nouvelle trame et que nous sommes pas dans la construction d'une trame ou la trame met trop longtemps a arrivé
          if((theByte == START_TRAME && newTrame == false) || (millis() - DEB_START_TRAME>OUT_OF_TIME_TRAME))
          {
            newTrame = true;  
            nbBuf = -1;
            DEB_START_TRAME = millis();
          }
          else {  
            // Cas ou nous sommes en construction d'une trame
            if(nbBuf>=BUFFER_SIZE)
            {
              // Cas ou le buffer et plein mais nous il ne s'agit pas de la fin de la trame
              // alors on sait que la trame est corrompu pas besoin de la traiter
              if(theByte != END_TRAME)
              {
                 newTrame = true; 
                 nbBuf = -1;
                 
                 for(int i = 0;i<BUFFER_SIZE;i++)
                 {
                   Serial.print(cmdbuf[i]);
                 }
                 Serial.println(" Erreur de trame - ");
              }
              else {
                 // Verification du checksum car la trame est valide
                 // On pointe sur le pointeur cmdbuf
       
                  if(cmdbuf[11] == checkSum())
                  {
                    // Checksum ok
                          Serial.println("Checksum ok");
                        // Ici la trame est ok donc on peux executer une commande en call back  
                        Serial.print("Commande : ");
                        Serial.print(cmdbuf[0]);
                        long val  = ((long )cmdbuf[9]) << 8;
                        val |= cmdbuf[10];
                        Serial.print("Val : ");
                        Serial.print(cmdbuf[9],DEC);
                  }
                  else {
                    // Checksum error $A0000000080I%
                      Serial.print("Erreur dans le check sum : ");
                      Serial.println(cmdbuf[11],DEC);
                      Serial.print("Compute checksum : ");
                    
                      Serial.println(checkSum(),DEC);
                  }
              }
              break; 
            }
            else {
             //Decodage d'une trame en cours 
              cmdbuf[++nbBuf] = theByte;
              
              Serial.print(nbBuf,DEC);
              Serial.print(" - ");
              Serial.println(theByte);
            } 
          }
      }
    

    In human readable way, the packet is : $A0000000080I%
    where $ start trame
    A command
    0000000080 Data
    I Checksum
    % end trame

    How to pass float data ?

  9. “How to pass float data ?”
    The arduino converts the value according to what it thinks it is, so declaring it a float arduino-side should work just fine..

    NICE post! far the most usable I have seen the past few days on the subject, tnx.. Btw making me a wifi robot :)

  10. hi guys,

    I’ve read kasper and todbot’s posts about the leds. I am trying to store 4 values in the arduino as well. I am using the code below, in which i expect to see the value for xStep printed. Instead of that I always get 32 returned by the serialmonitor, does anyone know why?

    cheers, tim


    #define stepPin 26
    #define dirPin 28
    #define enablePin 24

    byte cmd;
    byte xStep;
    byte yStep;
    byte zStep;
    int i = 0;

    void setup()
    {
    // We set the enable pin to be an output
    pinMode(enablePin, OUTPUT);
    pinMode(stepPin, OUTPUT);
    pinMode(dirPin, OUTPUT);

    digitalWrite(enablePin, LOW);
    digitalWrite(dirPin, LOW);
    digitalWrite(stepPin, LOW);

    Serial.begin(9600);
    Serial.print("Program Initiated\n");
    }

    void loop() {
    if( Serial.available() == 4 ) {
    cmd = Serial.read();
    xStep = Serial.read();
    yStep = Serial.read();
    zStep = Serial.read();
    Serial.println(xStep);
    }

    for(i=0; i<xStep; i++){
    stepx_p();
    delay(10000);
    }

    }

    void stepx_p(){
    digitalWrite(stepPin, HIGH);
    delayMicroseconds(2);
    digitalWrite(stepPin, LOW);
    delayMicroseconds(2);
    }

  11. Hi Tim,
    What does your code look like that is sending data to your Arduino?

  12. Nanopb (Google protocol buffers for microcontrollers) are an effective solution for a wide range of these problems:

    http://koti.kapsi.fi/~jpa/nanopb/docs/

    Note that when sending multiple messages to the Serial stream, there is nothing about a PB message that delimits it from the next one. Google has some great advice on how to deal with that here:

    https://developers.google.com/protocol-buffers/docs/techniques

  13. I wrote simple JSON-RPC (Remote Procedure Call) library for Arduino:
    http://www.cloud-rocket.com/2014/03/serial-json-rpc-server-arduino/

  14. I found another more reliable method. Basically just do nothing until the next character is available to be read. This gives a nice clean minimal input string:

    String inData ="";
    char ch;
    if (Serial.available() > 0)
    {
    //***********************
    //READ
    ch = Serial.read();
    while(ch != ';')//my end of message very reliable character. Then i don't have to worry about
    //newlines and carriage returns
    {
    inData += ch;
    while (Serial.available() <=0)
    {}
    ch = Serial.read();
    }

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>