RFID <--> Ethernet Shield Client <--> Rails

Last week A few months ago, I wrote an article on connecting the Parallax RFID reader to the the Arduino. The reason I was revisiting that device was because I am working on a system for our hackerspace which will allow people to find out who is in the hackerspace at any given time. The overall system requires the Arduino to connect to the internet as a client and tell the Ruby on Rails application that person with RFID tag XXXXXXXXXX has walked into or left the space. The Rails application will be able to ‘publish’ this information to different mediums [web, twitter, facebook, IRC, etc] by providing a simple API for other programmers in the space to utilize.

The whole system is too complicated to explain right now and I am not finished. I plan on writing my next article about that. For now, I just wanted to document some of the issues I had with the Arduino Ethernet Shield and how I resolved them as it may be of help to others trying to figure it out.

Arduino Ethernet Shield

Arduino Ethernet Shield

The ethernet shield, like the RFID reader, was not as simple to use as I hoped it would be. The information out there was decent, but nothing beyond explaining the examples. First I must explain my use cases to put context to the example code.

My goals for the code are to first send two pieces of information to the Rails application:

  1. A secret API key [to prevent fraud]
  2. The Tag code read by the RFID reader

Then I want to read the response and differentiate between four situations:

  1. The user has been marked as “Logged In”
  2. The user has been marked as “Logged Out”
  3. There is no user with the tag code that was sent
  4. The wrong API key was supplied

The ethernet shield allows your Arduino to operate as a Client or a Server. In this situation, I just want to use the client interface as I am connecting to a server to get information. Now let’s just look at the code required to make a simple HTTP GET request client with no parameters. I just modified this from the examples.

#include <Ethernet.h> // load ethernet SPI functions
 
// a MAC address that you make up to identify the Arduino
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
// this is the IP that will identify the Arduino
byte ip[] = { 192, 168, 0, 160 };
// this is the IP of the server that we are trying to connect to
byte server[] = { 192, 168, 0, 101 }; 
 
//create the client
//wide area HTTP networks are on port 80 but my local rails app is on 3000
Client client(server, 3000);
 
void setup()
{
  //start ethernet
  Ethernet.begin(mac, ip);
  Serial.begin(9600);
 
  delay(1000);
 
  Serial.println("connecting...");
 
  if (client.connect()) { 
    Serial.println("connected");
    //this constructs the GET request, it is the equivalent
    //going to the browser and entering http://192.168.0.101:3000/main/index
    client.println("GET /main/rfid HTTP/1.1");
    client.println(); // print a newline char to tell the server we are done?
  } else { // this is called if conenction fails
    Serial.println("connection failed");
  }
}
 
void loop()
{
  if (client.available()) { // need to see if response has been read into the buffer yet
    char c = client.read(); //get next character and print it
    Serial.print(c);
  }
 
  if (!client.connected()) { //if we lose connection
    Serial.println();
    Serial.println("disconnecting.");
    client.stop(); // disconnects from the server
    for(;;)
      ;
  }
}

Some of this warrants a little more explanation for some…

Configuring the ethernet shield

First is the MAC address.

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

For those unfamiliar with basic networks, a MAC address is a unique number assigned to your network card that never changes. You don’t need to know much about it other than you need to make it up. You should be good if you use the one I have just used in this example but keep in mind that if you have multiple Arduinos connected to the same network, you may need to make up unique MAC addresses for each one.

Next is your IP.

byte ip[] = { 192, 168, 0, 160 };

This array refers to the IP that you want your Arduino to take. The Arduino Ethernet Shield currently ?doesn’t support DHCP?, so you have to choose an unused IP yourself. There are a variety of ways and steps to do this. First what you need to do is figure out what the first three bytes in this array need to be. This is determined by the local router you are connecting to. For instance, Linksys is usually 192.168.1.XXX and D-Link, in my case, is usually 192.168.0.XXX. You can find out what your situation is by checking the IP of a computer on that network. Then you need to choose an unused number for the last byte, usually between 100 and 255. If you are on a network with few computers, around 150 is usually a safe bet. If you are not sure or have a lot of computers on the network, you should find a way to log into your router’s admin interface and check the DHCP information. It will tell you all the IP addresses that are currently assigned and you can make one up an unused one based off that.

Last is the server’s IP.

byte server[] = { 192, 168, 0, 101 };

This is the IP of the server that you are trying to connect to. In my case, it is a laptop running my Rails application connected to the same network. If you want to connect to a server outside your network, on the internet, you need to find it’s IP. Your computer usually does this under the surface using DNS but you will have to do it manually here. One way you can do this is by pinging the domain name and seeing what IP it resolves to.

Sending information to a server

So, this example works out for many situations, but how can I send information to the server where my Rails app is being run? In my case, I need to send the API key and and the RFID tag I just read. What we need to do is build a query string. If you aren’t sure what that means, you will have to read that wikipedia article carefully. Let’s reexamine the code:

//I changed this line
client.println("GET /main/rfid HTTP/1.1");
//to this line
client.println("GET /main/rfid?apikey=123&tag=0123456789 HTTP/1.1");

When I run this code, I see this on the serial port:

connecting…
connected
HTTP/1.1 200 OK
Connection: close
Date: Mon, 26 Oct 2009 01:12:43 GMT
ETag: “8da44df37e592a5020e852d50fc31a64″
X-Runtime: 8
Cache-Control: private, max-age=0, must-revalidate
Content-Type: text/html; charset=utf-8
Content-Length: 8

^=NOUSER

This is the HTTP response. The last bit:

^=NOUSER

Is what I have defined in the Rails application to tell the Arduino what has happened. Just for clarification, this is what the rfid function looks like in the Rails app:

  #login via rfid service, has no view
  def rfid    
    if params[:apikey] == $api_key
      @user = User.find_by_rfid_tag(params[:tag])
      if @user
        @user.in_space = !@user.in_space
        @user.save
        if @user.in_space
          render :text => "^=IN"
        else
          render :text => "^=OUT"
        end
      else
        render :text => "^=NOUSER"
      end 
    else
      render :text => "^=KEYFAILED"
    end
  end

I tested a few different query strings and made sure it worked for all conditions and it did. Now I need a way to dynamically send the tag that we read from the reader. As you might have read in my RFID post, I am storing the RFID tag in a 10 character array:

#define CODE_LEN 10
 
char tag[CODE_LEN];

I initially thought I could do this:

client.print("GET /main/rfid?apikey=123&tag=");
client.print(tag);
client.println(" HTTP/1.1");
client.println();

Notice that I am using a series of print()s to build a string and ending it with println() to close it out with a newline. The only problem is that this doesn’t work. My Mongrel server (the Rails server) was telling me that my HTTP request were invalid. It seems like a lot of other people were running into this problem as well. After temporarily giving up, I realized the problem. I was reading an essay by Brian Kernighan, one of the creators of the C programming language, about a simple regular expression parser that he uses to teach students about programming methodologies. Scanning through the code, I was reminded that a character array cannot be interpreted as a string without a terminal null character ‘\0′ as the last element. So, I handled this situation like this:

#define CODE_LEN 10
 
char tag[CODE_LEN + 1];
tag[CODE_LEN] = '\0';

This didn’t affect any of my other code and now the tag array can be interpreted as a legit string :) I love simple solutions.

Now I need a way to parse the response. The reason my responses had this format ‘^=XXXXXXX’ was so I could do something like this:

/**
 * Finds result that we are looking for in the returned HTTP response
 */
char getHTTPResult() {
  while (client.read() != '^') {}
  if (client.read() == '=') {
    return client.read();
  } 
  return 'x';
}

This would return the first character for the response I am looking for. For example, with the NOUSER message, I would get back the ‘N’ character. I implemented this but the problem is that it was really slow. Or at least, it was a lot slower than it could be. The problem is the amount of pointless HTTP data you have to read before you get to the actual message, or it could be that I am printing all that data? Doesn’t matter because I can make it faster. I could have configured my server to not spit out so much junk, or I could just place the message earlier in the response.

Reverse REST

The first thing you return is the status code so why not put it there? As I said before, there are only four states I am trying to identify and there are plenty of HTTP status codes to represent them. So I modified the Rails controller to look like this:

 
  #login via rfid service, has no view
  def rfid    
    if params[:apikey] == $api_key
      @user = User.find_by_rfid_tag(params[:tag])
      if @user
        @user.in_space = !@user.in_space
        @user.save
        if @user.in_space
          render :status => 400 #IN
        else
          render :status => 401 #OUT
        end
      else
        render :status => 402 #NOUSER
      end 
    else
      render :status => 403 #FAILED
    end
  end

I can’t help but wanting to call this ‘reverse REST’, LOL. Rerunning my simple “no user” test, I now get this:

HTTP/1.1 402 Payment Required
Connection: close
Date: Sat, 06 Feb 2010 02:45:19 GMT
X-Runtime: 12
Content-Type: text/html; charset=utf-8
Cache-Control: no-cache
Content-Length: 0

This is much quicker to parse. I probably could have used some nice code to make this more flexible, but just hardcoding the digit read process in is probably easier and just as reliable, if not more efficient:

char getHTTPResult() {
  while (client.read() != '4') {} 
  if (client.read() == '0') { //just to make sure
    return client.read();  // return either '0' or '1' or '2' or '3'
  } 
  return 'x';  // something bad happened
}

At this point a simple switch allows us to define the behavior of the response:

void sendToServer() {
   client.flush(); //just to be sure
   if (client.connect()) {
      client.print("GET /main/rfid?apikey=123&tag=");
      client.print(tag);
      client.println(" HTTP/1.0");
      client.println();
 
      Serial.println();
      switch (getHTTPResult()) {
        case '0':
           Serial.println("logged in"); break;
        case '1':
           Serial.println("logged out"); break;   
        case '2':
           Serial.println("no user"); break; 
        case '3':
           Serial.println("API key failed"); break;    
        default:
          Serial.println("broke"); break;
      }
      client.flush(); //just to be sure, probably not needed here
      client.stop(); //should stop it
  } else {
    Serial.println("connection failed");
  }
}

A few more troubleshooting tips (will keep updated as they come in)

  1. Be sure to use flush() when appropriate. This ensures that your input buffer is clean and that you don’t have any left over remnants.
  2. I noticed that sometimes the ethernet shield would not power up when I plugged in my Arduino. You can tell by the group of lights on the top. I could only get it to start up when I had unplugged anything that was leaching power from my board (in my case the RFID reader which is leaching current off the 5V pin). I am not 100% sure why this is but my guess is that it has something to do with some components on the ethernet shield not getting enough current to start it up. If this happens to you, you will often find that client.connect() will return false every time. I will try to investigate this further.

Conclusion

Hopefully this isn’t too far out of context for your applications. Feel free to post questions about this device and I will try to help out.

This entry was posted in Arduino, Tutorial/Documentation and tagged , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

14 Comments

  1. Posted February 8, 2010 at 8:04 am | Permalink

    I posted this to the hackerspaces.org mailing list and some folks replied with some similar projects that other spaces were doing:
    http://hackerspace.be/Pamela
    http://code.google.com/p/cerberus-prox/

  2. Posted March 28, 2010 at 10:38 am | Permalink

    Please remember that “Host: http://www.yourhost.com” header line is mandatory for HTTP 1.1, so you have to add it in your GET command so the server accepts your request.

  3. Peter
    Posted March 30, 2010 at 4:46 am | Permalink

    Hi,
    Firstly thanks for explaining your code, i’m trying to do something simular, but with php, and its sort of shown me what to expect to see so thanks.
    Secondly, in March there has been a DHCP and DNS library avaiable, its on the playground but it sends you here http://gkaindl.com/software/arduino-ethernet
    Very useful, but it does take up 6-15k of programming space!!!
    Thanks

    Pete

  4. Posted March 30, 2010 at 12:33 pm | Permalink

    @Peter,

    Thanks, Awesome tip! I am going to try to fit the DNS library in my sketch. It is pretty vital if you want to connect to a machine that uses a service like DynDNS.

  5. Mike Heckman
    Posted May 13, 2010 at 2:50 pm | Permalink

    I’d really recommend not using reserved HTTP status codes for things they’re not meant for. Why not always return a 204 (No Content) and change the status message? i.e. 204 IN, 204 OUT, 204 BAD CODE, 204 UNKNOWN USER. Then just parse the first byte.

    As it is now, you’re just one Authenticate configuration mistake away from the arduino thinking *any* RFID is valid and is an “OUT”.

    Also, I thought rails used :status for the message and :code for the… yep, code.

    Lastly, you’re doing one sided authentication with the same code repeatedly in plain text. I’m not sure the Arduino has the computing ability to do MD5 hashes but if it did, it would be substantially better to send the RFID code, epoch timestamp, and an MD5 hash of the RFID+timestamp+API code. At least then you wouldn’t be sending the code every time. The Ruby code can create the same MD5 hash using the shared “secret” code and verify that all is well. Ideally the server would sign its response with another hash, but if you’re having speed issues parsing individual bytes from the return stack, I doubt this is feasible.

  6. Posted May 13, 2010 at 4:01 pm | Permalink

    @Mike,

    1) that is a good point. It would be better to use 204.

    2) Never heard of the :code key. I just used the first thing that worked. I really don’t know rails too well and havent really written for the framework since about 2007.

    3) I considered hashing the message but I didnt really see a need to put in the time to get md5 to work on the Arduino. It is a breeze on the Ruby side though. The point is, if someone got a hold of the API key they wouldnt be able to do that much damage. I guess it depends on your application. I just wanted it to be as fast as possible and wanted to spend as little time coding as possible. If I wanted it to be professional grade I definitely wouldn’t have used an Arduino :)

  7. David
    Posted October 7, 2010 at 7:08 am | Permalink

    Thank you for sharing in a so detailed way.
    I have learnt a lot with your project.

  8. Posted February 6, 2011 at 5:32 am | Permalink

    Thanks for well written blog post and interesting code! However I noticed that on the getHTTPResult() function when you take this line alone:
    return client.read(); // return either ’0′ or ’1′ or ’2′ or ’3′
    It actually returns the ASCII number for the character your expecting and not the real character… Just if someone takes that line alone.. Gave me small headache before I realised my stupid mistake =)

  9. ritesh patel
    Posted June 10, 2011 at 8:43 am | Permalink

    can you explain me how can i connect link between my pc and arduino board through
    internet controlling device any where in the world.what type of free server use for this

  10. Folkes
    Posted September 3, 2012 at 11:49 pm | Permalink

    its verry good. Can help me to understand about arduino.
    I have one question, If I want to make sensor vibration and use accelerometer mma7361, arduino uno and ethernet shield, and I want to display vibration status display at PC (web basic), how I start that..??? how that sensor programming.?? tq

  11. Posted January 13, 2013 at 7:24 pm | Permalink

    “RFID Ethernet Shield Client Rails” honestly enables
    myself ponder a little bit further. I really treasured every individual piece of
    this blog post. Thank you -Erick

  12. Arenrylem
    Posted January 29, 2013 at 5:57 pm | Permalink

    There’s an answer for your lack of self confidence, there’s an answer to all of your worries, it does not even require the cost of a newspaper, just log on to the internet and withing minutes you’ll have secure your self a pair of life saving shoe lifts insoles

    http://www.bjmonline.com.br/forum/profile.php?mode=viewprofile&u=96730&sid=8eb5b40d072b452618cc2a51369b8f09

  13. Posted March 25, 2014 at 11:47 pm | Permalink

    When I originally left a comment I seem to have clicked on the
    -Notify me when new comments are added- checkbox and now every time a comment is added
    I recieve 4 emails with the same comment.
    There has to be a means you can remove me from that
    service? Many thanks!

  14. Posted April 1, 2014 at 12:38 am | Permalink

    I got this website from my buddy who informed me on the topic of this web page
    and at the moment this time I am visiting this web site and reading very informative articles or reviews at
    this place.

One Trackback

  1. By new sites here « Bilnir on March 20, 2012 at 2:25 pm

    [...] just wrote an article on the GumboLabs blog about using the Arduino Ethernet shield. Be sure to check it [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">