Structural-RNN: Deep Learning on Spatio-Temporal Graphs

Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many problems in computer vision inherently have an underlying high-level structure and can benefit from it. Spatio-temporal graphs are a popular flexible tool for imposing such high-level intuitions in the formulation of real world problems. In this paper, we propose an approach for combining the power of high-level spatio-temporal graphs and sequence learning success of Recurrent Neural Networks (RNNs). We develop a scalable method for casting an arbitrary spatio-temporal graph as a rich RNN mixture that is feedforward, fully differentiable, and jointly trainable. The proposed method is generic and principled as it can be used for transforming any spatio-temporal graph through employing a certain set of well defined steps. The evaluations of the proposed approach on a diverse set of problems, ranging from modeling human motion to object interactions, shows improvement over the state-of-the-art with a large margin. We expect this method to empower a new convenient approach to problem formulation through high-level spatio-temporal graphs and Recurrent Neural Networks, and be of broad interest to the community.

From Ashesh Jain, Amir Zamir, Silvio Savarese, Ashutosh Saxena @ Cornell.



Note to self: the Digi Series 2 ZigBee chips (XB24) require the XB24-ZB selection in X-CTU. You need to pick ZigBee Router AT and ZigBee Coordinator AT in order to update them to the latest firmware revision.

Also, to use picocom as suggested in the Wireless Sensor book, download and compile from source. The version available in Ubuntu is stale (1.4 vs. 1.6) and doesn’t support the –echo command.

Finally, note that the ATDL address for the two chips should be that of the other chip. You are specifying the destination address.

Got my two ZigBee’s to talk to one another!

Accessible Hardware Hacking


The Age of the Maker may be hand. Who knows whether this will be a flash in the pan, but there is an amazing number and variety of openly available design, plans, examples, forums, tools, and kits available for those who want to tinker.

I don’t want to debate whether past efforts (e.g. Basic Stamp) have been accessible or inaccessible, but a newer effort has certainly garnered a lot of attention in recent months: Arduino. Not that Arduino is new; if anything, they are merely refining their designs these days.

While I’ve had a couple of Arduinos for over a year now, I haven’t really done anything with them lately. The processors in Arduinos are limited, 8-bit Atmels that harken back to an earlier, more limited age of computing. One could argue that is what makes them an good basis for an introduction to electronics and programming, but I don’t think that is what makes them so special, as there are plenty of similar efforts. Instead, I think that Arduino has hit critical mass. For some reasons – economic or artistic, a cottage industry has grown up around them, and they seem to be (by popular mention) the dominant hardware hacking platform of the day. This largely online community is now largely self-sustaining.

I’ve made a couple of Arduino shields (daughterboards), but lately they have lain idle. Robert Fauldi‘s recently book, Building Wireless Sensor Networks, renewed my interest in using them in conjunction with ZigBee radios, low power transmitter/receivers that are a perfect match to the Arduino philosophy. There are many ZigBee shields that allow you to connect an Arduino board to the radio, and Fauldi’s book can walk you through not only that, but several example uses of increasing complexity.

In the weeks to come I’ll be documenting my own experiments. Stay tuned!

Erlang Remote Shells and Hot Swapping Code


Michael Hashimoto over at spawn_link is doing a wonderful service in providing examples of Erlang code.  One of his examples, Rules of Hot Code Swapping,  explains details behind one of the cooler features of the Erlang runtime – the ability to “hot swap”, that is change out executing code without stopping the service that using it.  In doing so, he glosses over a few steps that a beginning Erlang programmer might need, so I’ve augmented his example with a few instructions that can help a novice get his example to work.  You can pull the example from my git repository  (git clone git://github.com/jbgreer/Erlang.git), where I’ve added these instructions as comments.

First, I assume that you’ve installed a fairly recent version of Erlang and that erl and erlc are executable from your prompt.

  1. Edit hotswap.erl

    Bump the version number in -define(VERSION, 3).

    Edit the message in the io:format to something distinctive.

  2. Compile the module

    erlc hotswap.erl

  3. Start a Erlang shell. Here I set use two command-line options, sname and setcookie. Sname sets the “shortname” of the node; setting the node’s name is necessary if I want to connect to it later (and we do, so we can update the code running in it). Setcookie is a security mechanism used in connecting to nodes. Setting it like this on the command-line is an unrecommended practice but necessary if you don’t have a cookie file.

    Note that the values for sname and the cookie are arbitrary.

    erl -sname node1 -setcookie foo

  4. In the shell, start the server and test. You should get a response back formatted according to your io:format line. After you’ve tested this do not exit the Erlang shell. We will use it again in a minute.

    Pid = hotswap:start().

    Pid ! {echo, "foo"}.

  5. Open another command-line shell/terminal.
  6. Repeat steps 1 & 2. Try to edit the io:format string to make it visually distinctive from the current value.
  7. Connect to the current (remote) Erlang Shell. It’s called a remote shell even it isn’t on another machine because it is running on a separate node. By default, if you start an Erlang shell you’re starting a new node.

    erl -sname node2 -remsh node1@MACHINENAME -setcookie foo

    where MACHINENAME should be replaced with your host’s name.

  8. In the connected shell perform the code swap


  9. Test the change in the original Erlang shell. The first echo will likely show the existing message, while the second will show the new message.

    Pid ! {echo, "foo"}.

    Pid ! {echo, "foo"}.

RESTlet and JSR-311


Like many others I’ve been playing around with REST, an HTTP-based convention for performing CRUD calls against resources. One thing I’ve done is download RESTlet, a Java implementation of a REST framework. Of course, one concern in the tech world is that you’ll choose some approach only to discover that a competing technology disrupts the marketplace of ideas, or that an alternate approach becomes a standard. In the case of Java and REST the proposed standard is JSR-311. Looking back on it, I got lucky, in that I didn’t see any specific mention of JSR-311 on the RESTlet site, but I didn’t care: I was just playing around and wanted to see how light (or heavy) the framework was. Today I serendipitously found a mention of RESTlet and JSR-311 on the Noelios site here. If you’re looking for an example to get your feet wet, then try the RESTlet wiki, which has a short sample bit here.

So, in the words of Bill Murray, it’s got that going for it.

Paul Graham on Programming Language Design


Some of you know that Paul Graham of ViaWeb/Yahoo Stores fame has released his work-to-date on Arc, his new Lisp dialect. The response has been underwhelming(!), so much so that he felt compelled to respond to some of his detractors. After waxing lyrical on the beauties of append in Prolog last week, though, it was this paragraph that sank my battleship:

“This is one reason the source code of Arc itself is so short, incidentally. I did the same thing to it. But my first priority was making applications shorter, not the language. There are features, most notably Prolog-style pattern-matching, that seem to promise great savings in length, but turn out only to be useful for writing a few basic sequence operations like append, remove, and so on. Prolog is a great language for writing append; after that it’s all downhill.”

Ouch! Still, his arguments make for a good read of language design motivation and issues.



This is mainly a test to confirm that WordPress respects the pre tag such that I can post code snippets and that WordPress’ \LaTeX functionality works.

package fibonnaci;

object RecursiveFibonnaci {

    // Fibonnacci using if/else.  Note implicit return
    def fib(n: int): int = {
        if (n == 0)  0
        else if (n == 1) 1
        else fib(n-2) + fib(n-1)

That’s a simple example of the Fibonnaci function f(n) = f(n-2) + f(n-1) with f(0) = 0 and f(1) = 1.