1. Part of being programmer is constantly learning, and I wanted to evaluate python and the Google App Engine as a possible replacement for the PHP work I had already done.
2. I wanted to make my test app something real, though, so I needed a simple idea.
3. Beijing is a much dustier place than anywhere in the US – it can even be hazardous to breathe the air sometimes, and I was a bit worried about it when I first moved.
4. There was already a service on Twitter (@BeijingAir) that updates every hour with particulate matter and ozone statistics.
5. However, Twitter is blocked in China, so I can’t check it on my iPhone, plus Twitter is text-only, so there are no graphs.

By having Google’s servers request and cache the pollution data for me in the US, I figured I could create a database of historical pollution data, visualize it how I wanted, and access it on my iPhone because the data would now be served by a non-Twitter server.

So, I downloaded the Google development server, read up on Python, read about the Twitter API and put this together in a day or two.  Google’s toolchain made it quite easy to deploy the application.

It’s really pretty simple – in App Engine apps, you can create a process that will execute every X number of minutes/hours/days.  That meant I needed two pages for my app.  The first one would be a page that kicks off a process to access the latest updates from @BeijingAir, parse them, and store the data in Google’s DB.  The second is the home page, which accesses the data for the last 24 hours and displays it in a table, which then gets turned into a graph via javascript.  Every half hour, my script gets the latest twitter updates (throwing them away if they’re already in the DB) and that’s it.

If I ever get time to play around with it again, I’d like to make some cosmetic changes.  One would be to move the ‘About this page’ into the same fixed-width div as the pollution graphs.  That way, when I open it on my iPhone, the screen would automatically resize to fit the graphs perfectly.  Currently, I have to double click on them to get that behavior.

Secondly, I’d improve the graph display.  We really only need one graph, since the concentration and AQI for each pollutant are directly related.  Also, to make it more clear what range the graph is in, we should change its color (and the color of the description text) to match the colors defined on www.airnow.gov.  That would make it clear when one graph is going down and the other is going up that the one going down can still be a lot worse because it’s been bad for the entire day.

If you want, you can look at the code on github.com.  Feel free to send me any questions or code patches if you want to change anything.

It turns out that big part of my answer involves automating everything.  Part of my reasoning is that our team is very small now, so the more we automate, the more we can concentrate on just development.  Another part of my reasoning is that I’ve read a lot about continuous deployment lately and many of the principles like decreasing the time between developing a hypothesis and learning your answer make sense and can only be achieved through extensive automation.

To that end, I’ve decided that all the developers should have some common git hooks in their repositories that will do things like run tests, etc.  Moreover, since we’ll be spending a lot of time developing these hook scripts, I’d like them to be backed up and versioned as well – i.e. also in the repository.

Therefore, my first step was to create a folder in the repository called /git_hooks/ and symlink /.git/hooks -> /git_hooks/.  This way, any changes we make to the hooks will be automatically applied across all the developers’ machines.

However, this has a downside in that everyone’s hooks become the same and they are unable to add their own (I, for example, might like to add a spell checker to my commit comments, but that would be annoying if someone didn’t like it).  To solve that problem, I created a /git_hooks/personal/ directory which behaves just like the normal /.git/hooks/ directory, and any hooks a developer adds in there get executed as part of the common hook.  Here’s the code I used as the base of all of our common hooks to make it work:

#!/bin/sh

# Check if the individual developer has his own hook
CMD_NAME=`basename $0`
if [ -f $GIT_DIR/hooks/personal/$CMD_NAME ]
then
  # If so, run it. $@ passes all the command line arguments passed to this function
  # If the personal hook fails, fail this one as well
  if ! $GIT_DIR/hooks/personal/$CMD_NAME $@
  then
    echo "User hook '$CMD_NAME' failed"
    exit 1
  fi
fi

exit 0

Finally, if any developer does decide to add their own hooks in /git_hooks/personal/, they should be sure to add a line containing “/git_hooks/personal/*[^.sample]” to the end of their /.git/info/exclude file so that git will ignore their customizations.

You can download this at http://gist.github.com/357538.

I started out trying interface builder and got a basic interface to show up pretty quickly, but it was rather confusing and I didn’t really know what was going on.  Next, I thought I should probably just go through the tutorials in my book (iPhone SDK Application Development by O’Reilly) so that I could get an overview of what’s actually possible with the iPhone.  That way, when I restart building my own apps it will be much faster because I’ll know what I’m doing.

However, I’ve now been trying to get the TextView example in Chapter 3 (Hello World) to work for almost a week!  I typed in all the code correctly (twice, into two separate projects), and it compiles fine, but all I get on the iPhone simulator is a black screen!  After a while, I found the debugger console button in XCode and realized I was getting the following errors:

warning: Unable to read symbols for “/System/Library/Frameworks/UIKit.framework/UIKit” (file not found).
warning: Unable to read symbols from “UIKit” (not yet mapped into memory).
warning: Unable to read symbols for “/System/Library/Frameworks/CoreGraphics.framework/CoreGraphics” (file not found).
warning: Unable to read symbols from “CoreGraphics” (not yet mapped into memory).

These frameworks are quite obviously linked in XCode (because the code completion works and the project compiles fine), but for some reason they don’t show up in the simulator.  After searching online, I’ve gotten to the point where I think it has to do with the way they’re referenced in XCode and that when running on the simulator, the application doesn’t actually compile all the references together; it just starts running without them, so you have to run it from within XCode.   Some people have also said that it had to do with breakpoints in code.  However, I’m running from within XCode and I don’t have any breakpoints, so I think it must be a file location configuration issue.  Now, I’ve had no luck finding any solutions online or by experimenting…

I’ll let you know how it works out; I’m not going to give up on this.

—- Update – March 1, 2009 —-

Well, the Hello World example started working… First thing to note is that the “file not found” type warnings I noted above don’t actually seem to be related to the problem; now that it works, they still show up.

I fixed the blank black screen by cleaning all targets (in the Build menu), then rewriting Hello World to directly use a TextView instead of manually subclassing UIView myself. Also, I didn’t allocate a window object; I just started using it and let the synthesize keyword work for me (before, I was trying to remove Interface Builder from the project, like it recommended in the book. Here’s the code that first worked (I called the project “Arg” because I was so frustrated):

ArgAppDelegate.h:

#import

@end


ArgAppDelegate.m:

#import
#import
#import
#import "ArgAppDelegate.h"

- (void)dealloc {
[window release];
[super dealloc];
}
@end


This compiled fine and the text view showed up correctly, etc. Next, I slowly changed the code one line at a time back to the original code, compiling at each step, to see if I could find the line that was causing the problem. Unfortunately, I eventually came back to the original code and it worked…

ArgAppDelegate.h:

#import

@end


ArgAppDelegate.m:


#import
#import
#import
#import "ArgAppDelegate.h"

- (void)dealloc {
[myMainView release];
[window release];
[super dealloc];
}


main.m:


#import

NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
int retVal = UIApplicationMain(argc, argv, nil, @"ArgAppDelegate");
[pool release];
return retVal;
}


I hope this helps somewhat to anyone having the same problem. I found this incredibly frustrating and am not entirely happy about my solution here. I still don’t really know why it happened in the first place, so I’m worried it will come up again in the future. Let’s hope not!

As I thought about the problem, I realized that salsa dancing (and most partner dances) is really a finite-state machine. We have X number of states (the positions that you end up in on beat 1), and transitions between the states (twirls, etc.). Naturally, I thought of creating a diagram with states and transitions and tracing paths through the diagram to come up with new sequences. This way, if I could memorize a few positions and all the different transitions between them, I could come up with lots of sequences on my own without ever having to memorize a two-minute series of steps. Each set of steps would be at most 8 or 16 beats long.

Still, I didn’t know exactly what to call each state, what precisely is considered a transition, or most importantly, how to present this information in a usable manner. Last week, I was again thinking about this and realized that since a finite-state diagram is also a directed graph, and the web is a directed graph, a webpage made perfect sense for a presentation mechanism. Moreover, if I used a wiki, I could start adding pieces very easily, even if I don’t know the best terminology or have the perfect definition of each state. Plus, other people can add their own moves to the wiki and eventually, we can all learn from each other.

Thus, yesterday morning, I started a new website: salsa.wikidot.com. So far it’s not much; I have a few positions and moves described, and I plan to add more. I feel a little bit like I shouldn’t actually be the one adding content, since I’m so new at dancing and don’t really know how to describe everything. However, I also think that a lot of times we just need to start doing something even if we don’t know how, so there it is. I have it set up so that it will be easy to add photos and videos of the steps if I ever get that far. Should be fun. Hopefully this helps someone someday.

My project ended up being some work in story-analysis using common sense (provided by ConceptNet, another Media Lab project) as well as trying a new approach to anaphora resolution (basically determining antecedents for pronouns), which didn’t rely so much on common sense.

If you’re really interested, you can download the paper I wrote. I also am planning on releasing the source code to my project by the end of the month, though I want to get through my linear-algebra final first before I clean it up and put it on a webpage somewhere.

As usual, let me know what you think. I know the timeline is a little off and the slider has a few quirks as well; I’m working on polishing them up.

Also, if you are trying to few the site in IE and it doesn’t work, let me know. I don’t have a computer that can run IE handy, so I’m having trouble debugging it. Right now the site will probably only work in FireFox.

Tonight, I also wanted to mention that I’ve worked a little bit more on my map. After some trouble getting the Flickr REST API working because I didn’t realize that cross-domain xmlhttp requests were not possible, I finally have the site set up so that the photos come from a specified photoset on Flickr. For example, the set that I have summer pictures in is 72157594201283566, so the url you would use to see these on a map is:

http://web.mit.edu/mpdaugh/www/maps/mapPage.html?set=72157594201283566

To see other photosets mapped this way, just replace the number at the end of the url with the set id of your desired set. In order for my application to map the photos, though, they must be geotagged. This means that they must have two tags, one starting with “geo:lat=” and the other starting with “geo:lon=”. Check out one of my photos in the set Summer06 for an example.

Right now, the time scale on the bottom only goes from June-August of this year, but I plan on having it dynamically rescale based on the time of the photos in the input set.

In any case, I probably won’t be able to work on this until next weekend, so let me know if you think it’s interesting, and if so what features you’d like to see. Thanks for your feedback!

Steph has built up some vacation time over the last year, so she decided to come and visit San Jose for a week before spending the Fourth of July weekend with my in Las Vegas.

A few days after she went back home, my family came to visit for a week, so then I spent my time not at work with them.

They just left yesterday, and since then I’ve started to work on a little project visualizing all the photos that I’ve taken over the last few weeks. Check it out here (I can’t promise that it will be working when you look at it, since it’s still under construction): Mike’s Map

To use, drag the slider (the blue icon at the bottom of the page) to the right. As you move, you will see the current date range change and any photos taken within that range will show up on the map.

It’s such a beautiful day out right now, though, that I think I’m going to stop writing so I can go out and enjoy it. I’ll write more later.