Inflation adjusted analysis of mainland U.S. same home sales shows a cooling across Michigan and Massachusetts. For your enjoyment I have generated an animated heat map indicating the states that are burning above (red) and below (blue) the historical price trend for that area. The chart runs from 1975 through September 2007.

For every region I have done a regression analysis to determine the long-term historical growth rate. When local prices are way above sustainable long-term averages for that metropolitan area they contribute to a state being colored bright red. When home prices have fallen to be far below what they should be, they contribute to their state being colored deep blue.

Here’s how I did it. I built a mathematical model around the notion that a home price is dependent on volatility within a larger framework of inflation and regional factors such as population and income. To simplify the model I decided to reduce the regional factors into a single growth rate and set out to calculate the long term growth rate for every region.

Of course I needed data. I really liked that the U.S government’s Office of Federal Housing Enterprise Oversight (OFHEO) calculates a quarterly index based on same home sales. In order to calculate their trends they rely entirely on those homes that have sold two or more times. If there are two houses on the same street and one sold in 1980 for $100,000 and again in 1985 for the same price ($100,000) but a nearby house sold for $100,000 in 1980 and sold for $105,000 in 1986 then it can be inferred for that area that home prices did not change between 1980 and 1985 but increased 5% between 1985 and 1986. Clearly having access to lots of data helps establish trends, and that’s exactly what the OFHEO does.

Next I needed to remove inflation from the picture. To do that I chose to use the CPI data published by the Federal Reserve Bank of St. Louis to adjust prices into real dollars.

Then I logged the data and ran a least squared linear regression using the perl Regression.pm package off of CPAN using a constant weight for every data point. For each data point I then calculated the ratio of the prices relative to the trend line.

This was great. I had a massive spreadsheet and could look at any raw number I wanted. After a while of that I decided, “Hey, wouldn’t it be great to plot these trends?” Which I did. But it turned out that viewing a tangle of hundreds of metropolitan areas amounted to viewing a spaghetti of multicolored wiggles. Nonetheless I plowed ahead and graphed the discrete first derivative of every metropolitan area and noticed an unexpected smoothness in the data. So I analyzed the discrete second derivative and said to myself, “Wow, the time-variant trends are so predictable that this would make a great macroeconomic animation!!!”

But before I could make an animation I had to create even a single frame. I looked for charting software and found this amazingly helpful tutorial with sample code from IBM (Thank you IBM!!) that relies on the Cooperative Association for Internet Data Analysis (CAIDA) plot-latlong software (Thank you UCSD and NSF!!) to plot data onto maps.

I then annotated and animated the frames using ImageMagick convert. At this point I had an animated GIF, but I really wanted to let user move forward and back through the frames. So I converted the animation from GIF to FLA format using FFmpeg because that’s how all the cool cats like YouTube do it.

It was then I realized that I needed a bit more glue to make it all work. I researched for a couple hours and in the end decided to use the same javascript code as YouTube: Geoff Stearns’ SWFObject to play embedded shockwave files. To finish off what had started as a little project, I installed Jeroen Wijering’s FLV Media Player to provide the tidy controls and way cool full screen functionality.

All in all it was a nice “all nighter” project. I hope you find the animation educational and perhaps also a few coders out there can learn from some of the inside tricks on how I did it.