Spectacular Panoramic Google Earth

A few weeks ago my eldest son Thomas and I visited one of our friends at Google. Andy was a very gracious host and gave us a great tour. We were both very impressed, esp. Thomas as this was his first visit to the Googleplex.

Probably the coolest part of our visit was playing with a six screen, 180 degree, panoramic version of Google Earth. It was amazing. Extremely fluid and fast, very easy to control, and the 3D buildings made it a truly awesome experience.

Here's ace pilot Thomas as he travels the world. Many thanks to my wife Katrine who did a great job creating a movie out of my poor footage.

Remembering a time when Microsoft was Apple's underdog

Long long ago, in a galaxy really not so far far away, Apple's yearly revenues used to be four times higher than Microsoft's. No kidding.

I've been analyzing companies' revenue per employee from 1990-2010. That may become a blog post in itself but today I want to focus on Apple vs. Microsoft.

Last year, much was made of Apple of passing Microsoft in terms of revenue and capitalization.

What struck me looking at this diagram is that 20 years ago, Apple's revenues were almost five times larger than Microsoft's!

I'd grown so used to thinking of Apple being David to Microsoft's Goliath that I'd forgotten that this wasn't always the case. To be fair, in those early days Apple thought of itself as David to IBM's Goliath. As IBM went from being the PC vendor (in the 80s) to just another PC vendor (from the 90s onwards), and Microsoft's fortunes rose, Microsoft replaced IBM as Apple's main competitor.

High revenues are good, but profits are (at least in the long term :-) better. Apple was barely making money before 2005, while Microsoft was always profitable during this time period.

Here are Microsoft's and Apple's employee counts.

So what of revenue and net income per employee? Surprisingly Apple's revenue per employee has almost always been higher than Microsoft's. The latter's consistently larger employee base impacts this metric. Since 2005 Apple far surpassed Microsoft. Last year, each Apple employee generated 1.3 million dollars. Wow. (Google, BTW, was at 1.16 milion dollars per employee).

Looking at net income per employee however, Apple's only recently surpassed Microsoft.

The 2004-2005 period is the turning point in Apple's fortunes. All four metrics (Revenue, Net Income, Total, and per Employee) are on the rise reflecting a growing demand for its products. And remember the iPhone wasn't even out yet.

The market certainly reflects this. In 2005 Apple's stock price exceeded Microsoft's for the first time in almost 10 years.

All this makes me wonder...
  • Can Apple keep growing its revenue and income faster than employees? (They hired over 10,000 people in fiscal year 2010)
  • The market obviously values and rewards trends (i.e. first & second order derivatives). How much of a hit will the stock take when Apple's growth slows?
  • When a company's revenue, income, rev per employee, and income per employee are all rising and keep doing so for a few quarters... Is it high time to buy the stock?

Whatever the answers, let's hope Apple behaves itself well as our new Technology Goliath.

Stunning Norwegian Auroras

Love the way this was filmed. The smooth panning transforms the time lapse pictures. Norway is such a beautiful country (if a little cold ;-)

Auroras are the result of the solar wind colliding with gases in the earth's upper atmosphere (more details). A wonderful combination of art and science.

Why is California Building the World's Most Expensive Bridge?

I was inspired by Jack Dorsey's recent discussion on the importance of design. Many a blog post could be written on that topic. Jack's presentation also reminded me of a question that has nagged me for a while: how do the ballooning costs of the Bay Bridge replacement compare with the Golden Gate Bridge's construction costs?

Golden Gate Bridge
  • Construction time: 4.5 years (1933-1937)
  • Longest span: 4,200ft
  • Lanes 6
  • Cost: $76 million in 1933 (source), equivalent to $1.3 billion today (source)
  • Tons of steel: 83,000 (source)
  • Fun fact: The bridge opened to pedestrians one day before it opened to cars. At the time the toll was $0.50 each way and $0.05 extra if you had more than 3 passengers
  • Wikipedia page

Bay Bridge Eastern Span Replacement
  • Construction time: 9 years and counting (2002-2013?)
  • Span: 1,260ft
  • Lanes: 10
  • Cost: $6.2 billion (source)
  • Fun fact: The original Bay Bridge was also started in 1933 and finished six months ahead of the Golden Gate
  • Wikipedia page

Woah! The Bay Bridge Eastern Span Replacement is FOUR TIMES more expensive than the Golden Gate Bridge!

Why?

Here are possible differences that, in my mind, can be discounted.

Labor costs: "The Golden Gate Bridge was built during the depression, when workers were cheap". True, but nowadays workers are augmented by much more capable machines.

Material costs: "Steel costs much more now".  At first glance, there's evidence to back this up. USGS data states that a ton of steel cost $10 in 1940 vs. $165 in 2009. Big difference? Not when you adjust for inflation... $10 in 1940 is $153 in 2009.

Complexity: "The replacement has to link itself to existing infrastructure, with a minimum of impact to the current users of the Bay Bridge". Fair enough... But can this really account for four times the cost? Hard to believe, esp. when the "trickiest Bay Bridge work" only cost $140 million.

Destruction: "We have to destroy the old eastern span of the Bay Bridge". Sorry, nice try but this was taken out of the current budget. Yes, we'll need to pay even more than $6.2 billion if we want to get rid of the old bridge.

OK, so maybe bridges are just more expensive these days?

Let's take a look at the top three longest suspension bridges in the world:

  1. Akashi Kaikyō Bridge (Japan) completed in 1998 at a cost of Y500 billion or $6 billion (converting to USD using this data and then adjusting to 2011 dollars)
  2. Xihoumen Bridge (China) completed in 2009 at a cost of $363 million (wow!)
  3. Great Belt Bridge (Denmark) completed in 1998 at a cost of DKK21.4 billion or $4.1 billion (converting to USD using this data and adjusting to 2011 dollars)

All three have significantly longer spans than the Golden Gate Bridge, let alone the Bay Bridge. 

What's left?

I'm no expert on bridges. I may be missing something... It's just hard to find a reason why the Bay Bridge retrofit is so expensive. Other than mismanagement. On a massive scale.

Addendum (2011.3.26)

After writing this post I found an article, "The Most Expensive Bridge in the World", published in 2004 in Modern Steel Construction. Which bridge is it about? You guessed it! The Bay Bridge. The author, a structural engineer, called on CalTrans to make design changes to reduce costs. Ironically he already considered it the most expensive bridge in history when in 2004 it was only projected to cost us $4 billion...

Addendum 2 (2011.3.27)

I received a request for details regarding the project's evolution: How much was the work originally slated to cost? Why / When did it rise?

This article has a good summary. I'm quoting the main events it lists:

  • December 1996: Consultant report recommends replacement over retrofit. It estimates the cost at $843 million for a bridge that includes a single tower. Two Caltrans panels recommend building a new eastern span, saying it will be safer and more economical than a retrofit.
  • January 2002: At eastern span project groundbreaking, Caltrans says span will open in 2007.
  • March 2003: Caltrans increases eastern span cost estimate to $3 billion, citing the unique scale and complexity of the project.
  • May 2004: Single bid received to build a self-anchored suspension bridge at a cost up to $1.8 billion, which is double Caltrans’ $730 million estimate.
  • August 2004: Eastern span cost estimated at $5.1 billion, with $1.3 billion in overruns blamed on self-anchored suspension bridge.
  • December 2009: Eastern span cost estimated at $6.3 billion, including $2.3 billion for self-anchored suspension bridge.
  • February 2011: Construction crews begin to lift into place the fourth section of the span’s self-anchored suspension tower. Current projections have the entire self-anchored suspension span to be completed by late 2013.

Addendum 3 (2011.3.28)

A friend of mine, a Civil Engineer and expert on bridges, sent me his summary of bridge costs across the world. It further highlights the fact that we Californians are paying way more than we should for this replacement...

Essential Long Term Travel Equipment

Our family traveled extensively over the last eight months or so, spending two months in Europe (FranceItalySwitzerlandNorway) and another two in Australia. Check out the links if you want to see what a wonderful time we had :-)

Here's what ended up being essential equipment for us. I'll skip all the "you really don't need as many clothes as you'd think" advice. You can read that countless other places. (Though it's true, you really don't!).

External Wifi Card

This one's easy. I've already blogged about it. At $40 this gear helped us get online all over the place and was worth every penny.

12VDC Power Inverter

If you're planning to spend a lot of time driving during your vacation, one of these converters comes in really handy. You can charge laptops, iPads, iPods, camera batteries, DVD players, etc. They cost around $30.

Plug Converters and Power Strips

Traveling all over the place, you'll obviously need some converters. Can't say we've found any that we're crazy about but we took three of these with us and they held up pretty well. Bulky though. Bring a couple power strips with you too, you'll need them if you have anywhere near the level of gadgets we have. Why a couple? We've had at least one strip that couldn't take the 220VAC in Europe and consistently blew a fuse.

About Credit Cards
Call your credit card company before you leave and tell them
  • When you're leaving
  • When you're returning
  • What countries you'll be visiting
  • And that they better NOT decline your card when they see transactions from those countries 
If you keep your credit card receipts (you should!), bring a few envelopes with you. Put each country's or each location's receipts in a separate envelope. It'll make confirming that mystery purchase so much easier.

Teva Dozer Sandals

Love these sandals. Took them on a 7 week trip to Australia and wore them almost exclusively: beach, rain forest, desert, rocks, driving, and city. They performed admirably: lightweight, good grip, easy to slip on/off, stayed cool, and were still comfortable at the end of the day. FYI it took about a day for my feet to get used to them.

Toshiba Canvio 1TB Drives

These made great lightweight drives, run off USB, and aren't much larger than a pack of cards. My wife and I each had one to backup our Macbook Pros and provide additional storage. They're about $100 each.

A tip for safe data travel: swap drives while on the move. My wife and I backup regularly during our trips and when we travel she carries my drive, I carry hers. A loss of either of our backpacks doesn't lose all our data. Yes, I care about my laptop, but I care a lot more about the information that's on it.

GoPro HD Camera

Yes, that geek is me, diving off the Great Barrier Reef, with a GoPro on my head. If you're in to adventure sports, or just want a handy high quality wide angle HD camera, you'll be happy with the GoPro. For underwater use, get this housing.

iPad Apps
I love my iPad and it proved itself a great travel companion. Here are a few apps we liked:
  • GPS MotionX HD: A great GPS program, we used it a ton. Make sure you download maps well ahead of time because that external wifi card won't help your iPad get online :-)
  • AllSubway HD: If you're visiting a number of large cities, it's useful having these maps in one place
  • Wi-fi Finder: Can't say I've really used this yet, but it's on my iPad just the same. Internet connectivity is right up there with air, food, and water :-) Download the offline database before your trip starts
  • Living Earth: Wish I'd had this with me. It's a beautiful world map and weather app
  • Star Walk: Not a travel app but a wonderful way to admire new stars and make new friends (seeing the app in action attracts people like moths to a flame)
  • If multiple people will be using the iPad, install a few browsers and assign one to each person. That way your facebook, twitter, gmail cookies won't get mixed up

Visualizing One Hundred Years of Pacific Rim Earthquakes

Whenever I hear of a major earthquake, I always wonder when our turn will come. I've been asking myself that question way too frequently recently. My family and I live in the San Francisco Bay Area, prime earthquake country (or so I thought until I looked at Japan...). I started playing with NOAA's earthquake data after the New Zealand earthquake. After the recent Japan quake, I thought I'd publish a few graphics. 

Disclaimer: I'm no geologist, statistician, or expert on earthquakes. I don't even play one on TV. I don't think anyone can predict earthquakes with any certainty (though there is some interesting research) and I certainly won't try.

The Ring of Fire is the name given to the chain of mountains, volcanoes, and faults that ring the Pacific Ocean. Of the world's 16 largest earthquakes since 1900, 15 occurred in the Ring of Fire.

Here's an interesting graphic showing the earthquakes above 6.0 magnitude that have hit the Ring of Fire region since 1900. Earthquakes of magnitudes between 6 and 7 are in green, between 7 and 8 in blue, and 8 or higher in red.

Notice anything? Well, as a Californian, the first thing that struck me was: "we're getting off lightly!".

This movie gives you a different way to see the earthquakes. Same legend as before: magnitude 6+ green, 7+ blue, 8+ red. The video isn't the most exciting one you'll ever see. It helps to pick a point of interest on the map and imagine some elevator music in the background :-)

Let's dig deeper...

Japan, California, New Zealand, and Chile

Looking at the graph above it's clear California gets fewer earthquakes than many other parts of the Ring.

When I compare a circular area 2,000km around the center of Japan with the same size area around California (centered on San Francisco), Japan has been hit four times as often by large (i.e. 6+ magnitude) earthquakes than California (~200 vs. ~50).

Here are Japan's large earthquakes, with 6.x, 7.x, and 8+ magnitude earthquakes broken out (notice that the data for 6.x earthquakes in 1900-1950 is likely incomplete):

The equivalent map for the 2,000 km surrounding San Francisco looks like this (sorry, no, there are no 8+ earthquakes, NOAA has the 1906 one at 7.9):

Let's look at the "earthquake history" in the other recent hotspots: New Zealand and Chile.

Earthquakes from 1900-2011 in a 2,000km area centered on Christchurch, New Zealand.

And finally South America. 2,000km area centered on Santiago, Chile. (That 9.5'er in 1960 was a monster). 

BTW, I've only focused on a few of the Ring of Fire hotspots. Indonesia, Central America, etc. are all very active.

So are we Californians due for an earthquake?

As I wrote earlier: Who really knows? On the one hand 110 years of data tells us that our corner of the Ring of Fire experiences 25% as many earthquakes as Japan. On the other hand... It may be about time for a big one to hit us.

Simon Winchester (an author whose many books I'd recommend, esp. The Man Who Loved Chinawrote recently:

[The Chile, New Zealand, and Japan earthquakes]  involved more or less the same family of circum-Pacific fault lines and plate boundaries—and though there is still no hard scientific evidence to explain why, there is little doubt now that earthquakes do tend to occur in clusters: a significant event on one side of a major tectonic plate is often—not invariably, but often enough to be noticeable—followed some weeks or months later by another on the plate’s far side. [...]

Now there have been catastrophic events at three corners of the Pacific Plate—one in the northwest, on Friday; one in the southwest, last month; one in the southeast, last year.

That leaves just one corner unaffected—the northeast.

Are earthquakes really clustered? I haven't analyzed the data for correlations. Just eyeballing the graphs above, there are enough earthquakes happening around the Pacific Rim that you could claim some correlation exists.

If you want my advice... Better safe than sorry: Be prepared.

Technical Info

All graphs were created with Mathematica 8, one of my favorite pieces of software. It's a tremendously powerful package and, though it does have a bit of a learning curve, the help system is excellent at giving lots of examples.

There's a lot more that could, and probably should, be done with this data: time-based analysis, looking for correlations, leveraging more of the data (e.g. tsunamis, impact of earthquakes, etc.).

The graphs above are pretty simple. Mathematica can create much more sophisticated ones. Here's 3D version of the Japan-area earthquakes.

 

Learn the Zen of a Programming Language with Koans

I love the idea behind Ruby Koans: write a set of failing unit tests that teach you about the essence of ruby as make every test turn green. It's a brilliant idea. The tests themselves are usually simple and illustrative. You even get encouragement (or enlightenment? :-) as you fix them.

The good news is that this idea has spread beyond ruby. There are koans in many languages:


While learning a programming language is best achieved by writing a useful application, these koans are a very welcome (and fun!) addition.

Does Wealth Equal Happiness?

One of my favorite blogs is FlowingData. It's a great place to find all sorts of thought provoking (and sometimes weird) ways to visualize the world around us.

Yesterday, there was a reference to a very interesting article in the New York Times called "Mapping the Nation's Wellbeing". It measured Gallup's analysis of people in the United States well being across a number of variables: Happiness, Diabetes, Smoking, Exercise, Inadequate Food, etc. It's well worth viewing. Here's the summary graph (darker = greater wellbeing).


As you compare the different categories you'll be struck by how badly the South East of the US scores, which made me yearn for one piece of information that isn't included in this diagram: the per capita income of each state. Surely wealthy states are happier, right?

Wikipedia has data for 2009 but its graph dates back to 2006. That's pre-recession, things have changed since then. So I fired up Mathematica to create a 2009 version. (I won't include the code here but the notebook in this post made graphing the US very easy).

So does Wealth equal Happiness? It certainly seems to help but not universally: the South East is clearly poor and unhappy but Montana is about as poor as yet much happier. One surprise for me: Wyoming. I never realized it was so wealthy. Minerals and low taxes?

Whatever the correlation between happiness and wealth, smarter people than I have thought about this issue:

 “Money doesn’t make you happy. I have $50 million but I was just as happy when I had $48 million.”—Arnold Schwarzenegger

 “Money frees you from doing things you dislike. Since I dislike doing nearly everything, money is handy.”—Groucho Marx

I'll let you know how I feel when I get to $48 million :-)

Hashtables in Mathematica

I fondly think of Mathematica as a "kitchen sink language": other than that proverbial kitchen sink, it has functions for pretty much anything you can think of.

Why then does it not have a hashtable data type?

It turns out that it doesn't need one. Hashtables are built into the language at a fundamental level. Just start typing:

 h[foo] = 1;
 h[bar] = 2;
 h[foo]
 1
 h[bar]
 2

 

And you have a hashtable!

It's not quite that simple though. What if you want to list all the keys used in the hashtable? This function (from a handy StackOverflow answer) takes care of that:

 keys = DownValues[#][[All, 1, 1, 1]] &;
 keys[h]
 { bar, foo }

 

Recently I was playing with NOAA earthquake data in Mathematica provided in the form of a TSV (Tab Separated Values) file. Mathematica easily parses it into a list:

 ed = Import[NotebookDirectory[] <> "EarthquakeData", "TSV"]
 Take[ed, 5] // TableForm
 ID FLAG_TSUNAMI YEAR MONTH DAY ...
 8204 Tsu 2009 1 3 ...
 8211 Tsu 2009 1 3 ...
 8210 2009 1 8 ...
 8250 Tsu 2009 1 15 ...
 ...

 

This was a good start but the data wasn't in a very useful form. What I wanted was to be able to address the data by column name and row number, so I wrote this helper function:

 MakeHash[hash_, a_] := Module[
  {keys = First[a]},
  Clear[hash];
  Do[
   hash[keys[[i]], j - 1] = a[[j, i]],
   {i, 1, Length[keys]}, {j, 2, Length[a]}];
  hash[Dim] = {Length[a] - 1, Length[keys]};
  hash[Rows] = Length[a] - 1;
  hash[Cols] = Length[keys];
  hash[Keys] = keys;
  ]

 

The first parameter is the name of the hash to create, the second is the array to parse (assuming the first row represents column headers). It's now easy to access the elements you want.

 MakeHash[ehash, ed]
 ehash["ID",2]
 8211

 

You'll notice MakeHash adds some convenience entries in the hashtable. I even included one for keys, despite the function we defined earlier on. It ensures MakeHash is self contained and also deals with a limitation of the keys function as it stands. As we're dealing with a two dimensional hashtable, the keys function considers each key (i.e. ID,1 and ID,2 etc.) as distinct, so returns way too many of them.

 Length[ehash[Keys]]
 46 (* expected *)
 Length[keys[ehash]]
 2074 (* woah! *)
 (* Let's fix this by eliminating dupes with Union *)
 keys = Union[DownValues[#][[All, 1, 1, 1]]] &;
 Length[keys[ehash]]
 50

Why 50 keys and not 46? Because MakeHash added four more: Dim, Rows, Cols, and Keys.