This is one of those things about Washington I thought clearly established by the historical record -- that he said "so help me God" before being inaugurated. The "Christian Nation" crowd points out that Washington went so far as to kiss the Bible before uttering these words, apparently not realizing that this is a Freemasonic, not a Christian ritual. Indeed, according to the story, the Bible Washington used was borrowed from a Masonic lodge.
But it may not turn out to be true after all. See this post on Boston 1775, a great historical blog which documents American Revolution era Massachusetts.
Ongoing research has found the earliest statements that Washington added "So help me God" after taking his presidential oath of office date from the late 1850s, almost seventy years after the event. Oddly enough, that's also decades before Chester A. Arthur was first noted as doing so by a contemporary. (It might be noteworthy that he did not have a formal inauguration, but succeeded to office after James A. Garfield's death.) The Washingon Area Secular Humanists offer a little more info.
Also see this post which reproduces an email from Dr. Juretta Jordan Heckscher, an official with the Library of Congress:
This is in reply to Barbara Clark Smith's very interesting inquiry about Smithsonian NMAH [National Museum of American History] curators' attempts to find out when and by whom the phrase "so help me, God" was added the presidential oath of office prescribed by the Constitution.
Reference specialists on the Library of Congress's Digital Reference Team have done some research on this topic. In particular, my colleague Kenneth Drexler reports the following information:
"The question was whether or not there is primary-source evidence that Washington said 'so help me, God' in 1789. The short answer is that I could find no evidence that he did.
[Also,] according to a Washington Post article from [January 20,] 2001,'Whether Washington actually added "So help me God" to the oath is not supported by any eyewitness accounts, according to Philander D. Chase, editor of the Papers of George Washington project at the University of Virginia. "He may have said those words," Chase said.'
During my research I did obtain a copy of a letter by Tobias Lear to George Augustine Washington dated May 3, 1789 in which he described the inauguration.
I got the letter from Duke University. The letter makes no mention of 'so help me, God.'"
It's likely that the "so help me God" tradition didn't originate until Chester A. Arthur.
Update: Michael Newdow is on this and has a funny video about it. Brian Tubbs correctly notes that "Washington was most certainly a devout monotheist, who believed that the United States of America should indeed be under God." And it was for that reason, I had no problem believing Washington said "So Help Me God." However, an important point that Newdow's video raises is that Washington was very "rule oriented," and it's not likely that he would have just casually added words to an oath specified in the US Constitution, but rather would just recite the oath as written in the US Constitution, which, let us remember, does not have the words "so help me God."