Little by Little...

The album was released on the band's own Phonographic Records label on September 13, 2005. The CD version featured a second disc of B-sides and outtakes in deluxe packaging. In an effort to "embrace the indisputable fact of music in the 21st century", the band made the album available as a free download via BitTorrent and on the band's website. They pointed out that "it’s important that people understand the free download concept isn’t a frivolous act. It’s a key part of our promotional campaign, along with radio and press promotion, live shows, and videos. It’s a bet that the resources of the Internet can make possible a new way for musicians to find their audience, and forge a meaningful artistic career built on support from cooperative, not adversarial, relationships."

On April 19, 2006, the band announced on their MySpace blog that the album had been picked up by Olympia-based label Kill Rock Stars for nationwide release on July 25, 2006. The re-release's track listing, song order, and album art is different from the Phonographic Records release. The band did some limited touring to accompany the release.

The album title is taken from a quote by Melvyn Douglas's character Homer Bannon in the 1963 film Hud: "Little by little, the look of the country changes because of the men we admire."

The title of the song "Cream and Bastards Rise" is a quote by Paul Newman's character in Harper (1966). "Happiness Writes White" is a maxim first put forth by French writer Henry de Montherlant.

"The Piano Lesson" features someone attempting to play Beethoven's "Für Elise" before switching to Scott Joplin's "The Entertainer."

"Wine, Women and Song" is the first line of "Villanelle of the Poet's Road" by Ernest Christopher Dowson.

Little by Little... received generally positive reviews upon release, with AllMusic calling it "smart and compelling" and praising the band returning from their hiatus with "all their smarts (and hooks) intact". Paste further praised the album, calling it "gentle" and "melodious".

In a less positive review, Adam Moerder for Pitchfork praised the band's shift towards pop compositions while critiquing the album for retaining the lyrical style of the band's earlier indie-oriented work, finding it "leaving [listeners] with delightful vestiges [of 90's indie bands] though ultimately sounding like a Ben Folds album".

All songs written by Aaron Huffman, Jeff J. Lin and Sean Nelson.

Phonographic Records release (2005)

Kill Rock Stars release (2006)

Personnel per liner notes.