Google’s Matt Cutts, the head of search spam at Google, posted a video named Does Google use a set standard for manually removing webspam?

In short, he discusses as openly as possible, the philosophy used at Google on how they go about manually removing spam. He basically said in most cases the results are clear and simple. But he explains several points:

Google doesn’t ban sites just because they are being critical of Google.
Google has very clear webmaster guidelines
It is easy for clear cut cases
They have training to be consistent
They do “shadowing” to train
They always review new manual penalties set by new Googlers
They do random spot checks for quality
There are philosophical questions in the spam gray zone where they work together as a team to come to a consensus.
They are not bound by a particular view, they look at the whole holistic picture (repeat spammers, etc)

Here is the 4 minute plus video:

Forum discussion at Google+.