Interesting site with a bunch of in-depth info about Simplicity, Unification, Parsimony and Occam's Razor.

snippet :

This principle goes back at least as far as Aristotle who wrote "Nature operates in the shortest way possible." Aristotle went too far in believing that experiment and observation were unnecessary. The principle of simplicity works as a heuristic rule-of-thumb but some people quote it as if it is an axiom of physics. It is not. It can work well in philosophy or particle physics, but less often so in cosmology or psychology, where things usually turn out to be more complicated than you ever expected. Perhaps a quote from Shakespeare would be more appropriate than Occam's razor: "There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy.".

Simplicity, Unification, Parsimony, and Occam's Razor in Science