Isn't the mnemonic seed and the inability to change the words security holes?

Most people, when given the choice of their password (let alone passphrase), will either choose an easy-to-crack pass or use the same everywhere. The 25 words of our mnemonic seed are 24 words + a single word checksum. This gives you a 256-byte key generated from a 1626-word word list (position relative to the previous word is important, and they're generated in sets of 3 - i.e. 32 bits per set). You'd need the world's computational power and then more time than our sun will exist. (source: fluffypony).