Check the checkers

Always, always ensure that automated processed are checked by independent automated processes. If you don't do that then make sure the automated processes are very chatty -- email, SMS, smoke signals. etc. Well, err, perhaps not the last one as I am sure it would scare the data-center staff.