I have just read an article on the English 1881 Census, in Your Family Tree magazine, May 2014. It mentions that when it was completed officials noted a rise in the number of 'deaf & dumb' people. It seems that many enumerators had recorded babies as being deaf & dumb because they couldn't talk!
Something to keep in mind not just for that census but any records. Look at the persons age before making judgements.