FDR vs. Alf Landon
Faulty Sampling Method

A classic instance of faulty sampling occurred during the 1936 presidential campaign. About 10,000,000 questionnaires (the largest number ever) were distributed by Literary Digest, a respected magazine. For ease, the individuals surveyed were selected from automobile owners, telephone subscribers, and country club memberships. On the basis of 2,300,000 responses (only 23%), Literary Digest predicted that Republican Alf Landon (remember him?) would defeat Democratic candidate Franklin Roosevelt by a 3 to 2 margin. This would have been a landslide by election standards.

As you might have guessed (and since you have never heard of President Landon), Roosevelt won, and by a large margin. The erroneous prediction occurred because the voters used in the sample were not representative of the general voting population. In 1936, telephones and automobiles were unaffordable to the average voter, and "John Q. Public" didn't read Literary Digest either. Thus, the majority of voters didn't really care what Literary Digest had to say, and didn't even bother submitting the survey.

A similar bias takes place when considering questionnaires found on tables in restaurants, airlines, etc.
Only those with problems of some sort or those with exceptional experiences will typically respond, creating a set of data not truly representative of the majority of the population.

The moral of the story? Before taking a sample, try to consider what your population is like, and make all efforts to obtain a representative sample.