Homeland Security Policy Institute releases a report with questionable results.

"The report is essentially the findings of a survey conducted with less than 50 participants. You did not read that wrong. They are making attempts at generalizing statements made by a sample of less than 50 so-called experts, all of essentially homogenous opinion."

Even the authors of the report must point out this massive methodological flaw, although they employ classic Orwellian doublethink by claiming just moments later that it isn’t a flaw at all.

The basis for the report was a 44-question survey conducted by the George Washington University Homeland Security Policy Institute with funds from both GWU and the Ahmanson Foundation at a conference of the Intelligence Unit Commanders Group of the Major Cities Chiefs Association, which “is a professional organization of police executives representing the largest cities in the United States and Canada.”

One might argue that all law enforcement professionals with a focus on counterterrorism will make every effort to highlight the alleged threat terrorism poses because their job security relies on said threat.

The most astounding part of the study is that there were even less participants than there were questions on the survey. You don’t need to take a statistics course to realize that this is a bit unusual.

Would you consider a two-question study legitimate if they asked one person? Probably not, but the Homeland Security Policy Institute, or HSPI, thinks it is a reasonable way to go about compiling research.

For this 44-question study, the sample size was a measly 42 individuals. On page five of the report, they aptly point out that, “From a purely statistical standpoint, such a sample raises questions about the generalizability of HSPI’s findings.”

These findings are not even remotely generalizable. If someone attempted to get research published with such a meaningless sample size, they would get laughed out of the room. This is like polling a single class in a single high school then generalizing it to every high school student in the nation.


Homeland Security Policy Institute report:
http://www.gwumc.edu/hspi/policy/HSPI%20Research%20Brief%20-%20Counterterrorism%20Intelligence.pdf

http://www.activistpost.com/2011/09/homeland-security-policy-institute.html