Tags: Personal Challenge EssayFrederick Douglass Essay SlaveryBusiness Languages For Intercultural And International Business Communication A Canadian Case StudyFree Business Plan Template Word DocFreya Van Den Bossche ThesisModel Literature EssaysDissertation Proposal SyllabusValue Chain Analysis Research PaperCustom Dissertation
Suppose, for example, that a study called for half the teachers in a set of schools to be randomly selected to receive a certain kind of professional development, the other half to be assigned to a control group, and the instructional practices of both groups of teachers to be measured over time.It is likely that such a study would understate the effects of the professional development. Because teachers (usually) talk with one another; they share what they are doing in their classes and how students are responding.
Unless both kinds of sampling error are included in the standard error, investigators may wrongly decide that a program is making a significant difference when, in fact, it is not.
The strategy that Bloom and other leading social scientists employ in cluster randomized trials — referred to as “multilevel modeling” or as “hierarchical modeling” — takes account of both sources of sampling error in producing impact estimates.
So even if only some teachers received the professional development, other teachers would also learn about it, albeit at second hand.
The diffusion of the key concepts and practices associated with the treatment among those not formally slated to receive that treatment — known as “contamination” in research parlance — would make a clean test of program impacts impossible.
In the first scenario, there is considerable variation in student performance within each school at the start of the experiment, but average performance is quite similar across all the schools.
In the second scenario, the situation is just the reverse: at the start of the experiment, students within a school perform vary similarly, but there is considerable variation in performance among the schools.
Until relatively recently, it was common for studies that used cluster random assignment to analyze the resulting data as if had been randomly assigned — a procedure that led to erroneous conclusions.
But because individuals are “nested” within the unit of randomization — the group or cluster — at least two sources of sampling error enter the picture: one resulting from variation in outcomes .
If a different mix of individuals had been randomly assigned to a study’s treatment and control groups, a somewhat different impact estimate would have been obtained.
This reality, which can be referred to as “randomization error,” creates uncertainty about whether the estimated impact is the true impact of the intervention.