QUASI EXPERIMENT

INTRODUCTION

Quasi-experimental studies encompass a broad range of nonrandomized intervention studies. These designs are frequently used when it is not logistically feasible or ethical to conduct a randomized controlled trial. Examples of quasi-experimental studies follow. As one example of a quasi-experimental study, a hospital introduces a new order-entry system and wishes to study the impact of this intervention on the number of medication-related adverse events before and after the intervention. As another example, an informatics technology group is introducing a pharmacy order-entry system aimed at decreasing pharmacy costs. The intervention is implemented and pharmacy costs before and after the intervention are measured.

In medical informatics, the quasi-experimental, sometimes called the pre-post intervention, design often is used to evaluate the benefits of specific interventions. The increasing capacity of health care institutions to collect routine clinical data has led to the growing use of quasi-experimental study designs in the field of medical informatics as well as in other medical disciplines. However, little is written about these study designs in the medical literature or in traditional epidemiology textbooks. In contrast, the social sciences literature is replete with examples of ways to implement and improve quasi-experimental studies

DISCUSSION

A.    Definition of Quasi Experiment

A quasi experiment is a type of evaluation which aims to determine whether a program or intervention has the intended effect on a study’s participants. Quasi-experimental studies take on many forms, but may best be defined as lacking key components of a true experiment. While a true experiment includes (1) pre-post test design, (2) a treatment group and a control group, and (3) random assignment of study participants, quasi-experimental studies lack one or more of these design elements.

Since the most common form of a quasi-experimental study includes a pre-post test design with both a treatment group and a control group, quasi-experimental studies are often an impact evaluation that assigns members to the treatment group and control group by a method other than random assignment. Because of the danger that the treatment and control group may differ at the outset, researchers conducting quasi-experimental studies attempt to address this in a number of other ways (e.g., by matching treatment groups to like control groups or by controlling for these differences in analyses). This section focuses on two forms of quasi-experimental studies: a pre-post test design study without a control group and a pre-post test design with a control group.

Quasi-experiments are studies that aim to evaluate interventions but that do not use randomization. Similar to randomized trials, quasi-experiments aim to demonstrate causality between an intervention and an outcome. Quasi-experimental studies can use both preintervention and postintervention measurements as well as nonrandomly selected control groups.

Using this basic definition, it is evident that many published studies in medical informatics utilize the quasi-experimental design. Although the randomized controlled trial is generally considered to have the highest level of credibility with regard to assessing causality, in medical informatics, researchers often choose not to randomize the intervention for one or more reasons:

1.      ethical considerations,

2.      difficulty of randomizing subjects,

3.      difficulty to randomize by locations

4.      small available sample size.

B.  Types of Quasi Experiments

There are many types of quasi-experiments. Here we discuss just a few of the more common ones.

1.      Non-Equivalent Groups Design

A non-equivalent groups design includes an existing group of participants who receive a treatment and another existing group of participants to serve as a control or comparison group. 

2.      Pretest-Posttest Design

In a pretest-posttest design, a single group of participants is measured on the dependent variable both before and after the manipulation of the independent variable. 

3.      Interrupted Time-Series Designs

In an interrupted time-series design, a time series like this (the dependent variable) is interrupted (usually near the middle) by the manipulation of the independent variable. 

C.  Internal Validity

Many researchers (including us) see the difference between correlational studies, quasi-experiments, and experiments as one of degree rather than as one of kind. At one end of this continuum are ideal experiments in which only the independent variable differs across condition, so that it is perfectly clear that changes in the dependent variable were caused by the independent variable.  But as we move from ideal experiments to less ideal ones to quasi-experiments to correlational studies, there are more and more variables that differ across conditions, which makes it more and more difficult to see whether it was the independent variable that was responsible for changes in the dependent variable.

Researchers actually have a name for this continuum: internal validity.  To the extent that a study allows one to conclude that the independent variable affected the dependent variable, we say that it has good internal validity.  So an ideal experiment has perfect internal validity, experiments usually have good internal validity, quasi-experiments have somewhat less internal validity, and correlational studies often have poor internal validity.

D.  Methods of Quasi Experiment

1.      Identifying a research problem

The process starts by clearly identifying the problem you want to study and considering what possible methods will affect a solution. Then you choose the method you want to test, and formulate a hypothesis to predict the outcome of the test.

2.      Planning an experimental research study

The next step is to devise an experiment to test your hypothesis. In doing so, you must consider several factors.

3.      Conducting the experiment

At the start of an experiment, the control and treatment groups must be selected.

4.      Analyzing the data

The fourth step is to collect and analyze the data. This is not solely a step where you collect the papers, read them, and say your methods were a success.

5.      Writing the paper/presentation describing the findings

These are the structure of writing a paper/presentation:

• Abstract: Summarize the project: its aims, participants, basic methodology, results, and a brief interpretation.
• Introduction: Set the context of the experiment.
• Review of Literature: Provide a review of the literature in the specific area of study to show what work has been done. Should lead directly to the author's purpose for the study.
• Statement of Purpose: Present the problem to be studied.
• Participants: Describe in detail participants involved in the study; e.g., how many, etc. Provide as much information as possible.
• Materials and Procedures: Clearly describe materials and procedures. Provide enough information so that the experiment can be replicated, but not so much information that it becomes unreadable. Include how participants were chosen, the tasks assigned them, how they were conducted, how data were evaluated, etc.
• Results: Present the data in an organized fashion. If it is quantifiable, it is analyzed through statistical means. Avoid interpretation at this time.
• Discussion: After presenting the results, interpret what has happened in the experiment. Base the discussion only on the data collected and as objective an interpretation as possible. Hypothesizing is possible here.
• Limitations: Discuss factors that affect the results. Here, you can speculate how much generalization, or more likely, transferability, is possible based on results. This section is important for quasi-experimentation, since a quasi-experiment cannot control all of the variables that might affect the outcome of a study. You would discuss what variables you could not control.
• Conclusion: Synthesize all of the above sections.
• References: Document works cited in the correct format for the field.

E.  Analytic Applications Applied in Quasi experimental Research

Valentin made a claim that with an understanding of eight statistical procedures, it is reasonable to have an understanding of 90% of quantitative research. Experimental designs lend themselves to straightforward, often simpler, statistical analysis than quasiexperimenatal designs. Advanced statistical procedures are typically necessary in quasiexperimental research, largely due to the lack of randomization.

Two specific examples include multiple regression analysis and factor analysis. Multiple regression analysis is a statistical application that is utilized in studies in which impact is being measured. Using statistical methods, a control group is simulated, and multiple adjustments can be made for outside factors. Thus, the control that is in the design of an experiment is inserted through analytical techniques. Factor analysis is a useful technique when a study has a large number of variables. This statistical application allows for a reduction in the number of variables while detecting possible relationships between those variables of interest (Dimsdale & Kutner, 2004). It is commonly applied when data is collected through a survey, especially when the survey contains a large number of items.Analysis of covariance (ANCOVA) is yet another analytical technique employed to increase the strength of the quasiexperimental design. By making compensating adjustments, ANCOVA reduces the effects of the initial differences between groups. This again is an attempt to respond to the lack of randomization.

F.   Strengths and Weakness of Quasi experimental Research

When considering what type of design to employ in a study, it is important to consider both validity and practicality. In general, quasiexperimental research is more feasible, given the typical time and logistical constraints. At the surface level, an easily identifiable weakness of employing quasiexperimental research, in contrast to a true experiment, is the lack of random assignment. Without random assignment, internal validity is reduced, and causal claims become quite difficult to make.

On the other side, quasiexperimental designs tend to present the situation under investigation in real-world conditions, increasing the external validity. Typically, quasiexperimental designs are pre-existing constructions. Because of this, fewer variables are able to be controlled; yet another factor limiting the ability to make causal claims.

With the implementation of the No Child Left Behind statute, educational research put forth an agenda of scientifically based research. Shavelson and Towne outline criteria necessary for a scientific study, which include: direct, empirical investigation of an important question; consideration for the context in which the study took place; alignment with a conceptual framework; careful and thorough reasoning; and disclosure of results. Quasiexperimental research makes the mark by meeting each criterion listed. While the controlled, experimental design is the ideal, at least statistically, when an experiment is not possible or practical, the best approach is to identify and eliminate threats to validity through the implementation of a quasiexperi-mental approach.

CONCLUSION

Although quasi-experimental study designs are ubiquitous in the medical informatics literature, as evidenced by 34 studies in the past four years of the two informatics journals, little has been written about the benefits and limitations of the quasi-experimental approach. As we have outlined in this paper, a relative hierarchy and nomenclature of quasi-experimental study designs exist, with some designs being more likely than others to permit causal interpretations of observed associations. Strengths and limitations of a particular study design should be discussed when presenting data collected in the setting of a quasi-experimental study. Future medical informatics investigators should choose the strongest design that is feasible given the particular circumstances.

REFERENCES

http://psych.csufresno.edu/psy144/Content/Design/Nonexperimental/quasi.html

http://writing.colostate.edu/guides/page.cfm?pageid=1363

http://www.education.com/reference/article/quasiexperimental-research/

http://www.nationaltechcenter.org/index.php/products/at-research-matters/quasi-experimental-study/

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1380192/