Table 1 Variables and Definitions Used in Systematic Review
From: A systematic review of the quality of homeopathic clinical trials
Variable | Definition |
|---|---|
Sample size | Number of subjects who participated in the research at the initiation of the study |
Gender | Percent of sample male/female |
Age | Average age of sample |
Ethnicity | Percent of identified racial or ethnic categories |
Response rate | Sample size divided by the total number of people approached for participation |
Attrition rate | Number of subjects finishing study divided by sample size |
Sampling Frame | How sample was identified–random, systematic, mixed, none, not indicated |
Research Design | Prospective, cross-sectional or retrospective design |
Statistics | Based on highest level used: descriptive, univariate, or multivariate |
Funding Source | Who paid for research |
Control Group | Whether or not a control group was used in design |
Reliability | Any indication of the reliability of any measure used. |
Random assignment | Use of random assignment to groups |
Confounding variable | Any measurement or statistical applications that attempt to identify and control for variables that might influence outcomes independent of treatment. |
Threats to validity | |
Threats to Statistical Conclusion Validity | |
Low statistical power | Rated as any groups < 10 or correlations with fewer than 30 pairs. |
Violated assumption | Evidence of non-normal distributions with parametric statistics |
Fishing/error rate | More than 10 statistical tests without a Bonferroni (or similar) correction. |
Reliability of measures | Failure to test, note or reference information on measures |
Reliability of treatment | Failure to test or note consistently of the application of treatments |
Random irrelevancies | Absence of effort to make measurements reasonably consistent. |
Random heterogeneity representative | Absence of effort to ensure that sample is reasonably |
Threats to Internal validity | |
History | Repeated measures without temporal control |
Maturation | Repeated measures without temporal control on developmentally sensitive outcomes. |
Testing | Use of measures that are sensitive to the testing process. |
Instrumentation | Use of poorly or uncalibrated measures. |
Statistical regression | Study of cases selected from extremes without control group |
Mortality | More than 30% of sample did not complete study |
Interaction with selection | Evidence that selection into groups might interact with history, maturation, or testing. |
Ambiguity of cause | An association that can be interpreted in either direction vis-à-vis cause and effect. |
Diffusion of treatment | Contact between experimental and control subjects. |
Compensatory work to Equalization | Evidence that groups are knowledgeable about design and might Equal things out. |
Compensatory rivalry | Evidence that groups are knowledge about design and might compete with other group members. |
Resentful demoralization | Evidence that one group feels disadvantaged through group assignment process. |
Threats to Construct Validity | |
Inadequate explication | Evidence that choice of measurement operations does not represent the construct. |
Mono-operation bias | Use of only one question for central outcome |
Mono-method bias | Use of only one measurement approach (e.g self report) for central outcome |
Hypothesis guessing | Evidence that subjects might attempt to guess what results should be (absence of appropriate blinding procedures). |
Evaluation apprehension | Evidence that subjects might become anxious during assessments. |
Experimenter bias | Absence of controls to keep invested parties from participating in the measurement procedures. |
Confounding constructs | Range restriction in measurement |
With level of constructs | |
Threats to External Validity | |
Interaction of different Treatment | Failure to assess additional treatment received during study. |
Interaction of testing And treatment | Evidence that testing might be related to the treatment so that subjects complete tests differently after treatment. |
Restricted generalizability | Evidence of a limited measurement approach that may not generalize. |
Sample bias | Evidence of a poorly chosen sample that does not represent the population considered. |
Interaction of selection And treatment | Evidence that group assignment is influenced by variables related to the likelihood of response to treatment |
Interaction of setting And treatment | Evidence that treatment works only in some settings. |
Interaction of history And treatment | Evidence that the time during which the study was done may have an impact on findings. |