Table 4 Risk of Bias Assessment
From: Reflexology in oncological treatment – a systematic review
Reference | Study type | Standardized rating of risk of bias | Additional comments on methodology |
|---|---|---|---|
Mantoudi (2020) | RCT | RoB Randomised Assignment: low Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: randomization by independent person, testing for normal distribution, controlling for multiple testing, active control group Report quality: specification of effect sizes, information on other therapies CONTRA: Methodical quality: No homogeneity for chemotherapy between groups at 5% significance level (In order to reach homogeneity, significance level was decreased to 1% for this parameter), no power analysis, researcher conducted interventions, no blinding Report quality: No information where researcher learned how to apply reflexexology, No comparison of drug dosage at baseline, no specification on comorbidities |
Göral Türkcü, Özkan (2021) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: Homogeneity between arms Methodical quality: Approved by ethics committee, randomization via SPSS, power analysis, testing for normal distribution, Mann–Whitney U test for data without normal distribution, control for multiple testing CONTRA: Sample: only gynecological cancers (possibly limited carryover to other patient groups) Methodical quality: single blind (not possible), side effects only reported by researchers based on verbal and non-verbal responses of the patients, researcher applied intervention and collected data, no information on comorbidities, therapies or medication, short time frame of intervention Report quality: no information if researcher had formal training, no data on control group after 2nd week |
Murat-Ringot (2021) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: high Overall Risk of Bias: high | PRO: Sample: Homogeneity between groups Methodical quality: Intention-to-treat analysis for primary endpoint, power analysis, Sensitivity analyses for patients without VAS assessments during the 2nd cycle of chemotherapy, Categorical variables compared between groups, Comparison of nonparametric variables, reflexology applied by three certified reflexologists CONTRA: Methodical quality: no blinding, initially planned statistical method was altered, per protocol analysis for secondary outcome, for self-esteem two different scales were used at baseline and end, home application (no information how much) of reflexology not considered in analysis |
Sikorskii (2020) | RCT | RoB Randomised Assignment: low Deviations from the intended interventions: high Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: Homogeneity between arms Methodical quality: Associations among responses to multiple symptoms within patients were accounted for, dummy variable for differentiation between potentially different effects on different symptoms, patient level covariate analysis CONTRA: Methodical quality: no blinding Report quality: no data on dropouts, no information on ethics committee approval, no specification of location and type of pain |
Samancioglu (2019) | RCT | RoB Randomised Assignment: high Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: Active control group CONTRA: Sample: baseline differences between groups, small sample Methodical quality: no testing for normal distribution of data, per protocol analysis, no power analysis Report quality: no clear differentiation who dropped out of the study |
Wyatt (2021) | RCT | RoB Randomised Assignment: low Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: large sample, homogeneity between arms Methodical quality: patients with missing data points analysed due to LME model, inclusion of balancing factors for randomization, blinding of interviewers, Control group despite SMART, power analysis Report quality: Comparison of baseline values of dropouts, CONTRA: Sample: majority of sample are white women which are most interested in this kind of therapy according to research Methodical quality: bigger dropout for meditative practices (suitable protocol?) no correction for multiple testing, possibly varying frequency of intervention between patients, patients actively approached during hospital visits (possible bias), after week 4 high risk of bias due to differentiation between responders and non-responders Report quality: no information on other medication |
Sikorskii (2018) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Report quality: Effect sizes included CONTRA: Report quality: No differentiation between intervention and control group at baseline |
Rambod (2019) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: No dropouts Methodical quality: Power analysis, intervention by certified reflexologist, blinding of outcome assessor, testing for normal distribution, ANCOVA for comparison between groups CONTRA: Methodical quality: no controlling for multiple testing Report quality: no information on other treatments |
Nourmohammadi (2019) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: | PRO: Sample: Homogeneity between groups Methodical quality: Double blind, ANCOVA Report quality: information on belief in palliative care CONTRA: Methodical quality: hard to blind patients, no testing for normal distribution, randomization based on days of the week Report quality: No data directly after intervention periods, no information on other treatments and comorbidities, no information on who performed the intervention |
Hesami (20,219) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: Homogeneity among groups Methodical quality: power analysis, ANCOVA CONTRA: Methodical quality: short study period, researcher applied intervention, no testing for normal distribution, no follow up Report quality: no detailed information on other treatments, no information on blinding |
Dikmen (2019) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: Homogeneity between groups Methodical quality: Accounting for washout time of analgesic medications before intervention, power analysis, testing for normal distribution, ANOVA CONTRA: Methodical quality: randomization by researcher, intervention applied by researcher, researcher blinded for analysis (can he really be blind if he knew the allocation before?), blinding of patients not really possible, shorter sessions for progressive muscle relaxation, no controlling for multiple testing, effects of analgesics may last longer than the 30 and 60 min used in the study, dropout of patients because they didn’t match inclusion criteria (this could have been sorted out earlier) Report quality: few baseline information, full results only presented visually and not numerically (only p-values), no data on analgesic use, number of patients randomized is much bigger than number of patients participating (what happened?) |
Rezaei (2021) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: some concern Missing outcome data: some concern Measurement of the outcome: low Selection of the reported result: some concern Overall Risk of Bias: some concern | PRO: Sample: Methodical quality: power analysis, testing for normal distribution, Report quality: study registered in Iranian clinicaltrials.com registry CONTRA: Sample: Methodical quality: first author applied reflexology (not double blind), blinding of patients is not possible, researcher had direct contact with patients (high risk of bias), no controlling for multiple testing, very short time frame (one day) Report quality: no clear information if researcher was actually a trained reflexologist, process unclear (who evaluated the data?) |
Jahani (2018) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: Homogeneity between groups, power analysis CONTRA: Methodical quality: blinding is not really possible, no testing for normal distribution, no controlling for multiple testing Report quality: No information on other treatments or comorbidities, no information on dropouts, no further details on control group (probably only standard care then), process is not entirely clear from the text |
Kurt (2018) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: power analysis Report quality: information on chemotherapy CONTRA: Methodical quality: difference in number of patients in arms, no blinding, big dropout (problems with study design or recruiting?) Report quality: some patients apparently didn’t want to answer some questions precisely |
Dyer (2013) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: homogeneity between groups, low drop out Methodical quality: power analysis, Intention-to-treat analysis for primary outcome CONTRA: Methodical quality: no blinding, per protocol analysis for other outcome Report quality: outcome data for all only for primary outcome |
Hodgson (2012) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: low Selection of the reported result: low Overall Risk of Bias: some concern | PRO: Methodical quality: active control group, crossover design with washout CONTRA: Sample: homogenous sample Methodical quality: no blinding Report quality: statistical analysis incomprehensible |
Özdelikara (2017) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | CONTRA: Methodical quality: no control for multiple testing Report quality: no information on dropouts |
Özdelikara (2017) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | CONTRA: Methodical quality: no control for multiple testing Report quality: no information on dropouts |
Sharp (2010) | RCT | RoB Randomised Assignment: low Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: large sample size, homogeneity between groups Methodical quality: active control, Intention-to-treat analysis, control for multiple testing CONTRA: Report quality: one sided interpretation of results, risk for reporting bias |
Stephenson (2007) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: high Overall Risk of Bias: high | PRO: Sample: larger sample size, homogeneity between groups Methodical quality: statistical analysis CONTRA: Methodical quality: active but not completely comparable comparison group Report quality: reporting bias (only one of two scales for pain reported) |
Stephenson (2000) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: some concern Overall Risk of Bias: high | CONTRA: Sample: small sample size Methodical quality: incorrect statistical analysis for crossover design, for all except one outcome only part of the sample was analysed |
Uysal (2017) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: high Measurement of the outcome: high Selection of the reported result: high Overall Risk of Bias: high | PRO: Sample: active control Methodical quality: control for multiple testing, power analysis CONTRA: Sample: baseline differences, differences in groups regarding tumour grade Methodical quality: no blinding, differences in length of sessions between interventions Report quality: reporting bias |
Wyatt (2012) | RCT | RoB Randomised Assignment: low Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: large and multicentric sample Methodical quality: Intention-to-treat analysis CONTRA: Sample: Methodical quality: no blinding Report quality: no concrete results for comparison of active groups |
Tsay (2008) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: some concern Missing outcome data: low Measurement of the outcome: high Selection of the reported result: high Overall Risk of Bias: high | PRO: Sample: homogeneity between groups Methodical quality: power analysis CONTRA: Methodical quality: termed as double blind but no blinding possible, amount of narcotic analgesia consumption not evaluable Report quality: no information on adverse effects or conflict of interest |
Wyatt (2017) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Sample: large and multicentric sample Methodical quality: homogeneity between groups, Intention-to-treat analysis CONTRA: Sample: large dropout Report quality: no concrete information on activity in control group |
Anderson (2021) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: power analysis, reflexologist blinded so presession survey until after session CONTRA: Sample: small sample, no information on homogeneity Methodical quality: no direct comparison between groups, no comparison, no testing for normal distribution Report quality: no information on comorbidities |
Ross (2002) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: high Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: patients and interviewers blinded, correction for difference in group size, active control CONTRA: Sample: small sample Report quality: no information on data analysis |
Hodgson (2000) | RCT | RoB Randomised Assignment: some concern Deviations from the intended interventions: low Missing outcome data: low Measurement of the outcome: high Selection of the reported result: low Overall Risk of Bias: high | PRO: Methodical quality: single blind CONTRA: Sample: no information on homogeneity and cancer type, small sample Methodical quality: no testing for normal distribution Report quality: differences regarding timing of post intervention questionnaire in the beginning, due to printing error some items were left out of the questionnaire, no data on patients pre intervention, not all p-values disclosed, no information on other therapies |