Skip to main content

Methodological flaws on “manual therapy for the pediatric population: a systematic review” by Prevost et al. (2019)

A Correspondence to this article was published on 05 January 2021

Abstract

Prevost et al. published a systematic review evaluating the use of manual therapy for clinical conditions in the pediatric population in 2019. However, several methodological flaws in the conduct of the review limit the internal validity of its conclusions. We caution readers about the validity of the recommendations and suggest that the review not be used to inform the clinical management of pediatric patients.

Peer Review reports

Background

We reviewed the systematic review by Prevost et al [1] and are concerned about its methodological quality. We critically appraised the systematic review using the Scottish Intercollegiate Guidelines Network (SIGN) criteria for systematic reviews. Our appraisal of the review suggests that it suffers from numerous methodological flaws, and that it does not meet the minimal quality criteria of a valid systematic review. The methodological limitations refer to the formulation of its research question, the search strategy, the classification of the quality of evidence, and the evidence synthesis. These limitations restrict the interpretation and generalizability of this review.

Methodological flaws

Specifically, our first concern is with the stated objectives of the review: “…to evaluate the use of manual therapy for clinical conditions in the pediatric population, assess the methodological quality of the studies found, and synthesize findings based on health condition.” These objectives are vague and do not comply with accepted frameworks to formulate research questions to assess the effectiveness of interventions [2, 3] In particular, the clinical conditions should be identified in the research question since other conditions (e.g., cardiovascular disease, renal disease) were not included in the review. “Assess the methodological quality of the studies found” is not a research question/objective. Furthermore, “Synthesize findings based on health condition” is not clear, specific or representative, or reproducible.

Second, the search strategy presented in Table 1 is not comprehensive and does not meet the current standards for systematic review literature searching, as defined in the Peer Review of Electronic Search Strategies (PRESS) [4] Specifically, important search terms that could identify studies conducted in the pediatric populations were not included (e.g., newborn, infant, youth). Moreover, according to PRESS, the search strategy should be developed with the assistance of a health sciences librarian and reviewed by a second health sciences librarian using the PRESS checklist [5] Although the authors mention that three librarians were used, it is unclear whether and how the librarians reviewed each other’s work using the PRESS checklist. Furthermore, the authors claimed that they searched several databases (as listed in Table 1). However, we are concerned about the validity of this statement because ScienceDirect and McCoy Press are journal publishers, not databases. Therefore, we cannot exclude that the search strategy led to significant selection bias in the included literature.

Third, this review used cut-off points to classify the quality of evidence (i.e., “low quality study if the score was between 0 and 33.3%, medium quality if the score was between 33.4 and 66.6%, and high quality if the score was above 66.6%”). Although cut-points are commonly used to classify levels of evidence, it is important to understand the limitations of this methodology. One important limitation is that impact of biases identified by the items is not weighted (i.e., each item is given the same weight regardless of their impact on study results) [6] Therefore, a study with a high score can be deemed to be of high quality even if it suffers from a fatal methodological flaw (e.g., differential attrition rates leading to differential ascertainment of outcomes).

Fourth, the results of each study were categorized to indicate whether a treatment led to: 1) improvement; 2) no improvement; or 3) no difference in participants’ outcomes. However, the decisions used to make the judgements are not clear and cannot be critically appraised nor replicated. Specifically, what consideration was given to statistical significance versus clinical importance and to within- versus between-group differences? Also, the main descriptive results presented in data extraction tables (Tables 6–10) only provide a general summary of the results and the necessary information to interpret the results are lacking (e.g., control interventions, effect size, statistical significance, precision of estimates, clinical importance, etc.). We could not identify any supplemental tables providing such information about control interventions and effect size.

Conclusions

Based on our concerns, we caution readers about the poor internal validity of the review and suggest that its recommendations should not be used to inform the clinical management of pediatric patients.

Availability of data and materials

Not applicable.

Abbreviations

SIGN:

The Scottish Intercollegiate Guidelines Network

PRESS:

The Peer Review of Electronic Search Strategies

References

  1. 1.

    Parnell Prevost C, Gleberzon B, Carleo B, Anderson K, Cark M, Pohlman KA. Manual therapy for the pediatric population: a systematic review. BMC Complement Altern Med. 2019;19(1):60 https://bmccomplementalternmed.biomedcentral.com/articles/10.1186/s12906-019-2447-2.

    Article  Google Scholar 

  2. 2.

    Riva JJ, Malik KMP, Burnie SJ, Endicott AR, Busse JW. Commentary – what is your research question? An introduction to the PICOT format for clinicians. J Can Chiropr Assoc. 2012;56(3):167–71 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3430448/.

    PubMed  PubMed Central  Google Scholar 

  3. 3.

    Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

  4. 4.

    McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6 https://www.jclinepi.com/article/S0895-4356(16)00058-5/pdf.

    Article  Google Scholar 

  5. 5.

    McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

  6. 6.

    van der Velde G, van Tulder M, Côté P, Hogg-Johnson S, Aker P, Cassidy JD, et al. The sensitivity of review results to methods used to appraise and incorporate trial quality into data synthesis. Spine. 2007;32(7):796–806 https://www.ncbi.nlm.nih.gov/pubmed/17414916.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Affiliations

Authors

Contributions

HY: designed the work, critically appraised the systematic review (Prevost et al., 2019), drafted and revised the Correspondence. HS: designed the work, critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. PC: designed the work, critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. AT-V: designed the work, critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. SM: designed the work, critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. LV: critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. GC: critically appraised the systematic review (Prevost et al., 2019), reviewed, revised and approved the Correspondence. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Pierre Côté.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Shearer, H., Taylor-Vaisey, A. et al. Methodological flaws on “manual therapy for the pediatric population: a systematic review” by Prevost et al. (2019). BMC Complement Med Ther 21, 4 (2021). https://doi.org/10.1186/s12906-020-03145-6

Download citation

Keywords

  • Pediatric
  • Manual therapy
  • Manipulation
  • Mobilization
  • Systematic review