Skip to main content
  • Research article
  • Open access
  • Published:

A systematic review and quality assessment of complementary and alternative medicine recommendations in insomnia clinical practice guidelines

Abstract

Background

Sleep disorders encompass a wide range of conditions which affect the quality and quantity of sleep, with insomnia being a specific type of sleep disorder of focus in this review. Complementary and alternative medicine (CAM) is often utilized for various sleep disorders. Approximately 4.5% of individuals diagnosed with insomnia in the United States have used a CAM therapy to treat their condition. This systematic review identifies the quantity and assesses the quality of clinical practice guidelines (CPGs) which contain CAM recommendations for insomnia.

Methods

MEDLINE, EMBASE and CINAHL were systematically searched from 2009 to 2020, along with the Guidelines International Network, the National Center for Complementary and Integrative Health website, the National Institute for Health and Care Excellence, and the Emergency Care Research Institute. CPGs which focused on the treatment and/or management of insomnia in adults were assessed with the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument.

Results

From 277 total results, 250 results were unique, 9 CPGs mentioned CAM for insomnia, and 6 out of the 9 made CAM recommendations relevant to insomnia. Scaled domain percentages from highest to lowest were scope and purpose, clarity of presentation, editorial independence, stakeholder involvement, rigour of development, and applicability. Quality varied within and across CPGs.

Conclusions

The CPGs which contained CAM recommendations for insomnia and which scored well could be used by health care professionals and patients to discuss the use of CAM therapies for the treatment/management of insomnia, while CPGs which scored lower could be improved in future updates according to AGREE II.

Peer Review reports

Background

Sleep disorders are classified as a broad class of disorders which center around changes in quality and quantity of sleep, and can affect a patient's safety, productiveness, and overall quality of life [1, 2]. Sleep disorders include conditions such as insomnia, obstructive sleep apnea, narcolepsy, and restless leg syndrome [1, 2]. Insomnia is one of the most common sleep disorders and is defined as difficulty in initiating sleep, maintaining sleep, or obtaining good quality sleep [1,2,3,4]. Insomnia has been linked with many comorbid disorders such as hypertension, cardiovascular disease, depression, and diabetes [5]. Insomnia can also result in alterations to attention and episodic memory, and these cognitive impairments are clinically significant [6]. Thus, the treatment of insomnia is important for managing the overall health of an individual. Insomnia can be diagnosed as acute or chronic; furthermore, primary insomnia is defined as an individual developing sleep-preventing associations due to stress or emotions, while co-morbid insomnia is sleeplessness which arises from a primary illness [2, 7]. Specific diagnostic criteria for insomnia can be found in the International Classification of Sleep Disorders guideline [8]. Approximately 70 million Americans suffer from a sleep disorder in general, and more specifically, a 2018 study found that approximately 25% of the adult population experiences acute insomnia in the United States [9, 10]. One common group of therapies used by those suffering from insomnia include complementary and alternative medicine (CAM). Among adults experiencing insomnia, research has shown that 4.5% of them utilized CAM to treat their condition [11]. “Complementary medicine” is defined as practices which are considered to be non-mainstream, or methods which are not a part of the standard medical care. They are used in conjunction with conventional medical practices [12, 13]. In contrast, “alternative medicine” is defined as a non-mainstream practice which is used instead of standard medical care [12, 13].

CAM consists of a wide range of therapies which originate from various schools of thought as well as geographical regions of the world [14, 15]. Common CAM therapies used for insomnia include acupuncture, Ayurveda, melatonin, valerian, yoga, and mind-body practices [15,16,17,18,19,20,21]. Mind-body interventions such as mindfulness meditation and yoga assist in stress management/reduction and management of anxiety, which can improve sleep quality [17]. Melatonin, a highly studied CAM treatment for insomnia, targets the pineal gland and influences the circadian sleep-wake cycle, along with having possible sedative effects [18, 19]. Acupuncture and acupressure have been proposed to restore the normal sleep-wake cycle, and can also be used to increase the content of γ-amino butyric acid which enhances quality of sleep [20, 21]. Acupuncture has also been used to manage fatigue [20, 21]. While some individuals who suffer from insomnia utilize CAM, conventional healthcare providers generally receive little to no education or training about CAM therapies [22]. This lack of training can lead to miscommunication between healthcare providers and patients when treating insomnia with CAM, which can hinder the efficacy of the treatment plan [23].

Healthcare providers use evidence-based clinical practice guidelines (CPGs) to inform their practice decisions, especially in fields where their knowledge and expertise may be lacking [24]. Developers of CPGs create recommendations by reviewing current evidence, and assessing the benefits and risks associated with certain therapies [25]. The purpose of this present study was to conduct a systematic review to identify the quantity and assess the quality of CAM recommendations in CPGs for the treatment and/or management of insomnia using the AGREE II instrument.

Methods

Approach

In order to identify eligible CPGs focused on insomnia or containing a section on insomnia, standard methods and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) criteria were utilized [26]. A protocol was registered with PROSPERO, registration number CRD42020182236. Eligible CPGs containing CAM recommendations for insomnia were assessed with the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument, a tool which has been validated and widely-used [27]. CPGs which contained CAM recommendations were re-assessed with the AGREE II instrument and the assessors applied the 23 criteria to only the CAM sections of the CPGs. The AGREE II instrument contains 23 items grouped in 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence.

Eligibility criteria

Eligibility criteria for insomnia CPGs were based on the Population, Intervention, Comparison and Outcomes (PICO) framework. Populations which were considered eligible included adults aged 19 years and older with insomnia. With regards to interventions, we only included CPGs which 1) exclusively focused on the treatment and/or management of insomnia, or 2) focused on sleep disorders in general, but provided treatment and/or management recommendations for insomnia. CPGs which did not contain any mention of insomnia or CPGs which only provided recommendations for other sleep disorders (i.e. sleep apnea, restless leg syndrome) were excluded. Comparisons concerned the assessed quality of insomnia CPGs. The outcomes were scores from the AGREE II instrument which reflected CPG content and format. Additional inclusion criteria were applied to each CPG as follows: developed by non-profit organizations (this included academic institutions, government agencies, disease-specific foundations, or professional associations or societies); published in 2009 or later; published in the English language; and either publicly available or orderable via our library system. Publications which were deemed ineligible included protocols, abstracts, conference proceedings, letters and editorials. Additionally, primary studies (i.e., surveys, trials, case-control and cohort studies that evaluated an outcome relating to the treatment/management of insomnia, or focused on insomnia curriculum, education, training, research, professional certification or performance were all not eligible. The AGREE II instrument was applied twice to the eligible CPGs containing CAM recommendations: once for the overall CPGs, and then once for the CAM sections of the CPGs. This identified the domain score differences between the overall CPG and the CAM sections of the CPG. For eligible CPGs which only made mention of CAM, but did not contain CAM therapy recommendations, only demographic information was reported.

Searching and screening

MEDLINE, EMBASE and CINAHL were searched on April 17, 2020 from 2009 to April 16, 2020 inclusive. The search strategy (Supplementary File 1) included indexed headings and keywords that reflect terms commonly used in the literature to refer to insomnia. We also searched the Guidelines International Network, a repository of guidelines by using keyword searches based on the eligibility criteria, such as “insomnia” and “sleep” [28]. We then searched the NCCIH website which contained a single list of CAM CPGs [29]. The National Institute for Health and Care Excellence (NICE) Evidence Search was searched using the terms “sleep” and “sleep conditions” [30], and the Emergency Care Research Institute (ECRI) Guidelines Trust was searched using the search term “insomnia” and specifying the type of publication as “guidance” [31]. NDP and another research assistant screened titles and abstracts from all sources. Then, NDP and the other research assistant screened full-text items to confirm eligibility as per the PICO guidelines. JYN reviewed the screened titles and abstracts and full-text items to standardize screening, and assisted in resolving discrepancies between the 2 screeners.

Data extraction and analysis

Relevant data were collected from each CPG and summarised as follows: date of publication, country of first author, type of CPG publishing entity (i.e. research institutions, government departments, disease-specific foundations or professional associations or societies) and whether the specific CPG made any mention of or recommended any CAMs. Data extraction was performed on the type(s) of CAM listed, CAM recommendations made, CAM funding sources, and whether any CAM providers were part of the guideline panel in the event of CAM being mentioned in the CPG. Most of the aforementioned data was available in the CPG; in order to provide additional information, each developer’s website was searched for associated knowledge-based tools to support the implementation of the recommendations.

Guideline quality assessment

NDP and another research assistant completed all data extractions and AGREE II quality assessments of eligible CPGs. The initial step included NDP and the other research assistant participating in a pilot test of the AGREE II instrument by applying it to 3 separate CPGs, and discrepancies in scores were discussed with JYN and resolved. NDP and the other research assistant then proceeded to independently assess all eligible CPGs containing CAM therapy recommendations twice (i.e., once for the overall CPG, and once for only the CAM sections of the CPG). Six domains which contain 23 separate items were rated using a seven-point Likert scale from strongly disagree (1) to strongly agree (7) that the CPG contained the relevant criteria. The overall quality of each CPG (1 to 7) was also rated based on individual score items, and that information was used to recommend, recommend with modifications, or not recommend the CPG for use. The modified AGREE II questions used to score the CAM sections of each CPG are found in Supplementary File 2. NDP and the other research assistant then met with JYN and discussed and resolved discrepancies between the two appraisers' scores. For each CPG, each assessor's average of all 23 scored items was calculated separately and those two values were then combined to produce each CPG’s average assessment score. For each CPG, the total evaluation scores of both appraisers were combined to produce the average overall assessments. For each domain, scaled percentages were determined to allow for comparisons between the various aspects of each CPG. For each of the six domains, the scaled domain percentage was determined by taking the total of the two appraisers' individual scores within the domain, scaling it to the minimum and maximum possible score for that domain, and then translating it into a percentage. The average of the CPG’s scaled domain percentages was also created for each domain.

Results

Search results (Fig. 1)

Searches retrieved 277 items, 250 were unique, and 237 titles and abstracts were eliminated, leaving 13 full-text CPGs that were considered. Of those, 2 were not eligible because they did not contain a focus on insomnia (n = 2), 1 was not eligible primarily because it was not focused on CAM (n = 1), and 1 was focused on the evaluation/diagnosis of insomnia rather than the management/treatment of insomnia (n = 1), leaving 9 CPGs eligible for review [32,33,34,35,36,37,38,39,40]. Six out of the 9 CPGs made CAM therapy recommendations [32, 33, 35, 37,38,39]; of the remaining 3 CPGs, 1 CPG only contained mention of CAM [34], and 2 CPGs contained neither CAM mentions nor CAM recommendations [36, 40].

Fig. 1
figure 1

PRISMA Diagram

Guideline characteristics (Table 1)

Eligible CPGs were published from 2009 to 2020 in the United States (n = 3), Brazil (n = 1), Canada (n = 1), Germany (n = 1), Hong Kong (n = 1), India (n = 1), and Italy (n = 1). The CPGs were funded and/or developed by professional associations or societies (n = 7), academic (n = 1), and an international agency (n = 1). Seven CPGs made mention of CAMs. CAM recommendations were made in 6 CPGs, and included herbal medicines (n = 4), acupuncture (n = 4), yoga (n = 3), chiropractic use (n = 2), homeopathy (n = 2), mindfulness-based stress relaxation (n = 2), physical activity (n = 2), hypnotherapy (n = 1), and foot reflexology (n = 1). Only the CPGs with CAM recommendations were assessed using the AGREE II tool. CAM funding sources were not used in any of the CPGs, and 1 CPG included CAM providers as part of the guideline panel. In Fig. 2, a summary of CAM recommendations for insomnia are provided for the benefit of clinicians and researchers.

Table 1 Characteristics of eligible guidelines
Fig. 2
figure 2

Summary of CAM Recommendations in Clinical Practice Guidelines

Guidelines mentioning CAM without recommendations

One CPG contained CAM mention but did not make specific CAM recommendations [34]. This CPG mentioned mindfulness-based stress relaxation, hypnotic agents, and various herbal therapies [34].

Average appraisal scores, average overall assessments and recommendations regarding use of CPGs: overall CPG (Table 2)

On the seven-point Likert scale, the average appraisal scores for each of the 6 CPGs ranged from 3.8 to 5.9 (where 7 equals strongly agree that the item is met). All 6 CPGs achieved or exceeded a 3.0 average appraisal score, 4 CPGs achieved or exceeded a 4.0 average appraisal score, and 2 CPGs achieved or exceeded a 5.0 average appraisal score. The average overall assessment for the 6 CPGs ranged from 3.5 (lowest) to 6.0 (highest), which included 4 CPGs equal to or greater than 4.0 and 2 CPGs equal to or greater than 5.0.

Table 2 Average appraisal scores and average overall assessments of each guideline

Average appraisal scores, average overall assessments and recommendations regarding use of CPGs: CAM sections (Table 2)

On the seven-point Likert scale, the average appraisal scores for each of the 6 CPGs ranged from 3.0 to 4.8 (where 7 equals strongly agree that the item is met). All 6 CPGs achieved or exceeded a 3.0 average appraisal score, 4 CPGs achieved or exceeded a 4.0 average appraisal score. The average overall assessment for the 6 CPGs ranged from 2.5 (lowest) to 6.0 (highest), which included 5 CPGs equal to or greater than 3.0 and 3 CPGs equal to or greater than 4.0.

Overall recommendations: overall guideline (Table 3)

For overall recommendations, appraisers agreed in their overall recommendation for 3 out of 6 CPGs including Yes [39], Yes with modifications [33], and No [38]. Of the remaining 3 CPGs, 2 were rated Yes with modifications and Yes [35, 37], while the remaining CPG was rated No and Yes with Modifications [32].

Table 3 Overall recommendations for use of appraised guidelines

Overall recommendations: CAM sections (Table 3)

For the CAM sections of the CPGs, 3 out of 6 CPGs were given the same appraisal, with 1 CPG being Yes [39], and 2 CPGs being No [33, 38]. Of the remaining 3 CPGs, 2 were rated Yes with modifications and Yes [35, 37], while the remaining CPG was rated No and Yes with modifications [32].

Scaled domain percentage quality assessment (Table 4)

With regards to scaled domain percentages of the overall CPGs, scope and purpose scores ranged from 83.3 to 100.0%, stakeholder involvement scores ranged from 38.9 to 88.9%, rigor-of-development scores ranged from 24.0 to 91.7%, clarity of presentation scores ranged from 38.9 to 100.0%, applicability scores ranged from 10.4 to 50.0%, and editorial independence scores ranged from 37.5 to 95.8%. With regards to scaled domain percentages of the CAM sections, scope and purpose scores ranged from 72.2 to 100.0%, stakeholder involvement scores ranged from 22.2 to 61.1%, rigor-of-development scores ranged from 25.0 to 73.9%, clarity-of-presentation scores ranged from 44.4 to 86.1%, applicability scores ranged from 10.4 to 37.5%, and editorial independence scores ranged from 16.7 to 70.8%.

Table 4 Scaled domain percentages for appraisers of each guideline

Scope and purpose

The overall objectives and health questions were generally well-defined for both the overall CPGs and the CAM sections of the CPGs. All CPGs scored highly in providing the overall goals of the CPGs and for defining the target disease/condition in the overall CPG and CAM sections. For the population to which the CPGs were meant to apply, all CPGs scored highly for both the overall CPGs and CAM sections [32, 33, 35, 37,38,39].

Stakeholder involvement

In terms of the overall CPGs, most provided thorough details regarding the characteristics of the members of the guideline development group, including their names, professions, and institutional affiliation [32, 35, 37,38,39], while 1 did not [33]. However, for the CAM sections of the CPGs, only 2 CPGs included CAM experts in their guideline development group [32, 39]. In terms of the views and preferences of the target population, only 2 CPGs for both the overall and CAM sections detailed these views [32, 37], while the rest of the CPGs did not [33, 35, 38, 39]. For the overall CPGs, target users were typically well-defined and descriptions included items such as the type or specialty of practitioner [32, 33, 35, 37,38,39]. For the CAM sections, 3 CPGs defined their target users well [32, 35, 37], while the other 3 did not [33, 38, 39].

Rigor of development

With the exception of 1 CPG [38], systematic methods were used to search for evidence for the overall CPGs and the CAM sections of the CPGs [32, 33, 35, 37, 39]. For the overall CPGs, the criteria for selecting the evidence were clearly described for 4 CPGs [33, 35, 37, 38], while 2 CPGs did not contain clearly defined criteria for selecting the evidence [32, 38]. For the CAM sections of the CPGs, a few CPGs clearly defined their criteria for selecting evidence [33, 35, 37, 39], and a subset of the CPGs did not [32, 38]. For both the overall CPGs and CAM sections, the strengths and limitations of the body of evidence were clearly described in 4 CPGs [33, 35, 37, 39], while 2 of the CPGs did not identify strengths and limitations well for both the overall and CAM sections [32, 38]. Discrepancies existed between how the methods for formulating recommendations were outlined. For overall CPGs, some provided a sufficient amount of detail on how consensus was reached [35, 37, 39], while others provided little to no information [32, 33, 38]. This was similar for the CAM sections, as some CPGs adhered to the criteria outlined in the AGREE II instrument [35, 37, 39], while others did not [32, 33, 38]. Although quality varied between CPGs, all CPGs considered some of the health benefits, side effects, and/or risks in formulating their recommendations, for both the overall CPGs and CAM sections. All CPGs had an explicit link between their supporting evidence and the recommendations for both the overall and CAM sections. In terms of external review prior to publication, only 2 CPGs mentioned that they were externally reviewed [37, 39], while the rest of them were not [32, 33, 35, 38]. None of the CPGs were externally reviewed by CAM experts prior to publication. Half of the CPGs included a procedure for updating the CPGs [32, 37, 39], while the other half did not include this procedure [33, 35, 38], with respect to both the overall CPGs and CAM sections.

Clarity of presentation

Most CPGs offered specific and unambiguous recommendations for both the overall CPG and the CAM sections except for 1 CPG [38]. However, many of the CPGs did not include details in their recommendations, such as the identification of the intent/purpose, relevant population, or caveats. In terms of presenting different options for the management of the condition or health issue, 5 CPGs scored highly for both the overall CPGs and the CAM sections, with the exception of 1 CPG which scored poorly [22]. While key recommendations were generally easily identifiable for the overall recommendations in the CPGs [32, 35, 37,38,39], some CPGs did not have easily accessible key recommendations [33]. CAM recommendations were often more difficult to identify, and a few CPGs did not have easily identifiable key recommendations [32, 33, 37].

Applicability

One CPG discussed facilitators and barriers to the implementation of the recommendations for the overall CPG and CAM sections [39], while the other 5 CPGs did not for either the overall CPGs or the CAM sections [32, 33, 35, 37, 38]. In terms of providing advice and/or tools to support implementation of the recommendations, 1 CPG did this for both the overall CPG and the CAM section [39], while the rest of the CPGs did not [32, 33, 35, 37, 38]. One CPG addressed the resource implications of implementing the recommendations for both the overall CPG and CAM sections [39], while the other 5 CPGs contained little to no information [32, 33, 35, 37, 38]. No CPGs provided monitoring and auditing criteria for the overall CPGs or the CAM sections.

Editorial Independence

The 6 CPGs differed in their reporting of the funding source or competing interests of the members of the guideline development panel. Four out of the 6 CPGs stated that the views of the funding body did not influence the content of the CPG [33, 35, 37, 39], while 2 of the 6 CPGs did not state this explicitly [32, 38] for either the overall CPGs or the CAM sections. For the last question in this AGREE II instrument domain, 4 CPGs recorded and addressed the competing interests of the guideline development group [32, 35, 37, 39], while 2 did not address competing interests [33, 38] for neither the overall CPGs nor the CAM sections.

Discussion

The purpose of this study was to identify the quantity and assess the quality of CAM recommendations in CPGs for the treatment and/or management of insomnia to identify credible, knowledge-based resources for both patients and healthcare professionals to base therapy decisions upon. This study identified 9 CPGs published between 2009 and 2020 that were relevant to the treatment and/or management of insomnia. Out of those 9 CPGs, 6 CPGs made CAM therapy recommendations, 1 CPG mentioned CAM but did not provide CAM therapy recommendations, and 2 CPGs contained neither CAM mentions nor CAM recommendations. The quality of each CPG when assessed by the 23-item AGREE II instrument differed greatly both between the 6 overall CPGs and within the 6 specific domains for each CPG. In assessing the overall CPGs, 2 CPGs scored 5.0 or higher in both average appraisal score and average overall assessment [37, 39], and 4 CPGs scored 4.0 or lower in both of these metrics [32, 33, 35, 38]. In assessing the CAM sections of each CPG, 3 CPGs scored 4.5 or higher in both average appraisal score and average overall assessment [35, 37, 39], and 3 CPGs scored 4.5 or lower in both of these metrics [32, 33, 38] (1 = strongly disagree; 7 = strongly agree that criteria are met).

Of the 6 CPGs making CAM recommendations, yoga was recommended for the treatment and/or management of insomnia in 2 CPGs [38, 39]. These 2 CPGs focused on co-morbid insomnia with cancer as a comorbidity. Herbal remedies were also recommended for the treatment of insomnia in 2 CPGs [32, 33]. One of the CPGs provided recommendations in the context of co-morbid insomnia [32], while the other focused on the treatment of primary insomnia [33]. One guideline recommended against the use of acupuncture, chiropractic techniques, herbal medicines, homeopathy, yoga, mindfulness-based stress relaxation, and foot reflexology [35]. This recommendation against the use of CAM for insomnia may be due to the fact that low-quality evidence was consulted. In future updates of such guidelines, more recent and high-quality evidence should be consulted for the recommendation of CAM for insomnia. The remaining recommendations in the other 6 guidelines were either conflicting or unclear regarding the use of CAM therapies, or no recommendations were provided for the use of CAM therapies for insomnia altogether [32, 33, 35, 37,38,39]. In order to propose clearer recommendations, the authors of these CPGs should consult more recent and high-quality sources of evidence to inform their recommendations in future updates.

To our knowledge, no previous studies have determined the quantity and quality of CAM therapy recommendations in insomnia CPGs; this is the first study to assess the credibility and nature of such recommendations. In this study, the scaled domain percentages for the overall CPGs from highest to lowest were: scope and purpose (93.5%), clarity of presentation (73.2%), editorial independence (70.14%), stakeholder involvement (64.4%), rigour of development (59.5%), and applicability (26.0%). The scaled domain percentages for the CAM sections of the CPGs from highest to lowest were: scope and purpose (89.4%), clarity of presentation (67.1%), editorial independence (52.8%), rigour of development (52.1%), stakeholder involvement (44.9%), and applicability (20.8%). These findings are similar to the results of guideline assessments for  other conditions/diseases in the context of CAM recommendations. Various other studies which used a similar format to this current study found similar results in terms of scaled domain percentages for overall CPGs and CAM sections. For example, 1 study reported that across 17 low back pain  CPGs, scaled domain percentages from highest to lowest were as follows: scope and purpose, clarity of presentation, stakeholder involvement, rigor of development, editorial independence, and applicability [41]. Other studies assessing the quality of CAM recommendations across arthritis, lung cancer, hypertension, depression, and cancer-related pain CPGs also found similar trends to exist whereby the order of domains were similar with the scope and purpose and clarity of presentation domains typically scoring higher, and the domain of applicability typically scoring lower [42,43,44,45,46]. These findings reflect the fact that the variable and sometimes sub-optimal quality of CPGs is not a unique phenomenon.

Overall, this study revealed that very few CPGs containing CAM recommendations exist to support evidence-informed decision making among patients and their healthcare providers for the treatment and/or management of insomnia. Apart from a lack of evidence surrounding many CAM therapies, this lack of CAM recommendations for insomnia found in CPGs could also be explained by other factors which can impact the availability of CAM research, including the following: negative attitudes about CAM therapies [47,48,49,50,51], and a lack of CAM funding [52,53,54,55]. Despite this, CAM is utilized by more than 40% of the population in some parts of the world [56, 57]; for example, 36% of adults use some form of CAM in the USA [58]. Thus, this should serve as a reminder to researchers and clinicians alike that the importance of CAM research and recommendation implementation in CPGs comprises a large component of patient choice. It is therefore important for future CPG developers to utilize the available resources to optimize CPGs. The AGREE II instrument, along with various other principles, frameworks, criteria and checklists are available to assist CPG developers, including CAM CPG developers, to generate the highest-quality CPGs [59,60,61,62,63,64].

Strengths and limitations

One notable strength of this study included a comprehensive systematic review methodology to identify eligible CPGs which focused on insomnia management and/or treatment. Another strength included the use of the AGREE II instrument to assess CPGs, as it has been found to be both reliable and valid, and is the internationally-accepted gold standard for appraising CPGs. A possible limitation to the interpretation of our findings includes the fact that the CPGs were independently assessed by 2 assessors, instead of 4 as recommended by the AGREE II instrument instruction manual. To reduce discrepancies and standardize scoring, a pilot-test was conducted by JYN, NDP, and an another research assistant where 3 CPGs were independently appraised, scores were discussed, and consensus was achieved on how to apply the AGREE II instrument. After the 6 eligible CPGs were assessed, JYN met with NDP and the another research assistant to discuss and resolve uncertainties. Lastly, it should be acknowledged that we excluded non-English language CPGs, however, many traditional systems of medicine originate from regions of the world where English is not commonly spoken (i.e. traditional Asian medicine in China, Japan and Korea, among others).

Conclusions

This study identified 6 eligible CPGs published between 2009 and 2020 which provided CAM recommendations for insomnia, inclusive of acupuncture, yoga, herbal medicines, and mind-body practices. Following the appraisal of these CPGs with the AGREE II instrument, it was found that the quality varied within and across these CPGs. The CPGs which received higher scores could be utilized to inform healthcare providers about specific CAM therapies and could serve as the foundations for discussions involving the use of evidence-based CAMs for treating/managing insomnia. In future updates of CPGs, those which achieved lower domain scores and overall recommendations could be improved according to specifics in the AGREE II instrument, among other high-quality guideline development and implementation resources. At present, the lack of high-quality CAM recommendations across this subset of CPGs may lead to the continued use of potentially harmful CAM therapies, or the underuse of beneficial CAM therapies. These findings are important in justifying the need to understand why more CAM therapies have yet to be incorporated in insomnia CPGs despite great patient preference, and to determine where gaps in current knowledge exist. Future research should also seek to identify and incorporate into CPGs, CAM therapies which were not necessarily reviewed here, but which are supported by a sufficient evidence-base. Ultimately, this study suggests that while many patients utilize CAM for the treatment/management of insomnia, evidence-based recommendations are limited across CPGs, and further research is warranted to assist healthcare providers in standardizing the use of CAM therapies as part of patient care. 

Availability of data and materials

All relevant data are included in this manuscript.

Abbreviations

AGREE II:

Appraisal of Guidelines for Research & Evaluation II

CAM:

Complementary and alternative medicine

CPG:

Clinical practice guideline

ECRI:

Emergency Care Research Institute

NCCIH:

National Center for Complementary and Integrative Health

NICE:

National Institute for Health and Care Excellence

PICO:

Population, Intervention, Comparison and Outcomes

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

References

  1. Centers for Disease Control and Prevention. Sleep and Sleep Disorders. (2014). https://www.cdc.gov/sleep/about_sleep/key_disorders.html. Accessed 20 July 2020.

    Google Scholar 

  2. National Heart, Lung, and Blood Institute. Insomnia. (2020). https://www.nhlbi.nih.gov/health-topics/insomnia. Accessed 20 July 2020.

    Google Scholar 

  3. Morin C, Benca R. Chronic insomnia. Lancet. 2012;379:1129–41. https://doi.org/10.1016/S0140-6736(11)60750-2.

    Article  PubMed  Google Scholar 

  4. Sateia MJ. International classification of sleep disorders-third edition. Chest. 2014;146:1387–94. https://doi.org/10.1378/chest.14-0970.

    Article  PubMed  Google Scholar 

  5. Uchiyama M, Inoue Y, Uchimura N, Kawamori R, Kurabayashi M, Kario K, et al. Clinical significance and management of insomnia: clinical significance of insomnia. Sleep Biol Rhythms. 2011;9:63–72. https://doi.org/10.1111/j.1479-8425.2011.00496.x.

    Article  Google Scholar 

  6. Fortier-Brochu É, Morin CM. Cognitive impairment in individuals with insomnia: clinical significance and correlates. Sleep. 2014;37:1787–98. https://doi.org/10.5665/sleep.4172.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Monti J. Primary and secondary insomnia: prevalence, causes, and current therapeutics. Curr Med Chem. 2004;4(2):119–37. https://doi.org/10.2174/1568015043357039.

    Article  CAS  Google Scholar 

  8. American Academy of Sleep Medicine. International Classification of Sleep Disorders. 3rd ed. Darien: American Academy of Sleep Medicine; 2014.

    Google Scholar 

  9. Cleveland Clinic. Common Sleep Disorders. (2013). https://my.clevelandclinic.org/health/articles/11429-common-sleep-disorders. Accessed 20 July 2020.

    Google Scholar 

  10. University of Pennsylvania School of Medicine. One in four Americans develop insomnia each year: 75 percent of those with insomnia recover. (2018). https://www.sciencedaily.com/releases/2018/06/180605154114.htm. Accessed 12 Aug 2020.

    Google Scholar 

  11. Pearson NJ, Johnson LL, Nahin RL. Insomnia, trouble sleeping, and complementary and alternative medicine: analysis of the 2002 National Health Interview Survey Data. Arch Intern Med. 2006;166(16):1775–82. https://doi.org/10.1001/archinte.166.16.1775.

    Article  PubMed  Google Scholar 

  12. National Institutes of Health, National Centre for Complementary and Integrative Health (NCCIH). Complementary, alternative, or integrative health: What’s in a name? (2016). https://www.nccih.nih.gov/health/complementary-alternative-or-integrative-health-whats-in-a-name. Accessed 20 July 2020.

    Google Scholar 

  13. Ng JY, Boon HS, Thompson AK, Whitehead CR. Making sense of “alternative”, “complementary”, “unconventional” and “integrative” medicine: exploring the terms and meanings through a textual analysis. BMC Complement Altern Med. 2016;16(134):1. https://doi.org/10.1186/s12906-016-1111-3.

    Article  Google Scholar 

  14. Centre for Addiction and Mental Health. Complementary and Alternative Therapies (CAM). (2020). https://www.camh.ca/en/health-info/mental-illness-and-addiction-index/complementary-and-alternative-therapies. Accessed 22 July 2020.

    Google Scholar 

  15. Ernst E. Prevalence of use of complementary/alternative medicine: a systematic review. Bull World Health Organ. 2000; https://www.scielosp.org/pdf/bwho/2000.v78n2/258-266/en. Accessed 21 Nov 2020.

  16. National Institutes of Health, National Centre for Complementary and Integrative Health (NCCIH). Sleep Disorders: In Depth. (2015). https://www.nccih.nih.gov/health/sleep-disorders-in-depth. Accessed 21 Nov 2020.

    Google Scholar 

  17. Wang X, Li P, Pan C, Dai L, Wu Y, Deng Y. The effect of mind-body therapies on insomnia: a systematic review and meta-analysis. Evid Based Complement Alternat Med. 2019;2019:1–17. https://doi.org/10.1155/2019/9359807.

    Article  CAS  Google Scholar 

  18. Xie Z, et al. A review of sleep disorders and melatonin. Neurol Res. 2017;39(6):559–65. https://doi.org/10.1080/0161412.2017.1315864.

    Article  CAS  PubMed  Google Scholar 

  19. Sarris J, Byrne GJ. A systematic review of insomnia and complementary medicine. Sleep Med Rev. 2011;15:99–106. https://doi.org/10.1016/j.smrv.2010.04.001.

    Article  PubMed  Google Scholar 

  20. Cao H, Pan X, Li H, Liu J. Acupuncture for treatment of insomnia: a systematic review of randomized controlled trials. J Altern Complement Med. 2009;15:1171–86. https://doi.org/10.1089/acm.2009.0041.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Guo J, Huang W, Tang C, Wang GL, Zhang F, Wang P. Effect of acupuncture on sleep quality and hyperarousal state in patients with primary insomnia: study protocol for a randomised controlled trial. BMJ Open. 2016;6:e009594. https://doi.org/10.1136/bmjopen-2015-009594.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Patel SJ, Kemper KJ, Kitzmiller JP. Physician perspectives on education, training, and implementation of complementary and alternative medicine. Adv Med Educ Pract. 2017;8:499–503. https://doi.org/10.2147/AMEP.S138572.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Berretta M, Rinaldi L, Taibi R, Tralongo P, Fulvi A, Montesarchio V, et al. Physician attitudes and perceptions of complementary and alternative medicine (CAM): a multicentre Italian study. Front Oncol. 2020;10. https://doi.org/10.3389/fonc.2020.00594.

  24. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318:527–30. https://doi.org/10.1136/bmj.318.7182.527.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Graham R, Steinberg EP. Clinical practice guidelines we can trust. Washington, DC: National Academies Press; 2011. https://doi.org/10.17226/13058.

    Book  Google Scholar 

  26. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–9. https://doi.org/10.17226/1305810.7326/0003-4819-151-4-200908180-00135.

    Article  PubMed  Google Scholar 

  27. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P. AGREE II: advancing guideline development, reporting and evaluation in health care. Can Med Assoc J. 2010;182(18):E839–42. https://doi.org/10.1016/j.jclinepi.2010.07.001.

    Article  Google Scholar 

  28. Guidelines International Network. Home (2020). https://g-i-n.net. Accessed 20 July 2020.

    Google Scholar 

  29. National Centre for Complementary and Integrative Health. Clinical Practice Guidelines (2020). https://nccih.nih.gov/health/providers/clinicalpractice.htm Accessed 20 July 2020.

    Google Scholar 

  30. National Institute for Health and Care Excellence. NICE Evidence Search (2020). https://www.nice.org.uk/guidance Accessed 19 Nov 2020.

    Google Scholar 

  31. Emergency Care Research Institute. ECRI Guidelines Trust (2020). https://www.ecri.org Accessed 19 Nov 2020.

    Google Scholar 

  32. Lam WC, Zhong L, Liu Y, Shi N, Ng B, Ziea E, et al. Hong Kong Chinese medicine clinical practice guideline for Cancer palliative care: pain, constipation, and insomnia. Evid Based Complement Alternat Med. 2019;2019. https://doi.org/10.1155/2019/1038206.

  33. Silvestri R, Aricò I, Bonanni E, Bonsignore M, Caretto M, Caruso D, et al. Italian Association of Sleep Medicine (AIMS) position statement and guideline on the treatment of menopausal sleep disorders. Maturitas. 2019;129:30–9. https://doi.org/10.1016/j.maturitas.2019.08.006.

    Article  CAS  PubMed  Google Scholar 

  34. Gupta R, Das S, Gujar K, Mishra KK, Gaur N, Majid A. Clinical practice guidelines for sleep disorders. Indian J Psychiatry. 2017;59:S116–38. https://doi.org/10.4103/0019-5545.196978.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Riemann D, Baglioni C, Bassetti C, et al. European guideline for the diagnosis and treatment of insomnia. J Sleep Res. 2017;26(6):675–700. https://doi.org/10.1111/jsr.12594.

    Article  PubMed  Google Scholar 

  36. Sateia MJ, Buysse DJ, Krystal AD, Neubauer DN, Heald JL. Clinical Practice Guideline for the Pharmacologic Treatment of Chronic Insomnia in Adults: An American Academy of Sleep Medicine Clinical Practice Guideline. J Clin Sleep Med. 2017;13(2). https://doi.org/10.5664/jcsm.6470.

  37. Qaseem A, Kansagara D, Forciea MA, Cooke M, Denberg TD. Management of Chronic Insomnia Disorder in adults: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2016;165:125–33. https://doi.org/10.7326/M15-2175.

    Article  PubMed  Google Scholar 

  38. Denlinger CS, Ligibel JA, Are M, Baker KS, Demark-Wahnefried W, Friedman DL, et al. Survivorship: Sleep Disorders, Version 1.2014. J Natl Compr Cancer Netw. 2014;12:630–42. https://doi.org/10.6004/jnccn.2014.0067.

    Article  Google Scholar 

  39. Howell D, Oliver TK, Keller-Olaman S, Davidson J, Garland S, Samuels C, et al. A Pan-Canadian practice guideline: prevention, screening, assessment, and treatment of sleep disturbances in adults with cancer. Support Care Cancer. 2013;21:2695–706. https://doi.org/10.1007/s00520-013-1823-6.

    Article  PubMed  Google Scholar 

  40. Pinto LR Jr, Alves RC, Caixeta E, et al. New guidelines for diagnosis and treatment of insomnia. Arq Neuropsiquiatr. 2010;68(4):666–75. https://doi.org/10.1590/s0004-282x2010000400038.

    Article  PubMed  Google Scholar 

  41. Ng JY, Mohiuddin U. Quality of complementary and alternative medicine recommendations in low back pain guidelines: a systematic review. Eur Spine J. 2020;29:1833–44. https://doi.org/10.1007/s00586-020-06393-9.

    Article  PubMed  Google Scholar 

  42. Ng JY, Azizudin AM. Rheumatoid arthritis and osteoarthritis clinical practice guidelines provide few complementary and alternative medicine therapy recommendations: a systematic review. Clin Rheumatol. 2020. https://doi.org/10.1007/s10067-020-05054-y.

  43. Ng JY, Nault H, Nazir Z. Complementary and integrative medicine mention and recommendations: a systematic review and quality assessment of lung cancer clinical practice guidelines. Integr Med Res. 2021;10:100452. https://doi.org/10.1016/j.imr.2020.100452.

    Article  PubMed  Google Scholar 

  44. Ng JY, Gilotra K. Complementary medicine mention and recommendations are limited across hypertension guidelines: a systematic review. Complement Ther Med. 2020;50:102374. https://doi.org/10.1016/j.ctim.2020.102374.

    Article  PubMed  Google Scholar 

  45. Ng JY, Nazir Z, Nault H. Complementary and alternative medicine recommendations for depression: a systematic review and assessment of clinical practice guidelines. BMC Complement Med Ther. 2020;20(1):1–5. https://doi.org/10.1186/s12906-020-03085-1.

    Article  Google Scholar 

  46. Ng JY, Sharma AE. Guidelines for cancer-related pain: a systematic review of complementary and alternative medicine recommendations. Pain Practice. 2020. https://doi.org/10.1111/papr.12964.

  47. Singh S, Ernst E. Trick or treatment: the undeniable facts about alternative medicine. New York: WW Norton & Company; 2008.

    Google Scholar 

  48. Chatfield K, Partington H, Duckworth J. The place of the university in the provision of CAM education. Aust J Homeopathic Med. 2012;24(1):16–20.

    Google Scholar 

  49. Liem A. A comparison of attitudes towards complementary and alternative medicine between psychologists in Australia and Indonesia: a short report. Integr Med Res. 2019;8:195–9. https://doi.org/10.1016/j.imr.2019.07.004.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Offit PA. Studying complementary and alternative therapies. JAMA. 2012;307(17):1803–4. https://doi.org/10.1001/jama.2012.518.

    Article  CAS  PubMed  Google Scholar 

  51. Colquhoun D. Should NICE evaluate complementary and alternative medicines. BMJ. 2007;334:506–7. https://doi.org/10.1136/bmj.39122.551250.BE.

    Article  Google Scholar 

  52. Fischer FH, Lewith G, Witt CM, Linde K, von Ammon K, Cardini F, Falkenberg T, Fønnebø V, Johannessen H, Reiter B, Uehleke B. High prevalence but limited evidence in complementary and alternative medicine: guidelines for future research. BMC Complement Altern Med. 2014;14(1):1. https://doi.org/10.1186/1472-6882-14-46.

    Article  Google Scholar 

  53. Ernst E, Cohen MH, Stone J. Ethical problems arising in evidence based complementary and alternative medicine. J Med Ethics. 2004;30(2):156–9. https://doi.org/10.1136/jme.2003.007021.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Tais S, Oberg E. The Economic Evaluation of Complementary and Alternative Medicine. Nat Med J. 2013; https://www.naturalmedicinejournal.com/journal/2013-02/economic-evaluation-complementary-and-alternative-medicine Accessed 24 July 2020.

  55. Nissen N, Manderson L. Researching alternative and complementary therapies: mapping the field. Med Anthropol. 2013;32(1):1–7. https://doi.org/10.1080/01459740.2012.718016.

    Article  PubMed  Google Scholar 

  56. Clarke TC, Black LI, Stussman BJ, Barnes PM, Nahin RL. Trends in the use of complementary health approaches among adults: United States, 2002–2012. Natl Health Stat Rep. 2015;10:79.

    Google Scholar 

  57. Harris PE, Cooper KL, Relton C, Thomas KJ. Prevalence of complementary and alternative medicine (CAM) use by the general population: a systematic review and update. Int J Clin Pract. 2012;66(10):924–39. https://doi.org/10.1111/j.1742-1241.2012.02945.x.

    Article  CAS  PubMed  Google Scholar 

  58. National Center for Complementary and Alternative Medicine. The Use of CAM in the United States (2007). https://www.westernsydney.edu.au/__data/assets/pdf_file/0010/537409/CAM_in_the_United_States.pdf. Accessed 30 July 2020.

    Google Scholar 

  59. Schünemann HJ, Mustafa RA, Brozek J, et al. GRADE guidelines: 22. The GRADE approach for tests and strategies-from test accuracy to patient-important outcomes and recommendations. J Clin Epidemiol. 2019;111:69–82. https://doi.org/10.1016/j.jclinepi.2019.02.003.

    Article  PubMed  Google Scholar 

  60. Rosenfeld RM, Shiffman RN. Clinical practice guideline development manual: a quality-driven approach for translating evidence into action. Otolaryngol Head Neck Surg. 2009;140:S1–43. https://doi.org/10.1016/j.otohns.2009.04.015.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, Ventresca M, Brignardello-Petersen R, Laisaar KT, Kowalski S, Baldeh T. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J. 2014;186(3):E123–42. https://doi.org/10.1503/cmaj.131237.

    Article  Google Scholar 

  62. Gagliardi AR, Armstrong MJ, Bernhardsson S, Fleuren M, Pardo-Hernandez H, Vernooij RWM, et al. The clinician guideline determinants questionnaire was developed and validated to support tailored implementation planning. J Clin Epidemiol. 2019;113:129–36. https://doi.org/10.1016/j.jclinepi.2019.05.024.

    Article  PubMed  Google Scholar 

  63. Scottish Intercollegiate Guidelines Network. SIGN 50: A Guideline Developer’s Handbook (2015). https://www.sign.ac.uk/assets/sign50_2015.pdf Accessed 1 Sept 2020.

    Google Scholar 

  64. NICE. Developing NICE Guidelines: The Manual (2020). https://www.nice.org.uk/process/pmg20/chapter/introduction Accessed 1 Sept 2020.

    Google Scholar 

Download references

Acknowledgements

JYN was funded by a Research Scholarship and an Entrance Scholarship from the Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences at McMaster University. We gratefully acknowledge Archit Jain for assisting with screening and data extraction.

Funding

This study was unfunded.

Author information

Authors and Affiliations

Authors

Contributions

JYN: designed and conceptualized the study, collected and analysed data, drafted the manuscript, and gave final approval of the version to be published. NDP: assisted with the collection and analysis of data, critically revised the manuscript, and gave final approval of the version to be published.

Corresponding author

Correspondence to Jeremy Y. Ng.

Ethics declarations

Ethics approval and consent to participate

This study involved a systematic review of peer-reviewed literature only; it did not require ethics approval or consent to participate.

Consent for publication

All authors consent to this manuscript’s publication.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary File 1.

MEDLINE Search Strategy for Insomnia Clinical Practice Guidelines Executed April 17, 2020.

Additional file 2: Supplementary File 2.

Modified AGREE II Questions Used to Guide Scoring of CAM Sections of Each Guideline.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ng, J.Y., Parakh, N.D. A systematic review and quality assessment of complementary and alternative medicine recommendations in insomnia clinical practice guidelines. BMC Complement Med Ther 21, 54 (2021). https://doi.org/10.1186/s12906-021-03223-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12906-021-03223-3

Keywords