It has been a year and a half since the Institute of Medicine (IOM) published its standards for development of systematic reviews (SRs)¹ and clinical practice guidelines (CPGs).² In that time, the American Academy of Neurology (AAN) has updated its process for developing SRs and CPGs in accordance with the IOM standards. To test this new process, we chose two guideline projects as pilot efforts: immunizations in multiple sclerosis (MS) and disorders of consciousness.

To enable public comment, we worked with our marketing team to put together a strategy to reach AAN members and the public. Here are some ideas we implemented:

·         Email to all AAN members

·         Email to the Neurology Now® group for patients with MS

·         Pitch to public relations teams of major patient advocacy organizations

·         Posting of information on the Facebook page of Neurology Now, the AAN’s publication for patients

·         Posting of information on the largest Facebook groups for patients with MS

·         Posting of links on the AAN Twitter channel (@AANPublic)

·         Posting of links on the Neurology Now Twitter channel (@NeurologyNow)

·         Posting on AAN Google+ pages

The AAN’s first public comment was for the immunization in MS protocol, the project plan draft for the guideline development effort. We posted the protocol on the AAN website in August for 30 days. We had 63 individuals review the protocol and provide comment. Of those reviewers, 10 were AAN members, and 53 were nonmembers. A total of 79 comments were made.

The disorders of consciousness protocol underwent public comment from September to October. We had 18 individuals review the protocol and comment. Of those reviewers, 8 were AAN members, and 10 were nonmembers. A total of 41 comments were made.

After completing two public comment periods, we have found that this process is more complicated than we originally envisioned. It is difficult to coordinate communications with all of the parties we would like to invite for document review. In spite of this challenge, we have found that most of the reviewers have provided valuable comments.

We are excited to be able to incorporate feedback from these stakeholders in our development process. We still have some kinks to work out, but we believe this process will make for a much better product.

¹Institute of Medicine of the National Academies. Finding What Works in Health Care: Standards for Systematic Reviews. www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx. Released March 23, 2011. Accessed March 23, 2011.

²Institute of Medicine of the National Academies. Clinical Practice Guidelines We Can Trust: Standards for Developing Trustworthy Clinical Practice Guidelines (CPGs). www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx. Released March 23, 2011. Accessed March 23, 2011.

As recommended by the Institute of Medicine (IOM),^1,2 systematic review and clinical practice guideline (CPG) documents in draft form should be made available for a public comment period. However, the IOM standards did not specify what constitutes a public posting and how long the comment period should be:

Systematic Review¹

2.7.1 Provide a public comment period for the protocol and publicly report on disposition of comments.

5.2.2 Provide a public comment period for the report and publicly report on disposition of comments.

                Clinical Practice Guidelines²

7.4 A draft of the CPG at the external review stage or immediately following it (i.e., prior to the final draft) should be made available to the general public for comment. Reasonable notice of impending publication should be provided to interested public stakeholders.

Prior to bringing the issues to the physicians on our Guideline Development Subcommittee (GDS), I informed internal staff of the standards and my intention for the GDS to incorporate both sets of standards fully into our process. A concern was raised that pertained to the public comment period:

Physicians and patients may inappropriately follow draft recommendations and attempt to hold a medical society accountable for those recommendations even if the recommendations undergo substantive changes by the time they are published in final form.

When the AAN methodologist and I presented the standards to the GDS, they shared this same concern. We received varied opinions from the GDS on whether the AAN should adopt all the standards and include them in our development process or whether to leave out the incorporation of the public comment period. In the end, the GDS unanimously voted to approve incorporation of all the standards for development of systematic reviews and guidelines into our process. The AAN Practice Committee and Board of Directors also unanimously approved, and, in November 2011, we published a major revision to our process manual.³ The approval was subject to staff’s and physician leadership’s appropriately defining “public comment” and developing a process document on how to facilitate (and regulate) the public comment period.

Has your organization decided to incorporate into its development process the public comment recommendation from the IOM? If not, why? If so, how are you defining “public comment”? Have you developed a process for facilitating (and/or regulating) the public comment period? Has your organization addressed the concern noted above?

¹Institute of Medicine of the National Academies. Finding What Works in Health Care: Standards for Systematic Reviews. www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx. Released March 23, 2011. Accessed March 23, 2011.

²Institute of Medicine of the National Academies. Clinical Practice Guidelines We Can Trust: Standards for Developing Trustworthy Clinical Practice Guidelines (CPGs). www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx. Released March 23, 2011. Accessed March 23, 2011.

³American Academy of Neurology. 2011. Clinical Practice Guideline Process Manual, 2011 Ed. St. Paul, MN: The American Academy of Neurology.

The American Academy of Neurology (AAN) needs your expert opinion on eleven performance measures developed for Amyotrophic Lateral Sclerosis (ALS). The 30-day open Public Comment period on the ALS Measures is from 12:00 p.m. ET on Tuesday February 14, 2012, to 6:00 pm ET on Thursday March 15, 2012. It will be available only between these two dates. This is a 30-day period in which the draft measures will be made available for comment on the AAN website.  During the Public Comment period anyone may comment on each individual measure and the measurement set overall. *Note that all posted comments will become part of public record.

You can view and comment on the full measurement set at www.aan.com/view/comments.

The AAN is committed to enhancing quality of care and patient safety by taking the lead in the development, testing, and maintenance of evidence-based clinical performance measures for neurologic conditions. These measures may be used by neurologists and other practitioners for accountability or quality improvement initiatives unless otherwise noted.

If you have questions please contact Rebecca Swain-Eng, MS, Senior Manager, Measurement and Implementation at (651) 695-2808, quality@aan.com or rswaineng@aan.com.