2.1 Working group constitution and work distribution

The CPG leader designated the panel group and the external reviewers (Figure 1). This first phase was similar to other guidelines developed with different methodologies different to GRADE. Only 5 of the 18 components of the CPG panel group had previous experience with GRADE. It was therefore necessary to perform a specific training program for the rest of the group members. Including GRADE experts in the panel group was essential as they acted as consultants throughout the whole guideline development process. Another key element in the group constitution was the incorporation of document lists that performed all the bibliographic searches.

Figure 1: Working group constitution and work distribution.

All authors, collaborators and external reviewers of the CPG had to sign a conflict of interests´ form before participation in the CPG started. It is not clear how to manage potential conflicts of interest, common amongst groups of experts that frequently come together to develop CPG [9,10].

2.2 Defining scope and aim of the CPG

The first work of the CPG´s panel was to agree on the guideline´s aim and scope. For this, after the first face-to-face meeting of the whole group, a draft document was sent via email to each of the panel group members. The document was modified with the comments received.

2.3 Formulating clinical questions

This is one of the key processes for CPG elaboration. The GRADE system establishes that questions should be prioritized according to clinical relevance. However, it is unclear how this prioritization must be performed within a big panel group, nor how to resolve disagreements. To make this process our group followed the following steps: 1) Possible questions brainstorming in an open web environment 2) Pre-selection of questions and first classification, with elimination of redundant questions and reformulation in a PICO (population-intervention-comparison-outcome) format. 3) Bibliographic first search to assist in the screening of other relevant PICO questions not previously identified. 4) Questions and outcomes prioritization according to the GRADE methodology. For this last step, we decided to include a maximum of 3 primary and four secondary outcomes per question. To evaluate the clinical relevance of each question each member of the clinicians in the panel group rated from 1 to 9 each question or outcome in a blinded way. Following the GRADE methodology, questions or outcomes with mean scores of 1-3 were considered "not important," 4-6 "important" and 7-9 as questions or outcomes "key" in clinical decision making [11]. It was decided by consensus to select only those questions or outcomes with average scores above 6, 5. After this process questions were reduced to 14. One of the panel group members didn´t agree with the exclusion of a question with an average score of 6.42. Having considered all the arguments the panel group finally decided to include this questions and 15 questions were finally included in the guideline.

Question selection was slow and laborious. The anonymous prioritization system via email allowed clinically relevant prioritization. Patients and families´ views were unable to be incorporated at this stage as would be desirable since there was no published information available on this aspect when the CPG was being developed.

2.4 Bibliographic search

This was performed by expert documentalists. Their participation in the CPG elaboration allowed for a bigger accuracy in all the searches, allowing clinicians to focus on other tasks of the CPG´s development. Searches were conducted in three distinct stages (Table 1) and the publications found were included in folders on Refworks®. Searches didn´t take into account time period. Language was only an exclusive factor for recovering full text articles where Chinese, Japanese, Russian and Slavic languages were excluded. For the bibliographic search both MeSH (Medical Subject Headings in PubMed) and free-text terms were used.

Table 1: Databases and descriptors used for the bibliographic search.

The first stage of bibliographic search prioritized the identification of other GPC and systematic reviews on neonatal HIE. Weekly alerts were established in PubMed. In the second stage, meta-analysis (MA), systematic reviews (SR) and randomized clinical trials (RCT) were identified. Generic searches of economic evaluation studies were included. When additional evidence was required to answer a specific question the search was extended to observational studies. No particularities were observed in the literature search process for a GPC using GRADE compared to other methodologies.

Question selection was slow and laborious. The anonymous prioritization system via email allowed clinically relevant prioritization. Patients and families views were unable to be incorporated at this stage as would be desirable since there was no published information available on this aspect when the CPG was being developed.

2.5 Scientific evidence evaluation and synthesis

Collaborators performed critical appraisal of selected papers. Results were summarized on tables created specifically for this guide and based on those used on other guides CPG. AMSTAR [12] was used for SR´s quality assessment. For the other study types, criteria proposed by different groups were used.

Quality of the scientific evidence was assessed as recommended by the GRADE group [13]. Even if criteria for establishing evidence quality are clearly established, a degree of subjectivity exists in this phase. The GRADE advantage over other systems at this stage is however that reason for quality assessment must be clearly defined and is therefore easily to revise by the reader.

A challenge for the authors of this CPG was the inclusion of diagnostic tests in the clinical questions. There is still little experience on GRADE´s use for this. For these questions´ answers pre and post test probabilities were calculated and rates of false negatives / positives and true positives / negatives were presented [14-16]. An example of the final result for one of the questions is shown in Table 2.

Table 2: GRADE table example in diagnostic tests (prognostic value of amplitude integrated electroencephalogram at 24 hours of life, newborns treated with hypothermia).

Finally when the available evidence to answer a question was from outdated reviews or individual studies, the panel group performed new meta-analysis (final number of three).

2.6 Economic assessments

The GRADE system requires an economic assessment for each question including cost-effectiveness, cost-utility, cost-benefit or cost minimization aspects. For critical appraisal of these studies proposal by López Bastida and cols [17] was used. Literature search found very few economic studies for the clinical questions. Only economic evaluation studies on treatment with therapeutic hypothermia were identified.

2.7 Involvement of family members and caregivers in CPG development

GRADE includes in its final evaluation patient´s views. In the absence of publications that specifically addressed this aspect, our CPG group decided to conduct a qualitative study during the CPG development. This was developed during a year and was conducted with parents of newborns with HIE to investigate their feelings and needs. The qualitative study was led and coordinated by three members of the CPG panel group. The final aim was the development of a parents' guide and the incorporation of parent´s views in the answer of the clinical guideline's questions. Despite the importance of incorporating patient's and caregivers´ views, it is not clear in GRADE how this should be done and balanced in the final recommendations or how to proceed when, as in our case, bibliography of this aspect is absent.

2.8 Final recommendations

As suggested by GRADE [18] final recommendations were formulated taking into account scientific evidence quality, benefitrisk ratio, parents´ views and economic aspects. All these were written explicitly for each question, and thus easily identified by the reader (Table 3).

Table 3: Extracted from the CPG on management of perinatal HIE. Fragments of question 4 on therapeutic hypothermia.

Approval of all recommendations was performed in a final faceto- face meeting of the entire CPG panel group. Here all the questions recommendations were presented in a numerical order. A maximum of 30 minutes was assigned for each question's discussion.

This was another of the main phases for the CPG elaboration. The GRADE methodology doesn’t specify if a minimum consensus is necessary within the panel group to establish a recommendation. An alternative could be to include in the final recommendation a percentage of agreement within the group. How to present final recommendations continues to be subject of debate [19]. There is currently a European project (DECIDE) on how to best present these [20]. Our panel group decided to change the “good clinical practice” usual symbol (√) with an asterisk (*) as we thought that untrained readers in GRADE could mistake the first symbol with “good scientific evidence”.

2.9 External review

Final CPG draft agreed by the whole CPG panel was the submitted for external review. Scientific experts and all the scientific societies who are implied in these patient´s care were involved, including pediatricians, nurses, midwives, obstetricians, anesthesiologists, neurologists, methodologists and bioethicists. Patients´ associations could not be included as no such societies for infants with HIE exist in Spain. External review was key for the CPG accuracy and enrichment. Considering reviewers points and modifying previous errors was, however, a laborious process.

2.10 CPG implementation and update

An important aspect is guideline implementation once this has been developed. Tablet and smartphone use could provide an excellent tool for this yet unresolved aspect [21]. CPG´s update is programmed for 3 to 5 years after closure.