Reliability and Validity of an Assessment Algorithm to Predict the Coping Skills of 3-to 6-Year-old Children Undergoing Blood Sampling

Background: We developed an assessment algorithm to predict the coping skills of children aged 3 to 6 years who undergo blood sampling. The objective of this study was to determine the reliability and validity of this algorithm. Methods: Subjects were 41 children aged 3 to 6 years who were to undergo blood sampling in two hospitals, and their parents. To use the assessment algorithm, we used a questionnaire to obtain the following information from the parent before the child’s blood sampling: the child’s age, whether they had previously undergone blood sampling, and the parent’s prediction regarding the child’s coping behavior. The coping behavior of the children undergoing blood sampling was directly observed and scored by one researcher. In the examination of reliability, two different researchers independently estimated the coping skills of the child using the assessment algorithm based on the parent’s questionnaire, and the estimates were compared. We analyzed the predictive validity of the coping skills estimated by the assessment algorithm and the child’s actual coping behavior during blood sampling. We evaluated the sensitivity according to the number of children for whom the coping skills estimated by the assessment algorithm exceeded those exhibited in actual coping behavior. Results: The coping skills of the children estimated by the two researchers using the assessment algorithm were concordant (kappa = 1.000, p = 0.000). Spearman’s correlation coefficient between the coping skills estimated for the children using the assessment algorithm and their actual coping behavior was very high (r = -0.74, p = 0.000). Because the assessment algorithm we developed estimated a higher level of coping skill than was exhibited in actual coping behavior for five children, its sensitivity was higher than that of any other classification algorithm. Conclusion: Our analysis showed high reliability and validity of the assessment algorithm. Reliability and Validity of an Assessment Algorithm to Predict the Coping Skills of 3to 6-Year-old Children Undergoing Blood Sampling Publication History: Received: September 15, 2017 Accepted: October 30, 2017 Published: November 01, 2017


Introduction
Blood sampling is the most frequently performed test or procedure that causes pain, and it is performed routinely.It has been reported that a child's resistance to and refusal of blood sampling and the pain they feel are proportional to their level of anxiety and fear of the procedure [1,2].The coping behaviors of children who undergo blood sampling include effective and ineffective behaviors [3][4][5], and psychological preparation of the child is necessary for them to adopt effective behaviors [6][7][8].When a child undergoes blood sampling, it is important for medical staff to predict the child's coping skills because their coping skills determine their coping behavior.Psychological preparation suitable for the coping skills of the individual child is necessary [9,10].However, no specific method of assessing the coping skills of children has been established.An algorithm for a specific method of predicting the coping skills of children is needed.Development of such an assessment algorithm would likely enable us to more easily provide psychological preparation that suits the coping skills of the child.
We created an assessment algorithm (Figure 1) to predict the coping skills of children aged 3 to 6 years who undergo blood sampling [11].Children of that age are at an immature stage of their cognitive development, which makes psychological preparation before blood sampling even more important.The assessment algorithm was created in accordance with the Medical Information Network Distribution Service Hand book for Clinical Practice Guideline Development [12].The Medical Information Network Distribution Service handbook [12] describes the following procedure for guideline development: first, review the literature and extract the variables required for the algorithm; and second, examine methods of assessing the extracted variables.In accordance with this procedure, the literature on the factors related to coping behavior in children undergoing blood sampling was reviewed and assessment variables were extracted.The variables extracted were the age of the child, coping behavior of the child predicted by the parent before blood sampling ("parent's prediction" below), and past blood sampling experience.The most commonly reported factor related to the coping behavior of children undergoing blood sampling was the age of the child.A consistently obtained result was that the younger the child, the less adept they were at adopting coping behavior.Another consistent result concerned the parent's prediction.The blood sampling coping behavior of the child that was predicted by the parent was related to the child's actual coping behavior.One purpose of psychological preparation is to provide accurate information [13].Because such preparation is particularly important for children experiencing blood sampling for the first time, past blood sampling experience was also extracted as a variable.The child's temperament and illness and the anxiety of the parent were excluded as variables for the following reasons: the small number of related reports, lack of consistency in the results, and absence of an investigation in Japan.
To increase the content validity of the assessment algorithm, a second examination of the method used to assess the extracted variables was conducted by three pediatric nursing researchers,seven pediatric nurses, and two pediatricians at two hospitals.Past blood sampling experience was assessed as "yes" or "no".The parent's predictions were assessed by dividing them into the following three groups according to the total score for the Manifest Upset and Cooperation Scales developed by Wolfer and Visintainer [14] and partially revised by Koseki [15]: 2 points, effective; 4 to 6 points, moderately effective; and 8 to 10 points, ineffective.On reexamining the scales, we included a score of 4 points for the parent's prediction in the moderately effective group rather than the effective group to compensate for the gap between the actual and predicted coping behavior.The method used to assess the ages of the children was to classify the ages into two groups, a 3-year-old group and a 4-to 6-year-old group, based on the cognitive development of children according to Piaget [16].The three factors extracted and the method of assessing each factor were combined to create the assessment algorithm (Figure 1).Using the assessment algorithm, children with past blood sampling experience were divided into four categories based on the parent's prediction and the age of the child.Because this would be the first blood sampling for the children with no past blood sampling experience, they were classified into the category with the lowest coping skills.To assess the variables used in the assessment algorithm, an assessment sheet (Figure 2) was created.The children's coping skills were predicted with the assessment algorithm using the assessment sheet.
The objective of this study was to verify the reliability and validity of the assessment algorithm.Utilization of this algorithm would enable the coping skills of children undergoing blood sampling to be predicted specifically.Moreover, by defining the level of need for psychological preparation, it would enable psychological preparation suitable for the coping skills of the individual child to be provided.

Subjects
The study subjects were children aged 3 to 6 years who were to undergo blood sampling as pediatric outpatients at two hospitals (an advanced treatment hospital and a community hospital) and their parents.Children with developmental disabilities were excluded as subjects because the developmental stage could affect coping behavior.It was necessary for us to analyze predictive validity using a correlation coefficient in order to determine whether the coping skills of each child were predicted by the assessment algorithm.The sample size was calculated as 36 child-parent sets based on an estimated effect size (r) of 0.45, which is the lower limit for a moderately strong relationship, a two-tailed significance level of 7%, and statistical power of 80% [17].The dropout rate was estimated at about 10%.A sample size of 40 child-parent sets was therefore considered necessary for the study.

Procedure
To use the assessment algorithm, we used a questionnaire to obtain information from the parent about the assessment sheet variables.The parent's consent was obtained before blood sampling, and we then asked them to complete the questionnaire.The completed questionnaire was collected before the child entered the blood sampling room.Without having seen the parent's questionnaire, oneresearcher observed the coping behavior of the child during blood sampling.The coping behavior of all of the children during blood sampling was directly observed and scored by the researcher.After leaving the blood sampling room, the children were given several minutes to relax and then asked by the researcher to describe the level of pain they experienced during the procedure.After collecting these data, another researcher and a research assistant independently estimated the coping skills of the child using the assessment algorithm.The research assistant was a graduate student studying pediatric nursing who had agreed to collaborate in the study.To avoid researcher bias resulting from the observations performed during blood sampling, the coping skills of the child were estimated after blood sampling using the assessment algorithm based on the parent's questionnaire collected before blood sampling.We verified reliability and validity based on the relationship between the coping skills of the child estimated using the assessment algorithm and the coping

Coping skills of children estimated using the assessment algorithm (parent's questionnaire)
To enable the assessment algorithm to be used, the questionnaire asked the parent the child's age, whether they had previously undergone blood sampling, and the parent's prediction regarding the child's coping behavior.The scores on the Manifest Upset and Cooperation Scales developed by Wolfer and Visintainer [14] and partially revised by Koseki [15] are used to assess the coping behavior of children during blood sampling.We used these two scales for the parent's prediction.The parent was asked to use the two scales to predict the coping behavior of the child before the child's blood sampling.The scores for the two scales were determined and totaled.
The score for the Manifest Upset Scale is used to assess the child's level of anxiety and fear with respect to blood sampling.This scale consists of a single item scored as 1, 3, or 5 points.A high score indicates a high level of anxiety and fear in the child.
The Cooperation Scale is used to assess the child's level of cooperation with blood sampling during the procedure.This scale consists of a single item scored as 1, 3, or 5 points.A high score indicates highly uncooperative behavior by the child.

Children's coping behaviors observed during blood sampling
The coping behavior of the child during blood sampling was observed directly by oneresearcher using the scores for the Manifest Upset and Cooperation Scales developed by Koseki [15], which were also used for the parent's prediction.These two scales are easy to use and have been used in observations to assess the coping behavior of children in previous studies [5,10].The researcher observed the coping behavior of the children from the time they entered the blood sampling room until they exited the room.The observed behavior was scored using the Manifest Upset and Cooperation Scales, and the total score was determined.Before the start of the study, to enhance inter-rater reliability, the researcher observed the child's coping behavior during blood sampling together with the other researcher who had already received training.The concordance between the tworesearchers' scores for the two scales was then examined.
In addition, we observed the duration of blood sampling, the phlebotomist, the number of punctures, the posture of the child during blood sampling, and the involvement of the parent.

Level of pain reported by the children
To determine the criterion-related validity of the child's coping behavior observed during blood sampling, we examined the child's subjective level of pain using two scales.We first used the Faces Rating Scale developed by Wong and Baker [18], the reliability and validity of which have been established for children 3 years of age and older.In this scale, six round faces are shown, ranging from a happy, smiling expression to a sad, crying face.The children were asked to select the face that indicated how much pain they felt during blood sampling.The face was then scored from 0 points (happy) to 5 points (sad).
The second scale used was a 100-mm Vertical Visual Analogue Scale.The reliability and validity of this scale in measuring subjective pain have been previously described [19].In this study, the bottom end of the scale (0) was defined as representing no pain and the top end (100) as representing the most pain.The child marked a line at the point corresponding to the level of pain they felt during blood sampling.

Analysis
The statistical analysis was performed using the IBM SPSS Statistics 21 software.A two-tailed significance level of <5% was used.

Reliability
The kappa coefficient was used to assess reliability.We compared the estimates of the coping skills of the child that were independently made based on the parent's questionnaire by tworesearchers using the assessment algorithm with the assessment sheet.

Validity
Predictive validity and sensitivity were analyzed in the examination of validity.Predictive validity was analyzed by determining Spearman's rank-order correlation coefficient (exact test) for the coping skills estimated by the assessment algorithm and the child's actual coping behavior during blood sampling.The coping skills of the child estimated using the assessment algorithm and the pain level reported by the child were analyzed using Spearman's rank-order correlation coefficient.We evaluated the sensitivity of the assessment algorithm according to the number of children whose estimated coping skills were greater than those exhibited in actual coping behavior.The smaller this number, the greater the sensitivity of the assessment algorithm.

Ethical considerations
This study was approved by the ethics committee of Yamagata University Faculty of Medicine.All participants were informed of the purpose of the study.In addition, the following information was given to the participants both verbally and in writing: participation in the study was voluntary; a participant could withdraw from the study at any time, withdrawal from or refusal to take part in the study would result in no medical disadvantage; the obtained information would never be used for any purpose other than the study; personal information would remain anonymous and not be able to be connected to the source code; the results of the study would be presented in papers and at conferences after being processed so that the participants could not be identified individually; and that information, such as diagnoses, would be obtained from medical records.Only those who consented were included as participants in the study.Written consent was obtained from the children's parents after verbal consent to participate in the study was obtained from both the children and their parents.

Demographic data
Consent was obtained from 44 of the 46 child-parent sets asked to participate in this study.Three sets were excluded because the blood sampling order was canceled or an improper answer was given on the parent's questionnaire.Thus, 41 sets were included in the analysis (21 sets from one hospital, 20 sets from the other).Thirty-six of the parents were mothers, three were fathers, and two were grandmothers.The mean age ±standard deviation of the children (22 boys, 19 girls) was 5.1 ± 1.1 years (Table 1).Thirty-eight of the children had previously experienced blood sampling and three had not.

Coping skills of the children estimated using the parent's prediction and assessment algorithm
The parent's prediction before blood sampling and the actual coping behavior of the child during blood sampling are shown in Table 2.The median score (and range) for the parent's prediction was 3 points (1 to 5 points) for both the Manifest Upset Scale and Cooperation Scale.The assessment algorithm was used to classify the children according to four categories of coping skills.Thirteen children were classified as falling into each coping skill category except for Category 3, into which two children were classified (Table 3).

Conditions during blood sampling and actual coping behavior of the children
The mean ±standard deviation (range) for the blood sampling time was 3.8 ± 2.0 (1.8 to 13.1) minutes.Blood sampling at the two hospitals was performed by an attending physician or nurse in an outpatient treatment room.Blood sampling was performed by a pediatrician for 17 children and by a nurse for 24 children.For nearly all of the children, blood sampling was completed with a single puncture.Ten children were recumbent and 31 were sitting during blood sampling.The parent was allowed to be next to the child during blood sampling and, if necessary, hold their hand and encourage them.Twenty-four parents were in the blood sampling room with the child and provided support by hugging them and encouraging them verbally.Five parents were beside the child in the blood sampling room during blood sampling.Twelve parents did not enter the blood sampling room.The actual coping behavior of the child during blood sampling was scored using the scores for the Manifest Upset Scale and Cooperation Scale.The median score (range) for both scales was 3 points (1 to 5 points) (Table 2).The anxiety, fear, and cooperative behavior shown by the child during blood sampling were roughly concordant with the parent's prediction made before blood sampling.The mean level of pain ± standard deviation reported by the children after blood sampling was 2.2 ± 2.0 on the Faces Rating Scale and 3.6 ± 3.4 on the Visual Analogue Scale.

Reliability
We compared the coping skills of the children independently estimated by two researchers using the assessment algorithm.The assessments of the tworesearchers were concordant (kappa = 1.000, p = 0.000).Table 3: Distribution of the coping skills of children estimated based on the assessment algorithm (n = 41).

Validity Predictive validity
Using the assessment algorithm, the children with no past blood sampling experience were classified as having the lowest coping skills (Category 1).Consequently, those included in the analysis of predictive validity were the 38 children with past blood sampling experience.The coefficient of the correlation between the coping skills estimated for the children using the assessment algorithm and their actual coping behavior was very high (r = -0.74,p = 0.000) (Table 4).Children for whom poor coping skills were estimated using the assessment algorithm showed high levels of anxiety and fear and uncooperative behavior during blood sampling.A good correlation also was seen between the coping skills estimated using the assessment algorithm and the children's pain level reported using the Faces Rating Scale (r = -0.57,p = 0.000) and the Visual Analogue Scale (r = -0.62,p = 0.000).The lower the coping skills of the child estimated using the assessment algorithm, the more severe the pain reported.

Sensitivity
We could not calculate the sensitivity of the assessment algorithm because it is not a two-category variable.We examined the number of children with estimated coping skills that exceeded their actual coping behavior.With high sensitivity, the number of children with estimated coping skills that exceed their actual coping behavior is likely to be lower.The distribution of children based on the assessment algorithm and on actual coping behavior during blood sampling is shown in Table 4. Categories 3 and 4 indicated effective coping skills, Category 2 indicated moderately effective coping skills, and Category 1 indicated ineffective coping skills.The actual coping behavior of the child during blood sampling was classified according to the following three groups based on the total score on the Manifest Upset and Cooperation Scales: 2 to 4 points, effective coping behavior; 6 points, moderately effective coping behavior; and 8 to 10 points, ineffective coping behavior.Using these criteria, five children (15.2%) had coping skills estimated using the assessment algorithm that exceeded their actual coping behavior.Applying these categories (i.e., 2 to 4 points, 6 points, and 8 to 10 points) to the total score for the Manifest Upset and Cooperation Scales based on the parent's prediction, six children (18.2%) had estimated coping skills that exceeded their actual coping behavior when the coping skills were estimated.Another algorithm that categorized the score for the parent's prediction as 2 to 4 points, 6 to 8 points, and 10 points showed that seven children (21.2%) had estimated coping skills that exceeded their actual coping behavior.Because the assessment algorithm we developed estimated a higher level of coping skill than was exhibited in actual coping behavior for five children, the lowest number obtained, its sensitivity was higher than that of any other classification algorithm.

Coping behavior of children without past blood sampling
Three children had no past blood sampling experience.The blood sampling time was 6 minutes or longer for 1 child (case A) and 3 minutes or less for 2 children (cases B and C).All three children were in the blood sampling room with their parent, who encouraged the child verbally, hugged them, and held their hand.Case A asked his parent and the nurse several questions during blood sampling.He resisted during blood sampling and tried to run away before the needle was inserted.It took around 5 minutes to calm him, but he cooperated in the blood sampling once he became calmer.Case B grimaced the entire time they were in the blood sampling room.Case C cooperated calmly during blood sampling but cried after it was finished and the needle was removed.

Reliability and validity
The coping skills of the children estimated independently by two researchers using the assessment algorithm were concordant, verifying inter-rater reliability.The predictive validity of the algorithm was verified by the very good correlation of 0.7 or greater that was seen between the children's estimated coping skills and their actual coping behavior.When estimating the coping skills of a child, an estimate of high coping skills carries greater risk than an estimate of low skills because psychological preparation may be inadequate.Because the coping skills estimated using the assessment algorithm exceeded actual coping behavior for five children, the lowest number obtained, this algorithm was concluded to have a higher sensitivity than any other score-classification algorithms.
Using the assessment algorithm, the children with no past blood sampling experience were classified as having the lowest coping skills (Category 1).One child who experienced blood sampling for the first time in this study required time for his emotions to calm, and he asked several questions about the procedure.We think he required more psychological preparation.The other children who experienced blood sampling for the first time were one child who grimaced throughout the procedure and one child who cried after it was finished.If children experience pain or fear during the first blood sampling, they may experience greater anxiety, fear, and pain during the next blood sampling experience [20].It is therefore reasonable to classify children with no blood sampling experience in the lowest category of coping skills so that they are provided with adequate psychological preparation for blood sampling.
The findings described above verified the reliability and validity of this assessment algorithm.

Usability
This assessment algorithm can be used to predict the coping skills of children who are to undergo blood sampling.Psychological preparation that suits the coping skills of each child can be provided based on the results obtained with the algorithm.We think we can improve the quality of pediatric nursing care by using this algorithm.For example, for children predicted to have low coping skills, we can  4. Distribution of children based on the assessment algorithm and actual coping behavior during blood sampling (n = 38).Spearman's rank-correlation coefficient (exact test), r = -0.74,p = 0.000 Grey shading indicates the number of children whose coping skills were estimated higher than their actual coping behavior.allow time before blood sampling begins to provide adequate psychological preparation, such as by providing information, performing a simulation, and providing cognitive behavior therapy.On the other hand, for children predicted to have high coping skills, use of the assessment algorithm provides an opportunity to confirm that they are truly psychologically prepared.A lack of psychological preparation based on assessment of the background of children has been reported in Japan [21,22].Moreover, psychological preparation is often provided to children based on the skill of nurses [23].We think these problems can be resolved by using our assessment algorithm.The algorithm is not complicated and is easy to use in a short time.Consequently, it allows even nurses with little pediatric nursing experience to easily predict children's coping skills.In addition, it allows nurses to provide children with psychological preparation based on an assessment of their coping skills.

Limitations
The subjects of this study were pediatric outpatients and their parents.The reason for this was to ensure that their conditions were equivalent.Moreover, it is more difficult to predict coping skills for pediatric outpatients than for pediatric inpatients.This is because nurses have little contact with pediatric outpatients and their parents, and we must therefore assess children's coping skills in a short time.The use of an assessment algorithm for hospitalized children should be examined in the future.This study assessed the reliability and validity of an assessment algorithm as the first step of psychological preparation.It will be necessary in the future to develop psychological preparation appropriate for the coping skills of the individual and to verify the effectiveness of the assessment algorithm in randomized intervention studies.

Conclusion
Our analysis showed the assessment algorithm to be highly reliable and valid.The algorithm was useful for estimating the coping skills of children aged 3 to 6 years who are to undergo blood sampling.
Int J Nurs Clin Pract IJNCP, an open access journal ISSN: 2394-4978 Volume 4. 2017.257 Citation: Konta S, Sato Y (2017) Reliability and Validity of an Assessment Algorithm to Predict the Coping Skills of 3-to 6-Year-old Children Undergoing Blood Sampling.Int J Nurs Clin Pract 4: 259.doi: https://doi.org/10.15344/2394-4978/2017/259Page 3 of 7behavior of the child observed during blood sampling, and on the relationship between the estimated coping skills and the pain level reported by the child (Figure3).The child's sex and illness were transcribed from their medical records and included among their basic attributes.The illnesses were classified into several groups based on the 10th revision of the International Statistical Classification of Diseases and Related Health Problems.The data were collected from November 2012 to May 2013.

Figure 3 :
Figure 3: Procedures employed in the present study.
a Based on the 10th revision of the International Statistical Classification of Diseases and Related Health Problems.