1. Introduction

Success on the NCLEX-RN is of concern to nursing students and faculty as for both groups failure has serious consequences. Nursing school graduates must pass the NCLEX-RN before they become a practicing nurse and NCLEX-RN pass rates have emerged as one indicator of program quality for state boards of nursing and the nursing schools’ community of interest [1]. If students are not successful on the NCLEX-RN they are unable to pursue their chosen career resulting in loss of income and potential impact on self-esteem [2].

The NCLEX-RN test plan summarizes the scope and content that will comprise the NCLEX-RN, and it serves to guide both development of the exam as well as preparation of the candidates [3]. The NCLEXRN test plan and passing standard are updated periodically. In 2009 the NCLEX-RN passing standard was changed from -0.21 to -0.16 logits; then, in April 2013 was raised again to 0.00 logits. An increase in the passing standard means a student must be at a higher ability level to reach the standard and pass the exam. Following the 2013 passing standard change, the pass rate for first-time U.S. educated testtakers was 83%, down from ~90% in 2012. This drop is expected as, according to the NCSBN, pass rates are historically lower immediately following a passing standard increase. Typically, however, the rates rebound within 3 years [3].

The question of which student factors best correlate with NCLEXRN success has yielded research with interesting, although not completely surprising, findings. Comprehensive measures such as SAT scores, entrance examinations, overall nursing program GPA, measures of academic aptitude, science GPA, scores in advanced medical-surgical courses, and biology course grades have all been shown to be positively associated with NCLEX success [4,5,6]. In addition, research evidence on best practices in testing and assessment directs nurse educators to systematically evaluate student ability with more than one indicator especially when making important decisions, which affect students’ futures [7-10]. As Spurlock and Hunt (2008) noted: “Looking at a single, clinical-only indicator to represent students’ readiness for graduation devalues the rest of their education, whether it occurred in a community college, diploma school, or university setting [11]”. This means not making an important decision, like whether a student will graduate or not, on the basis of a single test score.

Non-academic factors also impact success rates for nursing students. For example, Eddy and Epeneter (2002) interviewed students who had not succeeded on the NCLEX and identified test anxiety along with lack of preparation to answer higher-level, critical thinking questions to be common factors [12]. Other factors such as self-efficacy, self-esteem, age, timing of test-taking, critical thinking skills and ethnicity have also all been found to impact NCLEX-success for some students [13].

Strategies for increasing NCLEX-RN pass rates range from modifying admission criteria, altering the number of times students can retake courses once in the program, and implementing remediation and progression policies. There does not appear, however, to be one single strategy which, when employed, can assure NCLEXRN success. Nor should we expect there to be. Just as we cannot make important decisions based on one single student indicator, nor should we expect there to be one “magic bullet” which will unequivocally determine NCLEX-RN success.

Nursing institutions sometimes use standardized testing products (e.g., ATI and HESI exams) to help try and predict student success in courses and on the NCLEX-RN. Indeed, in many cases nursing schools use a minimum score on a standardized exam as a criterion for student graduation (according to the National League for Nursing (NLN) (2012), one in five pre-licensure programs has this policy, [10]. Of course within-program standardized assessments (e.g. HESI or ATI) can serve other purposes as well; although many (including the HESI E2) are designed (and usually used) as summative, end-ofprogram measures, data from the exams may help students shape their remediation and studying efforts.

2. Digital Learning Materials

Another preparation strategy is a more personalized approach to out-of-class studying involving digital learning materials. These materials provide students the opportunity to combine ongoing learning with remediation in weaker areas. Basic online assessment tools have been studied with an eye to determining how they can best be utilized in out-of-class environments. Online quizzing systems are a way of efficiently providing the potential benefits of in-class formative assessment, but in an easier to administer and monitor environment. Online quizzing and practice also allows students to study and learn using the mastery approach proposed by Bloom [14,15]. The use of a system which helps students get practice and improve over time is one strategy shown to be beneficial to students [7,16]. By allowing students to take quizzes, study, retake quizzes, study again and so on, students may benefit from some of the learning gains shown in mastery learning programs [17] as well as the testing effect [18].

3. The Testing Effect

Most people assume that answering test questions is a way to evaluate learning and not much else. But there is a large body of research outlining the benefits of testing for learning-not just of learning [19]. Most students study in a very passive way-highlighting text books, or reading over class notes again and again [20]. The research indicates however, that passive studying techniques such as these are less effective than actually practicing what you will ultimately need to do to show you have learned something [21]. In addition, repeated reading can produce “illusions of competence” in which students may feel they comprehend a text to a higher degree than they actually do because repeated reading can create an increased ease of processing the text [22].

Practicing retrieval, on the other hand, enhances the long-term retention of information which benefits student learning [18,23]. When students have been tested on material, in the long term they remember more of it than if they had repeatedly studied it in the more traditional sense-this is referred to as the “testing effect” [22].

Put more simply, the idea is that rather than studying by repeatedly reading over notes, or highlighting text, a more effective method is to review content and take a practice quiz on that material. As students respond to practice questions they are actively trying to retrieve information from their brain and this changes how they can access it later. “When we use our memories by retrieving things, we change our access to that information. What we recall becomes more recallable in the future. In a sense you are practicing what you are going to need to do later” [24].

4. Self-regulated Studying

Even though there is clear evidence that repeated testing has greater learning benefits that repeated reading [25], it is not clear to what extent students understand and apply this principle in their own independent studying. And do students have metacognitive awareness of the impact of testing as a learning strategy? Many students seem not to have knowledge of evidence of effective learning strategies (not would we necessarily expect them to) and tend to improvise, or use intuition in terms of their study habits. Moreover, students make decisions about studying based on whatever is due the soonest, and tend not to ever return to material once a course has ended [26]. And in the event that students do quiz themselves (using flash cards for example), students tend to be too hasty in “dropping” the cards, or content they think they know. Students may feel they have learned something in the short-term, but if they abandon the content, or the flash card, they may not realize that as time goes by they are likely to forget that information [19,26].

The lack of student awareness of the testing effect may result in students not practicing retrieval when they are studying on their own. In one study, researchers asked students to list the strategies they used when studying. Only 11% of students (19/177) indicated they practiced retrieval while studying and only 1% of students (2/177) chose retrieval as their number one study strategy. The majority of students chose repeated reading of their books and notes as their number one study strategy [22]. The implication of these, and other findings, is that students may choose not to test themselves when they are directing their own studying efforts.

One implication of this body of research is that students must be made aware of the benefits of testing, and perhaps more importantly, be provided tools to help them implement the strategy. A possible reason students may not test themselves (even if they are aware of the benefits) is they may not have the understanding of how to do it, or which types of exercises, or resources to use. Another issue when considering self-regulated studying is students must make judgements about what they know and how to use this information to guide their studying behavior. Mistakes made in the former can lead to prioritizing the wrong topics, or an over-confidence in mastery which may result in less studying overall. Evidence suggests that having information from self-testing can improve students’ judgements of their own learning. So having a way to get information on relative strengths and weaknesses can be useful in helping students make more accurate assessments of their learning.

The project reported on here describes the use of an adaptive quizzing and learning system (AQS) [27] NCLEX-RN Pass Point (Wolters Kluwer) in a pilot program at the study school. The system implemented is designed to provide students an environment in which they can effectively and efficiently practice and learn nursing skills and concepts over time and had been previously implemented by the course instructor at a different institution. The system is not designed to serve as an end-of-course, entrance, or end-of-year exam. Rather, optimal use of the product is ongoing to help students engage in self-testing to better master curricular material and prepare for exams (such as the NCLEX-RN) in an efficient and effective way.

The AQS offers students both the opportunity to take regular, short, practice quizzes in desired Client Need categories, as well as longerlength exams which more closely resemble the actual NCLEX-RN experience. Students can select a number of questions (from 75 to 265) and practice sitting for different lengths of exam. Students do not know how long their actual NCLEX-RN will be, so it is important they practice taking longer exams to help build stamina and endurance. Taking an exam in as close as possible conditions to the actual exam helps familiarize students with the test-taking experience and increases confidence. As Dr. Sian Beilock points out in her book Choke (2010), “The key is to really be accustomed to dealing with whatever sort of reaction that you might have so that you’re ready in the high-stakes situation for whatever comes your way…. We often spend a lot of time, say, studying for a test, just going over our notes, but we rarely sit there and actually take a practice test, like, that’s very similar to what we’re going to find in the high-stress situation.” [28].

The AQS allows students to practice and learn in a low-stakes, authentic environment to help prepare for the high-stakes situationthe NCLEX-RN. In addition to the benefits described above, this type of practice can also be invaluable to populations such as English learner (EL) or limited English proficient (LEP) students, as well as those requiring extra support in content mastery and test-taking strategies. As Dudas (2011) noted, nursing schools associate college and nursing school entrance exam scores as measure of students’ ability to be successful within the program and on NLCEX exam [29]. However, for ESL students not much consideration is given to the impact of language. Evidence suggests that allowing students to engage in more independent, self-paced learning (by way of using interactive web-based tools) can help increase confidence as well support student learning [30,31].

The AQS also includes remediation links to content related to students’ individual strengths and weaknesses as indicated by their scores on the simulated NCLEX-RN Practice Exams or their dayto- day quizzing. The remediation links provide students a chance to review concepts that may be causing them difficulties before taking more practice quizzes or exams.

In this retrospective case study, we explored the impact of the implementation of the AQS during the final semester of a BSN program the southeast United States on nursing students’ performance. The AQS provides students a forum to actively practice for exams and master course content and in the study group, was employed primarily as a study and review tool for the NCLEX-RN. We hoped to gain insight into how much students would use the system if it were simply provided as an ancillary study tool and use was not associated with a grade, nor were there set requirements for use. Students were encouraged to use the AQS to prepare for class, unit exams, standardized exams (ATI), and the NCLEX-RN, but usage was voluntary. A set of practice quizzes were created and provided to the research participants to use as they studied, but were not graded activities. The research outlined above indicates that students do not frequently use self-testing as a study strategy, particularly when they are engaged in self-regulated studying. In this study we explored students voluntary use of a system designed to encourage and support self-testing within a nursing program.

5. Course and Students

The study sample consisted of students (N = 36) enrolled in a second semester senior-level course in a BSN program.

6. Implementation of the AQS

There were no parameters set to guide AQS usage. Students were encouraged to use the AQS to prepare for class, unit exams, standardized exams (ATI), and the NCLEX-RN, but usage was voluntary. Instructors created a set of optional practice quizzes which students could take, but no points were associated with usage.

7. Additional Requirements

The nursing program does not have an exit examination nor are there any “high-stakes exams” which may preclude graduation or progression. The school does, however, require a score on the ATI Comprehensive Exam of Level II or higher. During their fourth semester, students take the ATI Comprehensive Standardized Examination and based on their results follow one of two trajectories for the remainder of the course. Students who do not meet the benchmark must complete remediation and retake the exam. Therefore, based on initial ATI results, students were placed in Group I (scored Level II or above on 1st ATI attempt) or Group II (scored below Level II on 1st ATI Attempt). All students, Group I and II, continued to complete necessary coursework and prepare for the NCLEX-RN. This preparation focused on practice and reinforcing test-taking strategies. If a student scored at a Level III, they were not required to retest, but had the option to retake the exam if they choose to do so.

8. Materials and Methods

9. Results

9.1 Overall AQS usage and mastery

Overall AQS quiz usage and final ML are shown in Table 1. All 36 students used the AQS to some extent. Students answered an average of 574.11questions, took an average of 42.19 quizzes, and achieved an average final ML of 3.48. Students logged into the AQS an average of 31.5 times (with a range of 3-129). Given the large standard deviation for the number of quiz questions, a better measure is the median which was 419. Remediation link usage was low, over half of the students (58.3%) did not access any links at all. The remaining students accessed between 1 and 182 links (Table 1).

Table 1: AQS Quiz Usage and Mastery (N = 36).

Normality tests revealed three outlier cases for number of quiz questions answered. These cases were left in the data set and a log transformation on number of questions conducted. Following the transformation, question data was normally distributed as assessed by a Shapiro-Wilk test (p> .05).

9.2 Practice exam usage and mastery

The AQS provides students the opportunity to take short practice quizzes (Table 1) as well as longer-length practice exams covering the range of client need topics found on the NCLEX-RN. Table 2 shows usage of the practice exam feature of the AQS. Only 18 or the 36 students used the practice exam feature, with a median of four exams. One student took 38 practice exams, and the remaining 17 took between one and eight. Students achieved an average exam ML of 6.62 (SD = .69).

Table 2: Practice Exam Usage and Mastery (N = 18).

9.3 Course grades

Course outcomes for the Adult Health II course consisted of three unit and a final exam. The unit exams were administered in the spring with 3-4 week intervals in between each. The final exam was administered two weeks before the end of the semester. Course grades are shown in Table 3. Scores on exams 1 and 2, as well as the final exam had a larger range of performance, while for exam 3, students all scored between 93 and 100 points (M = 99.81, SD = 1.17).

Table 3: Course Exam Descriptive Statistics (N = 36).

9.4 ATI scores

As described above, students took the ATI comprehensive exam and had two attempts to achieve the benchmark score (Level II or > 65%). These attempts were in March and April 2016. The ATI Comprehensive exam is a 180-item test with questions across the same range of client need categories as the NCLEX-RN. Outcome data from this exam include individual scores, a “predicted probability to pass NCLEX-RN” as well as comparisons to national means. Students had previously taken other ATI exams (e.g., ATI Medical-Surgical exams) as part of the nursing program. Scores for all these exams as well as the number of students taking them are shown in Table 4.

Table 4: Scores on all ATI Exams taken during the program.

For the March 2016 exam, the group program mean (for programs similar to the study school) was 67.8% (compared to a national mean of 68.3%). For the April exam, the comparisons were the same. The adjusted group score, however, increased from 68.7% in March to 71.4% in April. The study school increased their percentile (within similar programs) from the 56^th percentile in March to the 75th percentile in April. By comparison, the previous cohort of students who took the exam in November 2015, had a group percentile rank of 57% (on the second attempt).

ATI reports the percentage probability of a student passing the NCLEX-RN. These percentages can be translated into the four levels (specified by ATI). On the first ATI Comprehensive attempt, 80% of students scored at Level II or higher. Of the remaining students, six scored a Level III and one scored Level IV. One student had taken the exam in the previous semester (and achieved the benchmark) and did not have to take it again.

On the second ATI Comprehensive attempt, 94.3% of students achieved the benchmark (or above). Two students achieved either a Level III or Level IV. Of the seven students who did not achieve the Level II benchmark on the first attempt, five did achieve it on the second attempt. The remaining two did not.

Each student’s raw score was converted to an adjusted individual total (to adjust for possible differences in assessment form difficulty). The adjusted individual total score is on a scale of 0% to 100%. We compared students’ adjusted scores between their two test attempts. On average, students improved by 2.63 points (SD =4.7), and some students (N = 11) received a lower score on their second attempt.

Students were divided into three groups based on the ATI scores:

1. Group 1: Achieved Level II (or above) on the first attempt (N = 28)
2. Group 2: Achieved Level II (or above) on the second attempt (N = 5)
3. Group 3: Did not ever achieve Level II (or above) (N = 2)

The difference in group sizes makes comparisons of these groups difficult. Scores on the course exams, however, decreased (for exam 1, 2 and the final) from Group1 to Group 2. As did overall usage and ML within the AQS (Table 5). Interesting to note was that Group 1 accessed the remediation links more than the other two groups.

Table 5: AQS Usage and ML by ATI Group.

10. Student Surveys

Students were given the opportunity to complete three online surveys. Of the 36 eligible students 25 responded to survey 1, 21 to survey 2, and 20 to survey 3. Twenty students responded to all three survey measures (a 55.5% response rate).

10.1 Summary of survey 1

Twenty-five students responded to the first student survey which focused on motivations for pursuing nursing as well as learning and study practices and attitudes. This survey included items focused on motivation, self-efficacy and learning and study strategies and was given to students at the beginning of their final semester. Items included on the survey were adapted from items on the LASSI (Learning and Study Strategies Inventory), the Student Opinion Scale and the Motivated Strategies for Learning Questionnaire [32].

The first set of questions asked students to respond to statements about their motivations and study habits (table 6).

Table 6: Summary for motivations and study habits questions.
Note: Response options ranged from (1) not at all true to (5) very true.

The most important motivations for students were improving skills and understanding course work. Most highly rated goals were to learn and master new skills, with all related items rated (on average) above 4.5/5. Less important to students were items relating to comparisons between themselves and other students.

The second set of questions focused on learning and study practices and attitudes. The most highly ranked item in the set was related to students finding relationships between what they are learning and already know (M = 4.28). Several items related to difficulties students may have studying. For these items (5, 8, 9, 10, 11, 12) a lower score indicated student did not think this behavior was typical of them (Table 7).

Table 7: Summary for study practices and attitudes questions.
Note: Response options ranged from (1) not at all typical of me to (5) very typical of me.

The third set of questions focused on study habits in general (Table 8). The most highly ranked general study habits chosen by students, were when studying they tried to determine which concepts they didn’t understand well (M = 3.96), and they tried to relate new material to what they already knew (M = 3.96). In keeping with the research literature, the lowest ranked study habits relate to the underlying concept of retrieval practice. Students were less likely to study by writing summaries of the main ideas they had learned (M = 2.52), or making up questions to focus studying (M = 2.32) This is not surprising as both of these tasks are labor intensive and many students may not make time to do them, or necessarily know how to best use these techniques. More common were study techniques we would expect to see-outlining and re-reading which are both more passive study techniques.

Table 8: Summary for general study habits questions.
Note: Response options ranged from (1) not at all true of me to (5) very true of me.

10.2 Summary of survey 2

Twenty-one students responded to the second student survey which focused on students’ reported use of the AQS as well as their opinions about it and features they preferred (Table 9).

Table 9: Survey 2 Student Response Summary (five survey items).

Students provided survey-feedback on their usage and opinions on the AQS. Of the 21/36 students who responded, the majority indicated that use of the AQS improved their performance in the course. The majority of students also indicated that the AQS was helpful in preparing for exams, getting feedback on strengths and weaknesses, increased knowledge of course concepts as well as preparing for the NCLEX. Students who provided additional information on perceived benefits of the AQS indicated factors such as helping to focus studying, improving test-taking strategies, as well as helping with performance on the ATI exams. The majority of students also reported to prefer an online studying tool vs. a paper and pencil, or pre-quiz bank. Benefits reported were similarity to the NCLEX experience, convenience, ability to track progress and ability to identify weaker areas to guide studying.

Students were also asked to indicate how much they valued certain features of the AQS. Responses are shown in Table 10.

Table 10: Student ratings of AQS features.

Over 85% of the respondents indicated that the large number of practice questions, the adaptive functionality, the ability to track progress throughout the year, and web-based convenience, and the easy to use interface were all “extremely important”. Mastery levels were seen as extremely important by 71.4% and close connection of the tool to the textbook by 76.2%of respondents.

10.3 Summary of Survey 3

Twenty students responded to the third student survey. Summary results are presented below.

Students reported taking the NCLEX between May 25th and June 14th, 2016. Eighteen students (90%) reported passing the NCLEX and two students reported not-passing. Students reported answering an average of 123.95 NCLEX questions (SD = 75.29) with a range from 75-265. Nine students reported answering 75 questions and four students reported answering 265.

Students were asked which tools/products they used to prepare for the NCLEX and responses are shown in Figure 1.

Figure 1: Tools/products used to prepare for the NCLEX.

11. Discussion

The descriptive retrospective analyses reported here explored the usage of the AQS within a final semester nursing course. The system was implemented to provide students an opportunity to bolster their preparation for the NCLEX. Use was optional, and students could use the system as part of their self-regulated studying. Ultimately, all students used the practice quizzing feature of the AQS and answered an average of 574 practice quiz questions with an overall quizzing ML of 3.48. Half of the students used the longer-length practice exam features, and those students took an average of 5.94 practice exams and had an overall exam ML of 6.62.

Thus we see that with increased usage of practice quizzes students were more able to correctly answer more difficult questions and mastery of the content improved. All students in the group passed the NCLEX-RN (on the first or second attempt). In the most recent two prior years before the AQS was implemented, the NCLEX-RN first time pass rates at the study school were 73.91% (2014) and 88.06% (2015). The 2016 study cohort had a 100% pass rate (considering both first and second-time test takers) and an 88.88% considering only first time.

We looked at the connections between usage and mastery within the AQS and other course outcomes. Quizzing ML was positively correlated with scores on the second attempt ATI Comprehensive exam, r (35) = .349, p < .05, but not ATI Comp 1 (first attempt). This may indicate that students who did not achieve the required score on the first attempt, increased their use of the AQS as they prepared for their second attempt. There was a strong, positive correlation between scores on the ATI Comp 1 and Comp 2, r (35) = .698, p < .001, indicating that student performance on their first attempt did not change much on the second attempt. There was no significant association between practice exam ML (for the 18 users) and score on the ATI Comprehensive exam. Two students did not achieve the target score on the ATI Comprehensive exam. Both ultimately passed the NCLEX, one on their first attempt and one on the second.

As we would expect, the ATI comprehensive exam can serve to provide students information to help inform their ongoing study and preparation. Results provide students and faculty valuable information on areas of curricular weakness (and strength) which can be used to focus remediation efforts. But students must actively use this information in order to improve outcomes on subsequent test attempts (if applicable). If they keep doing the same thing they have been doing, they are likely to get the same outcome.

If you receive information from a predictive test, the next question has to be-so what do I do now? This question could come from an instructor or a student. A percentage likelihood of passing a future exam might be one piece of helpful information, but to alter your path one needs tools to help address areas of weakness and provide the opportunity to practice, learn, improve and prepare for the NCLEX exam. Without them, the “predictive” information loses its value. Students don’t exist in a vacuum and we ought to hope they use information from a “predictive test” to alter their course (if necessary).

One important intervention strategy, based on information from a standardized exam is the development of an individualized remediation plan provided to students by faculty based upon student performance. Oftentimes, in our experience, students receive results from a standardized test and are left to try and make sense of them, and determine the best path forward. Students may not, however, be equipped to consider the context in which the test was taken, how to best interpret the findings, and how to use that information in a formative way.

The results of a test given before a student feels completely prepared can provide valuable formative feedback, but can also have a demoralizing effect. We feel is it crucial for instructors to help provide context and guidance around how to best use information on their current state of knowledge or preparedness for a high-stakes exam. Ought we to imagine that student performance during the final semester of a nursing program is indicative of their performance on a high-stakes exam looming sometime in the future? We argue, not necessarily.

In an ideal world, faculty would collaborate with students regarding their areas of weaknesses in order to further strengthen the effectiveness of the plan of remediation. When results are provided to the student without guidance, the next steps may prove overwhelming to the student. An unfortunate consequence is that students may tend to either address areas of strength in which they feel more confident, or invest all their remediation time reviewing areas of weakness, but neglect revisiting or reviewing areas of strength. This practice may lead to improvement in areas of weakness, but a potential decrease in score for areas of strength-as can be seen when comparing the first and second ATI standardized test results. Moreover, it may lead to more frustration, confusion, and despair among graduating seniorsfeelings which can negatively impact student confidence as they prepare to take the NCLEX RN licensure exam.

Utilization of the AQS can help to some degree by providing students the opportunity to answer questions on all components of the NCLEX-RN as well as ongoing, instant feedback on current state of mastery.

Given the benefits of self-testing as a learning strategy, we were interested to see if students would opt to engage in it in their selfregulated study practices. Results from the initial survey supported the findings from other studies in which students tend not to choose to test themselves when they are regulating their own learning. In keeping with the research literature, the lowest ranked study habits relate to the underlying concept of retrieval practice. Students were less likely to study by writing summaries of the main ideas they had learned (M = 2.52), or making up questions to focus studying (M = 2.32). This is not surprising as both of these tasks are labor intensive and many students may not make time to do them, or necessarily know how to best use these techniques. More common were study techniques we would expect to see-outlining and re-reading which are both more passive study techniques.

Students responded to these survey questions prior to taking the course, and using the AQS. So while many students initially reported not to use self-testing-based study strategies, ultimately all of them did so when the AQS was made available to them. There are, of course, many possible reasons why this is the case; it may be that students feel confident in their usual study habits and judge their mastery to be higher than it is. This phenomenon has been described by others as originating from illusions of competence which occur when studying in passive ways. By reading and re-reading text passages, students’ fluency and ease at which they process the text increases, and they may mistake this fluency with mastery of the material. While in actual fact, feelings of fluency when the text is in front of you has no relationship to future retention [23].

Students may also not be aware of the benefits of self-testing and the robust evidence that practicing retrieval enhances learning. Another possible reason for students not typically engaging in this sort of studying is that they may not know how to approach it, or may not have the tools to do so. Making flashcards, or other study aids may prove too time-consuming for students to embrace this strategy. And even if students utilize end of chapter practice questions (which tend to be included in textbooks), the tendency to peek at the answer before trying to respond independently can prove too tempting. Once one is exposed to the correct answer to a question, the initially question seems easier, and confidence in comprehension and knowledge can increase-both of which may cause students to think they know more than they do.

The AQS is built to offer students the opportunity to answer a large number of practice questions related to the course content in an environment where they must select the answers to all questions before receiving feedback on their performance. This is clearly beneficial as students are not likely to apply this technique in their own self-regulated studying in an effective way.

12. Conclusions

A clear implication from this, and other research, is that faculty should inform and educated students about the benefits of self-testing and the impact on enhancing learning. Students’ reliance on their own judgements of learning and mastery, may lead them to succumb to feelings of competence and notions that they know the content much better than they actually do. If they are constantly having to face situations in which they must actually choose answers to practice questions, and they don’t always get questions correct, they may end up with both a better sense of their competence and be likely to keep engaging in the practice.

The findings of this descriptive retrospective analyses reported here are consistent with the findings from another baccalaureate nursing program also located in Southeast Texas. Both studies support the utilization of adaptive quizzing as a self-regulated learning strategy for nursing students both during and after nursing school and indicate that as students actively study and learn in the system, their mastery of course content increases. Data from the AQS provide information and insight beneficial to all stakeholders. The AQS provide students an environment in which they can effectively and efficiently practice and learn nursing skills and concepts over time as well as to prepare for exams (like the NCLEX). It allows students to practice and learn in a low-stakes, authentic environment to help prepare for higherstakes exams (e.g. NCLEX-RN). This type of practice can also be beneficial to populations such as EL (English learner) or LEP (limited English proficient) students as well as those requiring extra support in content mastery and test-taking strategies. The AQS is a powerful tool for formative assessment and remediation, providing instructors with meaningful data that reveals student misconceptions and areas of weakness. This can help provide insight for faculty to evaluate both the learners’ understanding of content, critical thinking skills, and test taking abilities and help shape further learning and remediation.

Competing Interests

The authors declare that they have no competing interests.