1. Introduction

“Aggressiveness evolves from a desire or motive to overcome opposition forcefully, to fight, to revenge an injury, to attack another with intent to injure or kill, and to oppose forcefully or punish another [1,2]”. One of the most undesirable characteristics, aggressiveness is strongly associated with anything from minor unwanted behaviors such as lying [3,4], sabotage [5], absenteeism [6,7,8], grievances [7], cheating [4], and traffic violations [3], to serious evil behaviors such as stealing [9], fighting [10], and physical attacks [11]. Thus, the personality construct of aggression and its assessments has been the focus of considerable interest.

A number of methodologies such as behavioral measures and observation techniques have been proposed to study aggression. The approach that has attracted the greatest interest is the self-report method such as the questionnaire [12], which has been used to study disagreeableness, the most closely related subset of the Big Five traits to aggression [3,6,11]. While child aggression is often assessed by teacher and peer evaluations [13,14], adult aggression usually relies on self-report assessments. However, self-reporting may not produce an accurate assessment of an individual’s aggressiveness not only because individuals may not be able to perceive their own aggressive tendencies but also because they are generally less likely to report their aggressiveness in stressful situations [15]. Thus, self-report methods tend to generate information about how one perceives his/her own aggression or how one wants to be perceived rather than a true representation of one’s true aggressive disposition. Furthermore, selfreported aggression may not capture multiple facets of aggression. For example, self-attributed aggression more likely taps the explicit level of aggression, not the unconscious level of aggression.

2. The Conditional Reasoning Test for Aggression (CRT-A)

Individuals routinely perform activities based on what they believe is right or appropriate. This judgment, belief, or idea is not the same for everyone. Even in the same situation, people can make different judgments, and they act accordingly. Even if the actions or judgments may not seem acceptable or reasonable to others, most individuals are ready to justify their actions. Thus, aggressive individuals and nonaggressive individuals make different decisions in similar situations, and both parties have reasons for their actions that seem reasonable and rational to them. The reasoning biases that aggressive individuals use to make their actions appear rational and sensible are called “Justification Mechanisms”. James classified implicitly aggressive individuals’ biases into six JMs [16] (Table 1): hostile attribution bias, derogation of target bias, potency bias, retribution bias, victimization by powerful others bias, and social discounting bias. JMs are based on theory from previous research, not on empirical results exclusively. James contends that JMs are in place for implicitly aggressive individuals’ reasoning processes. These individuals are not only aggressive but also ready to justify their aggressive dispositions. These processes tend to happen outside of their awareness. Based on the six JMs, the CRT-Aggression (CRT-A) consists of what appears to be 22 inductive reasoning items, with three bogus items included for face validity. Each item has a short premise followed by four alternatives: One alternative is attractive to implicitly aggressive individuals, one is a pro-social alternative, and two are illogical alternatives. Individuals who endorse an aggressive response will score +1, a pro-social response will score -1, and an illogical response will score 0. James and his colleagues validated the measure, which showed promising validity in predicting employee absenteeism; counterproductive behaviors such as a theft, sabotage, and work performance [5]; perception of injustice; and obstructionism by basketball players [11].

3. The Present Study

We explored measurement (CRT-A) equivalence with Korean samples using two different models: factor analysis (FA) and item response theory (IRT). FA approaches to measurement invariance are different from IRT approaches, as FA investigates the construct from a scale level, while IRT explores it from an item level. Each approach has its own advantages. Kim, Kim, and Kamphous [17] stated that only a few studies used both FA and IRT to study measurement invariance. Thus, we adopted both approaches in this study.

We tested measurement equivalence with Korean populations because the CRT-A has not been validated in Asian cultures. To extend the generalizability of the CRT-A, it is critical to include more diverse populations than just populations from western countries. In addition, in Korea, psychological assessments that measure unconscious levels of aggression are lacking. In the selection process, Koreans use a variety of assessment as selection tools; however, most of those assessments are self-reporting instruments. Korean corporations are very interested in a new and innovative selection tool, and the idea of CRT-A assessing an individual’s aggressiveness is intriguing. The Korean version of the CRT-A uses the same idea (i.e., assessing unconscious aggressiveness using inductive reasoning) and operates on premises based on Korean culture, with the intent of providing valuable information in understanding Koreans’ aggressiveness and cross-cultural similarities and differences in individuals’ aggressiveness. Before we introduce the CRT-A to Koreans, it is important to test whether the CRT-A assesses Koreans’ aggressive traits in the same way that it measures that of US samples. This study is a cornerstone of introducing the Korean CRT-A.

In this study, to reduce any sample biases, we recruited participants from both Korea and the US. To explore the measurement equivalence we tried to control demographic information which might influence response patterns. We strived to include subjects of similar age and comparable gender ratio in both cultures.

4. Method

4.2 Measure

CRT-A: Implicit aggression was measured using the new CRT-A, which includes five more items than the original version. This test consists of 30 reasoning items including three bogus items. For each item, premises and reasoning tasks are followed by four possible solutions (alternatives). Different scoring systems can be used for the CRT-A (i.e., dichotomous or trichotomous), and this study adapted a dichotomous scoring system. Aggressive alternatives were scored +1, and pro-social and illogical responses were scored 0. High scores indicated highly aggressive personalities, while low scores indicated pro-social personalities.

Table 1 presents a sample item. In this question, alternatives (a) and (c) are illogical responses. The pro-social alternative from the sample item is (b): “It offers no way to settle a conflict in a friendly manner,” and the aggressive alternative is (d): “People have to wait until they are attacked before they can strike,” which is based on the retribution bias. Implicitly aggressive individuals are more interested in seeking retaliation than in seeking ways to maintain a relationship. From an aggressive individual’s perspective, the “eye for an eye” approach is problematic because of the need to wait to attack others, rather than resolving the issue in a friendly manner. As the retribution bias is embedded in the cognitive processes of unconsciously aggressive individuals’, they think their beliefs are reasonable and sound; thus, they justify their belief in retribution.

Table 1: Illustrative Conditional Reasoning Problems [6].

5. Results

5.1 Descriptive statistics

First of all, we analyzed proportion of aggressive responses in both Koreans and US participants. Koreans’ proportion of aggressive responses were ranged from .12 (problem no. 22) to .79 (problem no. 12). US participants’ proportion of aggressive responses were from .01(problems no. 7) to .72 (problem no. 27). Korean participants’ item biserial orrelation represent from .12 to .79 and US participants’ item biserial correlation ranged from .06 to .37 (Table 2). Koreans’ mean aggressive responses was 11.56 with 5.54 standard deviation and US participants’ mean was 6.53 with 2.44 standard deviation (Table 3).

Table 2: Comparison of item difficulties and item total correlations on Korean and US samples.

Table 3: Mean and Standard Deviation of the CRT-A on Korean and US samples.

5.2 Factor analyses

The principal factor axis using promax rotation with English CRT-A data is presented in Table 4. An eigenvalue greater than 1 criterion showed a four-factor structure, which was a little different from the CRT-A three-factor structure. The difference could be caused by adding five new items and perhaps sampling error due to the relatively small number of the students (N = 432) compared to the sample size for the three-factor structure (N = 4772). The four factors accounted for 76% of the total variance. Five CRT-A items loaded highest on Factor 1, eight items loaded on Factor 2, five items on Factor 3 and seven items on Factor 4. Compared to the three-factor structure, external controls (Factor 4) and internal controls (Factor 1) were moderately replicated. Based on the six JMs of the CRT-A, Factor 2 and Factor 3 were labeled as “hostility of powerful others” and “potency,” respectively.

Table 4: Factor Loadings (EFA) on the Conditional Reasoning Test- Aggression for US samples.

Thus, to understand the factor structure of the Korean CRT-A, another principal factor axis analysis was conducted. With the Korean sample one more factor was extracted, which represents a five-factor structure (Table 5). Three Korean CRT-A items loaded highest on Factor 1, three items on Factor 2, seven items on Factor 3, four items on Factor 4, and nine items on Factor 5. The different number of factors and different pattern of factor structure indicates that the CRT-A may not assess implicit aggressiveness among Koreans in the same way that it assesses US samples implicit aggressiveness. There may be construct bias, and this failure to replicate the factor structure of the CRT-A could be due to different cultural issues and/or due to the tetrachoric correlation matrix with binary data. Embretson and Reise [24] mentioned that, “Tetrachoric correlations are preferred over phi correlations because they correct for item difficulty effects… Adjusting whole matrix of item correlations to tetrachorics sometimes results in a singular correlation matrix, which is not appropriate for factor analysis” (p.37). A singular matrix was the case in this study, wherein a tetrachoric correlation matrix was entered, thus, factor analysis of the CRT-A may not provide meaningful information. Therefore, DIF analysis from the IRT model was used.

Table 5: Factor Loadings (EFA) on the Conditional Reasoning Test-Aggression for Koreans.

5.3 Differential item functioning

Again, the IRT likelihood ratio model indicated that the DIF model fit was better than the non-DIF model. G² of the invariance model was 6930.423 and G² of the baseline model was 6236.5613, which indicates at least one item showed DIF in the CRT-A. The Koreans’ difficulty in endorsing an aggressive alternative is different from that of the US participants. Item level analysis suggested that threshold differences between Koreans and US samples were greater than .3 for 26 items, meaning that DIF existed in almost all items on the CRT-A (Table 6). Only one item, CRT-A 24, did not show DIF; thus, its ICCs were similar across groups, while the CRT-A 18 ICC of reference (US participants) and focal (Koreans) groups was quite different. From the Koreans’ trait level, selecting an aggressive alternative on CRT-A Item 18 seemed to be easier than it was for the US samples’ trait level. Strong DIF on Item 18 could be due to the wording effect [25]. The back-translated pro-social alternative was “Hardworking employees receive bonuses and some time off.” In the pro-social alternative for the original CRT-A, “bonuses” was the subject of the sentence, but in the Korean CRT-A, “hard-working employees” was the subject of the sentence. Thus, to Koreans it seemed too obvious that employees who work hard receive bonuses, and they were less likely to think of an alternative reason for companies to use bonuses. Consequently, the pro-social option was less attractive to Koreans as a logical alternative; they were more likely to choose the aggressive alternative because the other two options did not sound sensible. The wording seemed to lead Koreans to choose the aggressive alternative more easily, regardless of their aggressiveness.

Table 6: IRT adjusted threshold parameters of the Conditional Reasoning Test-Aggression items between Korean and US samples.

DIF also occurred in Item 16, which raised concern before data collection. This Item referenced cultural familiarity of U.S. and Japanese carmakers. Accordingly, in the Korean CRT-A, U.S. was changed to Korea and Japan remained the same; however, Koreans tend to have animosity towards the Japanese from a long national history. Thus, apart from each Korean’s aggressive tendencies, different cultural attitudes of Koreans and US samples towards Japan seemed to cause DIF on this item.

DIF existed in CRT-A 25, which references World War II, and the aggressive alternative is “Only weak countries follow agreements.” Koreans tend to believe that Korea is a weak country while US participants tend to have pride in the US and believe the US is a strong country. Thus, Koreans and US samples are likely to have different perspectives regarding what constitutes a weak country, which seemed to affect DIF in Item 25.

6. Discussion

Most psychological assessments validate and test reliability among middle class white samples [26,27]. Validating the constructs of those assessments with broader populations should provide more information about their psychometric properties. Thus, in this research, the first study investigated measurement invariance of an implicit aggression assessment (CRT-A) across US samples and Koreans using EFA, and DIF.

For this study, five new items were developed and tested with Koreans. The four factors were extracted from a new version of the CRT-A with a five-factor structure. Results show that Koreans may have different underlying biases to justify their unconscious motives to be aggressive. On some of CRT-A items, Koreans found it easier to endorse aggressive alternatives (e.g., Items 3, 15, or 16) while they found it more difficult to choose aggressive responses on other items (e.g., Items 10, 18, or 29). DIF occurred in almost all CRT-A items. Each of the CRT-A items starts with a short premise and those premises seemed much more familiar among American cultures. Therefore, it may not assess Koreans’ implicit aggressiveness accurately. The differences on the CRT-A could be cultural differences, translation errors, or different latent variable relationships. The implicit assessment of aggression needs further studies to assess aggressiveness among the Korean population in the same way that it is assessed among US samples. Unless the assessments are developed by researchers from different cultures at the initial stage of development, it may be impossible empirically to meet all the equivalence conditions [28].

7. Limitations of the Study

As limitations of this study, criterion-related validity and translation need to be discussed. First, this study investigated cross-cultural issues at measurement levels without testing their predictive validity in the Korean population. To be a valid measure of different cultures, construct validity should be tested; testing its criterion related predictive validity would provide additional meaningful information. For instance, as previously mentioned, the CRT-A tends to predict US samples aggressiveness (i.e., sabotage, lying, absenteeism, stealing, obstructionism, etc.).

Second, for the translation process, this study only adapted one traditional method, back-translation, although it was additionally reviewed by a Korean professor. The Korean CRT-A did not show many errors from translation based on the back-translation process, and any discrepancies were resolved before conducting the study. However, the results suggest that some items were vague and unclear to Korean respondents. A little finesses and choice of word seems to affect participants’ response patterns.

8. Future Directions

Implicit personality assessment, and understanding unconscious levels of personality through assessments, is new and fascinating to Koreans and research areas in cross-cultural studies of implicit personality assessments are fruitful subject. For future studies we suggest as follow. First, US participants’ responses supported that AQ and the CRT-A are less likely to be correlated and they predicted different types of aggression (i.e., verbal hostility, physical aggression, obstructionism). Previous research suggests that understanding aggression using both explicit and implicit measurements provides much more meaningful information than does using one or the other in predicting individuals’ behaviors. Thus, it would be interesting to investigate the relationship between the self-reported AQ and the CRT-A in relation with criteria and the association between the two assessments among Koreans.

Second, a modified version of the CRT-A based on Korean culture would fit better with the Korean population and understanding their unconscious motives to be aggressive. The results of this study suggest that there were some CRT-A problems that may not be familiar to Korean culture; therefore, it would be intriguing to modify the CRT-A to align it more closely with Korean culture. The idea of the CRT-A assessing an individual’s unconscious motives through an inductive reasoning problem is fascinating and will truly provide valuable information in understanding Koreans’ unconscious level of aggressiveness. Premises that are more familiar to Korean culture will more accurately assess their implicit aggression.

Third, this study failed to replicate the four-factor structure of the AQ. As mentioned above, respondents from Hong Kong China also did not support the four-factor structure, but they showed a good fit with a 12-item model of the AQ. Therefore, for future study, exploring measurement invariance with a shorter version of the AQ and CRT-A (if possible) might produce different results. Furthermore, completing short versions of the AQ and the CRT-A will take less time than completing the full versions, thus making easier to recruit more participants, leading to a larger sample, which will create a more concrete factor structure.

Competing Interests

The author declare that there is no competing interests regarding the publication of this article.