Embarassibility Scale - Renshaw Works Crafts and Photography of

Use of the Personal Report of Communication Anxiety (PRCA-24) in Japanese
Contexts: Reliability, Structural Integrity, and Response Validity
Paper presented at the National Communication Association Convention
San Francisco, California, USA
November, 2010
Steven L. Renshaw
Kanda University of International Studies
Chiba, Japan
Abstract
The Personal Report of Communication Anxiety (PRCA-24) has been used extensively in a variety of
cultural contexts to measure apprehension in various communication situations. This study examined a
Japanese translation of this scale in order to assess reliability and aspects of validity including intrascale correlations, structural integrity, and measurement continuity.
The scale was administered to 350 Japanese college students. Alpha coefficients and item-scale
correlations for the instrument and subscales matched parameters found in development of the English
version. While results of these classic reliability measures were comparable, confirmatory factor
analysis showed weak goodness of fit relative to subscales, and Rasch analysis indicated that overall
use of the scale might fail to fully cover or discriminate levels of the trait.
Using classical assessments, the scale appears to provide a reliable and consistent measure of
communication anxiety. However, its apparent weak structural integrity and inability to fully cover the
purported trait indicate that caution should be exercised in its use for research in Japan. The uncritical
summing of items or summed groupings of the subscales, may fail to differentiate or integrate factors
that are particularly relevant to the underlying trait of communication anxiety as it is found in Japan
and other cultures. While it would be somewhat easy to delete items in order to create what appears to
be a valid scale, the need for attending to cultural constructs while preserving the theoretical integrity
of the measure seems evident if a valid assessment suitable for research is to emerge.
2
Use of the Personal Report of Communication Anxiety (PRCA-24) in Japanese
Contexts: Reliability, Structural Integrity, and Response Validity
Introduction
Communication anxiety has been an especially important variable in communication research in Japan
(Klopf, 1984; Tanaka, 1996). One of the most prominently used measures of this trait has been the
Personal Report of Communication Anxiety (PRCA-24), and previous studies have indicated
consistently high reliability for translated versions of this scale (see McCroskey, Gudykunst, and
Nishida, 1985; Pribyl, Keaten, Sakamoto, and Koshikawa, 1998). While these studies have supported
the overall viability of using the PRCA-24 for research in communication, little had been done to
assess the purported underlying dimensions of the scale or how well it reflects aspects of
communication apprehension. In a Likert based scale such as the PRCA-24, it is necessary that the
measure include a sufficient number of items necessary to incorporate cultural perceptions related to
the underlying trait. In addition, the measure should exclude those aspects that are irrelevant and so
preserve parsimony (John & Benet-Martínez, 2000; Judd & McClelland, 1998).
With developments in structural equation modeling (Arbuckle & Wothke, 1999) and item response
theory, most especially in the area of Rasch analysis (Wright, 1999; Wright & Masters, 1982), more
complex issues of scale measurement may be addressed. Specifically, confirmatory factor analysis
(CFA) allows assessment of presumed multidimensional structures, and Rasch analysis provides a
means of determining how accurately, continuously, and completely items in a scale are measuring a
purported continuous underlying trait. The primary purpose of this study was to assess a translated
version of the PRCA-24 in terms of its structural integrity and its ability to provide a continuous and
inclusive measure of trait anxiety.
Personal Report of Communication Anxiety (PRCA-24)
The PRCA-24 is a set of 24 items with Likert type responses that attempt to measure the latent trait of
communication apprehension in four state situations: group, meeting, dyadic, and public (McCroskey
1970, 1982; Daly & McCroskey, 1984; Richmond & McCroskey, 1998). A number of studies have
revealed strong content validity (McCroskey, 1978; McCroskey, Beatty, Kearney, & Plax, 1985) in the
3
original English version, and the PRCA-24 has been shown to correlate with most other measures of
trait anxiety. Consistent measures of α ≥ .90 and item-total correlations ranging from .40 to .69 have
been found with α for subscales greater than .75 (McCroskey, 1990, 1992). Using similar classical
measures, strong reliability has also been reported for Japanese translations (see McCroskey, 1978;
McCroskey, Gudykunst, & Nishida, 1985; Klopf, 1984).
Despite its use in Japan, Levine & McCroskey (1990) indirectly questioned the validity of the PRCA24 with concerns that some items might be culturally bound. However, summed totals and subscales
have continued to be used (McDowell, & Yotsuyanagi, 1996).
Replication of structures with the four subscales intact has been somewhat elusive, even in the English
version. Levine & McCroskey (1990) tested first and second order confirmatory factor analysis
models and reported strong support for a second order model in which the latent trait of
communication apprehension is reflected in the four subscale dimensions. Perhaps most important for
this study, they failed to replicate such goodness of fit in the one non-United States culture sampled.
This led them to question whether all items would be appropriate for research in cultures other than the
United States.
Pribyl, Keaten, Sakamoto, & Koshikawa (1998) were unable to replicate the four subscale factor
structure in a Principle Components analysis of their Japanese version. While they did not report
results of any confirmatory factor analysis or comparisons with other methods generally recommended
in exploratory analysis (see Johnson & Wichern, 1998), their four-component rotation showed meeting
items loading in various degrees on the public component. Despite its historical base in the theoretic
development of the construct, they recommend removal of the public dimension from Japanese
versions of the scale even though this component explained most variation in their model with public
items having relatively strong and pure loadings. These researchers concluded that the present version
of the PRCA-24 has doubtful use as a multidimensional measure of anxiety. Still, they indicated that a
summation of all items should be used as a general measure of communication anxiety. However, they
provided no real evidence for supporting the viability of using the PRCA-24 as a one-dimensional
measure of trait anxiety.
4
Research Questions
While the work mentioned in the previous section has assessed the reliability and validity of Japanese
versions of the PRCA-24, rigorous testing of the internal structure and ability of the scale to
discriminate along a continuum of trait anxiety has been lacking. No confirmatory analysis has been
conducted on their theoretical structure of the scale, nor has any item response theory based
assessment been made.
Given the discussion and rationale above, the following specific questions were the focus of this
research:
1. How reliable and internally consistent is a translated version of the PRCA-24? That is, can
reliability be further replicated with Japanese subjects responding to a Japanese version of the
scale?
2. How valid are the structural integrity and theoretic dimension or dimensions underling the
purported trait measured by the PRCA-24? Are the underlying dimensions represented by the
four subscales replicable in Japanese versions?
3. How viable is the PRCA-24 as a continuous measure of communication apprehension? Does
the PRCA-24 provide a measure that adequately assesses the trait from low to high levels?
The first research question includes analysis using classical measures of reliability. It is included to
provide a basic check of the data and to replicate research that has included classic assessments made
on Japanese versions.
5
Method
The PRCA-24 was administered with three other anxiety instruments. 1 While some of the scales have
been previously translated into Japanese, and some linguistic variability could affect comparisons with
those translations, multiple translators working on all four measures were used in order to preserve
internal consistency as much as possible. Three independent Japanese translators, including a
professor of English, a graduate student in English-Japanese studies, and a professional EnglishJapanese translator were asked to translate the scales. Another professional English-Japanese translator
did a back translation of the translated scales from Japanese into English. Following this work, all
translations were compared, and unanimous agreement was reached by all translators on all items.
The translated version of the PRCA-24 used in this study may be found in the Appendix. Scales were
administered to 350 Japanese college students (78 males, 272 females) consisting primarily of
freshmen, sophomores, and juniors at a Kanto area university. To avoid response bias, instruments
were presented in four different order combinations using mutually orthogonal Latin Squares. Both
Japanese and non-Japanese administered the instruments. Subjects were told that research was being
conducted by a professor in the university with no mention of that professor’s cultural background.
The measures, including all demographic items, were in Japanese.
Items with negative wording were reverse scored. Univariate tests for outliers (above or below 3.0
standard deviations) and Mahalanobis tests for multivariate outliers were conducted. Extreme
responses appeared consistent with having extreme levels of the trait variable, and no subject was
consistently an outlier on more than one scale. Further, analysis was conducted both with and without
outliers, and results were substantially the same. It was judged best to be conservative relative to
throwing out any data and consequent information. No subjects were removed from the study. There
were no missing values, and F-tests revealed no significant differences due to ordering of the
instruments.
1
Other scales included Embarrassability (Modigliani, 1968; Miller, 1987), Fear of Negative Evaluation (Watson
and Friend, 1969), and Interaction Anxiousness (Leary, 1983). A comparison and analysis of all four instruments
is anticipated in a forthcoming report.
6
Analysis proceeded in three phases based on the research questions mentioned earlier. In order to
provide replication for previously reported results (Research Question 1), classical item analysis was
conducted on the data including calculation of alpha coefficients and item-total correlations (Carmines
& Zeller, 1979).
To evaluate construct validity in terms of structural integrity (Research Question 2), confirmatory
factor analyses were then performed on the data. Attention was paid to the concerns raised by Levine
& McCroskey (1990) relative to the validity of the PRCA-24 and subtests for use in cultures outside
the United States. Hence, a first order model was initially used to assess this scale. A first-order model
with the four subscales as correlated factors was then tested, and this was followed by a second order
model with the four subscales as factors reflecting the latent trait of communication apprehension (see
Figures 1 and 2 in the Results Section).
Maximum Likelihood methods were used in all models, and a scale value of one was assigned to
unobserved variables. With these parameters, all models were identified. Several measures were
calculated and compared for goodness of fit, but the following were considered essential and reported
in the results (see Kline, 1998; Arbuckle & Wothke, 1999; Byrne, 2001):
1. CMIN – Minimum Value of the Discrepancy (Brown, 1982).
2. CMIN/df – CMIN divided by degrees of freedom. Values over 3 indicate weakness in the
model (Carmines & McIver, 1981; Marsh & Hocevar, 1985).
3. AGFI – Adjusted Goodness of Fit Index (Arbuckle & Wothke, 1999; Tanaka & Huba, 1985).
Values above .95 are common in adequate models, and values below .90 indicate poorly
fitting models.
4. CFI – Comparative Fit Index (Bentler, 1990). Values near one indicate good fit. Values below
.90 indicate poor fit.
5. RMSEA – Root Mean Square Error of Approximation (Browne & Cudeck, 1993). In general,
values of .05 or less indicate well fitting models; values above .05 and below .08 indicate
usable but weak models; values greater than .10 indicate unusable models.
Confirmatory factor analysis is normally conducted on models with strong theoretical bases and with
the assumption that any exploratory factor analysis has been performed in the scale’s original
7
development. Such an assumption was made in this study relative to testing the integrity of constructs
for the PRCA-24. Exploratory analyses and assessment of misfit indices of confirmatory factor
analysis is generally not advised (see Cliff, 1983; Kline, 1998), and extensive analysis of scale
revision is well beyond the bounds of this study. However, given failure of models to adequately fit
theorized constructs, exploratory analysis was conducted to gain a sense of dimensionality and provide
reference for assessment of errant items as an aid for future research. Consistent with the confirmatory
technique used earlier, a principle factors approach was taken. Rotated factors were chosen based on
theoretical assumptions outlined in original scale development. Given that underlying dimensions can
and perhaps should be correlated because they theoretically measure one latent trait, an oblique
(PROMAX) solution was derived (see Gorsuch, 1983; Maruyama, 1998; Johnson & Wichern, 1998;
Tabachnick & Fidell, 2001).
In order to assess the ability of the PRCA-24 to consistently and inclusively measure an underlying
trait of communication apprehension, a one-parameter Rasch analysis was conducted on the data
(Research Question 3). Developments in item response theory, and especially the application of Rasch
methods, have provided a means of assessing the ability of psychological measures to provide valid
and continuous measures of underlying traits (see discussions in Hambleton, Swaminathan, & Rogers,
1991; Fischer & Molenaar, 1995; Van der Linden & Hambleton, 1997; Embretson, & Reise, 2000).
Further, while Rasch Analysis has generally been applied in educational testing, its applicability to
assess measurement properties of rating based instruments, especially those that have Likert based
responses, has been amply demonstrated (Wright, 1999; Wright & Masters, 1982; Bond & Fox, 2001).
A full explication of the underlying assumptions and ensuing controversies in Rasch analysis is far
beyond the scope of this article; readers unfamiliar with the basic tenets are urged to consult the
sources cited.
Rasch analysis was chosen in this study in order to determine how well items in the PRCA-24 cover a
continuum of varying levels of trait anxiety among subjects responding to the scale. While the
theoretical multi-dimensionality of the PRCA-24 would be a direct contradiction of the assumptions of
Rasch modeling, this assessment provides a viable means of the testing the contentions of Pribyl,
8
Keaten, Sakamoto, & Koshikawa (1998) that summed items may provide a good overall measure of
communication apprehension.
SPSS was used for outlier, reliability, correlational, and factor analysis procedures. Confirmatory
factor analysis was conducted using AMOS (Arbuckle, 1994; Arbuckle & Wothke, 1999). WinSteps
(Linacre, 2001) was used for one-parameter Rasch analysis.
9
Results
Internal Consistency - Reliability
A summary of descriptive statistics and measures of reliability for Japanese subjects in this study may
be seen in Table 1. The instrument and subscales showed strong α coefficients and overall high itemscale correlations. Measures of α for the individual subscales of the PRCA-24 ranged from .72 to .83.
Though some item-total correlations were low, these results were consistent with earlier findings on
both English and Japanese versions of the scales, thus indicating that such metrics could be replicated
with this data (Research Question 1).
Table 1. Summary of reliability and descriptive measures.
.901
Lowest Item
Scale
Correlation
.242
Highest
Item Scale
Correlation
.676
Common
Inter-Item
Correlation
.275
Public
.812
.475
.651
.419
20.43
26.26
5.12
Group
.798
.416
.684
.397
19.73
22.83
4.78
Meeting
.830
.435
.647
.449
19.35
24.38
4.94
Dyadic
.721
.394
.560
.301
16.47
18.90
4.35
Measure
Cronbach’s
PRCA-24
α
10
Mean
Var
SD
75.98 228.21 15.11
Structural Integrity – Confirmatory Factor Analysis
A summary of measures for confirmatory factor analyses of the scale may be seen in Table 2. Models
for the first order and second order factor structures involving the four subscales may be seen in
Figures 1 and 2.
Table 2. Summary of Confirmatory Factor Analysis measures
Scale
(Number of
Components)
PRCA-24 (1)
CMIN
df
CMIN/df
AGFI
CFI
RMSEA
1566.83
252
6.218
.633
.692
.122
PRCA-24 (4)*
11082.49
246
4.400
.738
.785
.099
PRCA-24 (4)**
1129.77
251
4.501
.736
.748
.100
* First Order
** Second Order
Assessment revealed weaknesses in all models. All RMSEA terms were greater than .05. The
PCLOSE statistic (Browne and Cudeck, 1993; Arbuckle and Wothke, 1999) tests the null hypothesis
that the population RMSEA is not significantly higher than .05. This was less than .001 for all models,
thus leading to rejection of that hypothesis for all. Relative to the findings of Levine and McCroskey
(1990), both the first order and second order models of the PRCA-24 using four subscales as factors
showed weak fit with this data. These results indicate that structural models either assumed or tested in
earlier development of the scale could not be replicated in this study with confidence (Research
Question 2).
11
Figure 1. First Order Confirmatory Factor Analysis – PRCA-24 with four subscales;
Standardized estimates.
.43
e1
PRCA01
e2
PRCA02
e3
PRCA03
.40
PRCA04
.46
.37
.65
.68
.61
e5
.63
.49
.24
.77
PRCA05.60
e6
PRCA06
e4
Groups
.78
.28
e7
PRCA07
e8
PRCA08.51
e9
PRCA09
.53
PRCA10
e10
e11
e12
.40
.51
PRCA11.57
PRCA12
.53
.64
.71
.73
.71
.75
Meetings
.55
.17
e13
PRCA13
e14
PRCA14
.32
e15
PRCA15.25
PRCA16
e16
e17
e18
.42
.16
PRCA17
.56
PRCA18
PRCA19
e20
PRCA20
e21
PRCA21
.39
PRCA22
e22
e23
e24
.26
.50
.56
PRCA23.27
PRCA24
.57
.41
.65
.56
.50
.82
Dyad
.40
.75
.38
.60
e19
.40
.78
.51
.70
.62
.75
.52
12
Public
Figure 2. Second Order Confirmatory Factor Analysis – PRCA-24 with four subscales;
Standardized estimates.
.36
e1
PRCA01
e2
PRCA02
e3
PRCA03
e4
PRCA04
e5
PRCA05
e6
PRCA06
.46
.37
.38
.24
.60
.60
.68
.61
R1
.62
.49
Groups
.77
.63
.66
e12
PRCA12
e11
PRCA11
e10
PRCA10
e9
PRCA09
.75
.41
.64
Meetings
PRCA08
.50
.78
e8
.50
.52
.56
.81
.71
R2
.72
.25
e7
.79
.88
PRCA07
CA
.17
e13
PRCA13
.42
R3
.42
.65
e14
PRCA14
e15
PRCA15
.55
.25
.50
Dyad
PRCA16
.42
.28
e16
.31
.17
e17
PRCA17
.74
.55
e18
PRCA18
R4
.56
e19
e20
e21
e22
e23
PRCA19
.25
PRCA20
.47
PRCA21
.39
PRCA22
.56
.75
.50
Public
.69
.77
.63
.75
.52
PRCA23
.27
e24
PRCA24
13
.53
.88
Structural Integrity - Exploratory Factor Analysis
A Bartlett’s χ 2 of 3665.855 (p<.001) and a KMO index of .878 were obtained for the data, thus
indicating above criterion levels for both sampling and factorability. Results of Principle Axis factor
analysis may be seen in Table 3. Relatively large discrepancies in communality estimates were found
on all measures indicating lack of association for some items. In addition, multicollinearity indicated
potential problems with redundancy.
Table 3. Summary of Principle Axis factor analysis
Components
Extracted
Variance
Explained by
Components
Lowest
Communality
(Item)
Highest
Communality
(Item)
Determinant
4
45.830
.289 (Item 16)
.685 (Item 9)
0.000
The four rotated factors for the PRCA-24 may be seen in Table 4. The first factor appears to have a
mix of loadings relative to both meeting and public situations. The second factor seems most close to
the group situation, and the third factor contains items related to dyads. The fourth factor includes
items from two of the theoretic situations and appears to relate to communication with “new” people.
14
Table 4. Personal Report of Communication Anxiety - Structure Matrix
Factor
1
2
3
4
PRCA19 Fear Giving Speech
.754
.491
.250
.259
PRCA23 Prospect Speech Confidence
.709
.432
.260
.175
PRCA12 Relaxed Questions Meetings
.705
.573
.295
.215
PRCA10 Afraid Express Meetings
.703
.531
.574
.114
PRCA21 Relaxed Giving Speech
.699
.371
.054
.162
PRCA09 Calm, Relaxed Express Opinion Meeting
.674
.554
.243
.206
PRCA11 Communicating Meetings Uncomfortable
.658
.561
.524
.087
PRCA22 Thoughts Confused, Jumbled Speech
.635
.373
.292
.155
PRCA24 Speech Nervous, Forget Facts
.514
.265
.338
.142
PRCA20 Body Tense, Rigid Speech
.499
.251
.184
.143
PRCA06 Calm, Relaxed Groups
.453
.792
.171
.299
PRCA07 Participate Meetings
.364
.686
.297
.313
PRCA02 Comfortable Groups
.413
.662
.255
.214
PRCA01 Participating Groups
.419
.652
.299
.095
PRCA04 Involved Groups
.360
.630
.156
.058
PRCA08 Calm, Relaxed Meetings
.525
.617
.271
.200
PRCA03 Tense, Nervous Groups
.384
.605
.312
.297
PRCA18 Speak Up Conversation
.309
.299
.751
.204
PRCA15 Tense, Nervous Conversations
.161
.184
.649
.200
PRCA14 Fear Speaking Conversations
.396
.310
.533
.261
PRCA16 Calm, Relaxed Conversations
.128
.135
.526
.192
PRCA13 Conversation New Acquaintance
.160
.214
.316
.691
PRCA17 Conversing New Acquaintance Relaxed
.340
.287
.256
.647
PRCA05 Groups New People Tense, Nervous
.364
.492
.239
.510
Response Validity - Rasch Analysis
Results of goodness of fit for items in the Rasch model may be seen in Table 5. Looking at both infit
and outfit measures, several scale items (note especially 6, 8, 11, 12, 13, 15, 16, 20, 24) show
inadequate fit to a purported continuous measure of the underlying trait.
15
Table 5. Measures of Goodness of Fit to Rasch Model
+------------------------------------------------------------------------+
|ENTRY
RAW
|
INFIT | OUTFIT |SCORE|
|
|NUMBER SCORE COUNT MEASURE ERROR|MNSQ ZSTD|MNSQ ZSTD|CORR.| ITEMS |
|------------------------------------+----------+----------+-----+-------|
|
1
1062
347
.11
.06| .97
-.4|1.01
.2| .53| PRCA01|
|
2
1084
347
.04
.06| .88 -1.9| .89 -1.7| .54| PRCA02|
|
3
1121
347
-.07
.06|1.00
.1|1.02
.2| .53| PRCA03|
|
4
1123
347
-.08
.06| .94
-.9| .99
-.1| .47| PRCA04|
|
5
1255
347
-.51
.06|1.21
2.8|1.22
2.8| .50| PRCA05|
|
6
1193
347
-.30
.06| .75 -3.9| .75 -3.9| .58| PRCA06|
|
7
1125
347
-.08
.06| .86 -2.1| .87 -1.9| .55| PRCA07|
|
8
1116
347
-.06
.06| .69 -5.1| .73 -4.2| .57| PRCA08|
|
9
1264
347
-.54
.06| .87 -1.9| .91 -1.3| .59| PRCA09|
|
10
991
347
.33
.06| .86 -2.2| .84 -2.4| .64| PRCA10|
|
11
985
347
.34
.06| .77 -3.8| .77 -3.6| .63| PRCA11|
|
12
1225
347
-.40
.06| .66 -5.5| .67 -5.2| .62| PRCA12|
|
13
1163
347
-.20
.06|1.36
4.6|1.40
5.1| .36| PRCA13|
|
14
1018
347
.24
.06|1.05
.8|1.07
1.0| .48| PRCA14|
|
15
729
347
1.18
.06|1.30
3.8|1.50
5.6| .38| PRCA15|
|
16
782
347
.99
.06|1.63
7.4|1.74
8.1| .32| PRCA16|
|
17
1175
347
-.24
.06|1.01
.1|1.04
.6| .44| PRCA17|
|
18
833
347
.82
.06| .98
-.3|1.00
.0| .47| PRCA18|
|
19
1239
347
-.45
.06| .91 -1.4| .90 -1.4| .61| PRCA19|
|
20
1114
347
-.05
.06|1.39
5.2|1.44
5.5| .44| PRCA20|
|
21
1275
347
-.57
.06|1.06
.8|1.08
1.0| .51| PRCA21|
|
22
1163
347
-.20
.06|1.01
.1|1.03
.5| .53| PRCA22|
|
23
1188
347
-.28
.06| .90 -1.5| .88 -1.7| .57| PRCA23|
|
24
1105
347
-.02
.06|1.20
2.7|1.25
3.4| .47| PRCA24|
|------------------------------------+----------+----------+-----+-------|
| MEAN
1097.
347.
.00
.06|1.01
-.1|1.04
.3|
|
|
| S.D.
143.
0.
.45
.00| .23
3.2| .26
3.3|
|
|
+------------------------------------------------------------------------+
A graphical comparison of Rasch score distributions for subjects (persons) and items is presented in
Figure 3. Perusal of this comparison shows that the PRCA-24 is, overall, an “easy” measure in Rasch
terms. That is, with the exception of some items on the dyadic dimension (15, 16, 18), full coverage of
higher levels of the anxiety trait appears inadequate. Japanese respondents find it easy to agree with
the scale’s items, i.e., easy to indicate high levels of apprehension. This corresponds with previous
studies that have shown Japanese subjects to score consistently higher on the PRCA-24 than their
counterparts in the U.S. (see again McCroskey, Gudykunst, & Nishida, 1985; Klopf, 1984). Weakness
in the ability of the scale to adequately measure higher levels of the trait is further seen in Figure 4.
Here, item scores are plotted by increasing levels of the underlying trait.
16
Figure 3. Comparison of Person and Item Rasch Scores of PRCA-24.
4
3
2
1
0
-1
-2
-3
EACH
PERSONS MAP OF ITEMS
<more>|<rare>
. +
|
|
|
|
|
|
|
+
|
|
|
. |
|
. |
|
+
|
. |
. |
. T|
## |
#### |
.# | PRCA15
.### + PRCA16
.##### S|T PRCA18
.### |
###### |
.###### |S
######### | PRCA10
.######### | PRCA14
.########## M| PRCA01
.########## +M PRCA02
.######## | PRCA03
###### | PRCA06
.###### | PRCA12
.#### S|S PRCA05
.### | PRCA21
.### |
. |T
## +
# |
. T|
|
# |
. |
|
. |
+
|
|
# |
|
|
|
|
. +
<less>|<frequ>
'#' IS 3.
17
PRCA11
PRCA08
PRCA04
PRCA13
PRCA20
PRCA07
PRCA17
PRCA09
PRCA19
PRCA24
PRCA22
PRCA23
Figure 4. Continuity of PRCA items in measuring underlying trait apprehension
PRCA Items Sorted by Increasing Difficulty of Agreement
1.2
1
0.8
Difficulty
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
21 9
5 19 12 6 23 17 13 22 4
7
3
8 20 24 2
Item Number
18
1 14 10 11 18 16 15
Discussion
Several conclusions relative to a translated version of the PRCA-24 seem apparent from the results of
this study. First, while the translated scale shows internal consistency as measured by classical
reliability statistics, further analysis indicates weakness in terms of structural integrity and the
underlying theoretical assumption of four dimensions. Second, the ability of the scale to be used as an
overall measure of anxiety must be questioned. The ability of the scale to adequately cover and
discriminate higher levels of trait anxiety, i.e., the lack of items for which subjects find it more
difficult to agree, appears to be a prime weakness of the scale. Obviously, results of this study indicate
that care should be taken in using summed values of this scale for research or clinical assessment in
Japan.
Why does a Japanese version of the PRCA-24 show weakness in terms of clarity and coverage of
theoretical and internal constructs related to the trait of communication apprehension? Translation or
measurement anomaly cannot be eliminated as a source of error variation, but the high measures of
reliability indicate that reasons may lie elsewhere. As noted earlier, a stable structure for this scale has
been elusive, even in its English form. Thus, some aspects of model misfit in this study could be due
to instabilities or inadequacies in the theoretical foundation of the construct itself. For example, the
Dyadic dimension of the PRCA-24 shows lower item-total correlations as well as lower correlations in
both the first and second order confirmatory factor analysis models (Figures 1 and 2). This subscale
has also appeared to be a somewhat less stable factor than other subscales in prior research on the
English version (McCroskey & Beatty, 1984; McCroskey, 1990, 1992; Levine and McCroskey, 1990).
On the other hand, as seen in results of Rasch analysis, only items of this theoretic dimension appear
to have any ability to measure higher levels of communication apprehension. Obviously, for Japanese
subjects, augmentation of the scale to include constructs involving communication situations that
involve lower anxiety, and thus are more difficult to agree with, appears to be necessary in order for
the measure to adequately cover the a continuous range of the apprehension trait and sub-dimensions.
Results of the exploratory factor analysis of the PRCA-24 parallel those of Pribyl, Keaten, Sakamoto,
& Koshikawa (1998). As in their results, items related to meetings appear to load highly on the public
19
dimension (again see Table 4). Japanese terms for “meeting” can include aspects of both group and
public situations. Some “meetings” in Japan frequently involve formal presentations by participants
rather than discussion while other “meetings” involve strong levels of interpersonal contact. For
example, a regular “meeting” of full faculty at a university may require presentation skills on par with
a public speaking situation while what is still termed a “meeting” of a small committee may involve
much more interpersonal and less formal interaction. The grouping of items 17, 13, and 5 imply that
interacting with a “new” person or someone not within ones sphere of acquaintance may be the basis
of a state situation in itself and transcend the PRCA based contexts in which such interaction occurs.
Modification or construction of items reflecting these differences awaits not only further empirical
research in these specific interactions but in development of appropriate descriptive terms.
Pribyl, Keaten, Sakamoto, & Koshikawa (1998) have recommended discarding items related to the
public dimension since this form of communication is more rare in Japan than in the United States.
However, such seems rather drastic given the strong loading of items on this dimension and the fact
that such removal would destroy the historical and theoretical base of the scale. Further, the Rasch
analysis would indicate that if anything, items need to be added to the scale to adequately cover
situations which lie somewhere between conversations on the dyadic level and communication
situations involving more public performance. Overall, additions or different groupings of items that
maintain or enhance the fundamental theoretical situations and also account for different types of
communication in each such as performance, participation, conversation, or social adjustment
(establishing appropriate communication patterns with a new acquaintance) may be appropriate.
Clarification of the theoretical base for understanding various communication situations and terms
describing these situations seems necessary in future development of the PRCA and similar anxiety
measures for use in Japan. At present, a theoretical base for such is somewhat weak and lacking in
strong empirical support. Objective and empirical research into various forms of anxiety and modes of
communication to be measured in the PRCA may lead to further determination of appropriate
modifications or additions of items in this still potentially valuable instrument.
20
Conclusion
That future research with the PRCA-24 in its present form should be conducted with caution seems
evident. However, though it may be easy to factor analyze, delete, or force items of the PRCA-24 onto
a structure that has no clear theoretical base, such has been avoided in this study for obvious reasons.
Instead, these results should provide a base for reevaluating the theoretic base of the PRCA-24 as it
reflects communication apprehension in Japanese culture and should also aid in determining which
items should be modified, deleted, or augmented in order to produce the most parsimonious,
theoretically rich, and culturally independent measure.
21
Appendix
このアンケートの答え方：この質問表は人とのコミュニケーションに関するあなたの気持ちを尋ねる
２４の項目から構成されています。各項目の内容があなたにどの程度あてはまるのかを次の（１）ま
ったくその通りだと思う、（２）その通りだと思う、（３）どちらとも言えない、（４）そうは思わ
ない、(５)まったくそうは思わない、の中から選択し、各項目の前にある空欄に１から５までの数字
を記入して下さい。正しい答えや間違った答えといったものはありません。よく似た記述が繰り返さ
れても心配しないで下さい。時間をかけずに取り組み、あなたの第一印象を記入して下さい。
１.
２.
３.
４.
５.
６.
７.
８.
集団討論に参加することが好きではない。
たいてい、集団討論に参加している時は居心地がよく落ち着いた気持ちでいられる。
集団討論に参加している時は緊張して神経を使う。
集団討論に取り組むことが好きだ。
新しい人たちと集団討論を行うと緊張して神経質になる。
集団討論に参加している時は落ち着いてリラックスしている。
たいてい、会議やミーティングに参加しなければならない時は神経を使う。
いつも、会議やミーティングに参加している時は落ち着いた気持ちでリラックスしてい
る 。
９. 会議やミーティングで意見を述べるように求められた時、とても落ち着いた気持ちで
リラックスしている。
１０. 会議やミーティングで自分の意見を述べることが怖い。
１１. 会議やミーティングでの意見交換はいつも居心地が悪く落ち着かない。
１２. 会議やミーティングで質問に答える時、私はとてもリラックスしている。
１３. 新しく知り合った人と会話する時はとても神経を使う。
１４. 会話の途中で自分の意見を述べることにまったく怖さを感じない。
１５. 普段、会話する時はとても緊張し神経を使う。
１６. 普段、会話する時はとても落ち着いてリラックスしている。
１７. 新しく知りあった人と会話する時はとてもリラックスしている 。
１８. 会話をしている時に自分の意見を述べることに怖さを感じる。
１９. スピーチをすることはまったく怖くない。
２０.
２１.
２２.
２３.
スピーチをしている時、身体の一部分が緊張して硬直するように感じる。
スピーチをしている時、リラックスした気持ちでいる。
スピーチをしている時、頭の中が混乱し考えがまとまらない。
スピーチをしなければならない状況を前に、自信を持ってスピーチをする心の準備
ができている。
２４. スピーチをしている時、あまりにも神経が高ぶり、よく知っていることでさえも忘
れてしまう。
The following items are reversed: 2, 4, 6, 8, 9, 12, 14, 16, 17, 19, 21, 23
Subscales are calculated from raw scores as follows (instrument item number in parenthesis):
Group = 18 - (1) + (2) - (3) + (4) - (5) + (6)
Meeting = 18 - (7) + (8) - (9) + (10) - (11) + (12)
Dyadic = 18 – (13) + (14) – (15) + (16) – (17) + (18)
Public = 18 + (19) – (20) + (21) – (22) + (23) – (24)
Overall = Group + Meeting + Dyadic + Public
22
References
Arbuckle, J.L. (1994) AMOS: Analysis of moment structures. Psychometrika, 59, 135-137.
Arbuckle, J. L. & Wothke, W. (1999) Amos 4.0 user’s guide. Chicago: SmallWaters Corporation.
Bentler, P. M. (1990) Comparative fit indexes in structural models. Psychological Bulletin, 107, 238246.
Bond, T. G. & Fox, C. M. (2001) Applying the Rasch Model. London: Lawrence Erlbaum Associates.
Browne, M. W. (1982) Covariance structures. In D. M. Hawkins (Ed.) Topics in applied multivariate
analysis (pp. 72-141). Cambridge: Cambridge University Press.
Browne, M. W. & Cudeck, R. (1993) Alternative ways of assessing model fit. In K. A. Bollen & J. S.
Long (Eds.) Testing structural equation models (pp. 136-162). Newberry Park, CA: Sage.
Byrne, B. M. (2001) Structural equation modeling with AMOS. London: Lawrence Erlbaum
Associates.
Carmines, E. G. & McIver, J. P. (1981) Analyzing models with unobserved variables. In G. W.
Bohrnstedt & E. F. Borgatta (Eds.), Social measurement; Current issues. London: Sage.
Carmines, E. G. & Zeller, R. A. (1979) Reliability and validity assessment. London: Sage
Publications.
Cliff, N. (1983) Some cautions concerning the application of causal modeling methods. Multivariate
Behavioral Research, 18, 115-126.
Daly, J. A. & McCroskey, J. C. (Eds.) (1984) Avoiding communication: Shyness, reticence, and
communication apprehension. Beverly Hills, CA: Sage
Embretson, S. E. & Reise, S. P. (2000) Item Response Theory for psychologists. London: Lawrence
Erlbaum Associates.
Fischer, G. H. & Molenaar, I. W. (Eds.) (1995) Rasch Models: Foundations, recent developments, and
applications. New York: Springer-Verlag.
Gorsuch, R. L. (1983) Factor analysis. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991) Fundamentals of Item Response Theory.
London: Sage Publications.
John, O. P. & Benet-Martínez, V. (2000) Measurement: reliability, construct validation, and scale
construction. In H. T. Reis & C. M. Judd (Eds.) Handbook of research methods in social and
personality psychology (pp. 339-369). Cambridge: Cambridge University Press.
Johnson, R. A. & Wichern, D. W. (1998) Applied multivariate statistical analysis, Upper Saddle
River, NJ: Prentice-Hall.
Judd, C. M. & McClelland, G. H. (1998) Measurement. In D. T. Gilbert, S. T. Fiske, & G. Lindzey
(Eds.) The handbook of social psychology (4th Ed., Vol. 1, pp. 180-232). Boston: McGrawHill.
23
Kline, R. B. (1998) Principles and practice of structural equation modeling. New York: The Guilford
Press.
Klopf, D.W. (1984). Cross-cultural apprehension research: A summary of Pacific Basin studies. In J.
A. Daly & J. C. McCroskey (Eds.), Avoiding communication: Shyness, reticence, and
communication apprehension (pp. 157-172). Beverly Hills, CA: Sage.
Kondo, S. (2001) The perceived effectiveness of anxiety coping strategies in social situations. Human
Communication Studies, 29, 1-9.
Leary, M. R. (1983) Social Anxiousness: The Construct and Its Measurement. Journal of
Personality Assessment, 47, 66-75.
Leary, M. R. (1991) Social anxiety, shyness, and related constructs. In J. P. Robinson, P. R. Shaver, &
L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 161194). New York: Academic Press.
Leary, M. R. & Kowalski, R. M. (1987) Manual for the Interaction Anxiousness Scale. Social and
Behavioral Sciences Documents, 16, 2, Manuscript No. 2774.
Leary, M. R. & Kowalski, R. M. (1993). The Interaction Anxiousness Scale: Construct and criterionrelated validity. Journal of Personality Assessment, 61, 136-146.
Levine, T. R., & McCroskey, J. C. (1990). Measuring trait communication apprehension: A test of
rival measurement models of the PRCA-24. Communication Monographs, 57, 62-72.
Linacre, J. M. (2001) A user’s guide to WINSTEPS. Chicago: MESA Press.
Marsh, H. W. & Hocevar, D. (1985) Application of confirmatory factor analysis to the study of selfconcept; First and higher order factor models and their invariance across groups.
Psychological Bulletin, 97, 562-582.
Maruyama, G. M. (1998) Basics of structural equation modeling. London: Sage Publications.
McCroskey, J. C. (1970) Measures of communication-bound anxiety. Speech Monographs, 37, 269277.
McCroskey, J.C. (1978). Validity of the PRCA as an index of oral communication apprehension.
Communication Monographs, 45, 192-203.
McCroskey, J. C. (4th Ed.). (1982) An introduction to rhetorical communication, Englewood Cliffs,
New Jersey: Prentice-Hall.
McCroskey, J. C. (1992) Reliability and validity of the Willingness to Communicate Scale.
Communication Quarterly, 40, 16-25.
McCroskey, J. C., & Beatty, M. J. (1984) Communication apprehension and accumulated
communication state anxiety experiences: A research note. Communication Monographs, 51,
79-84.
McCroskey, J. C., Beatty, M. J., Kearney, P., & Plax, T. G. (1985) The content validity of the PRCA24 as a measure of communication apprehension across communication contexts.
Communication Quarterly, 33, 165-173.
24
McCroskey, J.M., Gudykunst, W.B., & Nishida, T. (1985). Communication apprehension among
Japanese students in native and second language. Communication Research Reports, 2, 11-15.
McDowell, E. E. and Yotsuyanagi, N. (1996) An Exploratory Study of Communication Apprehension,
Willingness To Communicate, and Sense of Humor between College Students from the United
States and Japan. (ERIC Document Reproduction Service No. ED396349)
Miller, R. S. (1987) The Nature of Embarrassability; Correlates and Sex Differences.
Unpublished Manuscript, Sam Houston State University, Huntsville, Texas.
Modigliani, A. (1968) Embarrassment and Embarrassability. Sociometry, 31, 313-326.
Pribyl, C. B., Keaten, J. A., Sakamoto, M., & Koshikawa, F. (1998) Assessing the cross-cultural
content validity of the Personal Report of Communication Apprehension Scale (PRCA-24).
Japanese Psychological Research, 40, 47-53.
Richmond, V. P. & McCroskey, J. C. (5th Ed.). (1998) Communication apprehension, avoidance, and
effectiveness. Boston: Allyn and Bacon.
Tabachnick, B. G. & Fidell, L. S. (2001) Using multivariate statistics, Needham Heights, MA: Allyn
and Bacon.
Tanaka, J. S. & Huba, G. J. (1985) A fit index for covariance structure models under arbitrary GLS
estimation. British Journal of Mathematical and Statistical Psychology, 38, 197-201.
Van der Linden, W. J. & Hambleton, R. K. (Eds.) (1997) Handbook of modern Item Response Theory.
New York: Springer-Verlag.
Watson, D. & Friend, R. (1969) Measurement of social evaluative anxiety. Journal of Consulting and
Clinical Psychology, 33, 448-457.
Wright, B. D. (1999) Fundamental measurement for psychology. In S. E. Embretson and S. L.
Hershberger (Eds.) New Rules of Measurement; What Every Psychologist and Educator
Should Know (pp. 65-104). London: Lawrence Erlbaum Associates.
Wright, B. D. & G. N. Masters (1982) Rating scale analysis; Rasch measurement. Chicago: MESA
Press.
25