Validation of the Electronic Compassion Competence Scale: Paper-and-Pencil versus Web-Based Questionnaires

Article information

J Korean Acad Fundam Nurs. 2023;30(3):382-390
Publication date (electronic) : 2023 August 31
doi :
1)Associate Professor, College of Nursing ․ Research Institute of Nursing Science, Jeonbuk National University, Jeonju, Korea
2)Associate Professor, College of Nursing ․ Research Institute of Nursing Science, Ajou University, Suwon, Korea
3)Assistant Professor, College of Nursing ․ Research Institute of Nursing Science, Pusan National University, Yangsan, Korea
Corresponding author: College of Nursing, Ajou University 164 World cup-ro, Yeongtong-gu, Suwon 16499, Korea Tel: +82-31-219-7011, Fax: +82-31-219-7020, E-mail:
*This study was financially supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Science and ICT (NRF-2017R1C1B2012357).
Received 2023 April 25; Revised 2023 August 6; Accepted 2023 August 18.

Trans Abstract


This study aimed to determine the validity and reliability of the electronic Compassion Competence Scale (e-CCS).


A cross-sectional, randomized, two-period crossover design was used. Nursing students from four South Korean universities were surveyed between June 2017 and April 2018. The participants were randomly assigned to one of three groups-paper/paper, electronic/electronic, and paper/electronic or electronic/paper-and a test-retest procedure was implemented. The reliability and validity of the e-CCS were evaluated using linear weighted kappa coefficients and intraclass correlation coefficients (ICCs). Internal consistency reliability was verified using linear weighted kappa coefficients and ICCs. Pearson's correlation coefficients between the initial test and retest scores were all statistically significant.


The newly developed e-CCS was found to have good reliability and validity. We suggest that future research should increase sample heterogeneity by recruiting diverse age groups, nurses working in different nursing fields, and students from multiple colleges.


This electronic instrument will help determine the differences in the level of compassion competence and devise interventions to improve compassion competence in nurses and nursing students. Further studies on enhancing compassion competence among nurses and nursing students may rely on the use of this electronic format.


Compassion in nursing care extends beyond empathy for patients’ suffering and seeks to empower them through everyday nursing practices [1]. Compassionate care, which includes active reactions to patients’ needs based on an understanding of their physical, mental, and emotional pain [2], is among the most valuable nursing qualities and skills [3]. Therefore, compassion may be a crucial factor in a patient's satisfaction with nursing care [4]. A previous study reported that patients’ level of satisfaction with the medical care provided was greater when they felt that they were being shown more compassion by nurses [5]. Additionally, compassion was reported to be the key factor that influenced the therapeutic relationship between patients and nurses and was found to play a crucial role in the provision of high-quality nursing care [6-11]. However, it has been reported that a lack of compassion resulted in an increase in missed nursing care [12] and a decrease in activities related to patient-centered positive experiences [11,14]. Therefore, more attention is being paid to nurses’ compassion competence and its improvement.

The increasing emphasis on compassion for patients can be ascertained from instruments that have recently been developed to measure it [15]. Various tools that measure compassion in the field of healthcare involve reports from patients’ perspectives [16]. The Compassionate Care Assessment Tool [17], Compassion Scale [18], Schwartz Center Compassionate Care Scale [19], and Sinclair Compassion Questionnaire [13] were developed to measure patients’ compassion. Currently, few scales can measure compassion from the perspective of healthcare professionals, especially nurses. Durkin et al. [20] conducted a systematic review of scales that can be used to measure compassion among nurses, nursing students, educators, and patients and chose three scales developed in the United States and one in Korea. Two scales have been developed to measure compassionate care as perceived by patients [17,18]. More specifically, the Calm, Compassionate Care Scale [21] focuses on the measurement of caregivers’ (e.g., dieticians, physicians, social workers, and nurses) confidence in providing compassionate care. The Compassion Competence Scale (CCS) [22] was developed as a self-report questionnaire that targeted nurses working in patient-facing nursing fields, and comprised 17 questions across three factors: communication, sensitivity, and insight [22]. The scale was developed based on the premise that compassion is a spe-cial ability employed by nurses when providing nursing care and has been used in studies that evaluate the relationship between compassion competence and nurses’ professional quality of life [17] and caring behavior [18]. It was developed in a paper-and-pencil format and psycho-metrically validated [15]. Unlike other scales that measure compassion, the CCS [9] focuses primarily on bedside nursing care that could be practiced and measured from the perspective of professional nurses. Given that compassionate care is no longer an option, but a patient's need that must be met, it is important to interpret compassion as a competency.

Recently, several researchers have begun to prefer electronic questionnaires to the traditional paper-and-pencil ones, as the former are easier for participants to access via electronic devices such as computers, tablets, and smart-phones [23]. Electronic questionnaires can be personalized by programming various functions, depending on the research protocol and target participants. Moreover, the data can be entered automatically in real time as the partic-ipant progresses through the survey, thereby eliminating errors that could occur if the data were to be manually entered. Electronic questionnaires are also less vulnerable to time and space constraints, thereby improving the convenience, speed, and accuracy of data input [23,24].

In keeping with this trend, recent studies on nurses worldwide have investigated the use of electronic surveys [25-27]. One of them compared survey methods used in studies on the nursing workforce and indicated that the advantages of Internet-based surveys include broader dissemination, greater accessibility, faster completion, and more rapid analysis [28]. However, it also pointed out dis-advantages such as lower response rates compared to traditional surveys and challenges in obtaining a representative sample [28]. Therefore, researchers must carefully consider the research topic, and study the population, cost, and timing when deciding on a method for obtaining survey data [28].

However, there are various concerns regarding electronic questionnaires. Even if an electronic questionnaire contains the same items as a paper-and-pencil questionnaire, there have been reports suggesting that the results may not be identical due to factors such as the device used for data collection, participants’ response styles, time spent on the questionnaire, and participants’ proficiency in using the device [23]. Therefore, before using an electronic questionnaire in an actual study, its validity and reliability should be verified by comparing the results obtained from the original tool. Additionally, to mitigate potential variations in responses based on the two survey sequences, this study employed a crossover design and administered two different survey methods to the subjects. This allowed for a comparison of the results between the groups to explore any potential differences in the use of paper and electronic surveys. Thus, this study aimed to develop an electronic questionnaire based on the paper-based CCS (p-CCS) and to verify the reliability and validity of the electronic CCS (e-CCS) through the comparison and analysis of the results.


1. Study Design

This study was designed as a randomized, two-period crossover study to develop and validate the e-CCS. The CCS was administered twice to three different groups for comparison: (a) two rounds of the paper-and-pencil format (Group A), (b) two rounds of the electronic format (Group B), and (c) electronic/paper mixed group, that is, the pa-per-and-pencil format followed by the electronic format, followed by the paper-and-pencil format (Group C).

2. Development of the e-CCS

The original scale, p-CCS [22], comprises 17 questions encompassing three primary factors of compassion competence: communication, sensitivity, and insight. To adapt the p-CCS into an electronic format, the e-CCS was developed based on the paper-and-pencil version and implemented using the online survey platform, SurveyMonkey. The interface was designed to present all items of the e-CCS to the participants. In addition to the compassion competence assessment, sociodemographic information such as age, university attended, and years of study was collected from the participants.

3. Participants

This study involved nursing students from four universities located in different regions of South Korea, including the Metropolitan area, South Chungcheong, and North Chungcheong. The sample size for scale validation was determined to be about five times the total number of items [29]. Since the electronic compassion competence scale (e-CCS) consists of 17 questions adapted from the original questionnaire, a desirable sample size of 85 subjects was expected for each group, totaling 255 participants. To account for a potential dropout rate of 33% during the retest, a total of 381 questionnaires were planned for distribution. Out of the 376 nursing students who registered and completed the first survey, accounting for a response rate of 98.2%, a total of 135 students opted not to participate in the subsequent survey round. This resulted in a dropout rate of 35.9%. The final sample included 241 participants (n=241): 71 in Group A (paper-and-pencil after paper-and-pencil group), 68 in Group B (electronic after electronic group), and 102 in Group C (paper-and-pencil after electronic group or electronic after paper-and-pencil group).

4. Data Collection and Procedure

The study was advertised on bulletin boards and university websites. Data were collected between June 2017 and April 2018. Students who agreed to participate were randomly assigned to three groups using a computerized random number generator. Information about the study and consent forms were provided by the researcher to the participants, who were first assigned to complete the pa-per-and-pencil format. A link was sent to the participants who were assigned to complete the electronic format. The link enabled the participants to access the questionnaire on a computer, smartphone, or any other Internet-based device. The study information and consent forms were provided in a PDF file before the students completed the electronic survey. The participants provided written or online consent before completing the questionnaire. Their phone numbers were collected to send them the survey link and a gift card upon the completion of the survey. This personal information was managed in a master file, separately from the datasets used for the analysis.

Following the initial survey, a minimum interval of two weeks was required before allowing the participants to complete the survey again [29]. For example, participants in Group A initially completed the paper-and-pencil format of the questionnaire and were then requested to fill out the next paper-and-pencil form two weeks later.

5. Data Analysis

SPSS (version 25.0; IBM Corp., Armonk, NY, USA) was used for data analysis. The demographic characteristics of each group were analyzed using descriptive statistics (e.g., frequency and percentage). The means and standard deviations of the scores were also analyzed using descriptive statistics. Linear weighted kappa coefficients were used to analyze the measurement agreement of item-by-item differences between the test and retest. Intraclass correlation coefficients were calculated to determine the internal consistency and reliability of each group. Finally, to evaluate convergent validity, we conducted Pearson's correlation coefficient analysis for the relationship between e-CCS and CLS.

6. Ethical Considerations

Participants selected on a voluntary basis among students at the school where the principal investigator works were involved in the development and validation of the questionnaire. Before the recruitment, the study received Institutional Review Board approval (IRB No. KNUTIRB-44). Students had the opportunity to provide feedback on the questionnaire and express freely their priorities, experiences, and preferences on the content of the questionnaire, including their views on its acceptability and specific written feedback on how to improve it.


1. Demographic Characteristics

Table 1 presents participants’ demographic characteristics. Most participants were women (84.6%) and sopho-mores (69.7%). Out of the total participants, 164 students (68.0%) were from two universities located in the Metropolitan area, while 77 students (32.0%) were from two universities in the other area. Seventy-one (29.5%) students were assigned to Group A, 68 (28.2%) to Group B, and 102 (42.3%) to Group C.

Demographic Characteristics of the Participants (N=241)

2. Reliability

To verify the consistency of the test-retest reliability, Table 2 shows the results of the item-by-item difference in the calculation of the linear weighted kappa coefficients. Notably, the test-retest results of Group A (paper-paper) and Group B (electronic-electronic) showed the reproduci-bility of each version (paper or electronic). Additionally, the results of Group C were used to verify the consistency of the results of the different versions; notably, the electronic version of the agreement was measured in the same manner as the paper version. The results of the linear weighted kappa coefficients confirmed that the consistency between the test and retest of each item was significant in all three groups.

Item-by-Item Differences between Pretest and Posttest (Linear Weighted Kappa Coefficients) (N=241)

To verify the test-retest internal consistency reliability between the groups, intraclass correlation coefficients were calculated (Table 3). The intraclass correlation coefficients were higher than.700, except for sensitivity in Group A (.66), insight in Group B (.67), and communication (.68) and insight (.69) in Group C. Despite a few low intraclass correlation coefficients, the differences between groups were not statistically significant (Table 3).

Mean Difference between Pretest and Posttest of Each Group (N=241)

3. Convergent Validity

To establish the convergent validity, we performed a Pearson's correlation coefficient analysis between the CCS and CLS within each of the three groups. The analysis revealed a statistically significant correlation in all three groups (p<.001) (Table 4).

Convergent Validity of the CCS with the CLS (N=241)


The original CCS was developed to measure nurses’ ability to provide compassionate care, and several studies have focused on how original the tool is in the measurement of compassion from the caregiver's perspective [13,16,20]. This study developed the e-CCS and verified its validity and reliability by analyzing the differences between the e-CCS and the original instrument. Due to the continuous commercialization of Internet-based devices, this study was conducted to improve the accessibility and convenience of questionnaire participation by addressing space and time-related constraints.

In most previous studies, participants were allocated to two groups; for example, a paper format followed by an electronic format or vice versa, and a test-retest procedure was implemented [23,30-32]. However, in this study, the newly developed electronic format of the questionnaire was validated more carefully through the implementation of the test-retest procedure in three groups: paper/paper format, electronic/electronic format, and paper/electronic format. This study design was chosen because the validity and reliability of the original scale were verified through a survey of registered nurses working in tertiary care hospitals, whereas this study was conducted among nursing students. This study also examined whether the instrument was valid for nursing students who were preparing to become nurses by learning theories and practicing nursing skills in college. As compassion has been rec-ognized an important ability for nurses, it needs to be addressed and taught more extensively in nursing studies curricula. Therefore, to make necessary improvements to the nursing education program, it is necessary to identify potential deficiencies in the existing nursing education curriculum through the measurement of the level of compassion competence in both nurses and nursing students.

The crossover design implemented in this study is commonly used in experimental research to assess the effects of different treatments [33]. Previous studies have utilized this design to compare paper and electronic surveys as distinct treatments, examining the reliability and validity of electronic surveys in particular [23,32]. In this research design, usually both methods (paper and electronic surveys) were administered to two groups. One group received a paper survey first and then an electronic survey, while the other group received the surveys in the opposite order. The differences in the results between the two groups are then analyzed to determine if there are variations in how the tools are utilized [23]. Alternatively, pop-ulations with different characteristics can be categorized into groups to explore differences in the results obtained from paper and electronic surveys [32]. These studies, em-ploying the same research design to compare paper and electronic surveys, have found no significant differences between the two methods [23,30,32], and have highlighted the advantages of electronic surveys [32]. In this study, a crossover design was adopted to develop the CCS from a paper format to an electronic format. Although the specific effect of administering the surveys in different orders was not explicitly examined in this study, it allowed for a comparison of the differences between groups that received either paper-paper, electronic-electronic, or paper-electronic surveys. The findings revealed no statistically significant difference in the use of paper and electronic formats among nursing students, indicating that both methods can be effectively utilized in various contexts.

The results of this study confirmed the reliability and validity of the instrument in all the groups. This could be because more than 60% of the participants were sopho-mores who were still learning about interpersonal relationships, as opposed to professional nurses, who had deep compassion for their patients and were trained to practice compassion. Additionally, the overall compassion competence score of the nursing students in this study was higher than 3.75 among groups, with 3.70 or more on each of the factors. This score was higher than the nurses’ score ranging between 3.50 and 3.60, as reported in previous studies [9,22]. Nevertheless, the somewhat lower reliability scores suggest that deviations among nursing students’ compassion competence levels may be greater than those among professional nurses. Insight measures the deep understanding of patients based on the nurses’ clinical experience and professional knowledge. Therefore, the reliability of the insight factor might have been lower among students owing to their lack of experience. This may be a limitation of this study, which thus underscores the need to expand the pool of participants in future research. In addition, there were groups with low scores in communication and sensitivity, and this difference is believed to result, at least to some extent, from differences in the composition of the groups. The three groups in this study were not completely homogeneous in composition by grade and location of the universities. Therefore, future studies should analyze the differences in the factors of compassion competence and the reasons for the difference with homogeneous groups.

It will also be helpful to compare the results after the elimination of the outliers and analyze in detail the causes of the existence of these outliers in order to understand the factors that affect the compassion competence of nurses and nursing students. Additionally, 135 of 376 participants dropped out of the study, which may have impacted the results because differences among groups may have occurred depending on the number of dropouts in the group and their demographic characteristics. Further analysis is required to confirm the potential effects of these factors. Despite the limitations of this study, the reliability of the instrument was high in all groups, which confirms that both the paper and electronic formats are reliable for the measurement of compassion competence.

In line with previous studies [24,30], the current results showed no significant differences between the original p-CCS and the newly developed e-CCS. Convergent validity was verified by testing the correlations between the paper and electronic formats of CCS; high correlations were found for each group individually. Furthermore, the intraclass correlation coefficients showed no significant differences between the two instruments, which suggested no difference in internal reliability between the instruments. Based on the findings of this study, it is recommended that future research replicates the study by measuring compassion competencies among nursing students and nurses in schools and hospitals across different regions of the country. Utilizing electronic surveys would enable data collection without the limitations of time and location restrictions. This approach would provide a broader perspective and enhance the generalizability of the results.

The convergent validity of the CCS was verified through correlation analysis using the CLS [34]. The CLS is a commonly used tool for the assessment of compassion and has been used to measure compassion in nurses [34] and healthcare professionals [35]. In this study, the CLS was used as the comparator instrument, and the correlation was significant when compared with both the paper-pencil and electronic versions of the CCS. Additionally, through the demonstration of a significant correlation among all factors, CCS was verified as a tool to measure compassion competence. This indicates that, through the comparison of the results of this study for nursing students and the CLS, the CCS was also verified as a tool to measure the compassion competence of nursing students. Therefore, an extended study should be conducted using CCS among nursing students. In particular, the limitations of this study can be supplemented through studies that analyze the differences in CCS by grade. It will be possible to prepare an education strategy in accordance with the level through research that reveals the difference in CCS by grade level and the cause of the difference.

The importance of compassion in nursing care is not limited to a few countries but is treated as an essential factor in patient care worldwide. Therefore, the compassion competence of nurses and nursing students in many more countries can be measured and compared using the e-CCS. Future studies are needed to examine if there are any cultural differences in compassion competence between countries, which will also help nurses develop strategies for the improvement of multicultural compassion competence.

There are several limitations to consider in this study. First, the participants were all college students who were relatively familiar with the use of electronic devices. Further studies involving older nurses are required to address this potential limitation. Second, the sample size of this study was relatively small; therefore, the results may not be readily generalizable. Third, the sample sizes and demographic characteristics of the participants assigned to each group were unequal because of the relatively high dropout rate. This may have caused differences in correlations and test-retest reliability; further research is needed to confirm this potential effect.

The voluntary nature of participation among students from four universities and the lack of specific encouragement to participate in the second survey at different times may have contributed to this high dropout rate. In particular, the high dropout rate of this study drew new im-plications that a research design that can encourage participants’ level of spontaneity should be considered in an on-line-based survey method. To address this limitation in future research, a more sophisticated research design and data collection strategy should be considered. Finally, we acknowledge that certain factors, such as the influence of the device used for survey participation, the time required to complete the questionnaire, entry accuracy, and response rates, were not examined in our study. However, we recognize the importance of investigating these factors in future research to obtain a comprehensive understanding of the advantages and limitations of the electronic survey method. Therefore, we recommend that future studies include these aspects to enhance the validity and reliability of electronic questionnaires.


The results of this study confirm that the newly developed e-CCS is a valid and reliable tool for measuring compassion competence in nursing students and is compar-able to the original scale. As discussed above, researchers can derive more accurate and meaningful results by con-ducting studies with careful consideration of the research participants, research setting, and the suitability of various approaches for gathering data. In addition, it is necessary for future research to extend the research sample to disparate age groups, nurses working in different nursing fields, and students from multiple colleges. This will help to identify differences in the level of compassion competence and establish strategies, such as education programs, to enhance compassion competence in nurses and nursing students.



The authors declared no conflict of interest.


Study conception and design acquisition - Chang HE & Lee Y; Data collection - Chang HE & Lee Y; Data analysis & Interpre-tation - Chang HE, Lee Y, & Jung S; Drafting & Revision of the manuscript - Chang HE, Lee Y, & Jung S; Study supervision: Chang HE, Lee Y, & Jung S.


The data that support the findings of this study are available from the corresponding author upon reasonable request.


1. . Schantz ML. Compassion: a concept analysis. Nursing Forum. 2007;42(2):48–55.
2. . Boyle DA. Countering compassion fatigue: a requisite nursing agenda. The Online Journal of Issues in Nursing. 2011;16(1):2.
3. . Burnell L. Compassionate care: a concept analysis. Home Health Care Management & Practice. 2009;21(5):319–324.
4. . Hill R. Compassion, quality and standards of care. Nurse Prescribing. 2010;8(3):100–101.
5. . Halldorsdottir S. Feeling empowered: A phenomenological case study of the lived experience of health. In : Fridlund B, Hildingh C, eds. Qualitative research methods in the service of health Sweden: Student Literature; 2000. p. 82–96.
6. . Ali S, Kleib M, Paul P, Petrovskaya O, Kennedy M. Compassionate nursing care and the use of digital health technologies: a scoping review. International Journal of Nursing Studies. 2022;127:104161.
7. . Dewar B, Nolan M. Caring about caring: developing a model to implement compassionate relationship centred care in an older people care setting. International Journal of Nursing Studies. 2013;50(9):1247–1258.
8. . Guan R, Jin L, Qian M. Validation of the empathy quotient - short form among Chinese healthcare professionals. Social Behavior and Personality: An International Journal. 2012;40(1):75–84.
9. . Lee Y, Seomun G. Role of compassion competence among clinical nurses in professional quality of life. International Nursing Review. 2016;63(3):381–387.
10. . Sinclair S, Hack TF, Raffin-Bouchal S, McClement S, Stajduhar K, Singh P, et al. What are healthcare providers’ understandings and experiences of compassion? the healthcare compassion model: a grounded theory study of healthcare providers in Canada. BMJ Open. 2018;8(3):e019701.
11. . Sinclair S, Kondejewski J, Jaggi P, Roze des Ordons AL, Kassam A, Hayden KA, et al. What works for whom in compassion training programs offered to practicing healthcare providers: a realist review. BMC Medical Education. 2021;21(1):455.
12. . Kim C, Lee Y. Effects of compassion competence on missed nursing care, professional quality of life and quality of life among Korean nurses. Journal of Nursing Management. 2020;28(8):2118–2127.
13. . Sinclair S, Hack TF, MacInnis CC, Jaggi P, Boss H, McClement S, et al. Development and validation of a patient-reported measure of compassion in healthcare: the Sinclair Compassion Questionnaire (SCQ). BMJ Open. 2021;11(6):e045988.
14. . Ghafourifard M, Zamanzadeh V, Valizadeh L, Rahmani A. Compassionate nursing care model: results from a grounded theory study. Nursing Ethics. 2022;29(3):621–635.
15. . Sinclair S, Kondejewski J, Hack TF, Boss HC, MacInnis CC. What is the most valid and reliable compassion measure in healthcare? an updated comprehensive and critical review. The Patient-Patient-Centered Outcomes Research. 2022;15(4):399–421.
16. . Sinclair S, Russell LB, Hack TF, Kondejewski J, Sawatzky R. Measuring compassion in healthcare: a comprehensive and critical review. The Patient-Patient-Centered Outcomes Research. 2017;10(4):389–405.
17. . Burnell L, Agan DL. Compassionate care: can it be defined and measured? The development of the compassionate care assessment tool. International Journal of Caring Science. 2013;6(2):180–187.
18. . Fogarty LA, Curbow BA, Wingard JR, McDonnell K, Somer-field MR. Can 40 seconds of compassion reduce patient anxi-ety? Journal of Clinical Oncology. 1999;17(1):371–379.
19. . Lown BA, Muncer SJ, Chadwick R. Can compassionate healthcare be measured? the schwartz center compassionate care scale ™. Patient Education and Counseling. 2015;98(8):1005–1010.
20. . Durkin M, Gurbutt R, Carson J. Qualities, teaching, and measurement of compassion in nursing: A systematic review. Nursing Education Today. 2018;63:50–58.
21. . Kemper KJ, Gascon G, Mahan JD. Two new scales for integrative medical education and research: Confidence in providing calm, compassionate care scale (CCCS) and self-efficacy in providing non-drug therapies (SEND) to relieve common symp-toms. European Journal of Integrative Medicine. 2015;7(4):389–395.
22. . Lee Y, Seomun G. Development and validation of an instrument to measure nurses’ compassion competence. Applied Nursing Research. 2016;30:76–82.
23. . Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. Journal of Asthma. 2003;40(7):751–762.
24. . Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported out-come measures: a meta-analytic review. Value in Health. 2008;11(2):322–333.
25. . Chang HE, Cho SH. Workplace violence and job outcomes of newly licensed nurses. Asian Nursing Research. 2016;10(4):271–276.
26. . Gill FJ, Leslie GD, Grech C, Latour JM. Using a web-based survey tool to undertake a Delphi study: application for nurse education research. Nurse Education Today. 2013;33(11):1322–1328.
27. . Tubbs-Cooley HL, Pickler RH, Younger JB, Mark BA. A descriptive study of nurse-reported missed care in neonatal in-tensive care units. Journal of Advanced Nursing. 2015;71(4):813–824.
28. . Reitz OE, Anderson MA. A comparison of survey methods in studies of the nurse workforce. Nurse Researcher. 2013;20(4):22–27.
29. . Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology. 2007;60(1):34–42.
30. . Im EO, Chee W, Bender M, Cheng CY, Tsai HM, Kang NM, et al. The psychometric properties of pen-and-pencil and internet versions of the midlife women's symptom index (MSI). International Journal of Nursing Studies. 2005;42(2):167–177.
31. . Junker U, Freynhagen R, Langler K, Gockel U, Schmidt U, Tolle TR, et al. Paper versus electronic rating scales for pain assessment: a prospective, randomised, crossover validation study with 200 chronic pain patients. Current Medical Research and Opinion. 2008;24(6):1797–1806.
32. . Ryan JM, Corry JR, Attewell R, Smithson MJ. A comparison of an electronic version of the SF-36 general health questionnaire to the standard paper version. Quality of Life Research. 2002;11(1):19–26.
33. . Maclure M, Mittleman AM. Should we use a case-crossover design? Annual Review of Public Health. 2000;21(1):93–221.
34. . Mersin S, İ brahimoğ lu O, Cağ lar M, Akyol E. Compassionate love, burnout and professional commitment in nurses. Journal of Nursing Management. 2020;28(1):72–81.
35. . Oman D, Thoresen CE, Hedberg J. Does passage meditation foster compassionate love among health professionals?: a randomised trial. Mental Health, Religion & Culture. 2010;13(2):129–154.

Article information Continued

Table 1.

Demographic Characteristics of the Participants (N=241)

Variables Categories Total (n=241) Group A (n=71) Group B (n=68) Group C (n=102) x2 p
n (%) n (%) n (%) n (%)
Gender Men 37 (15.4) 12 (16.9) 11 (16.2) 14 (13.7) 0.37 .829
Women 204 (84.6) 59 (83.1) 57 (83.8) 88 (86.3)
School grade Sophomore 168 (69.7) 71 (100) 35 (51.5) 62 (60.8) 45.4 <.001
Junior 73 (30.3) 33 (48.5) 40 (39.2)
Region of school Metropolitan area 164 (68.0) 71 (100) 68 (100) 25 (24.5) 154.2 <.001
Other 77 (32.0) 77 (75.5)

Note. Group A: paper-and-pencil after paper-and-pencil group; Group B: electronic after electronic group; Group C: paper-and-pencil after electronic group or electronic after paper-and-pencil group.

Table 2.

Item-by-Item Differences between Pretest and Posttest (Linear Weighted Kappa Coefficients) (N=241)

Item no. Group A (n=71) Group B (n=68) Group C (n=102)
Coefficient p Coefficient p Coefficient p
1 .49 <.001 .36 <.001 .32 <.001
2 .33 <.001 .48 <.001 .34 <.001
3 .49 <.001 .58 <.001 .61 <.001
4 .31 <.001 .36 <.001 .35 <.001
5 .54 <.001 .36 <.001 .29 <.001
6 .43 <.001 .47 <.001 .45 <.001
7 .29 <.001 .23 .003 .44 <.001
8 .49 <.001 .42 <.001 .37 <.001
9 .49 <.001 .56 <.001 .40 <.001
10 .43 <.001 .63 <.001 .40 <.001
11 .31 <.001 .50 <.001 .39 <.001
12 .32 <.001 .48 <.001 .37 <.001
13 .36 <.001 .44 <.001 .30 <.001
14 .40 <.001 .42 <.001 .34 <.001
15 .32 <.001 .24 .002 .32 <.001
16 .31 <.001 .38 <.001 .41 <.001
17 .39 <.001 .28 <.001 .42 <.001

Note. Group A: paper-and-pencil after paper-and-pencil group; Group B: electronic after electronic group; Group C: paper-and-pencil after electronic group or electronic after paper-and-pencil group.

Table 3.

Mean Difference between Pretest and Posttest of Each Group (N=241)

Variables Group A (n=71) Group B (n=68) Group C (n=102)
Paper-pencil Paper-pencil t (p) ICC Electronic Electronic t (p) ICC Paper-pencil Electronic t (p) ICC
Total 3.84±0.41 3.91±0.47 -1.88 (.064) .85 3.78±0.43 3.75±0.37 0.64 (.525) .85 3.90±0.47 3.85±0.48 1.05 (.297) .72
Communication 3.85±0.49 3.90±0.47 -1.10 (.274) .82 3.73±0.52 3.73±0.44 0.08 (.935) .83 3.90±0.51 3.85±0.56 1.00 (.319) .68
Sensitivity 3.89±0.52 3.97±0.60 -1.19 (.239) .66 3.92±0.47 3.85±0.46 1.73 (.088) .82 4.01±0.56 3.97±0.52 0.84 (.406) .76
Insight 3.74±0.55 3.85±0.56 -2.07 (.042) .81 3.68±0.51 3.69±0.47 -0.12 (.903) .67 3.76±0.63 3.72±0.58 0.71 (.478) .69

Note. Group A: paper-and-pencil after paper-and-pencil group; Group B: electronic after electronic group; Group C: paper-and-pencil after electronic group or electronic after paper-and-pencil group.

ICC=Intraclass correlation coefficient; M=mean; SD=Standard deviation.

Table 4.

Convergent Validity of the CCS with the CLS (N=241)

Variables Pearson's correlation coefficients with CLS
Group A (n=71) Group B (n=68) Group C (n=102)
Paper-pencil Paper-pencil Electronic Electronic Paper-pencil Electronic
Total .60* .49* .49* .55* .51* .58*
Communication .43* .46* .41* .50* .42* .51*
Sensitivity .42* .43* .45* .48* .52* .54*
Insight .66* .42* .40* .34* .39* .47*

p<.05; CCS=Compassion Competence Scale; CLS=Compassion Love Scale.