#### Manuela Scionic . <sup>a</sup> FISPPA Department, University of Padua, Padua, Italy. . **Measuring content validity of academic psychological capital and locus of control in fresh graduates**

, Luigi Fabbrisb

, Egidio Robustoa

,

, Daiana Colledania

**Measuring content validity of academic psychological capital and locus of control in fresh graduates** 

<sup>b</sup> Tolomeo Studi e Ricerche, Padua and Treviso, Italy. <sup>c</sup> Department of Statistics, University of Padua, Padua, Italy. Pasquale Anselmi, Daiana Colledani, Luigi Fabbris, Egidio Robusto, Manuela Scioni

### **1. Introduction**

Pasquale Anselmia

PETERE (Preferences for Employment and Training as Elected by REcent graduates) is a project of the University of Padua that investigated how fresh graduates interact with labour market to understand how to improve placement policies and support plans. One of the aims of the project was the identification of psychological patterns that could help graduates to stand the labour market in uncertain times. According to the literature, two sets of psychological variables have been identified that can be crucial to achieve academic and professional success.

The first set was developed within the framework of positive psychology (Seligman & Csikszentmihalyi, 2014) and is named "Psychological capital" (PsyCap; Luthans et al., 2007). PsyCap defines a positive psychological state characterized by feelings of self-efficacy, hope, optimism, and resilience. Self-efficacy (or confidence) describes the conviction of having all the abilities, motivation, and resources needed to successfully execute a specific task. Hope defines a positive motivational state that leads individuals to pursue their own objectives, redirecting, when it is necessary, the strategies employed to achieve them. Optimism is the subjective tendency to interpret situations and events positively. In the framework of PsyCap, this trait describes the propensity to carefully consider both positive and negative events to understand their causes and consequences (Youssef & Luthans, 2005). Optimistic individuals build positive expectancies that motivate them to persist toward their goals, dealing with difficulties, and reaching success (Chemers et al., 2001; Sharpe et al., 2011). The last trait included in PsyCap is resilience, which defines the ability to "bounce back" from adversity, failure, and uncertainty.

The second set of psychological variables is locus of control (LoC). It may be internal and external. The first defines the extent to which individuals perceive strong links between their actions and the following results. Individuals with internal LoC feel having control over their own fate. Conversely, external LoC defines the inclination to perceive a low control on ones' fate. Individuals with external LoC attribute personal outcomes to external and uncontrollable factors (Lefcourt, 2014; Rotter, 1966).

PsyCap and LoC have been extensively related to important work outcomes, including job satisfaction, job performance, and organisational commitment (e.g., Avey et al., 2010; 2011; Hansen et al., 2015; Judge & Bono, 2001). Moreover, these variables have been found to be associated with positive academic results, such as high performance and motivation, academic satisfaction, inclination to use effective and functional coping strategies, and ability to deal with stress (e.g., Clifton et al., 2004; Conti, 2000; Drago et al., 2018; Elias & Loomis, 2002; McKenzie & Schweitzer, 2001; Mohamed et al., 2018; Nunn & Nunn, 1993; Snyder et al., 2002). The attention towards these variables might be also due to the fact that they are "state-like" variables and can be modified through targeted interventions (Luthans et al., 2008; Stanton, 1982).

Scales for measuring PsyCap and LoC exist in the literature. The PsyCap Questionnaire (PCQ; Luthans et al., 2007) is meant for workers. As such, it might be inappropriate for assessing these traits among fresh graduates who were about to enter the world of work. With respect to LoC, there is a scale, called Academic Locus of Control Scale (Trice, 1985) which is intended for

Manuela Scioni, University of Padua, Italy, manuela.scioni@unipd.it, 0000-0003-3192-4030

FUP Best Practice in Scholarly Publishing (DOI 10.36253/fup\_best\_practice)

Pasquale Anselmi, Daiana Colledani, Luigi Fabbris, Egidio Robusto, Manuela Scioni, *Measuring content validity of academic psychological capital and locus of control in fresh graduates*, pp. 23-28, © 2021 Author(s), CC BY 4.0 International, DOI 10.36253/978- 88-5518-304-8.06, in Bruno Bertaccini, Luigi Fabbris, Alessandra Petrucci, *ASA 2021 Statistics and Information Systems for Policy Evaluation. Book of short papers of the opening conference*, © 2021 Author(s), content CC BY 4.0 International, metadata CC0 1.0 Universal, published by Firenze University Press (www.fupress.com), ISSN 2704-5846 (online), ISBN 978-88-5518-304-8 (PDF), DOI 10.36253/978-88-5518-304-8

<sup>15</sup> Pasquale Anselmi, University of Padua, Italy, pasquale.anselmi@unipd.it, 0000-0003-2982-7178 Daiana Colledani, University of Padua, Italy, daiana.colledani@unipd.it, 0000-0003-2840-9193 Luigi Fabbris, University of Padua, Italy, luigi.fabbris@unipd.it, 0000-0001-8657-8361 Egidio Robusto, University of Padua, Italy, egidio.robusto@unipd.it, 0000-0002-7583-2587

students. However, this instrument is founded on a unidimensional conceptualization of LoC, which is not supported by research in this field. Levenson (1981), for instance, found that internal and external LoC are two distinct dimensions.

Recently, two brief instruments have been appositely developed for measuring PsyCap and LoC among fresh graduates: the Academic PsyCap and the LoC scales (Robusto et al., 2019). These two scales showed significant relationships with the occupational status of respondents, with their entrepreneurial disposition, and with the number of actions taken when they were looking for a job. Although the two scales showed satisfactory psychometric properties, there was room for some improvement pertaining to the content validity and the length of the six subscales (i.e., self-efficacy, hope, optimism, resilience, internal LoC, and external LoC). With respect to the former, the analysis of the content of the items included in each subscale suggested that they did not adequately cover relevant operationalizations of the different psychological variables. With respect to the latter, the length of some subscales was too small (e.g., internal LoC subscale contained only 3 items). To this purpose, in the present work new items were developed for each of the six subscales with the aim of increasing their length and improving the coverage of additional relevant operationalizations.

# **2. Method**

### **Participants and procedure**

To test the functioning of the new scales, a study was conducted on 1105 graduates (Males 36.7%, Mean age = 24.92, *SD* = 4.66) at the University of Padua. They were surveyed in the context of the PETERE project within one month after graduation. The survey was administered via a CAWI (Computer-Assisted Web-based Interviewing) system. Students from medicine and nursing courses were not included in the sample. To analyse the data on the Academic PsyCap and LoC scales, the total sample was randomly split into two subsamples including 550 (Males 35.9%, Mean Age = 25.11, *SD* = 4.84) and 555 (Males 37.1%, Mean Age = 24.72, *SD* = 4.47) participants, respectively.

### **Measures**

A total of 37 items were used to measure the four facets of PsyCap: resilience (11 items, 5 of them being new), self-efficacy (9 items, 2 of them being new), optimism (9 items, 2 of them being new), and hope (8 items, 2 of them being new).

To evaluate internal and external LoC, 12 items were used, six for each subscale (3 items of internal LoC and 2 items of external LoC being new).

All items were scored on a four-point Likert scale (from 1 "Completely disagree" to 4 "Completely agree").

### **Analytic approach**

The factor structure of Academic PsyCap and LoC scales was tested through Exploratory Structural Equation Models (ESEMs; Asparouhov & Muthén, 2009) and Confirmatory Factor Analyses (CFAs). The ESEMs were run in the first subsample (*n* = 550) whereas CFAs in the second (*n* = 555). The ESEMs were performed on all the 37 and 12 items of Academic PsyCap and LoC scales (defining four and two factors, respectively), and allowed for the identification and exclusion of poorly performing items (i.e., items with large cross-loadings or low factor loadings on the intended scale). After having removed the items with unsatisfactory performance, the factor structure of Academic PsyCap and LoC was confirmed through CFAs. ESEMs and CFAs were run using the WLSMV estimator (weighted least squares mean and variance-adjusted; Muthén & Muthén, 2012); this method is recommended for categorical observed data (e.g., Flora

& Curran, 2004; Brown, 2006). The goodness of fit of the models was evaluated by means of several fit indices: χ<sup>2</sup> , root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardized root mean square residual (SRMR). A solution fits the data when χ<sup>2</sup> is non-significant (*p* > .05). Since this statistic is sensitive to sample size, the other fit measures were also taken into account in the evaluation of the models. Specifically, CFI indices close to .95 (.90 to .95 for reasonable fit), SRMR values less than .08, and RMSEA smaller than .06 (.06 to .08 for reasonable fit) are indicative of good model fit (Marsh et al., 2004).

Composite reliability was computed to measure the internal consistency of the scales. This coefficient is conceptually similar to Cronbach's α but more accurate and can be easily computed in the structural equation modeling framework (Raykov, 2001). Composite reliability ranges from 0 to 1. The closer the value to 1, the larger the internal consistency.

# **3. Results**

*Academic PsyCap* 



Note. All factor loadings and correlation coefficients were significant *p* < .001

The four-factor ESEM run on the 37 items of the Academic PsyCap scale obtained a

successful fit. Although χ<sup>2</sup> was significant due to sample size (χ<sup>2</sup> (524) = 1298.107, *p* ൏ .001), the other indices satisfied the rules of thumb (RMSEA = .052 [.048, .055]; CFI = .953; SRMR = .045).

The inspection of factor loadings, modification indices, and item content suggested excluding 13 items from the final version of the scale. In particular, one item of the self-efficacy scale was excluded since its content was very close to that of another item of the same scale but it was characterized by a weaker factor loading. Three items of the optimism scale, one of the resilience scale and two items of the hope scale were excluded since they exhibited weak loadings on the intended factor. One item of the self-efficacy scale and three items of the resilience scale were excluded because they exhibited high cross-loadings. Finally, two items, one from the selfefficacy scale and the other from the resilience scale, were excluded according to indications of modifications indices. The new self-efficacy, optimism, resilience, and hope scales contained 6 items each, out of which: 1 item of self-efficacy, 2 items of optimism, 2 items of resilience, and 1 item of hope were new.

The results of the CFA run in the second sample, on the remaining 24 items, are reported in Table 1. The model showed an adequate fit: χ<sup>2</sup> (246) = 930.574, *p* ൏ .001; RMSEA = .071 [.066, .076]; CFI = .941; SRMR = .066. Composite reliability was satisfactory for all scales: .89, .83, .84, and .79 for self-efficacy, optimism, resilience, and hope, respectively.

### *LoC*

The two-factor ESEM run on the 12 items of the LoC scale obtained a successful fit. Despite χ<sup>2</sup> was significant due to sample size (χ<sup>2</sup> (43) = 182.343, *p* ൏ .001), the other indices were satisfactory (RMSEA = .077 [.065, .088]; CFI = .930; SRMR = .054). The inspection of factor loadings, modification indices, and item content suggested excluding only two items, one for each subscale. In particular, the item of internal LoC was excluded because it did not load on the intended factor, whereas the item of external LoC was excluded because it had the lowest loading on the factor. The new LoC scales contained 5 items each (2 items of internal LoC and 1 item of external LoC were new).

The results of the CFA run in the second sample, on the remaining 10 items, are reported in Table 2. The model showed an adequate fit: χ<sup>2</sup> (41) = 138.393, *p* ൏ .001; RMSEA = .075 [.062, .088]; CFI = .940; SRMR = .068. Composite reliability was satisfactory for both scales: .62 and .80 for internal and external LoC, respectively.


**Table 2. Factor loadings (**λ**) from the CFA of the LoC scale**

Note. All factor loadings and correlation coefficients were significant *p* < .001

## **4. Discussion**

The two scales for measuring Academic PsyCap and LoC introduced by Robusto et al. (2019) have been administered to a new large sample of fresh graduates in order to develop new items and evaluate their performance. The new Academic PsyCap scale contained 24 items, 6 for each of the four subscales. The new LoC scale contained 10 items, 5 for each of the two subscales. One to two items of each subscale were new.

On the whole, the psychometric properties of the new instruments are in line with those of the original ones. However, the content validity of the new scales was improved due to the introduction of items that investigate additional relevant operationalizations of the psychological variables. Moreover, in the new version of the instruments, the subscales were balanced for item length: the four Academic PsyCap subscales contained 6 items each, while the two subscales of Academic LoC contained 5 items each. This was especially useful for the internal LoC subscale that, in the previous version, contained only three items.

### **References**

