Psychometric Properties of the Short ASSIST Scales
- Side: 164-181
- DOI: https://doi.org/10.18261/issn.1893-8981-2018-02-07
- Publisert på Idunn: 2018-06-05
- Publisert: 2018-06-05
- Creative Commons (CC BY 4.0)
Background: The shortening of measurement scales may improve their feasibility, but may also affect the scales’ measurement properties. This study investigated the psychometric properties of the short Norwegian Approaches and Study Skills Inventory for Students (ASSIST) among occupational therapy students.
Methods: The students (n = 148) completed the Norwegian ASSIST and provided socio-demographic data. Scale structure was examined with Principal Components Analysis (PCA). Scale consistency was assessed with Cronbach’s α. Bivariate associations between the full and short scales were examined with Pearson’s r. Linear regression analyses were used to assess associations between scale scores and the students’ average exam grade.
Results: Three factors were confirmed, but one item did not fit within the deep approach scale. Cronbach’s α for the scales ranged 0.64–0.71. The full-length scales were strongly associated with the corresponding short scales, r ranging 0.85–0.87. For the surface scales, the full-length and short scale scores were both associated with the students’ average exam grade. For the strategic scale, only the full-length scale scores were associated with exam grade.
Conclusions: The short scales of the Norwegian ASSIST have satisfactory psychometric properties and represent a feasible way of assessing students’ study approaches in Norwegian higher education.Keywords: factor analysis, higher education, occupational therapy, psychometrics, students
Bakgrunn: Å forkorte måleinstrumenter kan gjøre instrumentene mer anvendelige, men kan også påvirke måleegenskapene deres. Denne studien undersøkte måleegenskapene til den forkortede norske versjonen av Approaches and Study Skills Inventory for Students (ASSIST) i et utvalg ergoterapistudenter.
Metode: Studentene (n = 148) fylte ut den norske ASSIST og oppga sosiodemografiske data. Skalastruktur ble undersøkt med prinsipal komponentanalyse. Skalaenes interne konsistens ble undersøkt med Cronbach’s α. Bivariate sammenhenger mellom skalaene i fullversjon og kortversjon ble undersøkt med Pearson’s r. Lineære regresjonsanalyser ble benyttet for å undersøke direkte sammenhenger mellom studentenes skåringer på skalaene og deres gjennomsnittlige eksamenskarakter i studiet.
Resultater: Tre faktorer ble bekreftet, men ett spørsmål passet ikke inn på skalaen dyp tilnærming. Cronbach’s α for skalaene varierte mellom 0.64 og 0.71. Skåringene på skalaene fra fullversjonen var sterkt forbundet med skåringene på kortversjonens skalaer, og r varierte mellom 0.85 og 0.87. Begge skalaene for overflatisk tilnærming var forbundet med studentenes gjennomsnittlige eksamenskarakter i studiet, mens kun den fullstendige skalaen for strategisk tilnærming var forbundet med karakterer.
Konklusjon: De forkortede skalaene i den norske ASSIST har tilfredsstillende måleegenskaper og representerer en anvendelig måte å vurdere studenter i norsk ergoterapiutdanning med henblikk på deres tilnærminger til læring.Nøkkelord: faktoranalyse, høyere utdanning, ergoterapi, psykometri, studenter
There is general agreement that students’ own study behaviors and engagement with the study materials are of great importance for their subsequent study results (Diseth & Martinsen, 2003; Kusurkar, Ten Cate, Vos, Westers, & Croiset, 2013; Mattick, Dennis, & Bligh, 2004; Richardson, Abraham, & Bond, 2012; Salamonson et al., 2013). Building on the work of Marton and Säljö (1976) four decades ago, Entwistle and Ramsden (1983) introduced a much used conceptual framework in an attempt to distinguish between the qualitatively different approaches to studying that students employ. Three approaches to studying were identified: the deep, the surface, and the strategic approaches, and students may be characterized by their dominant study approach (Entwistle & Ramsden, 1983).
Students who are mainly driven by a desire to understand, and who seek personal meaning by connecting and distinguishing between the ideas they find in the study materials, are denoted as deep learners. The surface learners, on the other hand, are students who are mainly driven by a desire to avoid failure, and who emphasize rote learning strategies in order to be able to reproduce knowledge at exams. The third group of students, the strategic learners, are mainly driven by a desire to perform well and achieve the best possible results. These students may draw on a wide range of learning strategies derived from both the deep and the surface approaches whenever these are found to be consistent with the ultimate aim of achievement. However, classifying students as using one study approach or another is overly simplistic (Entwistle, 2007), as is inferring that one approach is uniformly better than others (Beattie, Collins, & McInnes, 1997). Approaches to studying have been linked with age and experience (Beccaria, Kek, Huijser, Rose, & Kimmins, 2014; Bonsaksen, Sadeghi, & Thørrisen, 2017; Zeegers, 2001) and with relatively stable personal traits like self-efficacy (Bonsaksen, Sadeghi, et al., 2017; Prat-Sala & Redford, 2010). Nonetheless, a range of research studies have also found study approaches to be related to aspects of the learning environment (Baeten, Kyndt, Struyven, & Dochy, 2010; Kreber, 2003; Lizzio, Wilson, & Simons, 2002; Richardson, 2010; Sadlo & Richardson, 2003; Trigwell, Prosser, & Waterhouse, 1999). For example, a recent study with occupational therapy students found that higher scores on all the subscales of a course experience questionnaire (i.e., appropriate assessment and workload, clarity of goals and standards, emphasis on independence, generic skills, and good teaching) correlated positively with the deep and strategic approaches to studying, and correlated negatively with the surface approach (Sun & Richardson, 2016).
Approaches to studying are frequently measured with the Approaches and Study Skills Inventory for Students (ASSIST; Tait, Entwistle, & McCune, 1998). The ASSIST consists of 52 statements to which the respondent indicates his or her level of agreement on a 1–5 scale. The items comprise three main scales (the deep, surface, and strategic approaches to studying), each of which is comprised by four to five subscales. Factor analytic studies across a range of disciplines have consistently reproduced the theoretically proposed three-factor structure (Byrne, Flood, & Willis, 2004; Diseth, 2001; Kreber, 2003; Richardson, 2005, 2010; Valadas, Goncalves, & Faísca, 2010), and the same structure was also recently found in a cross-cultural sample of occupational therapy students (Bonsaksen, Småstuen, et al., 2017). Internal consistency has been found to be high for the main scales, while it has been deemed acceptable or in the lower range for the subscales (Ballantine, Duff, & Larres, 2008; Bonsaksen, Småstuen, et al., 2017; Reid, Duvall, & Evans, 2005; Reid, Evans, & Duvall, 2012; Richardson, 2010; Richardson, Gamborg, & Hammerberg, 2005; Valadas et al., 2010).
Despite the consistent psychometric findings indicating very good validity and reliability of the ASSIST, its length may decrease its feasibility in practical use. As Pettersen (2010: p. 239) noted, “the longer the inventory, the less care students may take in completing it and the less likely it is that staff will use it”. Therefore, an 18-item short version of the ASSIST was also developed (Entwistle, McCune, & Tait, 2006). Because of its brevity in comparison to the full instrument, the subscales known from the ASSIST are not in use.
The short version of the ASSIST has been used in research on students with mixed discipline backgrounds (Heinström, 2005) and on students within fields like medicine (Feeley & Biggerstaff, 2015; Liew, 2015), nursing (Martyn, Terwijn, Kek, & Huijser, 2014), psychology (Prat-Sala & Redford, 2010) and mathematics and computer science (Bälter, Cleveland-Innes, Petterson, Scheja, & Svedin, 2013; Svedin & Bälter, 2016). Heinström (2005) reported that the short scales correlate strongly with the respective full scales (strategic scales: r =0.91; deep and surface scales: r = 0.93). The short scales have also been shown able to predict outcomes – for example, studies have found that higher scores on the surface approach scale significantly decreased the probability of completing an online course, whereas higher scores on the deep approach scale increased the same probability (Bälter et al., 2013; Svedin & Bälter, 2016). Another study showed the short study approach scales to be associated with different information-seeking behaviors in a mixed sample of master’s degree students (Heinström, 2005).
Internal consistency of the three short ASSIST scales has been found to differ between studies. Cronbach’s α was satisfactory (ranging 0.72–0.76) in the original study (Noel Entwistle 1999, personal communication, cited by Heinström, 2005) and even better in more recent studies (Liew, 2015; Prat-Sala & Redford, 2010), but below satisfactory (α < 0.70) in others (Heinström, 2005). In spite of adequate internal consistency estimates, however, one should not assume that the scale items uniformly belong to only one latent dimension. We have not been able to identify any studies, Norwegian or international, where the short ASSIST’s underlying dimensions have been empirically examined. Thus, the factor structure of the short ASSIST instrument appears to have been taken for granted based on the original results (Entwistle, 2017).
Within the Norwegian context, Pettersen (2010) examined the properties of four different approaches to studying measures, but not the 18-item version as developed by Entwistle and coworkers (2006). Diseth and coworkers (2006) developed their own ASSIST short version with 24 items, with items being selected based on which provided the best validity and reliability estimates in the given sample. However, the exact items used in that study were not reported, and the degree of correspondence between the full and the short versions is therefore unknown. As the psychometric properties of an instrument depend on its specific items used in a specific sample within a specific language- and professional culture, there is reason to investigate the properties of the Norwegian 18-items ASSIST with occupational therapy students – a sample not previously assessed with this instrument. The lack of known measurement properties represents a substantial weakness related to the short ASSIST, and constitutes the rationale for the current study.
The aim of the study was to confirm the factor structure of the Norwegian short ASSIST version in a sample of undergraduate occupational therapy students, and to establish measures of internal consistency related to each of the short scales. In addition, the study examined the strength of associations between the short scales and their corresponding full version scales, and examined the short scales’ ability to predict outcome in comparison to the full version scales.
Design and setting of the study
The study had a cross-sectional design. It was a part of a larger inquiry into the study approaches among occupational therapy students in a cross-cultural sample, taking place at four different universities in four different countries (Brown et al., 2016). However, the short ASSIST scales have not been used in any previous studies from the project. Given the aim of examining the short version of the Norwegian language ASSIST instrument, the current study employed only the data from the Norwegian students.
Recruitment and participants
Students were included as participants in the study given that they were enrolled in the occupational therapy education program in Oslo and provided informed consent to participate in the study. A non-teaching member of staff, who distributed the questionnaires to the students during breaks between scheduled classes, collected the data in 2015. In line with the instructions on the ASSIST forms (Diseth, 2001; Entwistle et al., 2006), the students were encouraged to respond according to “your actual ways of studying”, and in the way that best describes “how you go about learning and studying”. Given that the questionnaires were completed at the university, and between scheduled classes, it seems fair to assume that the students had their current line of study in mind when responding.
The sample was comprised of 148 occupational therapy students in Oslo, representing students in the first year (n = 52, 35.1 %), second year (n = 46, 31.1 %) and third year cohorts (n = 50, 33.8 %). The sample mean age was 23.9 years (SD = 4.4 years) and there was a predominance of female students (n = 120, 81.1 %) compared to male (n = 28, 18.9 %).
Approaches to studying
In this study, the ASSIST items concerned with approaches to studying were used (Entwistle et al., 2006; Tait et al., 1998), and the Norwegian students used Diseth’s previously developed translation of the full instrument (Diseth, 2001). The ASSIST consists of 52 statements to which the respondent is asked to rate his or her level of agreement (1 = disagree, 2 = disagree somewhat, 3 = unsure, 4 = agree somewhat, 5 = agree). The instrument has a proposed structure, based on theory and research, where the items are organized into three main scales. These scales are commonly referred to as the deep, strategic, and surface approaches to studying (Tait et al., 1998), and previous research has uniformly confirmed the three-factor structure of the ASSIST (Byrne et al., 2004; Diseth, 2001; Entwistle, Tait, & McCune, 2000; Kreber, 2003; Richardson et al., 2005; Valadas et al., 2010). In the present sample, the internal consistencies of the full-length scales were 0.81 (deep approach), 0.80 (strategic approach), and 0.77 (surface approach).
Eighteen items from the complete version comprise the short version of the ASSIST, with six items belonging to each of the deep, strategic, and surface scales. The items and proposed scales are shown in Table 1, and scale scores are calculated by adding the scores on the relevant items (scale score range is 6–30). The factor structure of the short version is purported to be identical to that of the full version, as the chosen items were those with the highest loadings on the scales in the full version (Entwistle, 2017). The short deep scale consists of two items from the seeking meaning subscale, two items from the relating ideas subscale, and two items from the use of evidence subscale. The short strategic scale consists of three items from the time management subscale, two items from the achieving subscale, and one item from the organized studying subscale. Finally, the short surface scale consists of three items from the unrelated memorizing subscale, two items from the fear of failure subscale, and one item from the lack of purpose subscale.
|Scale||Item #||Item||M (SD)|
|2||When I’m reading an article or a book, I try to find out for myself exactly what the author means||3.12 (1.07)|
|6||Before tackling a problem or assignment, I first try to work out what lies behind it||3.59 (1.00)|
|10||When I’m working on a new topic, I try to see in my own mind how all the ideas fit together||3.80 (0.97)|
|12||Often I find myself questioning things I hear in lectures or read in books||3.33 (1.10)|
|15||Ideas in course books or articles often set me off on long chains of thought of my own||3.30 (1.11)|
|17||When I read, I examine the details carefully to see how they fit in with what’s being said||3.18 (1.18)|
|3||I organize my study time carefully to make the best use of it||2.93 (1.13)|
|5||I work steadily through the term or semester, rather than leave it all until the last minute||3.00 (1.25)|
|7||I’m pretty good at getting down to work whenever I need to||3.80 (1.13)|
|9||I put a lot of effort into studying because I’m determined to do well||3.46 (1.13)|
|11||I don’t find it at all difficult to motivate myself||2.77 (1.08)|
|13||I think I’m quite systematic and organised when it comes to revising for exams||3.44 (1.18)|
|1||I often have trouble in making sense of the things I have to remember||2.39 (1.02)|
|4||There’s not much of the work here that I find interesting or relevant||1.96 (0.96)|
|8||Much of what I’m studying makes little sense: it’s like unrelated bits and pieces||2.40 (1.05)|
|14||Often I feel I’m drowning in the sheer amount of material we’re having to cope with||4.02 (1.00)|
|16||I’m not really sure what’s important in lectures, so I try to get down all I can||3.30 (1.33)|
|18||I often worry about whether I’ll be able to cope with the work properly||3.43 (1.27)|
Note. Scores are interpreted as 1 = disagree, 2 = disagree somewhat, 3 = unsure, 4 = agree somewhat, 5 = agree.
Average exam grade
Academic performance was operationalized as the students’ average exam grade (related to the occupational therapy study program) based on their completed exams at the time of the data collection. Thus, academic performance scores were based on the qualitative descriptors related to the students’ exam grade (The Norwegian Association of Higher Education Institutions, 2011): Fail = 1, sufficient = 2, satisfactory = 3, good = 4, very good = 5, and excellent = 6. In addition, information regarding the participants’ age and gender was collected using a brief demographic questionnaire.
With the purpose of confirming the proposed factors latent in the short version of the questionnaire, a confirmatory approach to Principal Component Analysis (PCA) using forced factor extraction was applied. The forced extraction of three factors was based on theory and subsequent research on the full ASSIST instrument (Byrne et al., 2004; Diseth, 2001; Kreber, 2003; Tait et al., 1998; Valadas et al., 2010). In addition, we visually inspected the scree-plots, assessed Eigenvalue (λ) estimates, and assessed the proportion of variance explained by the extracted factors. Each additional extracted factor should be able to explain about 10 % of the total variance in the data. The factors were expected to be correlated to one another, therefore the Direct Oblimin rotation method was used to obtain a clearer structure matrix. All analyses were performed using IBM SPSS version 24 (IBM Corporation, 2016).
The items included in the short ASSIST were expected to load on the three extracted factors, the factors representing the three approaches to studying. First, the analysis was performed on all 18 items, followed by analyses of the items belonging to the derived factors separately. The Kaiser-Meier-Olkin (KMO) measure of sampling adequacy (Kaiser, 1974) and Bartlett’s test of sphericity (Bartlett, 1954) were used to assess whether this dataset was eligible for factorization. The KMO value was to exceed 0.60 in order to proceed with factor analysis (Cerny & Kaiser, 1977; Kaiser, 1974), and Bartlett’s test of sphericity was to reach statistical significance, thus indicating that the correlation matrix was different from zero (Bartlett, 1954). The reported statistical measures include Eigenvalues, communalities (the proportion of variance of each variable explained by the three factors together) and factor loadings (estimates of the impact from a given variable on each factor). Factor loadings > 0.40 were interpreted as high (Field, 2005), and we specified that items should display loadings of at least this size in order to load on a factor. The internal consistency of the scales was assessed with Cronbach’s α.
Bivariate associations between the short scale scores and the full-length scale scores were assessed with Pearson’s correlation coefficient r. Two linear regression analyses were conducted to enable the comparison of the full and the short scales’ ability to predict students’ average exam grade, while controlling for gender and cohort. Thus, the models were constructed as (1) Average grade = gender + cohort + full deep scale score + full strategic scale score + full surface scale score; and (2) Average grade = gender + cohort + short deep scale score + short strategic scale score + short surface scale score. Comparisons were made with a view to the resulting standardized β values related to each of the scales, and with a view to the proportion of exam grade variance explained by the models. For all analyses, statistical significance was set at p < 0.05.
Approval for conducting the study was obtained from the Norwegian Data Protection Official for Research (project number 40314). The students were informed that completion of the questionnaires was voluntary, that their responses would be treated in confidence, and that there would be no negative consequences from opting not to participate in the study. All data were de-identified and analyzed on a group basis, hence anonymity of the participants was ensured.
Factor structure and internal consistency
Including all 18 items in the PCA with forced extraction of three factors, the KMO value was 0.75 and Bartlett’s test of sphericity was statistically significant (p < 0.001), both of which indicating that the data were appropriate for factor analysis. Five factors had Eigenvalues above the threshold level of λ = 1. Only the first two factors explained more than the recommended 10 % of the data variance. However, the third factor explained more than 9 %. The three extracted factors explained 41.7 % of the variance in the data. The communalities of the items after the extraction of three factors were between 0.27 (item # 16: “I’m not really sure what’s important in lectures, so I try to get down all I can”) and 0.64 (item # 9: I put a lot of effort into studying because I’m determined to do well”).
Table 2 shows the factor structure resulting from the PCA with Oblimin Rotation, with factor loadings sorted by size. All items, excepting item # 17 (“When I read, I examine the details carefully to see how they fit in what’s being said”), loaded on the three factors in line with theory. This item cross-loaded with the strongest loading on Factor 1 (0.53), representing the strategic approach, but with a high loading on Factor 3 (0.48), representing the deep approach. For the subsequent analyses, this item was therefore retained as part of the deep approach scale, in line with theory. The correlation between factor 1 and 2 was –0.12, it was 0.18 between factor 1 and 3, and it was –0.08 between factor 2 and 3.
Then, PCA was conducted for each of the resulting factors to ascertain that the items loaded appropriately on the respective scales. According to the main analytic strategy of factor confirmation, we used forced extraction of one factor only. The analysis of the items belonging to Factor 1, the strategic approach scale, revealed that all items loaded appropriately on the factor. The strongest loading was 0.79 (item # 9: “I put a lot of effort into studying because I’m determined to do well”) and the weakest loading was 0.52 (item # 11: “I don’t find it at all difficult to motivate myself”). The factor explained 41.3 % of the data variance, and internal consistency of the items was α = 0.71. Removal of any item would result in decreased internal consistency.
Similarly, the analysis of the items belonging to Factor 2, the surface approach scale, revealed that all items loaded appropriately on the factor. The strongest loading was 0.70 (item # 1: “I often have trouble in making sense of the things I have to remember”) and the weakest loading was 0.53 (item # 4: “There’s not much of the work here that I find interesting or relevant”, and item # 16: “I’m not really sure what’s important in lectures, so I try to get down all I can”). The factor explained 37.3 % of the data variance, and internal consistency of the items was α = 0.65. Removal of any item would result in decreased internal consistency.
|Items||Factor 1||Factor 2||Factor 3||Communalities|
|Cronbach’s α 3||0.71||0.65||0.64|
|Explained variance||20.4 %||12.1 %||9.2 %|
|Total explained variance||41.7 %|
Note. Results derived from Principal Component Analysis with a forced 3-factor solution, using Direct Oblimin rotation with Kaiser Normalization. Factor loadings are taken from the structure matrix.
1 Item # 17 cross-loaded on Factor 1 and Factor 3, and was retained within Factor 3 in line with theory.
2 The subsequent PCA of the items belonging to Factor 3 revealed that item # 12 did not load appropriately on the factor, and it was therefore removed from the scale.
3 Considering the results for items # 12 and # 17, the Cronbach’s α estimate for Factor 3 included item # 17, but excluded item # 12.
The analysis of the items belonging to Factor 3, the deep approach scale, showed that item # 12 (“Often I find myself questioning things I hear in lectures or read in books”) did not load appropriately on the factor (loading 0.29), and that the internal consistency of the items would increase by removing it from the scale. After removal of this item, the subsequent analysis showed that all remaining items loaded appropriately: factor loadings were between 0.74 (item # 2: “When I’m reading an article or a book, I try to find out for myself exactly what the author means”) and 0.50 (item # 15: “Ideas in course books or articles often set me off on long chains of thought of my own”). The factor explained 41.8 % of the data variance, and internal consistency of the items was α = 0.64. Removal of any additional items would result in decreased internal consistency.
Bivariate associations between the short and the full-length scales
The short deep approach scale correlated r = 0.86 with the full deep approach scale. Similarly, the short and full strategic approach scales correlated r = 0.85, and the short and full surface approach scales correlated r = 0.87.
Prediction of average exam grade
The results from the two linear regression analyses are shown in Table 3. When controlling for gender and cohort, higher scores on the full version strategic approach scale were associated with better average grades. The short strategic scale showed a trend in the same direction, but this result did not reach statistical significance. Higher scores on both of the surface approach scales were associated with poorer average grades. The analysis with the full-length scales produced, for the most part, larger effect sizes and more credible probability estimates than the short scales. In addition, the full scales accounted for 13.2 % of the variance in exam grades, compared to the 11.2 % of the variance accounted for by the short scales. Neither gender, nor cohort, was significantly associated with exam grades.
|ASSIST scales||Average exam grade|
|Analysis 1: The full scales||β||p|
|Explained variance||2.2 %||0.19|
|Strategic approach||0.27||< 0.01|
|R2 change||11.0 %||< 0.001|
|Explained variance||13.2 %||< 0.01|
|Analysis 2: The short scales||β||p|
|Explained variance||2.6 %||0.15|
|Surface approach||–0.23||< 0.01|
|R2 change||8.6 %||< 0.01|
|Explained variance||11.2 %||< 0.01|
Note. Results derived from linear regression analyses. Table content is standardized β weights, their corresponding p-values, and the outcome variance proportions explained by the models. Higher scores on gender indicates female, higher scores on cohort indicates more advanced levels.
This study examined the factor structure and internal consistency of the items included in the short version of the ASSIST. Moreover, associations between the short and full version scales were examined, and the scales were compared with a view to their ability to predict outcome, operationalized as the students’ average exam grade.
Factor structure and internal consistency
The study showed that the items included in the short ASSIST scales conformed to the theoretically proposed factor structure (Entwistle, 2017; Entwistle et al., 2006). All the six items on the strategic and surface scales loaded appropriately on the scales with no cross-loadings. One item (# 17) on the proposed deep scale had the highest loading on the strategic scale, but had also high loading on the deep scale (see Table 2). The item is concerned with the student connecting the studied literature with “what’s being said” (see Table 1), presumably referring to the content of lectures and seminars. This aspect of connecting and contrasting ideas, and looking for evidence to support them, corresponds well with the theoretical description of the deep approach to studying, as provided by Entwistle and coworkers (Entwistle et al., 2006; Entwistle & Ramsden, 1983; Tait et al., 1998). Thus, for theoretical reasons, this item was retained within the deep scale. The strategy of retaining cross-loading items within their theoretically proposed scale is in agreement with current advice on factor analysis (Pett, Lackey, & Sullivan, 2003).
The fact that one item cross-loaded on the deep and the strategic scales is hardly surprising. Positive associations between the deep and strategic scales are frequently found in studies where the full ASSIST has been used (Bonsaksen, Småstuen, et al., 2017; Byrne et al., 2004; Diseth, 2001; Entwistle et al., 2000; Richardson, 2005, 2010; Richardson et al., 2005; Valadas et al., 2010). In addition, researchers have generally avoided trying to verify the factor structure of the ASSIST at the item level, and have rather confirmed the main scales based on the structure of the 13 subscales (e.g., Bonsaksen, Småstuen, et al., 2017; Kreber, 2003; Valadas et al., 2010). The magnitude of items comprising the full ASSIST tend to produce more factors than the three factors as theoretically proposed. In view of this, some cross-loading of single items also seems logical.
When repeating the PCA for each of the three scales, using forced extraction of one factor, the structure of the strategic and surface scales was as expected from theory. Internal consistency was at a satisfactory level for the strategic scale but in the lower range for the deep and surface scales (see Table 2). The analysis of the items proposed to belong to the deep factor, however, revealed that item # 12 did not load appropriately on the factor. Moreover, the internal consistency of the scale increased when this item was deleted. The item describes the student as often “questioning things I hear in lectures or read in books” (see Table 1). In comparison to the other items on the deep scale, this item may be interpreted as describing the student with more of a critical stance toward the course and those providing the course lectures and the educational activities in general. If this interpretation was commonly held in the sample, it may contribute to explain why the scores on this particular item were different from the pattern of scores on the other deep approach items.
Nonetheless, the internal consistency estimates for the deep and surface scales were low. Scales with few items are generally found to produce lower internal consistency estimates than scales with more items (Ponterotto & Ruckdeschel, 2007; Streiner & Norman, 2008), and this consideration applies to the short ASSIST scales. Moreover, these results mirror the findings of a previous study using the short ASSIST scales with mixed-discipline master’s degree students (Heinström, 2005), where Cronbach’s α was 0.66 (deep approach), 0.63 (surface approach) and 0.67 (strategic approach). Taken together, it appears that relatively low consistency between scale items is to be expected when using the short ASSIST scales.
The strategic, surface and deep factors explained 20.4 %, 12.1 % and 9.2 % of the data variance, respectively (total explained variance 41.7 %; see Table 2). These proportions of explained variance are lower than the variance proportions explained in studies of the full version ASSIST, and it may be that the somewhat different scale composition of the short ASSIST can contribute to explain this. In the short ASSIST, no items are taken from the subscales interest in ideas (deep approach), monitoring effectiveness, alertness to assessment demands (both strategic approach), or syllabus-boundness (surface approach). On the other hand, the subscales time management (strategic approach) and unrelated memorizing (surface approach) are given more weight to their respective scales, as these subscales (in the full version) each contribute three items. The main scales of the full-length ASSIST frequently explain between 55 % and 65 % of the total subscale variance (Byrne et al., 2004; Kreber, 2003; Richardson, 2005). Similar results have been produced in studies where the Norwegian version of the full ASSIST has been used in a mixed sample of university students (Diseth, 2001) and in a sample of occupational therapy students (Bonsaksen, Småstuen, et al., 2017). Summarizing the results of the PCA, we can therefore conclude that the factor structure of the short ASSIST is largely in agreement with the theoretically proposed structure. However, smaller proportions of the data variance were explained by the underlying factors, and removing one item from the deep approach scale produced a more coherent, and therefore better functioning, scale.
Associations between the short and full scales
As the scales of the full ASSIST version are partly comprised by the same items that comprise the short scales, strong associations between the short and full scales were expected. In this study, the strength of the associations was r = 0.86 the deep scales, r = 0.85 the strategic scales, and r = 0.87 the surface scales. To our knowledge, similar analyses of the associations between the short and full version scales have only been reported in one previous study (Heinström, 2005). In that study, correlation coefficients ranged between 0.91 (strategic scales) and 0.93 (deep and surface scales), thus indicating slightly stronger associations between scores on the short and full scales than those found in the current study. Nonetheless, in line with Pallant’s (2010) and Field’s (2005) discussions on multi-collinearity (variables correlated > 0.70 or > 0.80, respectively), we consider the strength of the associations as indicating that the short and full version scales do in fact measure the same concepts.
Predictive ability of the scales
In line with the results indicating strong associations between the short and full version scales, we also expected them to be relatively equal in terms of their ability to predict outcome. This expectation was confirmed as the pattern of associations between the short scales and the students’ average exam grades was similar to the pattern shown for the full version scales (see Table 3). However, the effect sizes and proportion of explained outcome variance were somewhat larger when using the full ASSIST scales, in comparison to the short scales. The significance levels associated with these findings were also different from each other – the probability of falsely rejecting the null-hypothesis was lower when using the full version scales than when using the short scales. Similar comparisons of the predictive ability of the short and full version scales appear not to have been reported previously.
The study is limited by its relatively small sample (n = 148), in particular in relation to the initial PCA conducted with the 18 items of the short ASSIST. Nunally (1978) suggested a 10:1 ratio between subjects and variables included in multivariate analyses, and for the analysis of the 18-items instrument, that was not the case in this study. The internal consistency estimates for the resulting scales were in the lower range. Cronbach’s α > 0.70 is usually considered a good level of internal consistency, but this level is known to vary by the number of items included on the scale. The convenience sample and the recruitment of students at only one university are also limitations of the study. Therefore, the results of the study should be interpreted with caution. However, the sample mean age and the proportion of female students largely mirror the sample used in a cross-cultural study of occupational therapy students in four countries (Brown et al., 2016).
Students in different year cohorts may have different response patterns to a questionnaire like the ASSIST, and if so, this may play a role for the ability to generalize the results to the larger population. However, recent research suggests that there is little difference in ASSIST scale and subscale scores between cohorts of undergraduate occupational therapy students during their first three years of study (Bonsaksen, Thørrisen, & Sadeghi, 2017; Brown & Murdolo, 2016). In turn, this may indicate that the response patterns were similar across the year cohorts investigated in this study.
More generally, the reliance on self-report methodology may also represent a limitation of the study. Self-report measures of behaviors and attitudes have been used in the social and behavioral sciences for decades, and have received substantial criticisms for lack of validity (Podsakoff & Organ, 1986). In particular, criticisms have fallen on measures that cannot be verified by some objective means, on measures that introduce a consistency motif, and on measures within which some responses comply with social norms, whereas others do not. The latter aspect of “social desirability” has been widely discussed within practice fields where norms are particularly relevant, for example in the field of drug and alcohol abuse (Whitford, Widner, Mellick, & Elkins, 2009). However, study behaviors among students may also be norm-laden, such that the students participating in this study may have felt that some responses were desired whereas others were not. A consistency motif may have played a part in the students’ response-giving, and we do not have access to other measures or observations that can verify the student’s self-reports. Future studies may employ an integrated approach, combining self-report measures with other measurement techniques, as recently advised (Fulmer & Frijters, 2009).
This study showed that the short version of the ASSIST functioned largely as expected from theory. All 18 items taken from the full version ASSIST loaded on the appropriate scale, and only one item cross-loaded. In the separate analyses of the resulting scales’ factor structure, one item on the deep approach scale failed to load appropriately on the underlying factor and contributed to reduced internal consistency between the scale items. As a result, this item was removed from the scale. There were very strong associations between the short and the full version scales, indicating that they measure the same concept. The short scales predicted outcome in a similar way compared to the full version scales, but stronger associations with the outcome, more credible significance levels, and larger explained variance proportions were found for the full version scales. In summary, the study showed that the short ASSIST had good psychometric properties in the employed sample. Thus, the instrument represents a feasible and trustworthy method of assessing approaches to studying among Norwegian occupational therapy students.
The author would like to express his gratitude to the students who volunteered to participate in the study.