In their 2018 article Academic Engagement and Student Success: Do High Impact Practices Mean Higher Graduation Rates, Johnson and Stage hypothesized the following:
- First-year seminars, writing requirements, learning communities, and service learning, as high impact practices, would have a positive correlation with graduation rates.
- “Freshman seminar would have the largest association with graduation rates” (Johnson & Stage, 2018, p. 761).
Johnson and Stage (2018) spent a considerable portion of their article on the literature review providing rationale for why they chose the institutions, why they chose the institutional characteristics, and outlining and defining the high impact practices identified by the Association of American Colleges & Universities (AAC&U).
Credibility
To assess the credibility of an article, Coughlan, Cronin, and Ryan (2007) suggest analyzing whether an article describes and follows appropriate guidelines for the research process. Steps that researchers should follow to influence the credibility of a study include: being well written, having an abstract, title, and author qualifications listed; and having the following logistical order:
- literature review;
- theoretical framework;
- research question;
- methodology;
- data analysis; and
- Discussion (Coughlan, Cronin, & Ryan, 2007).
Digging further, Coughlan, Cronin, and Ryan (2007) recommend having peer-reviewed articles that are less than five years old in the literature review. Due to the robustness of this study, Johnson and Stage (2018) referenced many articles that did not meet this requirement. However, based on the other guidelines provided by Coughlan, Cronin, and Ryan (2007), Johnson and Stage’s (2018) study meets the criteria of being a credible study.
Applicability and Generalizability
Johnson and Stage (2018) placed focus on whether these high impact practices should be widely adopted due to financial institutional influences and constraints. This study by Johnson and Stage (2018) focuses on “student outcomes on multiple college campuses” (p. 754) with these campuses categorized based on their level of competitiveness from Barron’s Profile of American Colleges.
Through collection of primary data from 244 institutions, with enrollment at minimum of 10,000 students, and a response rate of 41.4%, Johnson and Stage (2018) provided institutions with four response choices: “required for all students,” “required for some students,” “optional for students,” and “not offered.” For regression analysis, the data from this study shows that for six year graduation rates, all else being equal, institutions with freshman seminar are more likely to have higher graduation rates and have a higher rate of students receiving federal financial aid. The coefficients for these variables were statistically significant at p < .01. For Spearman correlation, freshman seminar was negatively and significantly correlated to four year graduation rates for most-selective institutions.
The authors acknowledge that “no study that includes a limited population can address the institutional factors that might influence time to degree or an institution’s implementation of high-impact practices” (Johnson & Stage, 2018, p. 774). In addition, data from the regression models used in the study did not associate high-impact practices with institutional or student characteristics (Johnson & Stage, 2018). These admissions negatively influence a reader’s perception regarding the hypotheses presented being addressed. Based on this information, I believe the Johnson and Stage’ (2018) hypotheses were satisfied even if disproven. I do not believe that the hypotheses will hold true for a larger population sample and therefore is not generalizable however, this study did satisfy the research questions.
References
Coughlan, M., Cronin, P., & Ryan, F. (2007). Step-by-step guide to critiquing research. Part 1: quantitative research. British Journal of Nursing, 16(11), 658-663.
Johnson, S. R., & Stage, F. K. (2018). Academic engagement and student success: Do high-impact practices mean higher graduation rates? The Journal of Higher Education, 89(5), 751-781. DOI: 10.1080/00221546.2018.1441107.