Browse Topics

• Human Subject Research and IRB
• Literature and Systematic Review
• Program and project evaluation
• Research Design
• Data Collection and Analysis
• Datasets
• Funders reports

Human Subject Research and IRB

It is important to consider the extent to which OER related assessments are done as program improvements or qualitative improvement and are not “research” requiring an IRB exemption or approval. Even if the assessment activity is “human subject research”, when data is collected anonymously, the research may fall within the exempt categories 45 CRF 46.101(b)(1) (established educational settings) and (b)(2) (anonymous data collection).

• The CUNY Human Research Protection Program (HRPP)
• HHS.gov Quality Improvement Activities FAQs

Literature and Systematic Review

• Open Education Reader (edited by David Wiley)
• Open Education Group’s The Review Project provides a summary of all known empirical research on the impacts of OER adoption. This is similar to the journal article on empirical research relating to perceptions and efficacy at the college level (Aug. 2019)
• On the Sustainability of Open Educational Resource Initiatives in Higher Education (D Wiley)
• OER Hub publication list of research articles, reports, conference proceedings, and books
• Open to What? A Critical Evaluation of OER Efficacy Studies by Ian McDermott (2020)
• Open Educational Resources open access edition [ala.informz.net] includes six sections — Introduction, Literature Review and Bibliography, Analysis and Discussion of Survey Results, Survey with Results, Resources from Survey Respondents, and Sample Documents. It provides context for OER initiatives in college libraries along with insights and strategies for librarians who are new to the topic, part of an existing OER initiative, or looking to form a program at their institution. (ALA.org)  

CUNY OER and related studies

CUNY OER and related assessments on CUNY Academic Works include:

• But What Do The Students Think: Results of the Cross-Campus Zero-Textbook Cost Student Survey” (2019)
• How Do Open Educational Resources (OERs) Impact Students? A Qualitative Study at New York City College of Technology (2016) (master’s thesis)
• Textbook Evaluation Toolkit (2019)
• Student textbook purchasing: the hidden cost of time” (2019)
• A quantitative analysis of the relationship between an online homework system and student achievement in pre-calculus (for a fee-platform)

Questions to consider in reviewing the literature

1. What is the problem statement? How is the problem, question, or issue framed?
2. Are the students sampled in the studies similar to those at LGACC (Overall, in 2018, 22% Asian, 17% Black, 48% Hispanic, 11% White and 2% Other/Unknown); 55% full-time; 46% part-time; 59% foreign born; 59% received financial aid)? See 2019 Institutional Profile – LaGuardia Community College
3. Is the study at the community college level?
4. What is the sample size/response rate?
5. What is the theory of change? What are the hypotheses? What are the assumptions?
6. What types of OER are included in the study?
7. What disciplines are included?
8. What is the research design? 
9. What/who are the comparison groups, if any?  Is there a controlled group? What comparisons are being made?
10. Has there been a change in learning or teaching? What is/are the change(s)?  Positive, negative, neutral/none?
11. Are the data and instruments available for review?

Program and Project evaluation

• New York Open Educational Resources Funds CUNY OER Year One Report; CUNY OER Year Two Report
• Babson Survey Research Group OER reports on OER awareness and use (annual since 2012)
• OER initiatives may have program evaluations posted or to share. See OER and Textbook Affordability Initiatives (list for North America) or a similar SPARC list
• The Evolution of Affordable Content Efforts in the Higher Education Environment: Programs, Case Studies, and Examples
• Achieve the Dream OER Degree
• MIT 2005 Program Evaluation Findings Report
• Lessons Learned – Adaptive Courseware
• OER at Scale: The Academic and Economic Outcomes of Achieving the Dream’s OER Degree Initiative (final report, 2020), 2018 report, and 2017 report.

Research Design

• OER Research Guidebook (PDF, Word, InDesign), COUP Framework
• Do open educational resources improve student learning? Implications of the access hypothesis. (Online, PDF)
  The article recommends surveying students before the intervention to see who did not get the textbook and after to see if their access changed. The study should have an explicit hypothesis about why using OER would result in improved learning outcomes.  Under the authors’ access hypothesis, access — getting the textbook — is the primary mechanism for why there might be improved learning. This assumes that if students get the textbook, they will use it, with the caveat that different types of usage can results in different learning outcomes.

Data Collection and Analysis

• Just-in-Time Diagnostic Skills-Pre-Test, Math Attitudes Inventory short form, and Student Feedback Survey on MyOpenMath. Faculty Guide
• CUNY OER Research Program Assessment Template
• First cross-campus survey of CUNY student opinions on Zero Textbook Cost (ZTC) course. Survey instrument is at goo.gl/ZY7ZcG
• The OER Research Toolkit (Open Education Group)
  • Open Education Group Student Survey (based on Bliss et al. (2013))
  • Open Education Group Faculty Survey (based on Bliss et al. (2013))
  • Faculty Survey (Jhangiani, Jhangiani, 2017).
  • OER Hub Student Survey
  • OER Hub Faculty Survey
  • Digital Open Textbook Questionnaire (from Jhangiani, Dastur, Le Grand, & Penner, article currently submitted)
  • Sample Data Collection Template with Definitions
• US Office of Education Technology Research and Innovation
• OER Adoption Impact Calculator (by Lumen Learning and Open Education Group)
• OER Hub survey of 7000 OER users and their perceptions of OER impact on teaching and learning (survey questions, interview questions)
• Achieving the Dream’s OER Degree Initiative instructor survey and student survey questions.

Datasets

• First cross-campus survey of CUNY student opinions on Zero Textbook Cost Data (2018 survey data download)
• OER Hub survey of 7000 OER users and their perceptions of OER impact on teaching and learning (Data CVS , xlsx)

Funder Grantmaking and Reports

William and Flora Hewlett Foundation

• Open Educational Resources Advancing Widespread Adoption to Improve Instruction and Learning
• OER Technology Assessment
• Grants database