National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Viewpoints

We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

 

Making Assessment Count
Dr. Maggie Bailey, Vice Provost Program Development and Accreditation, Point Loma Nazarene University

 

Most universities have highly developed processes for using assessment data to inform faculty how well students are achieving the program’s desired learning objectives, as well as identifying those areas needing improvement.  However, this key evidence is too often not incorporated in other institutional decision-making processes such as program review, budgeting, program development, and strategic planning.  PLNU’s Assessment and Program Review committees decided to tackle this issue by linking assessment findings directly into the program review process.  The program review findings, including assessment, are then embedded in the program review Memorandum of Understanding (MOU), and used to inform budgeting and strategic planning. 

When the two Committees began their discussions around harmonizing their processes, there were already key practices embedded in the PLNU assessment culture that made this “next step” in closing the assessment loop an easy transition.  First, in spring 2012 PLNU adopted the NILOA Transparency Framework for curricular and co-curricular assessment reporting on the web.  The PLNU adaptation of the NILOA Framework we refer to as the PLNU Assessment Wheel.  Through the use of the wheel, the culture of accountability and transparency has been widely accepted in the PLNU ethos.  Second, PLNU was well into revising both the Program Review and Assessment policies and practices to move the university away from a compliance model toward a continuous improvement cycle based on a culture of evidence.  Third, the Program Review Committee had recently adopted a culminating Memorandum of Understanding (MOU) linking the outcomes of the program review self-study and external review to university budgeting and strategic planning.  Therefore, a stronger and more intentional integration of the Assessment Committee’s review into the program review process was a natural next step in the development of the two Committees’ collaboration. 

The PLNU Assessment and Program Review Committee faculty noted that when academic units go through program review, the focus shifts from internal course level assessment to external program level analysis, and the academic program self-study rarely closes the loop in articulating how assessment evidence is informing program improvements.  Additionally, the Program Review Committee is focused on academic unit level issues informed by financial, enrollment, retention rates and other data and the important assessment findings are not always fully considered in the Program Review Committee’s decision-making.  The message sent to faculty is that while assessment data is very important to the faculty culture and curriculum improvement, it is not essential to resourcing decisions or strategic priorities of the institution. 

The Academic Assessment and Program Review Committees agreed the first step would be to harmonize the rubrics the committees use to conduct their evaluations. The Committees found in their respective rubrics that there were gaps and overlaps, as well as confusion in the assessment language.  The committees delineated responsibilities and agreed to revise the rubrics to avoid redundancy, mixed messages, and gaps in accountability.  In addition, the Committees decided to sequence and coordinate their reviews so that academic units moving through program review would first receive an in-depth Assessment Committee review and report.  The Assessment Committee report (4 to 6 pages) includes commendations, recommendations for improvement, and a summary of the overall quality of the assessment work being done in the program.  The report is then used by the program faculty to assist them in revising their assessment planning and activities as part of the program review process.  The assessment report must be received by the Program Review Committee before the Program Self-study will be accepted by the Program Review Committee.    

The revised process includes the following steps: 

STEP 1 - ASSESSMENT WHEEL: Academic unit maintains all assessment planning and activities documentaton in their Assessment Wheel (NILOA Transparency Framework) on the PLNU website.

wheel

STEP 2: ASSESSMENT REVIEW AND SEQUENCING: PLNU has a three-year assessment cycle and a six-year Program Review Cycle. Each program goes through two complete assessment cycles between each program review. All major curriculum changes must be accompanied by a current Assessment Committee report and Program Review MOU.

chart

STEP 3: ASSESSMENT REPORT: At the end of the second 3-year assessment cycle the Assessment Committee conducts an in-depth review followed by a Committee report (4-6 pages) on their findings, commendations, and recommendations. The Committee report is sent to the academic unit leadership, Program review Committee, curriculum committees, College Dean(s), and Provost.

STEP 4: PROGRAM REVIEW COMMITTEE: The Program Review Committee receives the Assessment Committee report and Assessment Committee rubric criteria scores on the “Quality of Assessment” are then embedded in the Program Review Self-Study Rubric.

rubric

STEP 5: MEMORANDUM OF UNDERSTANDING: The Program Review process culminates with a MOU and is linked to future budgeting, planning and resource allocation as well as a program improvement time line. The MOU contains three parts: (1) summary of academic unit and program improvements needed including assessment, (2) the action plan for the academic unit to improve the program with specific targets, and (3) a commitment from the University to provide resources, support, and faculty to achieve the agreed upon student learning outcomes.

The assessment of student learning is the foundation for program review with the focus of improving educational quality based on the culture of evidence. This assessment process is a continual, formative process of data collection, analysis, reflection and improvement.  Below is a graphic depiction of the new reporting structure linking assessment to strategic planning.

process

PLNU began using the revised process in spring 2014, and while the evidence is still preliminary there are some early indications that this more coordinated approach holds out a promise to more directly align resources to academic assessment findings or at the very least having assessment data as important evidence informing Institutional decision-making.  

 

We invite you to contribute your thoughts regarding this Viewpoint here.

Check out our past Viewpoints:

Making Assessment Count
Maggie Bailey

Some Thoughts on Assessing Intercultural Competence
Darla K. Deardorff

Catalyst for Learning: ePortfolio-Based Outcomes Assessment
Laura M. Gambino and Bret Eynon

The Interstate Passport: A New Framework for Transfer
Peter Quigley, Patricia Shea, and Robert Turner

College Ratings: What Lessons Can We Learn from Other Sectors?
Nicholas Hillman

Guidelines to Consider in Being Strategic about Assessment
Larry A. Braskamp and Mark E. Engberg

An "Uncommon" View of the Common Core
Paul L. Gaston

Involving Undergraduates in Assessment: Documenting Student Engagement in Flipped Classrooms
Adriana Signorini & Robert Oschner

The Surprisingly Useful Practice of Meta-Assessment
Keston H. Fulcher & Megan Rodgers Good

Student Invovlement in Assessment: A 3-Way Win
Josie Welsh

Internships: Fertile Ground for Cultivating Integrative Learning
Alan W. Grose

What if the VSA Morphed into the VST?
George Kuh

Where is Culture in Higher Education Assessment and Evaluation?
Nora Gannon-Slater, Stafford Hood, and Thomas Schwandt

Embedded Assessment and Evidence-Based Curriculum Mapping: The Promise of Learning Analytics
Jane M. Souza

The DQP and the Creation of the Transformative Education Program at St. Augustine University
St. Augustine University

Why Student Learning Outcomes Assessment is Key to the Future of MOOCs

Wallace Boston & Jennifer Stephens

Measuring Success in Internationalization: What are Students Learning?
Madeleine F. Green

Demonstrating How Career Services Contribute to Student Learning
Julia Panke Makela & Gail S. Rooney

The Culture Change Imperative for Learning Assessment
Richard H. Hersh & Richard P. Keeling

Comments on the Commentaries about "Seven Red Herrings"
Roger Benjamin

Ethics and Assessment: When the Test is Life Itself
Edward L. Queen

Discussing the Data, Making Meaning of the Results
Anne Goodsell Love

Faculty Concerns About Student Learning Outcomes Assessment
Janet Fontenot

What to Consider When Selecting an Assessment Management System
R. Stephen RiCharde

AAHE Principles of Good Practice: Aging Nicely A Letter from Pat Hutchings, Peter Ewell, and Trudy Banta

The State of Assessment of Learning Outcomes Eduardo M. Ochoa

What is Satisfactory Performance? Measuring Students and Measuring Programs with Rubrics
Patricia DeWitt

Being Confident about Results from Rubrics Thomas P. Judd, Charles Secolsky & Clayton Allen

What Assessment Personnel Need to Know About IRBs
Curtis R. Naser

How Assessment and Institutional Research Staff Can Help Faculty with Student Learning Outcomes Assessment
Laura Blasi

Why Assess Student Learning? What the Measuring Stick Series Revealed
Gloria F. Shenoy

Putting Myself to the Test
Ama Nyamekye

From Uniformity to Personalization: How to Get the Most Out of Assessment
Peter Stokes

Transparency Drives Learning at Rio Salado College
Vernon Smith

Navigating a Perfect Storm
Robert Connor

Avoiding a Tragedy of the Commons in Postsecondary Education
Roger Benjamin

In Search for Standard of Quality
Michael Bassis

It is Time to Make our Academic Standards Clear
Paul E. Lingenfelter