National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Viewpoints

We’ve invited learning outcomes experts and thought leaders to craft a Viewpoint. We hope that these pieces will spark further conversations and actions that help advance the field. To join the conversation, click the link below the Viewpoint. You can also sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

 

Learning-Oriented Assessment in Practice
David Carless

 

What does good classroom assessment look like in the contemporary university? What kinds of assessment-related activities encourage productive student learning processes? How do excellent teachers manage assessment? These are some of the issues discussed in my recent book Excellence in University Assessment: Learning from award-winning practice (Carless, 2015a).

In the book, I frame the discussion with the concept of learning-oriented assessment: assessment with a primary focus on promoting productive student learning processes. In learning-oriented assessment, learning comes first (literally!). Learning-oriented assessment is worth distinguishing from other related terminologies, such as formative and summative assessment.

The formative/summative distinction has a long lineage dating back to Scriven (1967) but risks downplaying the need for summative assessment to encourage productive learning. Other potentially useful terminologies: assessment for/of/as learning can end up becoming an exercise in prepositions! The essence of learning-oriented assessment is that all assessment whether predominantly summative or formative in orientation is focused on developing effective student learning processes (Carless, 2015b).

Learning-oriented assessment involves three inter-related components. First, productive assessment task design when students are assessed on meaningful tasks which require higher order learning outcomes. Second, activities which support students in developing understandings of what quality work looks like: going beyond rubrics and lists of criteria to explore quality academic performance. And third, approaches to feedback processes which focus less on telling and more on entering into different forms of dialogue about student work, so that students are primed to engage with and act on feedback messages.

The three components of this learning-oriented assessment framework served as a lens to analyze the practices of five teachers who had been awarded internal or external awards for teaching excellence. These teachers came from the disciplines of Architecture, Business, Geology, History and Law. The critical analysis of their practice and students’ perceptions of it facilitate the portrayal of assessment in context.

For instance, one of the key features of assessment task design was authentic assessment or what I prefer to call ‘assessment mirroring real-life applications of the discipline’. Architecture students designed a village house. Business students tried to sell their innovative product ideas through assessed oral presentations. Geology students engaged in experiential learning through field trips. History students visited a museum of their own choice and analyzed how history was portrayed. Law students went to Court to observe legal debate in context and wrote up their reflections.

A striking issue in the design of assessment was student flexibility and choice. Students felt empowered to develop their best performance when they had some autonomy in what they worked on and how they approached assessment tasks. Students could often choose topics; there was also some flexibility between individual, pair and group assignments. The History professor permitted web-based presentations as alternatives to traditional essays. Students could upload work in progress to Facebook if they wished to interact via social media. In Law, students had the option of adjusting the weighting of their examination performance by carrying out alternative assessment tasks.

In relation to the second element of learning-oriented assessment: how students come to appreciate what quality work looks like, students did not find rubrics particularly useful. Lists of criteria in a standardized form were perceived as vague and repetitive. Furthermore, students often did not feel that the rubric represented how they would actually be assessed. They felt that in reality, criteria would be outweighed by teachers’ personal feelings and subjective preferences. This resulted in students trying to identify what individual teachers sought: a phenomenon identified as long ago as the 1960s (Becker et al., 1968).

Instead, students expressed positive perceptions of the analysis of samples of student work. Such exemplars could help students understand what was required and what quality academic performance looked like. Productive dialogues about exemplars could help students articulate their judgments of student work; discuss how samples could be improved; and relate them to their own work in progress. Students also expressed a wish for more teachers to analyze samples during class.

Feedback is an essential component of progress but difficult to carry out effectively in undergraduate education. Challenges include timeliness, ability to act on feedback and the inevitable emotional consequences of assessment (Evans, 2013). Possible ways forward are centered on dialogic feedback processes. These include peer review, internal feedback (inner dialogue) to promote self-regulation and technology-facilitated dialogue. These strategies are consistent with the principal goal of feedback as supporting students in refining their capacities to self-evaluate their work in progress.

So in brief, what might good assessment look like? It should entail assessment tasks which encourage higher order thinking and deep approaches to learning. Students would need to be actively involved in generating, applying or engaging with criteria and quality. For feedback as dialogue to be carried out effectively, feedback processes need to reduce the burden on teachers and make students more pro-active in generating and using feedback. Importantly, to implement a learning-oriented assessment approach shared expertise around meaningful assessment needs to be developed for both faculty and students: an ongoing challenge and possible future agenda.

References

Becker, H., Geer, B. & Hughes, E. (1968). Making the Grade: The Academic Side of College Life. New York: John Wiley and Sons.

Carless, D. (2015a). Excellence in University Assessment: Learning from award-winning practice. London: Routledge.

Carless, D. (2015b). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.

Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70-120.

Scriven, M. (1967). The methodology of evaluation. In R. Tyler, R. Gagne & M. Scriven (Eds.), Perspectives on Curriculum Evaluation (pp. 39-83). Chicago, IL: Rand McNally.

 

Check out our past Viewpoints:

Learning-Oriented Assessment in Practice
David Carless

Moving Beyond Anarchy to Build a New Field
Hamish Coats

The Tools of Intentional Colleges and Universities: The DQP, ELOs, and Tuning
Paul L. Gaston, Trustees Professor, Kent State University

Addressing Assessment Fatigue by Keeping the Focus on Learning
George Kuh and Pat Hutchings, NILOA

Evidence of Student Learning: What Counts and What Matters for Improvement
Pat Hutchings, Jillian Kinzie, and George D. Kuh, NILOA

Using Evidence to Make a Difference
Stan Ikenberry and George Kuh, NILOA

Assessment - More than Numbers
Sheri Barrett

Challenges and Opportunities in Assessing the Capstone Experience in Australia
Nicolette Lee

Making Assessment Count
Maggie Bailey

Some Thoughts on Assessing Intercultural Competence
Darla K. Deardorff

Catalyst for Learning: ePortfolio-Based Outcomes Assessment
Laura M. Gambino and Bret Eynon

The Interstate Passport: A New Framework for Transfer
Peter Quigley, Patricia Shea, and Robert Turner

College Ratings: What Lessons Can We Learn from Other Sectors?
Nicholas Hillman

Guidelines to Consider in Being Strategic about Assessment
Larry A. Braskamp and Mark E. Engberg

An "Uncommon" View of the Common Core
Paul L. Gaston

Involving Undergraduates in Assessment: Documenting Student Engagement in Flipped Classrooms
Adriana Signorini & Robert Oschner

The Surprisingly Useful Practice of Meta-Assessment
Keston H. Fulcher & Megan Rodgers Good

Student Invovlement in Assessment: A 3-Way Win
Josie Welsh

Internships: Fertile Ground for Cultivating Integrative Learning
Alan W. Grose

What if the VSA Morphed into the VST?
George Kuh

Where is Culture in Higher Education Assessment and Evaluation?
Nora Gannon-Slater, Stafford Hood, and Thomas Schwandt

Embedded Assessment and Evidence-Based Curriculum Mapping: The Promise of Learning Analytics
Jane M. Souza

The DQP and the Creation of the Transformative Education Program at St. Augustine University
St. Augustine University

Why Student Learning Outcomes Assessment is Key to the Future of MOOCs

Wallace Boston & Jennifer Stephens

Measuring Success in Internationalization: What are Students Learning?
Madeleine F. Green

Demonstrating How Career Services Contribute to Student Learning
Julia Panke Makela & Gail S. Rooney

The Culture Change Imperative for Learning Assessment
Richard H. Hersh & Richard P. Keeling

Comments on the Commentaries about "Seven Red Herrings"
Roger Benjamin

Ethics and Assessment: When the Test is Life Itself
Edward L. Queen

Discussing the Data, Making Meaning of the Results
Anne Goodsell Love

Faculty Concerns About Student Learning Outcomes Assessment
Janet Fontenot

What to Consider When Selecting an Assessment Management System
R. Stephen RiCharde

AAHE Principles of Good Practice: Aging Nicely A Letter from Pat Hutchings, Peter Ewell, and Trudy Banta

The State of Assessment of Learning Outcomes Eduardo M. Ochoa

What is Satisfactory Performance? Measuring Students and Measuring Programs with Rubrics
Patricia DeWitt

Being Confident about Results from Rubrics Thomas P. Judd, Charles Secolsky & Clayton Allen

What Assessment Personnel Need to Know About IRBs
Curtis R. Naser

How Assessment and Institutional Research Staff Can Help Faculty with Student Learning Outcomes Assessment
Laura Blasi

Why Assess Student Learning? What the Measuring Stick Series Revealed
Gloria F. Shenoy

Putting Myself to the Test
Ama Nyamekye

From Uniformity to Personalization: How to Get the Most Out of Assessment
Peter Stokes

Transparency Drives Learning at Rio Salado College
Vernon Smith

Navigating a Perfect Storm
Robert Connor

Avoiding a Tragedy of the Commons in Postsecondary Education
Roger Benjamin

In Search for Standard of Quality
Michael Bassis

It is Time to Make our Academic Standards Clear
Paul E. Lingenfelter