National Institute for Learning Outcomes Assessment |

National Institute for Learning Outcomes Assessment

NILOA Guest Paper Response

We’ve invited thought leaders to provide a response to Occasional Papers that NILOA crafts. We hope that these pieces will spark further conversations and actions that help advance the field. Please sign up here to receive monthly newsletters that headline these pieces along with NILOA updates, current news items, and upcoming conferences.

Response to Equity and Assessment: Moving towards Culturally Responsive Assessment

Jodi Fisler
Associate for Assessment Policy and Analysis at the State Council of Higher Education for Virginia (SCHEV)

Montenegro & Jankowski’s paper, Equity and Assessment: Moving Towards Culturally Responsive Assessment, offers a valuable opportunity to think about how our approaches to assessment can contribute to—or diminish—students’ sense of worth and belonging in their classrooms and in a broader community of learners.

I agree with the authors that most learning can be demonstrated in a number of ways, and that we do a gross disservice to students when we suggest there is one and only one way to show mastery of a skill or body of knowledge. The authors, as well as several respondents, make a compelling argument for the importance of expanding our mental models regarding assessment and, indeed, making the entire assessment process more inclusive, from the identification and articulation of learning outcomes right through to the interpretation and use of findings.

At the same time, this paper prompted a number of thoughts and questions in my mind. I offer some of them here in the spirit of genuine inquiry and in the hope that they might deepen our collective thinking on this important issue.

1. Cultural responsiveness vs. student responsiveness

I found myself wondering throughout the paper why the authors chose to frame this topic in terms of cultures as opposed to individuals. Cultures reflect group norms, experiences, and characteristics. As Montenegro & Jankowski acknowledge, many cultural influences intersect to shape a given individual’s experiences and needs. Once we start talking about intersectionality, the characteristics of groups become less meaningful because each new layer of cultural identity changes how a given individual relates to any one of the contributing cultures that make up who they are.

Culture is important, of course, and my question is not about ignoring real and legitimate patterns of thinking, behavior, and experiences that are shared by members of cultural groups. It is important to learn about these patterns and to keep them in mind. However, we shouldn’t see them as determinative. I may make some initial assumptions about specific students’ experiences based on what I know (or think I know) of their cultural identities, but I should always be prepared to adjust my thinking.

So, if we can’t use our knowledge of particular cultures to reliably inform how we work with individual students from those cultures, why refer to culture at all? Perhaps it’s important because there are still people who deny (or who have never considered) the relevance of cultural norms, or who dismiss culturally-related ways of knowing as inferior. In that sense, explicitly recognizing the fact of culture and its influence—as complex and multifaceted as that is—may be critical, even if in practice we tailor our assessments around individual students’ needs.

2. Students and faculty need a (reasonably) common language for both form and content.

I co-taught a class on women in leadership a few years ago. The final assignment asked students to draft their own “leadership manifesto,” articulating their personal vision of leadership and the commitments they would make to themselves and others in light of what they had learned in the course. My co-instructor and I told the students they could present their manifesto in whatever form they felt was most appropriate to express what they wanted to say. Most students wrote traditional reflection papers, a few did PowerPoint presentations. One student presented her manifesto in the form of a Zumba routine.

Neither I nor my co-instructor were familiar with Zumba. I was intrigued by the idea though. The student designed the routine and led me through it in a one-on-one session. Afterward, I asked her to explain how the routine drew on what she had learned in class, which she did with conviction. She also told me that she had ADHD and was excited to have been allowed to present her final assignment through physical activity. It was clearly meaningful for her.

The experience left me thinking about the nature of learning and about various aspects of assessment. Although I was pleased to be able to give this student an option that spoke to her, had she really met the expectations of the assignment? She had chosen to communicate her learning through the language of Zumba, which was a language I didn’t speak. So I had to ask her to translate for me and explain how what she had done related to the course outcomes. Was her grade, therefore, based on her Zumba routine or on her ability to express her routine in words? Did it matter?

As we think about giving students greater latitude to demonstrate or communicate their learning in less conventional ways, we need to think about the ability of faculty members to evaluate what the students do. We wouldn’t want to give students the option to be creative in their demonstrations of learning if the faculty lack sufficient knowledge of the form or style of expression to give an accurate assessment of the content.

3. Sometimes the form of expression is part of the learning.

Although there is much to gain by applying culturally responsive assessment strategies, we should also give some thought to what might be lost. Many institutions place a high priority on developing students’ skill in written communication, for example. As with any skill, learning to write well requires a good deal of practice. Having students demonstrate their learning through essays or research papers gives them an opportunity to practice writing beyond whatever the topic of the course might be.

As more and more faculty adopt inclusive approaches to assessment, how will institutions ensure students get sufficient practice in core skills to be prepared for whatever they choose to do after college? As much as students might prefer to demonstrate their learning through alternative means, is that necessarily a good thing in every course? I’m not referring here to practical skills like repairing a car engine or drawing blood; we could probably all agree that learning of that sort would need to be demonstrated in a particular way. But would we be comfortable if a student graduated without ever writing anything substantial (outside of a course dedicated to writing, perhaps)? If all faculty were to use culturally responsive assessment in their courses, a considerable level of coordination and communication would likely be necessary to make sure that students didn’t unintentionally (or intentionally) miss out on key learning experiences over the course of their program.

4. What are the practical implications of this for faculty?

Culturally responsive assessment, at least as Montenegro & Jankowski describe it, requires an investment of time and energy by faculty to (1) engage students in the conversation about learning outcomes, assignments, etc, (2) review and revise tests, rubrics, and other evaluation tools in accordance with student needs, and (3) actually evaluate whatever students submit. Even assuming positive regard on the part of the faculty, how much can we realistically expect them to add to what they are already doing? Would we expect an instructor to negotiate learning outcomes and assignments with an entire class of 200 students every semester, for instance? That would seem to be the ideal, at least in theory, if the goal is to help students feel involved in their own learning. But that also seems unreasonable from a practical standpoint. Some faculty who teach large classes have already abandoned assessments that take a lot of time to grade in favor of multiple-choice or short-answer exams that can be graded quickly by teaching assistants. So, given current realities in higher education, what are the optimal levels of student involvement and faculty responsiveness?

5. What is the appropriate role of a state or other governing/oversight body?

Since I work at a state coordinating agency and am responsible for developing and overseeing policies regarding assessment, I wonder how the ideas laid out by Montenegro & Jankowski should inform what I do at the state level. In Virginia’s new assessment policy (still being finalized at the time of this writing), we are asking institutions to disaggregate their student learning assessment data so that they can monitor how well different student populations are developing the desired core competencies. Should we be doing more? If so, what?

These comments and questions are based on my understanding of culturally responsive assessment as presented by Montenegro & Jankowski. I readily accept that my understanding may be flawed. I found myself wishing for concrete examples at several points in their paper, which brings me to my final thought. I’m sure no one would dispute that higher education in the United States is under a great deal of scrutiny right now, and there are strong concerns about quality and accountability among many lawmakers, employers, and members of the public, as well as among some educators. Some people see efforts to increase equity as sacrificing rigor in favor of an “anything goes” mentality. For those of us who believe that equity is still an unmet goal, and that higher education can do a better job of recognizing legitimate learning by our students, we need to be very clear and specific about what we are (and are not) talking about when we advocate for inclusive assessment practices.

I appreciate the invitation to be part of this conversation, and I am eager to see how it develops.

About the author:

Jodi Fisler has served as the Associate for Assessment Policy and Analysis at the State Council of Higher Education for Virginia (SCHEV) since February 2016. Prior to that, she worked for six years as the Director of Student Affairs Planning & Assessment and Assistant to the Vice President for Student Affairs at the College of William & Mary. Jodi received her B.A. in German Studies from Williams College and her M.Ed. and Ph.D. degrees in Educational Policy, Planning and Leadership (Higher Education concentration) from the William & Mary School of Education.

For additional responses to this paper please click below:

Video response by Dr. Eboni Zamani-Gallaher

Response by Dr. Thomas F. Nelson Laird & Dr. Allison BrckaLorenz

Response by Melissa Wright

Response by Pamela Felder

Response by Jan McArthur

Response by Ereka Williams

Response by Joseph D. Levy and Ciji A. Heiser

Response by Sheri Williams and Frank Perrone