In 2021, UCLA’s Center for the Advancement of Teaching initiated a new project to help departments improve the way they evaluate teaching. The motivation for this project is simple: to improve student learning and students’ academic experience by helping faculty implement excellent teaching practices. Across our campus, there is widespread recognition that excellent teaching is too multifaceted and nuanced to be documented using a single source of evidence like student surveys. Thus, we developed a framework that we refer to as Holistic Evaluation of Teaching or HET: a way to think about and document teaching that includes self-reflection, student input, and peer review. Importantly, this new approach emphasizes faculty professional development and efforts to improve teaching. It is adaptable to the particular teaching challenges and cultures of each unit across campus, and it has been developed with broad input from faculty, the Senate, and academic leadership.
Working in tandem with the HET initiative, the Center for Educational Assessment is revising the end-of-quarter surveys that students complete for each class. Students using the new surveys are asked about their experience of evidence-based instructional practices, and they are encouraged to share feedback that instructors can use to improve the course. These revisions aim, among other things, to make these surveys a more robust form of evidence that can be used within the larger framework of HET.
We began working with our first cohort of departments in AY 22-23, and a second cohort is joining us for AY 23-24. Participating departments are listed here. Each department determines an appropriate process and timeline for adapting the HET framework, building consensus with department colleagues, and beginning to use HET. The steps below are common.
It seems like HET is designed to evaluate both (i) the instructor of a huge, undergraduate chemistry course and (ii) the instructor of a small graduate seminar in literary analysis. How is that possible, given that teaching looks so different in the two contexts?
There are two answers to this question. First, in departments that decide to use HET, faculty spend a lot of time tailoring the “off-the-shelf” HET resources to fit their disciplinary context. So, HET tools in Nursing might refer to students’ clinical skills, HET tools in Engineering might refer to problem sets or design projects, and HET tools in English might refer to essays or term papers. Second, the HET tools do not prescribe a single, lockstep way to teach. Instead, they help instructors and evaluators focus on general principles of good pedagogy, like the idea that students benefit from feedback about their learning as that learning is underway. For example, we would discourage a department from using an evaluation criterion like, “instructor grades and returns student work within a week and with at least three comments per student.” A more useful criterion could be, “instructor provides regular and timely feedback on student work,” along with the opportunity for the instructor to explain what this criterion looks like in their teaching practice.
What about cherry-picking evidence in HET or misrepresenting one’s teaching in a self-statement?
In HET, as in most teaching evaluation systems in higher education, an instructor might present a student assignment that is carefully scaffolded and aligned to learning objectives, has a detailed grading rubric, and incorporates student choice… but this might be atypical in their teaching practice. Or an instructor might write in their statement, “I use a range of strategies to promote equitable airtime among students” when, in fact, a few talkative students dominate every class session.
HET is grounded in the idea that principles of sound pedagogy play out differently in different contexts. An evaluator might have their own idea of what it looks like to “incorporate student choice” into a course, but the instructor they’re reviewing might use different strategies. One way that HET prioritizes the instructor’s autonomy and unique pedagogical identity is by soliciting self-reported data. The limitations of these data are precisely what the question above names: the risk of cherry-picking and misrepresentation. This tension (between the value of hearing directly from an instructor and the risk of misleading data) is not unique to HET. It’s a tension that’s inherent in any evaluation system that includes self-reported data, a tension HET does not claim to have resolved.
What if there’s a colleague on the departmental teaching-review committee who has a tense relationship with the candidate they’re reviewing, and because of that tension, they review and interpret HET materials with a negative bias?
Job performance in higher education is evaluated through peer review, and a clear limitation of this approach is the potential for preexisting interpersonal relationships to color evaluator judgments. HET does not eliminate this concern, but it does aim to mitigate it. HET provides an explicit structure and process for evaluation, one familiar to both the instructor and evaluator. In simplest terms, evaluators are tasked to look for evidence of teaching practices X, Y, and Z… criteria of which instructors are aware as they prepare their materials. The idea is that this precision and transparency reduces opportunities to make vague, unfairly negative assessments (or positive ones, sometimes referred to as “departmental love letters” about teaching).
Does HET require a lot of extra work compared to what my department has done in the past to evaluate teaching?
Whether HET is more work than what a department has done in the past depends on two factors: what the department has been doing to evaluate teaching and what the department’s version of HET looks like. Consider a hypothetical department that previously required a lengthy teaching statement and a classroom observation by a colleague as part of teaching evaluation. If this department adopted a streamlined version of HET (e.g., without these requirements and with simple rubrics), teaching evaluation might take less time. Alternatively, a department that previously evaluated teaching primarily by considering student evaluations of teaching might adopt a version of HET that asks instructors to provide more and richer evidence of their teaching. This will likely require more work the first time an instructor uses HET (the work should decrease in subsequent HET reviews).
As departments adapt the HET materials and process to fit their needs, one of their central tasks is to balance the desire for fairer, more substantive evaluation that supports instructors in improving against the time this requires. Each department determines the best balance for their context.
What is the relationship between student evaluations of teaching and HET?
One thing the word “Holistic” is meant to convey in “Holistic Evaluation of Teaching” is that different perspectives on an instructor’s teaching are important, including those of the instructor, their peers, and students. This means that student evaluations are part of the evidence an instructor may use to showcase their teaching in HET—but they’re not the only evidence. Put differently, HET encourages evaluators to consider a “basket of evidence” that contains more than just student evaluations.
Does HET require observation of my teaching?
No. The UCLA campus requires peer evaluation, but not necessarily peer observation. Peer evaluation might take the form of peers reviewing an instructor’s teaching materials, for example. Departments that adapt and use a version of HET decide whether to require peer observation as part of teaching evaluation.
F 2021-W 2022
Groundwork
Understand HET research base, HET efforts at other universities
Convene HET advisory, steering committees
W-Sp 2022
Needs Assessment, Initial Design
Interviews with 30 UCLA stakeholders
Input from advisory and steering committees
Design framework
Sp-Su 2022
Revise Framework, Preparation for Pilot
Revise framework based on feedback from campus leaders
Design departmental adaptation process, identify collaborating departments
F 2022-Sp 2024
Implementation Pilot
HET adaptation and adoption in select departments
F 2024-Sp 2026
Implementation Scale-Up
Wider HET adaptation and adoption across campus
Glory Tobiason, Co-Lead
Dr. Glory Tobiason is Clinical Faculty in Education and a Research Scientist at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Working under the aegis of CAT, Dr. Tobiason staffs and co-leads the Holistic Evaluation of Teaching project. She has a background in formative assessment, standards-based instructional design, and teacher quality. She also developed UCLA’s Peer-Assisted Reflections on Student Learning (PAROSL) program.
Adrienne Lavine, Co-Lead
Adrienne Lavine served as the Associate Vice Provost, Center for the Advancement of Teaching from July 1, 2017 to September 30, 2022, and she is a faculty member in Mechanical and Aerospace Engineering. She has served as chair of the Academic Senate and as chair of her department. She was the Director of Education and Outreach for the NSF-sponsored Center for Scalable and Integrated Nano-Manufacturing, directed the Clare Boothe Luce Undergraduate Scholars Program, and chaired the SEAS ABET Accreditation Committee in 2002-2006. She was awarded the Henry and Susan Samueli Teaching Award in her department and is the co-author of an internationally respected heat transfer textbook.
Michelle Chen, Program Manager
Michelle Chen is a project manager in the Center for the Advancement of Teaching (CAT). In addition to HET, projects and programs she manages include the Instructional Improvement Grants, Peer Assisted Reflections on Student Learning, Distinguished Teaching Awards, and Night to Honor Teaching event. She also supports various faculty committees and provides administrative support for CAT’s departmental operations.
Marc Levis-Fitzgerald, Liaison to HET from Center for Educational Assessment
Dr. Marc Levis-Fitzgerald is the Director of the Center for Educational Assessment. His research interests include curriculum reform and evaluation, student and faculty development, and institutional transformation. He is particularly interested in documenting the experience of undergraduate students, and he directs the development and implementation of the UCLA Senior Survey. He heads several undergraduate science initiative assessment projects funded by NSF and HHMI. In addition, he leads program review efforts related to the Transfer Alliance Program, a partnership between UCLA and community colleges in California.
Kumiko Haas, Liaison to HET from Instructional Improvement Programs
Dr. Kumiko Haas works with individual faculty and oversees the development and implementation of programs, services and activities across all UCLA schools and colleges in support of innovative instructional improvement. She develops and teaches intensive training programs, workshops, and seminars on issues of pedagogy, course design, and instructional methods for new and experienced instructors. Her current responsibilities include development and implementation of faculty engagement programs and teaching assistant training program. These include the New Faculty Teaching Engagement (NFTE), the Collegium of University Teaching Fellows Program (CUTF), TA Training Program (TATP), Teaching Assistant Consultant (TAC) Program and the TAC Academy, and Test of Oral Proficiency (TOP) and oversight of the Instructional Improvement Grant Programs (IIP Grants). She is also a serves as the chair of the Distinguished Teaching Awards Selection Committee of the Campuswide Distinguished Teaching Awards.
Because of the importance of this project, CAT is devoting significant resources of its staff. To date, the following individuals have contributed to the project: Anita Han, Molly Jacobs, Amy Liu, Casey Shapiro, Erin Sparck, Lucia Tabarez, and Brit Toven-Lindsey.
Paul Barber, Ecology and Evolutionary Biology
Elizabeth Bjork, Psychology
Clarence Braddock III, David Geffen School of Medicine
Jessica Collett, Sociology
Al Courey, Chemistry and Biochemistry
David MacFadyen, Comparative Literature
Victoria Marks, World Arts and Cultures / Dance
Jose-Felipe Martinez, Education
Muriel McClendon, History
Christopher Mott, English
Josh Samani, Physics and Astronomy
Lisa Felipe, Excellence in Pedagogy and Innovative Classrooms
Rachel Kennison, Center for Education Innovation and Learning in the Sciences
Tami Kremer-Sadlik, Social Science IDP
Shanna Shaked, Center for Education Innovation and Learning in the Sciences
Sirui Wang, UCLA Online Teaching and Learning
Juliet Williams, Social Science IDP
The HET initiative is part of an effort to establish a culture in which faculty continually examine their pedagogy and strive to innovate and improve… towards the goal of enabling all students — across the diversity of their backgrounds, levels of preparation, and access points (freshman vs. transfer entry) — to learn.
The academic personnel reward structure must align with the goal of teaching excellence, so that faculty see that their efforts are both appreciated and rewarded. Here, CAP currently faces a challenge with lack of robust evidence for the quality of teaching. CAP members have noted that peer evaluations and letters from students are almost uniformly positive and therefore not helpful. Student evaluations are the only metric that displays a range of opinions. Unfortunately, while there is value in getting input from students, research shows that students are not the best judge of their own learning and that student evaluations are subject to bias based on gender, ethnicity, and other factors.
Because evaluation of teaching is associated with the high-stakes decisions of merits and promotions, it is not surprising that individuals and departments tend to speak in glowing terms about the candidate’s teaching, without serious attention to solving problems that may exist. We believe it is preferable to take a developmental approach in which the candidate is rewarded for professional development and efforts to improve teaching—particularly efforts that research suggests can enhance student learning.
This is consistent with CAP’s Teaching page, which states that “The articulation of a candidate’s pedagogical approach, the description of new teaching techniques tried out and their results, the efforts to constructively and imaginatively address problematic issues cited by students, all testify to the candidate’s engagement in the educational mission.”
Those developing Holistic Evaluation of Teaching initiatives at other universities have made it clear: the work must be grounded in the priorities and perspectives of the campus community. For this reason, an initial phase of our work (see timeline below) was to conduct interviews with about thirty faculty across campus, including several current and past chairs and deans. These individuals represent a range of disciplines, career stages, and racial/ethnic backgrounds. What we learned from these interviews is at the heart of the HET framework.
We are grateful to have incorporated insights from Interim Executive Vice Chancellor and Provost Michael Levine, now Vice Chancellor, Academic Personnel, who says of the project: “I believe this is a useful approach to the evaluation of teaching, especially in terms of providing information about pedagogical improvement.” Additionally, we are grateful for the insights of former Interim Vice Provost, Academic Personnel Kathleen Komar and members of the Academic Senate’s Council on Academic Personnel and Committee on Teaching. Our work is also informed by insights from our Advisory Committee (faculty from across campus), our Steering Committee (leaders from other campus teaching-and-learning centers), the experience of other UC campuses, and our engagement with the Bay View Alliance and their TEval project. We are grateful for the generous financial support and institutional endorsement of the current and immediate past College Deans: Miguel García-Garibay, Darnell Hunt, Tracy Johnson, David Schaberg, Alexandra Stern, and Abel Valenzuela Jr.
Engaging in initial HET adaptation work |
Completed initial HET adaptation work |
Presented initial adapted version of HET to faculty |
Implementing HET: voluntary and / or formative use |
Implementing HET: use by some faculty in personnel review |
Implementing HET: use by most faculty in personnel review |
|
| Life Sciences Core Education |  |
|
||||
| Linguistics | |
|
||||
| Nursing | |
|
||||
| Sociology | |
|
The departments below began working on their version of HET in AY 22-23.
Life Sciences Core Education
Beth Lazazzera (department chair): “This initiative to adopt a more holistic method to evaluate teaching by faculty is desperately needed and will help create a system that can reward faculty contributions to teaching. As the LS Core Education Department has a large group of outstanding educators, we felt we could help this process become more widespread by being involved early in the HET initiative.”
Working Group:
Deb Pires |
Jukka Keranen |
Rana Khankan |
Jay Phelan |
Linguistics
Megha Sundara (department chair): “We joined the HET initiative to make teaching evaluations in Linguistics transparent and consistent. Through these discussions we hope to articulate a shared understanding of departmental expectations about teaching, and additionally, to inform ourselves about best practices for both teaching evaluation and teaching effectiveness.”
Working Group:
Dylan Bumford |
Jesse Harris |
Stefan Keine |
Megha Sundara |
Kie Zuraw |
Nursing
Lauren Clark (Associate Dean of Academic Programs): “In nursing, we know that one kind of evaluation doesn’t tell the full story about teaching effectiveness. We have big classes, small classes, clinical, and lecture classes. Developing holistic evaluation methods lets us appraise our teaching performance more fully and fairly.”
Working Group:
Theresa Brown |
MarySue Heilemann |
Charlene Niemi |
Dorothy Wiley |
Sociology
Abigail Saguy (department chair): “As a department that prizes teaching excellence, we welcome the opportunity to customize new ways of assessing effective teaching that go beyond student reviews and encourage self-reflection.”
Working Group:
Jessica Collett |
Chris Herring |
Megan Sweeney |
Stefan Timmermans |
The departments below began working on their version of HET in AY 23-24. The rest of Cohort 2 will be announced soon!
Molecular, Cell, and Developmental Biology
HET is organized into a 3-part framework, which includes dimensions of excellent teaching, sources of evidence, and evaluation lenses. Based on research, examples of other universities, and feedback from UCLA stakeholders, we have designed “off-the-shelf” versions of each of these three elements. These are versions that reflect discipline-agnostic principles about how people learn and common features of student-centered, inclusive pedagogy. (These are described in detail in this document.)
But of course, disciplines and departments have their own pedagogical practices, priorities, cultures, etc. To reflect those, departments that implement HET may choose to adjust the off-the-shelf versions of the three elements.
Using provided HET guidance and examples, instructors write their self/teaching statement and assemble their teaching portfolio. Evaluators review these materials, interpret them, and make a judgment about the instructor’s teaching, using the provided HET rubric(s). They describe their judgment in a departmental letter that becomes part of the case. Evaluators may also provide private, no-stakes feedback to the instructor to help them reflect on and improve their teaching.
In the HET framework, excellent teaching is understood to comprise four principles. Excellent teaching (i) engages students, (ii) is equitable, (iii) is learning centered and responsive, and (iv) involves striving to improve.
Engaging Students
DEFINITION. Teachers engage students by creating courses and lessons that are inviting, coherent, and clearly organized. They interact with students and support their learning beyond the delivery of lectures. Courses and lessons include appropriate content, reflect scholarship about how people learn, and are designed to engage students as active participants.
This includes the following components:
Equitable
DEFINITION. Equitable teaching means designing instruction, interacting with students, and maintaining learning environments in ways that enable all students to learn. Instructors convey that all students are capable scholars and valued members of an academic community and acknowledge differences as contributions to the learning community.
This includes the following components:
Learning Centered and Responsive
DEFINITION. Learning-centered and responsive teaching means instructors design lessons and courses starting with what they intend students to learn, rather than which content they will present. Assessment makes gaps between students’ current and intended learning visible; after formative assessment, feedback and subsequent instruction narrow those gaps. Instructors regularly refine teaching materials based on what they observe about student learning and feedback from students.
This includes the following components:
Striving to Improve
DEFINITION. Teachers who strive to improve recognize that teaching expertise is not static, and it is always possible to learn more about teaching. Over time, there are shifts in student demographics, background learning, and interests; the social context in which we teach; what we know about how people learn; and disciplinary content (in some fields). Excellent teachers keep an eye out for ways their pedagogy may also need to shift, and they support colleagues to do the same.
This includes the following components:
The examples below might be helpful to an instructor who’s considering how to provide evidence of each dimension of excellent teaching. Note that these are suggestions for inspiration, not requirements, and the lists are not comprehensive.
Engaging Students
Equitable
Learning Centered and Responsive
Striving to Improve
Instructors write a self/teaching statement (which can be part of their standard self-evaluation) in which they give examples of their teaching in each dimension. They may also choose to provide additional context (e.g., reflect overall on their teaching, discuss their teaching philosophy, provide a brief, self-evaluative narrative). The HET framework provides guidance and structure to assist faculty with creating a more substantive self/teaching statement.
Two forms of evidence are typically used in evaluation cases at UCLA: the Teaching Record (a form that summarizes courses developed and taught) and Student Evaluations of Teaching (SETs). An additional source of evidence is required (APM 210), and several options are listed below.
These should be thought of as suggestions to spark inspiration, not requirements. The intention is not that most (or even many) of these should be in every instructor’s portfolio. What makes a strong portfolio is the inclusion of evidence that illustrates what the instructor wants the evaluators to consider.
Evidence from students beyond SETs might include:
Evidence from peers might include a colleague’s description of:
Teaching artifacts might include:
Evidence of efforts to improve teaching might include documentation of:
| Proficiency Lens | Improvement Lens | |
 “Where are you on a map of excellent teaching?” |  “How are you moving further along the road to excellent teaching?” | |
| Purpose | Evaluate the state of an instructor’s teaching (i.e., how proficient they are at teaching) | Evaluate the instructor’s efforts to improve their teaching |
| What instructors do | Provide evidence of the static quality of their teaching | Provide evidence of engagement in improvement efforts |
A cornerstone of HET is the idea that evaluation should take into account not just the current state of an instructor’s teaching, but their efforts to improve their teaching, as well. In the HET framework, this idea is described in terms of two lenses.
When evaluators look through the proficiency lens, they are asking, “How proficient at teaching is this instructor?” They consider proficiency in three of the HET dimensions: “Engaging Students,” “Equitable,” and “Learning Centered and Responsive.” When evaluators look through the improvement lens, they are asking, “To what extent is this instructor trying to improve their teaching?” This aligns with the dimension, “Striving to Improve.”
Put simply, looking through the proficiency lens means focusing on quality of teaching at a moment in time, while looking through the improvement lens means focusing on growth over time. By using HET, evaluators can be sure they have considered an instructor’s teaching through both lenses.
