Introduction to the Project
UCLA’s Center for the Advancement of Teaching has initiated an exciting new project to improve the way UCLA evaluates teaching. Across our campus, there is widespread recognition that excellent teaching is too multifaceted and nuanced to be documented using a single source of evidence like student surveys. Thus, we have developed a framework that we refer to as Holistic Evaluation of Teaching: a way to think about and document teaching that includes self-reflection, student input, and peer review. Importantly, this new approach emphasizes faculty professional development and efforts to improve teaching. It is adaptable to the particular teaching challenges and cultures of each unit across campus, and it has been developed with broad input from faculty, the Senate, and academic leadership.
Excellent Teaching and Student Success
The motivation for this project is simple: to improve student learning and students’ academic experience by helping faculty implement excellent teaching practices. We believe this can be accomplished in part by establishing a culture in which faculty continually examine their pedagogy and strive to innovate and improve, with the goal of enabling all students — across the diversity of their backgrounds, levels of preparation, and access points (freshman vs. transfer entry) — to learn.
Better Documentation of Teaching Excellence
The academic personnel reward structure must align with the goal of teaching excellence, so that faculty see that their efforts are both appreciated and rewarded. Here, CAP currently faces a challenge with lack of robust evidence for the quality of teaching. CAP members have noted that peer evaluations and letters from students are almost uniformly positive and therefore not helpful. Student evaluations are the only metric that displays a range of opinions. Unfortunately, while there is value in getting input from students, research shows that students are not the best judge of their own learning and that student evaluations are subject to bias based on gender, ethnicity, and other factors.
Working in tandem with the HET initiative, the Center for Educational Assessment is revising the end-of-quarter surveys that students complete for each class. Students using the new surveys are asked about their experience of evidence-based instructional practices, and they are encouraged to share feedback that instructors can use to improve the course. These revisions aim, among other things, to make these surveys a more robust form of evidence in the evaluation of teaching.
Because evaluation of teaching is associated with the high-stakes decisions of merits and promotions, it is not surprising that individuals and departments tend to speak in glowing terms about the candidate’s teaching, without serious attention to solving problems that may exist. We believe it is preferable to take a developmental approach in which the candidate is rewarded for professional development and efforts to improve teaching—particularly efforts that research suggests can enhance student learning.
This is consistent with CAP’s Teaching page, which states that “The articulation of a candidate’s pedagogical approach, the description of new teaching techniques tried out and their results, the efforts to constructively and imaginatively address problematic issues cited by students, all testify to the candidate’s engagement in the educational mission.”
Insights, Support, and Guidance from the UCLA Community
Those developing Holistic Evaluation of Teaching initiatives at other universities have made it clear: the work must be grounded in the priorities and perspectives of the campus community. For this reason, an initial phase of our work (see timeline below) was to conduct interviews with about thirty faculty across campus, including several current and past chairs and deans. These individuals represent a range of disciplines, career stages, and racial/ethnic backgrounds. What we learned from these interviews is at the heart of the HET framework.
We are grateful to have incorporated insights from Interim Executive Vice Chancellor and Provost Michael Levine, now Vice Chancellor, Academic Personnel, who says of the project: “I believe this is a useful approach to the evaluation of teaching, especially in terms of providing information about pedagogical improvement.” Additionally, we are grateful for the insights of Interim Vice Provost, Academic Personnel Kathleen Komar and members of the Academic Senate’s Council on Academic Personnel and Committee on Teaching. Our work is also informed by insights from our Advisory Committee (faculty from across campus), our Steering Committee (leaders from other campus teaching-and-learning centers), the experience of other UC campuses, and our engagement with the Bay View Alliance and their TEval project. We are grateful for the generous financial support and institutional endorsement of the current and immediate past College Deans: Miguel García-Garibay, Darnell Hunt, Tracy Johnson, David Schaberg, and Abel Valenzuela Jr.
Timeline
← Ongoing consultation with the Senate and academic leadership →
Stage 1
F 2021-W 2022
Groundwork
Understand HET research base, HET efforts at other universities
Convene HET advisory, steering committees
Stage 2
W-Sp 2022
Needs Assessment, Initial Design
Interviews with 30 UCLA stakeholders
Input from advisory and steering committees
Design framework
Stage 3
Sp-Su 2022
Revise Framework, Preparation for Pilot
Revise framework based on feedback from campus leaders
Design departmental adaptation process, identify collaborating departments
Stage 4
F 2022-Sp 2024
Implementation Pilot
HET adaptation and adoption in select departments
Stage 5
F 2024-Sp 2026
Implementation Scale-Up
Wider HET adaptation and adoption across campus
Project Team
Glory Tobiason, Co-Lead
Dr. Glory Tobiason is Clinical Faculty in Education and a Research Scientist at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Working under the aegis of CAT, Dr. Tobiason staffs and co-leads the Holistic Evaluation of Teaching project. She has a background in formative assessment, standards-based instructional design, and teacher quality. She also developed UCLA’s Peer-Assisted Reflections on Student Learning (PAROSL) program.
Adrienne Lavine, Co-Lead
Adrienne Lavine served as the Associate Vice Provost, Center for the Advancement of Teaching from July 1, 2017 to September 30, 2022, and she is a faculty member in Mechanical and Aerospace Engineering. She has served as chair of the Academic Senate and as chair of her department. She was the Director of Education and Outreach for the NSF-sponsored Center for Scalable and Integrated Nano-Manufacturing, directed the Clare Boothe Luce Undergraduate Scholars Program, and chaired the SEAS ABET Accreditation Committee in 2002-2006. She was awarded the Henry and Susan Samueli Teaching Award in her department and is the co-author of an internationally respected heat transfer textbook.
Michelle Chen, Program Manager
Michelle Chen is a project manager in the Center for the Advancement of Teaching (CAT). In addition to HET, projects and programs she manages include the Instructional Improvement Grants, Peer Assisted Reflections on Student Learning, Distinguished Teaching Awards, and Night to Honor Teaching event. She also supports various faculty committees and provides administrative support for CAT’s departmental operations.
Marc Levis-Fitzgerald, Liaison to HET from Center for Educational Assessment
Dr. Marc Levis-Fitzgerald is the Director of the Center for Educational Assessment. His research interests include curriculum reform and evaluation, student and faculty development, and institutional transformation. He is particularly interested in documenting the experience of undergraduate students, and he directs the development and implementation of the UCLA Senior Survey. He heads several undergraduate science initiative assessment projects funded by NSF and HHMI. In addition, he leads program review efforts related to the Transfer Alliance Program, a partnership between UCLA and community colleges in California.
Kumiko Haas, Liaison to HET from Instructional Improvement Programs
Dr. Kumiko Haas works with individual faculty and oversees the development and implementation of programs, services and activities across all UCLA schools and colleges in support of innovative instructional improvement. She develops and teaches intensive training programs, workshops, and seminars on issues of pedagogy, course design, and instructional methods for new and experienced instructors. Her current responsibilities include development and implementation of faculty engagement programs and teaching assistant training program. These include the New Faculty Teaching Engagement (NFTE), the Collegium of University Teaching Fellows Program (CUTF), TA Training Program (TATP), and Test of Oral Proficiency (TOP) and oversight of the Instructional Improvement Grant Programs (IIP Grants). She also oversees the UCLA Community Based Learning Programs.
Because of the importance of this project, CAT is devoting significant resources of its staff. To date, the following individuals have contributed to the project: Anita Han, Molly Jacobs, Amy Liu, Erin Sparck, Lucia Tabarez, and Brit Toven-Lindsey.
Advisory Committee Members
Paul Barber, Ecology and Evolutionary Biology
Elizabeth Bjork, Psychology
Clarence Braddock III, David Geffen School of Medicine
Jessica Collett, Sociology
Al Courey, Chemistry and Biochemistry
David MacFadyen, Comparative Literature
Victoria Marks, World Arts and Cultures / Dance
Jose-Felipe Martinez, Education
Muriel McClendon, History
Christopher Mott, English
Josh Samani, Physics and Astronomy
Steering Committee Members
Lisa Felipe, Excellence in Pedagogy and Innovative Classrooms
Rachel Kennison, Center for Education Innovation and Learning in the Sciences
Tami Kremer-Sadlik, Social Science IDP
Shanna Shaked, Center for Education Innovation and Learning in the Sciences
Sirui Wang, UCLA Online Teaching and Learning
Juliet Williams, Social Science IDP
Cohort 1
We are working with our first cohort of departments in AY 22-23. During this year, a faculty working group from each department meets regularly with a HET facilitator to tailor the HET approach, making sure it reflects the priorities and disciplinary context of their department.
Life Science Core Education Department
Beth Lazazzera (department chair): “This initiative to adopt a more holistic method to evaluate teaching by faculty is desperately needed and will help create a system that can reward faculty contributions to teaching. As the LS Core Education Department has a large group of outstanding educators, we felt we could help this process become more widespread by being involved early in the HET initiative.”
Working Group:
Deb Pires |
Jukka Keranen |
Rana Khankan |
Jay Phelan |
Sociology
Abigail Saguy (department chair): “As a department that prizes teaching excellence, we welcome the opportunity to customize new ways of assessing effective teaching that go beyond student reviews and encourage self-reflection.”
Working Group:
Jessica Collett |
Chris Herring |
Megan Sweeney |
Stefan Timmermans |
School of Nursing
Lauren Clark (Associate Dean of Academic Programs): “In nursing, we know that one kind of evaluation doesn’t tell the full story about teaching effectiveness. We have big classes, small classes, clinical, and lecture classes. Developing holistic evaluation methods lets us appraise our teaching performance more fully and fairly.”
Working Group:
Theresa Brown |
MarySue Heilemann |
Charlene Niemi |
Dorothy Wiley |
Linguistics
Megha Sundara (department chair): TBD
Working Group:
Dylan Bumford |
Jesse Harris |
Stefan Keine |
Megha Sundara |
Kie Zuraw |
HET Framework
HET is organized into a 3-part framework, which includes dimensions of excellent teaching, sources of evidence, and evaluation lenses. Based on research, examples of other universities, and feedback from UCLA stakeholders, we have designed “off-the-shelf” versions of each of these three elements. These are versions that reflect discipline-agnostic principles about how people learn and common features of student-centered, inclusive pedagogy. (These are described in detail in this document.)
But of course, disciplines and departments have their own pedagogical practices, priorities, cultures, etc. To reflect those, departments that implement HET may choose to adjust the off-the-shelf versions of the three elements.
Conducting A Review Using the HET Framework
Using provided HET guidance and examples, instructors write their teaching statement and assemble their teaching portfolio. Evaluators review these materials, interpret them, and make a judgment about the instructor’s teaching, using the provided HET rubric(s). They describe their judgment in a departmental letter that becomes part of the case. Evaluators may also provide private, no-stakes feedback to the instructor to help them reflect on and improve their teaching.
Dimensions of Excellent Teaching in HET
In the HET framework, excellent teaching is understood to comprise four principles. Excellent teaching (i) engages students, (ii) is equitable, (iii) is learning centered and responsive, and (iv) strives to improve.
Engaging Students
DEFINITION. Teachers engage students by creating courses and lessons that are inviting, coherent, and clearly organized. They interact with students and support their learning beyond the delivery of lectures. Courses and lessons include appropriate content, reflect scholarship about how people learn, and are designed to engage students as active participants.
This includes the following components:
- Being appropriately available to students and communicating respectfully and promptly;
- Communicating (e.g., in syllabus, course website) student responsibilities, grading conventions, and course logistics (e.g., where to find materials, how to communicate with the instructor) in an accessible, clear, and explicit way; and
- Selecting appropriate course content and designing activities that reflect scholarship about how people learn: actively; with frequent opportunities to practice and apply; and with appropriate challenge, scaffolding, and support.
Equitable
DEFINITION. Equitable teaching means designing instruction, interacting with students, and maintaining learning environments in ways that enable all students to learn. Instructors convey that all students are capable scholars and valued members of an academic community and acknowledge differences as contributions to the learning community.
This includes the following components:
- Designing learning activities to be accessible for all students, including creating and maintaining structures that allow all students to contribute;
- Incorporating student voice and choice into learning activities;
- Modeling a commitment to inclusive teaching in decisions, comments, and behaviors; and
- Ensuring that course content reflects diversity and surfaces issues of equity and social justice that are relevant to the discipline.
Learning Centered and Responsive
DEFINITION. Learning-centered and responsive teaching means instructors design lessons and courses starting with what they intend students to learn, rather than which content they will present. Assessment makes gaps between students’ current and intended learning visible; after formative assessment, feedback and subsequent instruction narrow those gaps. Instructors regularly refine teaching materials based on what they observe about student learning and feedback from students.
This includes the following components:
- Making the intended learning for courses and lessons explicit and aligning materials, activities, and assessments to that intended learning;
- Conducting frequent, low-stakes, formative assessment to make student thinking visible and adjusting subsequent instruction and giving students feedback in response;
- Designing synchronous lessons with enough flexibility and student participation that decisions and adaptations can be made in real time, in response to how learning is unfolding in a particular lesson, for particular students; and
- Asking students and observing how syllabi, assignments, activities, assessments, readings, etc. support learning, and then refining these materials.
Striving to Improve
DEFINITION. Teachers who strive to improve recognize that teaching expertise is not static, and it is always possible to learn more about teaching. Over time, there are shifts in student demographics, background learning, and interests; the social context in which we teach; what we know about how people learn; and disciplinary content (in some fields). Excellent teachers keep an eye out for ways their pedagogy may also need to shift, and they support colleagues to do the same.
This includes the following components:
- Reflecting on the success of current teaching strategies and adjusting them to better serve students;
- Participating in ongoing professional development;
- Observing or being observed by a colleague in order to give or get feedback about teaching; and
- Contributing to conversations about and efforts to improve teaching and learning in the department, on campus, and beyond.
Sources of Evidence in HET
Teaching Statement
This brief document serves two purposes. First, it serves as a roadmap for the contents of the teaching portfolio. Second, it is an opportunity for the instructor to self-evaluate and reflect on their teaching. As with current practice in many departments, this can be part of the candidate’s overall self-evaluation letter; the HET framework provides guidance and structure to assist faculty with creating a more substantive teaching statement.
Teaching Portfolio
The teaching portfolio, includes the following, which are consistent with current requirements at UCLA:
- A Teaching Record (a form that summarizes courses developed and taught);
- SETs; and
- One or more of the following additional sources of evidence:
- Evidence from students beyond SETs (e.g., student work, feedback about a course that the instructor has acted upon, etc.),
- Evidence from peers (e.g., description of classroom observation or review of course materials; description of collaborative curricular planning of related courses),
- Teaching artifacts (e.g., syllabi, lesson materials, major assignments or assessments, websites, etc.), and
- Record of Efforts to Improve Teaching (a form that summarizes improvement efforts, including but not limited to participation in professional development; participation in formative peer observation; service on a curriculum or assessment committee; teaching-related publications, presentations, or grant applications; etc.).
Evaluation Lenses in HET
| Proficiency Lens | Innovation Lens | |
 “Where are you on a map of excellent teaching?” |  “How have you moved further along the road to excellent teaching?” | |
| Purpose | Evaluate the state of an instructor’s teaching (i.e., how proficient they are at teaching) | Evaluate the instructor’s efforts to improve their teaching–that is, the extent to which they have innovated in their practice |
| What instructors do | Provide evidence of the static quality of their teaching | Provide evidence of growth over time in their teaching |
Departments are provided with guidance for combining two evaluation “lenses”–two ways to view and judge an instructor’s teaching. Looking through the proficiency lens, evaluators consider the state of an instructor’s teaching (i.e., how proficient they are at teaching). The innovation lens helps evaluators render a judgment about the instructor’s efforts to improve their teaching–that is, the extent to which they have innovated in their practice. By focusing on changes in teaching practice and efforts to improve, the innovation lens prioritizes growth over time, rather than static quality of teaching.
Departments tailor these lenses and decide how best to combine them. HET-developed evaluation rubrics, which departments may choose to customize, help evaluators render judgments about an instructor’s teaching, which they summarize in a narrative.
The teaching statement and evidence in the teaching portfolio are organized to illustrate proficiency in the first three dimensions of excellent teaching and innovation as part of the fourth dimension.
