Holistic Evaluation of Teaching at UCLA

Introduction to the Project

UCLA’s Center for the Advancement of Teaching has initiated an exciting new project to improve the way UCLA evaluates teaching. Across our campus, there is widespread recognition that excellent teaching is too multifaceted and nuanced to be documented using a single source of evidence like student surveys. Thus, we have developed a framework that we refer to as Holistic Evaluation of Teaching: a way to think about and document teaching that includes self-reflection, student input, and peer review. Importantly, this new approach emphasizes faculty professional development and efforts to improve teaching. It is adaptable to the particular teaching challenges and cultures of each unit across campus, and it has been developed with broad input from faculty, the Senate, and academic leadership.

Excellent Teaching and Student Success

The motivation for this project is simple: to improve student learning and students’ academic experience by helping faculty implement excellent teaching practices. We believe this can be accomplished in part by establishing a culture in which faculty continually examine their pedagogy and strive to innovate and improve, with the goal of enabling all students — across the diversity of their backgrounds, levels of preparation, and access points (freshman vs. transfer entry) — to learn.

Better Documentation of Teaching Excellence

The academic personnel reward structure must align with the goal of teaching excellence, so that faculty see that their efforts are both appreciated and rewarded. Here, CAP currently faces a challenge with lack of robust evidence for the quality of teaching. CAP members have noted that peer evaluations and letters from students are almost uniformly positive and therefore not helpful. Student evaluations are the only metric that displays a range of opinions. Unfortunately, while there is value in getting input from students, research shows that students are not the best judge of their own learning and that student evaluations are subject to bias based on gender, ethnicity, and other factors.

Because evaluation of teaching is associated with the high-stakes decisions of merits and promotions, it is not surprising that individuals and departments tend to speak in glowing terms about the candidate’s teaching, without serious attention to solving problems that may exist. We believe it is preferable to take a developmental approach in which the candidate is rewarded for professional development and efforts to improve teaching—particularly efforts that research suggests can enhance student learning.

This is consistent with CAP’s Teaching page, which states that “The articulation of a candidate’s pedagogical approach, the description of new teaching techniques tried out and their results, the efforts to constructively and imaginatively address problematic issues cited by students, all testify to the candidate’s engagement in the educational mission.”

Insights, Support, and Guidance from the UCLA Community

Those developing Holistic Evaluation of Teaching initiatives at other universities have made it clear: the work must be grounded in the priorities and perspectives of the campus community. For this reason, an initial phase of our work (see timeline below) was to conduct interviews with about thirty faculty across campus, including several current and past chairs and deans. These individuals represent a range of disciplines, career stages, and racial/ethnic backgrounds. What we learned from these interviews is at the heart of the HET framework.

We are grateful to have incorporated insights from Interim Executive Vice Chancellor and Provost Michael Levine, now Vice Chancellor, Academic Personnel, who says of the project: “I believe this is a useful approach to the evaluation of teaching, especially in terms of providing information about pedagogical improvement.” Additionally, we are grateful for the insights of Interim Vice Provost, Academic Personnel Kathleen Komar and members of the Academic Senate’s Council on Academic Personnel and Committee on Teaching. Our work is also informed by insights from our Advisory Committee (faculty from across campus), our Steering Committee (leaders from other campus teaching-and-learning centers), the experience of other UC campuses, and our engagement with the Bay View Alliance and their TEval project. We are grateful for the generous financial support and institutional endorsement of the current and immediate past College Deans: Miguel García-Garibay, Darnell Hunt, Tracy Johnson, David Schaberg, and Abel Valenzuela Jr.

Timeline

Convene Advisory Committee (January, ‘22)

Advisory Committee provides input on multiple, potential frameworks (March, ’22)

Interviews with ~30 campus stakeholders for input on multiple, potential frameworks (March, April, ‘22)

Advisory Committee provides input on draft framework (May, ‘22)

Steering Committee provides input on draft framework (May, ‘22)

Seek input from Senate and campus leadership (April – August, ’22)

Pilot draft framework in select departments (‘22-’23 AY)

Project Team

Glory Tobiason

Dr. Glory Tobiason is Clinical Faculty in Education and a Research Scientist at the National Center for Research on Evaluation, Standards, and Student Testing (CRESST). Working under the aegis of CAT, Dr. Tobiason staffs and co-leads the Holistic Evaluation of Teaching project. She has a background in formative assessment, standards-based instructional design, and teacher quality. She also developed and leads UCLA’s Peer-Assisted Reflections on Student Learning (PAROSL) program.

Adrienne Lavine

Adrienne Lavine served as the Associate Vice Provost, Center for the Advancement of Teaching from July 1, 2017 to September 30, 2022, and she is a faculty member in Mechanical and Aerospace Engineering. She has served as chair of the Academic Senate and as chair of her department. She was the Director of Education and Outreach for the NSF-sponsored Center for Scalable and Integrated Nano-Manufacturing, directed the Clare Boothe Luce Undergraduate Scholars Program, and chaired the SEAS ABET Accreditation Committee in 2002-2006. She was awarded the Henry and Susan Samueli Teaching Award in her department and is the co-author of an internationally respected heat transfer textbook.

Marc Levis-Fitzgerald

Dr. Marc Levis-Fitzgerald is the Director of the Center for Educational Assessment. His research interests include curriculum reform and evaluation, student and faculty development, and institutional transformation. He is particularly interested in documenting the experience of undergraduate students, and he directs the development and implementation of the UCLA Senior Survey. He heads several undergraduate science initiative assessment projects funded by NSF and HHMI. In addition, he leads program review efforts related to the Transfer Alliance Program, a partnership between UCLA and community colleges in California.

Kumiko Haas

Dr. Kumiko Haas works with individual faculty and oversees the development and implementation of programs, services and activities across all UCLA schools and colleges in support of innovative instructional improvement. She develops and teaches intensive training programs, workshops, and seminars on issues of pedagogy, course design, and instructional methods for new and experienced instructors. Her current responsibilities include development and implementation of faculty engagement programs and teaching assistant training program. These include the New Faculty Teaching Engagement (NFTE), the Collegium of University Teaching Fellows Program (CUTF), TA Training Program (TATP), and Test of Oral Proficiency (TOP) and oversight of the Instructional Improvement Grant Programs (IIP Grants). She also oversees the UCLA Community Based Learning Programs.

Because of the importance of this project, CAT is devoting significant resources of its staff. To date, the following individuals have contributed to the project: Michelle Chen, Anita Han, Molly Jacobs, Amy Liu, Erin Sparck, Lucia Tabarez, and Brit Toven-Lindsey.

Advisory Committee Members

Paul Barber, Ecology and Evolutionary Biology

Elizabeth Bjork, Psychology

Clarence Braddock III, David Geffen School of Medicine

Jessica Collett, Sociology

Al Courey, Chemistry and Biochemistry

David MacFadyen, Comparative Literature

Victoria Marks, World Arts and Cultures / Dance

Jose-Felipe Martinez, Education

Muriel McClendon, History

Christopher Mott, English

Josh Samani, Physics and Astronomy

Steering Committee Members

Lisa Felipe, Excellence in Pedagogy and Innovative Classrooms

Rachel Kennison, Center for Education Innovation and Learning in the Sciences

Tami Kremer-Sadlik, Social Science IDP

Shanna Shaked, Center for Education Innovation and Learning in the Sciences

Sirui Wang, UCLA Online Teaching and Learning

Juliet Williams, Social Science IDP

Circle with three sections with the names of the three elements of HET in each section: "Dimensions of Excellent Teaching," "Sources of Evidence," and "Evaluation Lenses."

HET Framework

HET is organized into a 3-part framework, which includes dimensions of excellent teaching, sources of evidence, and evaluation lenses. Based on research, examples of other universities, and feedback from UCLA stakeholders, we have designed “off-the-shelf” versions of each of these three elements. These are versions that reflect discipline-agnostic principles about how people learn and common features of student-centered, inclusive pedagogy, etc. (These are described in detail in this document.)

But of course, disciplines and departments have their own pedagogical practices, priorities, cultures, etc. To reflect those, departments that implement HET may choose to adjust the off-the-shelf versions of the three elements.

Conducting A Review Using the HET Framework

Using provided HET guidance and examples, instructors write their teaching statement and assemble their teaching portfolio. They align these materials to the appropriate evaluation lens. Evaluators review these materials, interpret them, and make a judgment about the instructor’s teaching, using the provided HET rubric(s). They describe their judgment in a departmental letter that becomes part of the case. Evaluators may also provide private, no-stakes feedback to the instructor to help them reflect on and improve their teaching.

Dimensions of Excellent Teaching in HET

In the HET framework, excellent teaching is understood to comprise four principles. Excellent teaching (i) engages students, (ii) is equitable, (iii) is learning centered and responsive, and (iv) strives to improve.

Engaging Students

stick figure next to open door

DEFINITION. Teachers engage students by creating courses and lessons that are inviting, coherent, and clearly organized. They interact with students and support their learning beyond the delivery of lectures. Courses and lessons include appropriate content, reflect scholarship about how people learn, and are designed to engage students as active participants.

This includes the following components:

  • Being appropriately available to students and communicating respectfully and promptly;
  • Communicating (e.g., in syllabus, course website) student responsibilities, grading conventions, and course logistics (e.g., where to find materials, how to communicate with the instructor) in an accessible, clear, and explicit way; and
  • Selecting appropriate course content and designing activities that reflect scholarship about how people learn: actively; with frequent opportunities to practice and apply; and with appropriate challenge, scaffolding, and support.

Equitable

DEFINITION. Equitable teaching means designing instruction, interacting with students, and maintaining learning environments in ways that enable all students to learn. Instructors convey that all students are capable scholars and valued members of an academic community and acknowledge differences as contributions to the learning community.

This includes the following components:

  • Designing learning activities to be accessible for all students, including creating and maintaining structures that allow all students to contribute;
  • Incorporating student voice and choice into learning activities; 
  • Modeling a commitment to inclusive teaching in decisions, comments, and behaviors; and
  • Ensuring that course content reflects diversity and surfaces issues of equity and social justice that are relevant to the discipline.

Learning Centered and Responsive

stick figure with two gears above head

DEFINITION. Learning-centered and responsive teaching means instructors design lessons and courses starting with what they intend students to learn, rather than which content they will present. Assessment makes gaps between students’ current and intended learning visible; after formative assessment, feedback and subsequent instruction narrow those gaps. Instructors regularly refine teaching materials based on what they observe about student learning and feedback from students.

This includes the following components:

  • Making the intended learning for courses and lessons explicit and aligning materials, activities, and assessments to that intended learning;
  • Conducting frequent, low-stakes, formative assessment to make student thinking visible and adjusting subsequent instruction and giving students feedback in response;
  • Designing synchronous lessons with enough flexibility and student participation that decisions and adaptations can be made in real time, in response to how learning is unfolding in a particular lesson, for particular students; and
  • Asking students and observing how syllabi, assignments, activities, assessments, readings, etc. support learning, and then refining these materials.

Striving to Improve

stick figure holding briefcase walking up stairs

DEFINITION. Teachers who strive to improve recognize that teaching expertise is not static, and it is always possible to learn more about teaching. Over time, there are shifts in student demographics, background learning, and interests; the social context in which we teach; what we know about how people learn; and disciplinary content (in some fields). Excellent teachers keep an eye out for ways their pedagogy may also need to shift, and they support colleagues to do the same.

This includes the following components:

  • Reflecting on the success of current teaching strategies and adjusting them to better serve students;
  • Participating in ongoing professional development;
  • Observing or being observed by a colleague in order to give or get feedback about teaching; and
  • Contributing to conversations about and efforts to improve teaching and learning in the department, on campus, and beyond.

Sources of Evidence in HET

graphic of a document representing teaching statement and a folder with papers labeled "Teaching Record," "SETs," "Additional Evidence" representing teaching portfolio

Teaching Statement

This brief document serves two purposes. First, it serves as a roadmap for the contents of the teaching portfolio. Second, it is an opportunity for the instructor to self-evaluate and reflect on their teaching. As with current practice in many departments, this can be part of the candidate’s overall self-evaluation letter; the HET framework provides guidance and structure to assist faculty with creating a more substantive teaching statement.

Teaching Portfolio

The teaching portfolio, includes the following, which are consistent with current requirements at UCLA:

  • A Teaching Record (a form that summarizes courses developed and taught); 
  • SETs; and
  • One or more of the following additional sources of evidence:
    • Evidence from students beyond SETs (e.g., student work, feedback about a course that the instructor has acted upon, etc.),
    • Evidence from peers (e.g., description of classroom observation or review of course materials; description of collaborative curricular planning of related courses),
    • Teaching artifacts (e.g., syllabi, lesson materials, major assignments or assessments, websites, etc.), and
    • Record of Efforts to Improve Teaching (a form that summarizes improvement efforts, including but not limited to participation in professional development; participation in formative peer observation; service on a curriculum or assessment committee; teaching-related publications, presentations, or grant applications; etc.).

Evaluation Lenses in HET

Quick Guide to HET Evaluation Lenses
Proficiency LensInnovation Lens
graphic of map with drop pin“Where are you on a map of excellent teaching?”graphic of hilly road with trees“How have you moved further along the road to excellent teaching?”
PurposeEvaluate the state of an instructor’s teaching (i.e., how proficient they are at teaching)Evaluate the instructor’s efforts to improve their teaching–that is, the extent to which they have innovated in their practice
What instructors doProvide evidence of the static quality of their teachingProvide evidence of growth over time in their teaching
What evaluators (departmental colleagues) doUse four rubrics to assign a rating of developing, proficient, or exceptional to the instructor’s practice in each dimensionUse one rubric to assign a rating of modest, substantial, or exceptional to the instructor’s degree of engagement in innovation
Organization of teaching statementArranged by indicators from the four rubrics used by evaluators (departmental colleagues)Arranged by innovation

Departments are provided with two models for evaluation–two “lenses” through which to judge an instructor’s teaching. The innovation lens helps evaluators render a judgment about the instructor’s efforts to improve their teaching–that is, the extent to which they have innovated in their practice. By centering evaluation around changes in teaching practice, the innovation lens prioritizes growth over time, rather than static quality of teaching. The proficiency lens is used to evaluate the state of an instructor’s teaching (i.e., how proficient they are at teaching).

Departments have the option of tailoring these lenses or using both, if they have the time and faculty will to do so. Regardless of which lens is used, evidence is assembled, described, and evaluated using the dimensions of excellent teaching.

Sample HET-developed evaluation rubrics​, which departments may choose to customize, are provided for each lens. These rubrics help evaluators render judgments about an instructor’s teaching, which they summarize in a narrative.