Teaching Effectiveness

​

Defining Teaching Effectiveness

In his book Understanding Teaching Excellence in Higher Education (2005), Alan Skelton argues that teaching effectiveness is a contested, value-laden concept. He points out that “students, teachers, politicians and employers may all have different understandings of teaching excellence at any given moment in time within a particular system of higher education,” and that while “policies which seek to promote teaching excellence may claim to be neutral and value-free, they intentionally or unintentionally connect with particular values and interests” (p. 11).

A useful definition of teaching effectiveness, then, should be intentionally linked to the specific context where teaching is evaluated. Communities should explicitly identify the values and assumptions that underpin their understanding of what it means to be an effective teacher and that inform what they define as best practices. For example, a definition might reflect a university’s mission, the unique practices of an academic discipline, or the values that inform a certain teaching award.

There are three elements to consider when evaluating teaching effectiveness within a particular context:

1. Criteria – attributes of effective teaching
2. Evidence – documentation of teaching considered in the review process
3. Standards – expectations of quality and quantity

Criteria

In her practical guide Peer Review of Teaching: A Sourcebook (2007) Nancy Chism points to several sets of criteria that have been proposed to evaluate teaching effectiveness and combines these sets into a master list where the criteria are organized into categories and characteristics (pp. 57-8). You can use this resource to identify possible criteria that describe your particular definition of effective teaching.

Category	Characteristics
Content Knowledge	Mastery/accuracy, currency, knowledge of other fields and perspectives, tolerance of diverse points of view
Course Design	Focus on student learning outcomes, clarity of learning outcomes, appropriate challenge for student population, appropriateness of methods used to assess student learning, alignment of learning experiences with learning outcomes, alignment with program curriculum, alignment with university mission and vision, intentionally-sequenced activities
Teaching Methods	Informed by research on learning, worthwhile activities, variety of approaches, well-organized class activities, effective speaking, effective facilitation methods, effective student interaction, inclusive of diverse student populations, accommodating disabilities
Student Communication	Accessibility to students, responsiveness, sensitivity to diversity of student population, knowledge of student resources and services, rapport with students
Materials	Appropriateness for student group, currency, alignment with learning outcomes, depth, creativity, sets high standards
Use of Technology	Appropriate use of medium, technology functionality, interactivity, attractiveness of design, alignment with learning outcomes
Student Learning Results	Evidence of student learning using direct measures, evidence of student engagement
Leadership around Teaching in Department, School &Nation	Quality of contribution, quantity of contribution
Scholarship of Teaching and Learning	Based on important problems or issues, appropriateness of research methods, quality of data collection and analysis, clarity of presentation of findings, peer review
Teaching Development	Reflective practice, participation in professional development events and programs, attention to current research on learning and teaching
Mentoring & Advising	Accessibility to students, quality of advice, knowledge of options and resources for students, interpersonal skills
Special Contexts	Modeling of rapport with patients or clients, quality of relationship with onsite preceptor or supervisor, collaboration with co-teacher

Evidence

There are many sources of evidence available that document teaching effectiveness. Evidence should include contributions from students, colleagues, and the instructor.

Evidence collected by students might include student ratings and letters from students who have been supervised by the instructor. Evidence collected from colleagues might include reviews of course materials and classroom observation reports. Evidence collected from instructors themselves might include course materials, examples of student work, reflections on teaching, evidence of participation in professional development activities, or a teaching portfolio.

Below is a non-exhaustive list of additional sources of evidence organized by category. This list is an expanded version of Chism’s suggestions (pp. 61):

Category	Possible Sources of Evidence
Content Knowledge	Classroom observation records, record of scholarly research in subject area, reading lists
Course Design	Course proposals, course materials (e.g. syllabus, assignment prompts, exams), philosophy of teaching statement, instructor's reflections on the effectiveness of the course
Teaching Methods	Classroom observation records, video recordings of teaching, lecture outlines or lesson plans, handouts, problem sets, student course evaluations, feedback solicited from students at midterm, student testimonials, instructor’s reflective statement on how she incorporates technology into teaching
Assessment	Exams, quizzes, assignment prompts, samples of written feedback on graded work, graded student exams or papers
Student Learning Results	Pretests and post tests that show student progress towards meeting the learning outcomes, comparisons of student work at the beginning and end of term to document growth, sample student projects, students’ lab books or other workbooks
Leadership around Teaching & Scholarship of Teaching and Learning	Publications in teaching journals, presentations given on teaching, service on teaching and learning committees, work on curriculum revision or development, assistance to colleagues on teaching matters, teaching awards, testimony of colleagues
Teaching Development	Philosophy of teaching statement, participation in professional development workshops or programs, reflections on implementing new methods of teaching or assessing learning, description of instructional improvement projects
Mentoring, Advising & Student Communication	Documentation of advising record, samples of emails and letters, records of student achievements, student testimonials

See alternative lists organized by goal (e.g. documenting student learning) and by information source (e.g. students, colleagues).

Standards

Standards articulate expectations of quality or quantity for each evaluation criterion. As with criteria and sources of evidence, the corresponding standards should fit the context in which teaching is evaluated. For instance, a teaching award for graduate teaching assistants might articulate high standards for student learning results yet reflect lower expectations for departmental and institutional leadership around teaching.

The following are examples of levels of achievement:

• "Compelling evidence... some evidence... no evidence”
• “Innovative... current... outdated”
• “Extensive... specific... vague”
• “Frequent... regular... infrequent”
• “Above average... average... below average”
• “Excellent... satisfactory... unacceptable”
• “Exceptional... adequate... needs work... absent”
• “Excellent... very good... good... fair... poor
• “Distinguished... sufficient... minimal”
• “Exemplary... beginning... developing”
• “Advanced... professional... novice”

Bringing It Together

One way to articulate a definition of teaching excellence and communicate that definition to teachers and peer evaluators is to create a rubric. Here are two examples:

• Teaching Portfolio Rubric. This rubric explicitly identifies the evaluation criteria, the types of evidence that should be included in the portfolio, and the evaluation standards.
• Scholar-Educator Rubric from Introduction to Rubrics (Stevens & Levi, 2013, p. 164). This rubric articulates criteria and standards, but does not specify what types of evidence are considered. If you do not address types of evidence in your rubric, you should explicitly identify what is acceptable elsewhere.

Considerations When Evaluating Teaching Effectiveness

If you wish to develop a system for evaluating teaching effectiveness within a particular context, such as a teaching award for your department, be mindful of the following considerations:

Purpose

Decide if the primary purpose of the evaluation is formative or summative and use that distinction to guide the evaluation process. For example, a peer teaching observation conducted for formative purposes should not be used to make summative judgements in merit evaluation because it can compromise the relationship between the instructor and the observer.

Formative Evaluation	Summative Evaluation
is used for personal development, i.e. giving instructors feedback on their teaching and providing them with resources they can use to improve	is used for personnel decisions, such as hiring, promotion, tenure, or merit awards
provides instructors with information that is private and confidential	involves the public inspection and review of information
is informal and ongoing	is formal and conducted at a given interval

Be transparent about the purpose of the evaluation. Communicate the evaluation criteria, sources, and standards to stakeholders as early as possible.

Faculty and students are key stakeholders in the evaluation of teaching. Invite them to participate actively in defining the underlying values and purposes of the evaluation.

Validity

To get a complete picture of an individual’s teaching performance, gather information from multiple sources, use multiple methods, and consider multiple points in time. Student ratings are useful, but they should only be one of many sources of evidence considered.

When documenting student learning, make sure to consider direct measures that provide tangible, compelling evidence that students have achieved a specific learning outcome. Keep in mind that grades and surveys asking students to report what they learned are indirect measures of student learning, meaning they are only proxy signs that students are probably learning.

Consider Portfolios

Consider using teaching portfolios to evaluate teaching effectiveness. Portfolios offer faculty more control over representing their teaching because they can select and curate sources of evidence. In addition to providing more control, teaching portfolios encourage faculty to reflect on their assumptions about learning, their past experiences, and their future goals.

Further Resources

• The Office for Teaching Learning and Assessment (TLA) can provide advice, guidance, and resources to help you define teaching effectiveness for your particular program, department, or award. TLA also works directly with instructors to provide formative, confidential feedback on their teaching. TLA does not participate in the summative assessment of instructors for tenure, promotion, or teaching award purposes.
• Peer Review of Teaching: A Sourcebook (2007) by Nancy Chism. This is an accessible, research-based sourcebook that provides administrators with guidance on developing a system for peer review, conducting peer observations, and implementing teaching portfolios. The book includes many sample forms and other materials that you can use or adapt.
• Summaries of Research on Student Evaluations. An annotated bibliography created by Vanderbilt University’s Center for Teaching that reviews research on the reliability, validity, and potential biases of student course evaluations.

---