Extraordinary Teaching and Learning: Opportunities for the UGA Teaching Academy

Josef M. Broder, Assistant Dean for Academic Affairs, College of Agricultural and Environmental Sciences and Executive Committee Chair, UGA Teaching Academy

I am pleased to be here today to talk about the University of Georgia Teaching Academy, its role in planning this Symposium and in serving as a mechanism for implementing the recommendations of this Symposium. This Twelfth Annual Academic Affairs Faculty Symposium represents a testament to the University's teaching mission and to the administration's efforts to support faculty in this endeavor. Having transcended two administrations, I believe that this symposium series has been institutionalized as part of the University's academic culture. I hope today's conversation will not only engage those in attendance but will be carried to the larger University Community. The goal of this symposia is to give us time to reflect on the status and potential of our teaching enterprise. Our task today is not so much to find solutions to our problems but to ask relevant and pressing questions about our University.

The theme of today's Symposium is bold and significant for the University of Georgia and one that I could not have imagined as a young faculty member. Our theme, "The Challenge of Becoming Extraordinary Teachers" suggests that the institution is not satisfied with its reputation and potential as a teaching institution. Yet, this theme provides a vision for the University becoming a premier institution for teaching and learning or, in the spirit of Fred Stephenson's book, "an extraordinary institution for teaching and learning."

Allow me to digress on the more general purpose of symposiums and what is expected of us today. The word symposium derives from the Greek symposion which means to "drink together" [Merriam-Webster]. Webster's first definition of symposium reads: "(a) a convivial party (as after a banquet in ancient Greece) with music and conversation (b) a social gathering at which there is free interchange of ideas." A second, less engaging definition reads, "(2) a formal meeting at which several specialists deliver short addresses on a topic or on related topics." With these definitions in mind, I hope you will use this occasion to converse and share ideas about excellence in teaching and learning.

My task here today is to share with you the history of the University of Georgia Teaching Academy, its role in this Symposium and how the Academy can facilitate the recommendations of this Symposium. An overview of my presentation is as follows. First, in the spirit of a symposium, we will start with a reflective exercise. This will be followed by some thoughts on the motivation for this Symposium. Why are we here? Next, I give you a brief history of the UGA Teaching Academy [Broder and Kalivoda]. I'll close with some critical issues for teaching and learning and the role the Teaching Academy can play in addressing these issues.

Reflective Exercise

In preparation for this presentation, please take a few minutes to reflect on the following:

Think about your daily interactions with colleagues. How often and in what context do you discuss teaching and learning at UGA

In my experience, discussions of teaching and learning do not occur as often as they should. We discuss teaching and learning when required by external reviewers or administrative mandates. The discussion tends to be reactive to problems that have occurred. Discussions tend to be private since public conversations about teaching and learning are perceived to be negative or risky. On this occasion, we have a unique opportunity to talk about teaching and learning in constructive and collaborative context. Within this context, let me speak to the motivation of today's Symposium.

Motivation

People are generally drawn together when they feel threatened or when they seek opportunities for collaboration. This Symposium is motivated by both threats and opportunities. Perceived threats to the teaching enterprise come from external groups who seek to hold faculty accountable for their activities. Among the more recent threats include the Board of Regents Faculty Workload initiative which sought, unsuccessfully, to impose workload standards on faculty activities. Next, we adopted a System wide program of post tenure review, with mixed results. More recently, we have just completed a University Self-study and SACS review which, has proven to be one of the University's more constructive review exercises. Then, we have "lurking in the background" student interest groups who would like to make our student evaluations public and for the world to see on the Web. While threatening to faculty, one must ask why these initiatives continue to emerge. At issue is why have faculty groups been slow to respond to the mandates and concerns expressed by these groups?

In all fairness, faculty have responded to the concerns of these groups, but our response is not well known outside the teaching circles at the University. Stories that tend to make the headlines are more likely to include rising admissions' standards (rejection rates) or allegations of rape, hazing and other misconduct. Still, we have some great (although, less sensational) stories to tell about teaching and learning. First, there are the many fine programs sponsored by the Office of the Vice President for Instruction and its support for this Symposium. Next, are the many programs and activities of the Office of Instructional Support and Development and the Institute of Higher Education. Then we have the energy and activities of the Teaching Academy. Finally, we have contributions of individual faculty such as those by Fred Stephenson and the many authors of his book, "Extraordinary Teachers." Thus, I feel the proper motivation for this Symposium is for us to share the success stories and explore opportunities for collaboration that will advance the teaching and learning enterprise at the University of Georgia. In the process and the events that follow, we can address the concerns raised by our critics.

Problem Setting

As a public research institution, the University of Georgia has not been immune to criticism. Historically, public research universities have depended largely upon their research programs to achieve and sustain national prominence. In periods of expanding federal and state funding for research, this focus on research productivity could be justified in the name of growth and competition. Many universities promoted and rewarded research to the detriment to other scholarly activities. Activities that did not directly lead to publications, patents, grants and contracts were given second class status while young faculty were warned not to over invest in these secondary activities. Teaching and public service activities tended to suffer. Excellence in teaching and service were not considered scholarly activities and were recognized only superficially in the promotion and tenure process. These universities expected all faculty to be excellent researchers and tolerated and often promoted poor teachers

In the late 1980s, higher education professionals began to question this focus on research with its narrow view of scholarship. The call for a more balanced definition of scholarship was perhaps, best articulated by Ernest Boyer and the Carnegie Foundation for the Advancement of Teaching. Boyer's message in Scholarship Reconsidered: Priorities of the Professoriate, was as simple as it was elegant [1990]. He argued that research is but one function of public research universities and that research scholarship, which he coined the scholarship of discovery, is but one part of the scholarly process. Faculty at public universities are also engaged in the scholarship of integration, the scholarship of application and the scholarship of teaching. Boyer argued that the discovery of knowledge is meaningful only to the extent that knowledge is integrated, applied and taught to others. In the absence of this continuum of scholarship, public universities cease to be relevant.

Boyer's writings have had a profound impact on the professoriate and have challenged many public universities to reexamine their practices. A number of national initiatives emerged from these conversations on scholarship including the Peer Review of Teaching Project, funded by the Hewlett Foundation and the Pew Charitable Trust and more recently, the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) [Hutchings]. These programs have been coordinated by the American Association of Higher Education which created the Forum on Faculty Roles and Rewards, an annual conference to discuss the changing role of the professoriate. A primary goal of these national teaching initiatives has been to promote conversations on teaching and scholarship at the university, department and faculty level. The Peer Review Project sought to promote peer collaboration of teaching with the goal of improving teaching and learning. The Peer Review Project evolved into the Carnegie Academy which seeks to link conversations on teaching across departments, disciplines and institutions. CASTL's programs in higher education include

1. The Pew National Fellowship Program for Carnegie Scholars
2. The Teaching Academy Campus Program
3. Work with the Scholarly and Professional Societies.

The University of Georgia's efforts to develop a campus-wide teaching academy was motivated in part by the Carnegie's Teaching Academy Campus Program.

Background

The formation of teaching academies across universities is likely to be influenced by institutional differences. For this reason, the University of Georgia's efforts to establish a teaching academy were different from those of other universities. A key factor in the growth and development of teaching academies is the extent to which the institution has nurtured a culture or climate for teaching. In the University of Georgia's case, this culture of respecting and honoring teaching faculty was created by a series of teaching initiatives that raised the visibility and support of teaching on the campus. A brief background of events that preceded and gave impetus to the formation of the Teaching Academy merit discussion.

Instructional Support and Development

In the late 1960s, the University sought to achieve national prominence as a major research institution. Faculty were hired and rewarded for their research expertise while teaching was viewed as a secondary activity. As the population and status of research faculty increased, there was a growing concern that these faculty members lacked the necessary teaching skills. Given that graduate schools offered little training in teaching and advising, these new faculty entered the classrooms with virtually no teacher training. In response to these concerns, the Vice President of Academic Affairs, Virginia Trotter, created an Office of Instructional Development (OID) in 1979 to provide instructional leadership and support to faculty, administrators and graduate teaching assistants across disciplines. Prior to this time, instructional support at the University was limited primarily to providing audio visual equipment for classroom instruction.

The University's Office of Instructional Development, under the leadership of Ron Simpson and Bill Jackson, emerged as one of the nation's most successful and enduring offices of its kind. While many such offices were established, many were underfunded or funded on soft money and failed to live up to their full potential. The secret to OID's success lies in their working with faculty and graduate student groups in a collaborative manner. OID services were available to faculty and graduate students at no charge and on a voluntary basis. The office sought to create environments for faculty to enhance their teaching skills and establish forums for faculty to share teaching ideas. Among the early faculty forums were the Instructional Advisory Committee, the Senior Teaching Fellows and the Lilly Teaching Fellows. These programs were instrumental in creating a campus culture for teaching and learning. In 1997, the Office of Instructional Development was reorganized into the Office of Instructional Support and Development (OISD).

Campus-wide Teaching Awards

Sensing a need to promote and recognize teaching excellence, the University established a campus-wide teaching award in 1984, in honor of the University's second president, Josiah Meigs. This award was open to faculty across all disciplines. Award winners were recognized at a campus-wide award ceremony and received discretionary funds to support their teaching programs. Despite the program's goals of recognizing outstanding teachers, the early Meigs Award lacked the monetary rewards associated with the many long-established research awards. Consequently, the status of these awards was raised in 1988 by the Vice President for Academic Affairs, William Prokasy, who added a permanent salary increment of $5000 in addition to the one time $1000 discretionary allowance. The salary increment has since been raised to $6000. Up to five Josiah Meigs Teaching Awards could be given each year.

The enhanced Meigs Teaching Awards were an instant hit with the students and teaching faculty. The award recipients were recognized at a campus-wide teaching award banquet. Video biographies of the award winners were produced and made available for public relations purposes, their stories were captured in the campus newspaper and on some occasions, the winners rode in the homecoming parade. The University sought to maximize the public relations benefits of this enhanced teaching award. This salary increment was bold and somewhat controversial for a research institution, especially when the salary increment rivaled that for many of the campus-wide research awards. These teaching awards were significant because they both recognized and rewarded teaching, while conveying to young faculty and graduate students that the University is serious about teaching. These awards also gave the Vice President a means for addressing salary differences between outstanding teachers and outstanding researchers. Lacking in the awards program was a mechanism for award winners to share their teaching expertise with others or with the larger university community. For the most part, award winners, received their awards and returned to their private lives as teachers.

Peer Review Project

While the Office of Instructional Development and the Meigs Teaching Awards touched the lives of many teachers, the University lacked a mechanism for these participants to continue the conversation on teaching. With few exceptions, teaching was considered a private matter. Evidence of excellent or poor teaching was documented with student evaluations of teaching, but was not a matter of public record or discourse. While research has enjoyed a long history of peer review and affirmation, teaching has lacked these features. Responding to a need for collaboration among teachers, the American Association of Higher Education initiated a national project, "From Idea to Prototype: The Peer Review of Teaching," which sought to improve teaching through peer review and collaboration. The University of Georgia was one of 12 institutions invited to participate in the Peer Review Project. University faculty from various disciplines attended national forums on peer review, developed pilot projects in their respective departments and sponsored campus-wide workshops [Roberts 1998]. While activities of the Peer Review Project were not readily adopted campus-wide, some departments developed teaching circles while others incorporated peer review into their promotion and tenure process. From the standpoint of campus culture, the peer review project offered a mechanism for greater collaboration among teachers. AAHE's Peer Review Project ended in 1997 and was superceded by the Carnegie Academy of Teaching and Learning (CASTL), which includes the Teaching Academy Campus Programs [CASTL].

The University of Georgia Teaching Academy

In March of 1999, Lee Shulman, President of the Carnegie Foundation for the Advancement, visited the campus and encouraged us to establish a teaching academy at the University of Georgia. Faculty who had participated in OISD's teaching fellows programs, recipients of campus-wide teaching awards, Regents Professors and University Professors were contacted to solicit their interest in establishing such an organization. Thirteen faculty identified in the survey were asked to establish the foundations of the University of Georgia Teaching Academy and later became charter members (Table 1). The charter members first considered adopting the teaching academy structure from other institutions across the country. Yet, they felt that the academy concept should not be just an end-product but a process of shared ownership and development. They wanted a structure that was unique to the University of Georgia and one that would build upon the many good works in teaching and faculty development that had been taking place on campus for the past 15 or so years. In many ways, the conditions and climate for establishing a teaching academy at the University of Georgia were ideal. After deliberating and planning for some 15 months, the charter members established the University of Georgia Teaching Academy in October 1999. The charter members comprised the Teaching Academy's Executive Committee, the governing body of the organization.

Robert L. Anderson - Arts & Sciences
Jeanne A. Barsanti - Veterinary Medicine
Josef M. Broder - Agriculture & Environmental Sciences
Ronald L. Carlson - Law
Joe W. Crim - Arts & Sciences
Sylvia M. Hutchinson - Institute of Higher Education
William K. Jackson - Instructional Support & Development
Patricia L. Kalivoda - Instructional Support & Development
Jeremy Kilpatrick - Education
Patricia Bell-Scott - Family & Consumer Science
Peter J. Shedd - Business
Frederick J. Stephenson - Business
Susette M. Talarico - Arts & Sciences

Mission - The mission of the Academy is to promote and celebrate excellence in teaching and to foster learning through inquiry.

Goals - The Academy will promote faculty leadership to enhance teaching and learning, to advocate for effective educational environments, and to foster a community of scholars.

Core Values - We believe that educating students is a fundamental responsibility of every faculty member of the University of Georgia and that teachers are catalysts for effective learning.

A Preamble was drafted to integrate the Teaching Academy's mission statement into the historical context of the University's mission (Figure 1.)

Summary of Programs and Activities

In its first year of operation the UGA Teaching Academy sponsored a number of projects and activities to promote interaction among teachers across campus. The working philosophy of the Academy has been to encourage opportunities for participation and engagement at all sponsored events. This section describes some of the major programs and activities sponsored by the Academy.

Inaugural Workshop

At the outset the Academy wanted to convey that membership was not merely honorific but that its members were expected to contribute to and participate in the Academy's activities. Hence, a workshop was held as part of the induction ceremony for the inaugural class. The workshop, "Taking Teaching Seriously: An Agenda for the UGA Teaching Academy," asked the membership to discuss the proper role of the Academy and to develop goals and action plans to promote teaching and learning on campus. Workshop recommendations included:

Make teaching "community property"
Create a center for undergraduate teaching opportunities
Establish chaired teaching professorships
Engage teaching talents and experience of retiring faculty
Establish interdisciplinary teaching circles
Establish teacher mentoring programs
Sponsor workshops and seminars to promote excellence in teaching and learning

Whereas, the great purpose of higher education is the strongest obligation to form the youth, the rising hope of our land, to render the like capable and dedicated to glorious and essential service.

Whereas, the faculty of the University of Georgia share a special commitment to the value and practicality of higher education and an obligation to promote a culture of inquiry, a passion for learning, a community of scholars and a contempt for ignorance, apathy and indifference.

Whereas, the University was founded on the vision of a land-grant institution with its roots in and commitment to serving all people of the state with the knowledge and skills that uplift the economic, cultural and spiritual well-being of the common citizen.

Whereas, the public's expectations and sensibilities of the University's commitment to teaching and learning are great and prudent.

Whereas, the teaching and learning mission of the University is the strength and only non-proprietary enterprise of the University.

Whereas, the University's tripartite mission in teaching, research and service enhances and enriches the learning environment and whereas faculty engaged in such activities have the fortune and obligation to share the fruits of their knowledge and activities in the glorious enterprise of teaching.

Therefore ,we solemnly commit ourselves to establishing a University of Georgia Teaching Academy to celebrate and engage the larger University community to embrace the joy, passions and rewards of teaching and learning. The charter members of the University Teaching Academy duly enact and embrace this assembly of scholars and promote its just and necessary causes and ambitions on this twenty-seventh day of October in the year nineteen hundred ninety-nine. We declare by these signatures that this body be hereby created.

Teaching Academy Forums

For faculty engagement to be effective and sustaining, opportunities for frequent interaction must be created. To promote interaction among its members, the Academy sponsors or cosponsors major teaching forums each semester. These forums feature prominent speakers and include meal functions, small group sessions, a major lecture, panel discussions and reception. Preference is given to speakers who are renowned for their contributions to teaching and learning. A summary of Teaching Academy speakers to date are shown in Table 2.

Attendance at Teaching Academy forums has exceeded expectations and the quality of interaction presentations has been excellent. Still, not all Academy members fully participate in these forms because of scheduling problems. Hence, other opportunities for interaction were explored.

Teacher Mentoring Program

Recognizing that faculty members are busy people, the Executive Committee explored ways for Teaching Academy members in interact on their own schedules. After all, the Academy's goals are to promote interaction between members and between members and nonmember. That is, the energy and expertise in teaching and learning lies with the Academy's members and interaction at the member level is thought to be the most productive. With this in mind, the Executive Committee surveyed the membership to explore their interest in various mentoring arrangements.

Advisory Functions

The Teaching Academy consists of the University's most outstanding and recognized teachers. Many of its members are involved in faculty governance at the department, college and university levels. The expertise and service of Academy members are in high demand. Recognizing these demands, the Executive Committee has been guarded in requests made of its members. Thus far, the Academy has been reluctant to assume complete responsibility for ongoing campus teaching activities. Instead, efforts have been made to tap the expertise of the Academy without asking them to commit large amounts of time. Three such requests have been made of the Academy. First, the Academy was asked to recommend books that would be assigned reading for entering freshmen. A Civil Action by Jonathan Harr emerged from the Academy's recommendations and was be assigned to students entering the University's Freshmen College, Summer 2001. Second, the Academy was asked to recommend topics and issues for the University's annual Academic Affairs Faculty Symposium. For the past 10 years, the Provost has sponsored a two-day symposium at an off-campus location to discuss critical issues in academic affairs. Each year, some 60 - 80 faculty across the University are invited to participate in these symposia. The recommendations of the Teaching Academy will be used to plan future symposia. Third, the Teaching Academy was asked by the Provost to evaluate the following proposals:

• Should the University create Josiah Meigs Professorships in addition to or in place of the Josiah Meigs Teaching Awards?



• Should the University create separate promotional tracks for faculty who are primarily engaged in teaching?

Implications for the University Community

The formation of the University of Georgia Teaching Academy represents a significant milestone in the University's history. The Teaching Academy will, no doubt, serve different purposes for different people and have a spectrum of supporters and detractors. What are the implications of the Teaching Academy for the larger university community?

First, the Academy is an affirmation by a major research university that teaching and learning are central to the mission of the institution. From a historical perspective, this affirmation is bold and necessary. Given the University's mixed reputation for teaching undergraduates, the Academy is good for the University.

Second, the Academy can and should celebrate and promote teaching and learning. The Academy should avoid notions of superiority or elitism and reach out to the larger campus community. Teaching and learning are not the exclusive domain of classroom teachers. The Academy should identify and celebrate teaching and learning excellence in all forms and settings across the University.

Third, the Academy should elevate and promote the scholarship of teaching as complementary not competitive to other forms of scholarships on campus. The Academy must bridge the divisiveness between teaching as scholarship and research as scholarship.

Fourth, the Academy creates a community of scholars who value and promote teaching and learning, a community that can energize and celebrate teaching and learning on a larger scale than has been possible on this campus.

Fifth, the Academy is a logical and opportune outgrowth of the many faculty development initiatives in the Office of Instructional Support and Development. The Academy can build upon and enhance the impact of these initiatives.

Sixth, the Academy establishes an ongoing forum for campus conversations on teaching and learning. That is, the Academy represents an institutionalized forum for peer collaboration of teaching and learning.

Seventh, the Academy, as a faculty organized and faculty directed organization, democratizes the vision for teaching and learning on campus and gives faculty ownership to setting the teaching and learning agendas on campus.

Eight, the Academy has the potential to foster a true sense of spirituality in teaching and learning. Many of us entered higher education not for the fame and fortune, but for a sense of service and kinship to the community of learners. We define our lives not by who we are but by what we have done for others.

Challenging the Teaching Academy

Given the core values and mission of the Teaching Academy, what role can the Academy they play in supporting the recommendations that follow from this Symposium. Let's examine the Academy's credentials. First, the Academy is a faculty organized and faculty driven association whose members are personally committed to promoting excellence in teaching and learning.

Second, the Academy includes the University's best and most motivated teachers. Third, members have collaborated with the Office of Institutional Support and Development and the Institute of Higher Education. This partnership between a faculty-driven organization, teaching support unit and teaching academic unit is a powerful coalition for change. Fourth, the Academy provides an ongoing forum on teaching and learning that transcends periodic self-studies and external reviews. Finally, the Academy serves in an advisory capacity to inform the administration of campus-wide teaching and learning initiatives. With these credentials, the Teaching Academy can serve in an advisory capacity to the recommendations that will follow from this Symposium.

How Do We Get There from Here?

The task before us today is one of moving the University from the ranks of the ordinary to the extraordinary, where teachers and students move beyond routine expectations for teaching and learning. This is a bold move for the University and one that will require bold thinking. Let me share with you some of my thoughts on this journey.

The quest for extraordinary teaching and learning begins with the recognition that teaching and learning are community property and the shared responsibility of the larger University Community.

The journey to the extraordinary will not happen as long as teaching is a private matter and the responsibility for teaching and learning falls on a handful of "teaching faculty." In fact, the concept of a teaching faculty versus research or service faculty is outdated and unproductive at a public research university.

Extraordinary teaching is fostered by a culture of teaching and learning that transcends the limitations of faculty governance and administrative mandates.

Although external factors play an important role in support of teaching and learning, they are not sufficient to foster a culture of excellence in teaching and learning. Administrative views and tasks are often too broad while faculty governance is too narrow to coordinate and implement sustained programs in teaching and learning.

The University should create an environment where extraordinary teaching is promoted, supported and rewarded.

Still, the University and the larger University System should promote, support and reward extraordinary teaching. This support is especially important in the formative years of faculty careers. The University's Teaching Fellows Program, Governors Teaching Fellows and Preparing Future Faculty Program are cases in point that warrant continued support and enhancements.

Assessment of Teaching and Learning

A university that is not serious about teaching will not be serious about learning and the assessment process

Until now, I've avoided the "A" word that many of you are so fond of. The "A" word being assessment. Yet, if we are serious about becoming an extraordinary institution for teaching and learning we need a sense of where we are now, where we want to be, how do we get there and how will we know how far we have traveled. I understand that this Symposium intends for us to think beyond assessment, yet we cannot avoid the subject all together. Thus, in the final part of my presentation, I want to touch upon the assessment of teaching and learning.

In the 25 years that I have been at the University, I have seen few serious efforts to critically assess teaching and learning process at either the philosophical or the mechanical level. Yet, the literature on the assessment of teaching and learning is considerable and there are lessons to be learned. Here are some of the issues and lessons from the experts. First, can and should there be a separation between the teacher/mentor and the evaluator/assessor [Shulman]? One school of thought would argue that the two are indistinguishable. Another school would argue that faculty and departments need help in this process.

Second, the assessment literature supports multiple measures of teaching and learning. While student evaluations are useful, other forms of evaluation provide useful information that may be more conducive to faculty development. Third, improvements in teaching and learning require both formative and summative assessments. Formative assessments are essential to young faculty who aspire to be extraordinary teachers. Fourth, in the absence of mentoring and collaboration, any assessment form and process is likely to be ineffective, if not a waste of time. Evaluation data alone may not help young faculty become better teachers. Lastly, the assessment of teaching and learning must ultimately focus on student learning. Many of our current instruments focus on faculty performance and do little to measure what happens to students. In the words of Nancy Cantor, Chancellor, University of Illinois at Urbana-Champaign, how has higher education been a "life changing experience for our students" [Cantor].

Of course, we must deal with the more mundane, if not outdated, instruments for assessing teaching and learning. Extraordinary teaching will require us to know more about our teaching than we have known previously. Instruments that are being uses to varying degrees include:

Student Evaluations or Teaching
Peer Evaluations of Teaching
Administrative Assessments
Teaching Portfolios
Scholarship of Teaching and Learning
Student-Centered Assessments

Over the years I have spent a great deal of time looking for the perfect student evaluation of teaching, one that I referred to as the "Holy Grail" of assessment [Broder and Taylor]. My hopes were that if we found the perfect instrument that we can create perfect teachers. Still, I learned a great deal from a close examination of student evaluations. In a paper published in Research in Higher Education, we reported the following from a study of student evaluations in my home department [Broder and Dorfman]. At the time, we were revising our departmental end-of-course evaluation, which had some 35 separate questions about the teacher and course. We were looking for a way to simplify the form.

Data for the study were taken from student evaluations from departmental courses over several years. Incidentally, we met some resistance to getting access to the evaluations. Our analysis basically asked, of the many different survey items, which are most correlated with the overall student rating of the teacher or course. Using a method of restricted-least squares, we measured the percentage contributions of individual items to the overall teacher or course rating. We also tested if students were consistent and rational in their evaluations. Our analysis found that overall ratings of teachers were determined by four identifiable factors (Figure 2). Likewise, overall ratings of courses were determined by three identifiable factors (Figure 3).

The methods and findings of this study are presented as an example of how student evaluations can better be used to improve teaching and learning. Far from being popularity contests, student ratings can provide a wealth of information on how and why learning takes place in our classes. These are the kinds of issues that need to be addressed as the University aspires to be extraordinary.

Of course, one criticism of traditional evaluations is that they tend to be teacher or course centered. That is, they measure more about teacher and course performance than about student learning [Maki]. Proponents might argue that course examinations should test for student learning. However, routine course examinations cannot address the broader learning objectives. Understanding and measuring the impacts of higher education on the student experience will require a much more creative instrument than is currently available on campus. The issue of where and when such an instrument should be administered must also be addressed. By student experience, I am referring to the learning outcomes that were identified by the University's recent Self-Study [The University of Georgia]. In that exercise, University students were expected to:

Identify and articulate life goals
Take an active leadership role in society
Appreciate range of disciplines/mission of UGA
Understand/appreciate tradition and diversity
Think critically and understand the value of inquiry
Possess effective communication skills
Master information technology for lifelong learning

If the University is serious about extraordinary learning, we need to determine what skills and life-changing attributes do we want our students to have, how can they be achieved and how can they best be measured.

Closing Comments

I have given you a brief description of the University of Georgia Teaching Academy and how the Academy might help advance the University from the ordinary to the extraordinary. I have also raised some teaching and learning issues that need to be addressed. I have spoken more about programs and activities and less about people, personal commitments and the remarkable journeys of faculty who have entered the realm of the extraordinary. While the University has room for improvement, we have a growing core of faculty who have achieved excellence in teaching and learning. Thanks to the efforts of Fred Stephenson, the trials and triumphs of these teachers have been documented for all of us to see. In closing, I want to express my greatest appreciation to Fred for giving us this book and for setting the stage for this Symposium. Fred's book, Extraordinary Teachers: The Essence of Excellent Teaching, has shown the larger University community, the positive, instructive and enlightening aspect of teaching and learning at the University. May his book serve to move the University to become an extraordinary place of learning.

Acknowledgments

The authors would like to thank fellow charter members of the UGA Teaching Academy, Robert Anderson, Jeanne Barsanti, Ron Carlson, Joe Crim, Sylvia Hutchinson, Bill Jackson, Patricia Kalivoda, Jeremy Kilpatrick, Pat Bell-Scott, Peter Shedd, Fred Stephenson and Susette Talarico for their valuable contributions to this paper. The author owe a special thanks to Ron Simpson for his inspiration and leadership as Director of the Office of Instructional Development and for his dedication and commitment to teaching and learning at the University of Georgia.

References

Boyer, Ernest L. Scholarship Reconsidered: Priorities of the Professoriate. Princeton, New Jersey: The Carnegie Foundation for the Advancement of Teaching. 1990.

Broder, Josef M. and Jeffrey H. Dorfman, "Determinants of Teaching Quality: What's Important to Students." Research in Higher Education, Vol. 35, No. 2, 1994, pp. 235-249.

Broder, Josef and Patricia Kalivoda, "Creating a Forum for Interaction Among Teachers Across Disciplines." Interactive Teaching and Learning Across Disciplines and Cultures: Case Method and Other Techniques. Edited by Hans Klein, Needham (Boston), MA: World Association of Case Method Research and Application, 2001, pp.249-260.

Broder, Josef M. and William J. Taylor. "Teaching Evaluation in Agricultural Economics and Related Departments." American Journal of Agricultural Economics, Vol. 76, No. 1, 1994, pp. 153-162.

Cantor, Nancy. "Purposes and Possibilities: Why the Scholarship of Teaching and Learning Matters" panel presentation, Carnegie Academy for the Scholarship of Teaching and Learning meeting, Chicago, March 15, 2002.

CASTL, Carnegie Academy for the Scholarship of Teaching and Learning. www.carnegiefoundation.org/CASTL/highered/index.htm, March 1, 2001.

The University of Georgia. "Creating a Climate of Inquiry: The Undergraduate Experience at a Public Research University and its Relationship to the University's Mission." Athens: The University of Georgia, January 2001, Vol. 1. p. 18.

Harr, Jonathan. A Civil Action. New York: Random House Inc.,1995

Hutchings, Patricia. Summer 1996. "The Peer Review of Teaching Progress, Issues and Prospects," Innovative Higher Education, Vol 20, No. 4. Pp.221-234.

Maki, Peggy L. "Developing an Assessment Plan to Learn about Student Learning." Journal of Academic Librarianship, January 2002. Pre-published on American Association of Higher Education web site: www.aahe.org/Assessment/assessmentplan.htm.

Merriam-Webster Online Dictionary. www.m-w.com/, April 2, 2002

Roberts, Beth. "Peer Review of Teaching," Columns, Athens, Georgia: The University of Georgia Office of Communications, www.uga.edu/columns/020998/focus1.html February 9, 1998

Roberts, Beth. "Teaching Faculty A Lesson," Columns, Athens, Georgia: The University of Georgia Office of Communications, www.uga.edu/columns/001127/campnews1.html November 27, 2000.

Shulman, Lee. "A Taxonomic Trek: From Student Learning to Faculty Scholarship." Plenary address, American Association of Higher Education Meeting, Chicago, IL, March 17, 2002.

Stephenson, Fred. Extraordinary Teachers: The Essence of Excellent Teaching. Kansas City, MO: Andrews McMeel Publishing. October 2001.