Posted 12 Jan 1993. J. E. Armstrong (Joe) - BIOLAB Sysop

----------------------------------------------------------------

But students' anonymous opinions about teachers on in-course questionnaires do not provide an objective measure of what they have learned. The process of final assignments and exams, where much or most of the course's learning achievement becomes clear, have not been completed. Moreover, it is consistent with student opinion of a teacher's teaching as excellent that in truth the teaching is inferior. The teacher could, for example, ingratiate him or herself to the students to secure high ratings, and not demand logical rigour, precision of concept, attention to text and detail, and so on because these demands might offend the students, especially students who are used to getting what they want. As in the market system, what counts here is not the competence of the students, but the comsumers' ability to demand satisfaction. This point is not lost on students.

The persistent grade inflation we find in our educational institutions seems related to this phenomenon. Grades inflate as student opinion and market requirements replace student learning and performance as the determiner of educational evaluation.

On the other hand, it may be consistent with a student or students' negative rating of a teacher that the teacher succeeds in achieving advanced or maximum development of student learning. The old-fashioned archetype of a "really tough teacher who taught me a lot" come to mind here. In bygone days, being "strict", "no-nonsense", "forcing you to learn", and so on was the standard, perhaps too one-sided a one. But now in the universities, it's how high or low one's student opinion polls are which sets the standard. Hard evidence of what students have in fact learned, or not learned, does not enter the picture.

How are we to resolve the problem of arbitrary judgement substituting for objective evidence in our assessment of teaching quality? Class visits by administrators, or even peers, do not provide reliable evidence, because they cannot assess the actual learning of the students over a course's instruction. Their evidence is confined to the oral exchanges of a mid-course class.

At the time it first started coming in about 20 years ago, the Teaching Evaluation Questionnaire seemed to be a way of grounding judgements of instructional capability on classroom performance. It took the first step towards making external judgements by administrative units more answerable to the reality of the classroom. Now there are library shelves of publications on teaching evaluation. Increasingly technical and scientific in appearance, the TEQ has become a major growth industry in the academy. But where in this burgeoning literature does the focus move beyond people's opinions about teaching and learning achievement to what has actually been learned by students under a teacher's direction in a subject-matter?

It is surely time the demonstrated outcomes of teachers' instruction were the basis of evaluating teaching. The consumer- market model of education is in principle inadequate because it conceives of education as a "product" to be "delivered" to students, whose quality can be assessed by the satisfaction of student clients or consumers. If we are serious about the much more demanding business of teaching and learning, which requires that people do not consume but earn their knowledge, we will have to get underneath opinion counting to what students have in fact learned in their courses.

We are used to the political game of "making people feel good" substituting for reality, and the underlying public relations strategy of manipulating perceptions rather than addressing actual problems. We deplore this pattern in politics and media. But the same tidal movement to images of what is going on rather than study of what is going on, may have taken over our understanding of education. We find it in the tirelessly repeated idea that universities and schools "must deal with public perceptions", "must adapt to the demands of the market", and so on. At some point, educators will need to move past the surface play of perceptions to the actual learning achievements of the students they have instructed. Then we will be able to understand and to demonstrate what we are in truth doing, and how well, or badly, we are doing it.

The distinction between public images and reality is a problem that has been examined since Lao Tzu and Plato. It is perhaps the ultimate problem of education. Current standardized tests, however, cannot ascertain learning achievements because their multiple choice format does not test for what is most basic and important to eductional achievement. They do not test, for example, whether students can in fact construct a coherent sentence or paragraph on their own. Uniform written examinations are no less problematic. They select for rote memorization of canonical truths, and against diverse points of view and curricula. Their imposition violates academic freedom. We must look elsewhere for our measure of what has or has not been learned. What can provide the evidence we require?

The objective evidence for assessing learning outcomes is already available in final written assignments and final exams. There are two problems here: the selection of what is to be judged, and what it is to be evaluated in comparison with. The first problem is solved by selection which is appropriately representative. The instructor and the department chair, for example, could choose one example from among what the instructor judges are the top, the middle and the lower performances, or one paper could be selected from each grade level (a maximum of six papers).

This representative selection could be supplemented by any paper whose grade is appealed by a student. Appeal-papers as evidence of good or bad teaching would provide ideal test cases, and have the advantage of encouraging consistently careful assessment. If teachers knew that the quality of their most important work -- evaluative feed-back on completed student performances -- was open to peer review at any time (a mental exercise good instructors already follow), their attention would be focussed on doing this work as conscientiously as possible.

The problem remains a base-line performance to get a measure of what students have in fact learned from the teaching of a course. An "entry performance test" would provide this base-line information. It would also provide an invaluable diagnostic tool from which to begin the teaching of any course, identifying the general strengths and weaknesses, the learning problems as well as capabilities of every student being taught. It is extraordinary that among all of the helping professions, teaching is the only one which does not begin with diagnosis. If initial diagnosis is too difficult to manage in large classes, then this is yet another reason why large classes are pedagogically inferior.

Liberation from being evaluated on the basis of subjective opinions would also depoliticize the teaching evaluation game, and help to protect academic freedom from the powers of censure by conventional doctrine and group opinion. If teachers knew that actual learning outcomes were the basis of their teaching evaluation, they would be better motivated to devote their attention to advancing students' learning, not to currying people's favour.

Teaching well entails that students learn from mistakes, which requires that these are continuously identified and corrected, which in turn incurs negative perceptions as well as positive ones independently of the learning achieved. Teaching well also often requires advancing arguments which unsettle accepted opinions and even concepts of self. If these necessities of teaching well are to be encouraged rather than made dangerous, we will need to return the evaluation of teaching back to consideration of the actual academic outcomes. Otherwise, we will continue to drift on a sea of quick-fix flotsam that knows no end -- looking to standardized questionnaires, peer visits, committee votes, administrative fiats, non-writing tests, large class techno-solutions, or whatever else the edu-market serves up next in place of the bottom line of learning achievement. At the same time, we may see academic freedom being increasingly sold out by teachers for the rewards and recognition of higher opinion-poll results.

----------------------------------------------------------------

The views expressed are solely those of the author and not of CAUT.