This webbooklet offers students in colleges of education a look at the development of some strongly, but rationally, contested, interrelated disagreements about teaching, about assessment, and about assessing the quality of prospective teachers. The discussion presented here took place from November 18, 1993, to January 14, 1994, on EDPOLYAN, a professional listserv for education (and the predecessor of EDPOLICY), based at Arizona State University. It is hoped that presenting the issues this way will help make them more meaningful to students than essays, articles, or typical textbook material on them normally do. It is my view that until one sees, and appreciates a problem, "answers" to that problem are not often very meaningful. Some problems are definitely displayed here.

Other purposes of this site are also to show:
1) how extremely difficult communication can be without persistent discussion that tries to clarify and resolve differences, some of which are based on, often at first unrecognized, mutual misunderstanding and some of which are based on focusing on different evidence, and
2) how complex (though not necessarily difficult) seemingly simple issues can be.
If you contrast this discussion with a typical journal article or book, I think the difference will be clear.  It is my contention that a discussion of this sort goes much deeper than a typical journal article. I think that too often (education) journal articles are considered definitive when they should instead be discussed and scrutinized.

This particular discussion sprang from questioning what is necessary (and sufficient?) for good teacher training, given that some states were starting to offer alternative progams for teacher certification. This led to the question of quality of teacher education in traditional programs and to questions about how to assess the quality of newly certified teachers. That, in turn, evolved into the issue of evaluating students in the classroom in general.
Rick Garlikov

The "****" symbol in front of a line signifies the line is quoted from a previous post.

The following participated in the discussion:

 Josh Barbanel Eugene Bartoo Bolland, Kathy Greg Camilli Cindy Cotter Andrew Coulson John F. Covaleskie Kevin Drumm Jill Ellsworth Mark Fetler John V. Gallagher Rick Garlikov David Gibson Joan Gipson-Fredin Gene Glass Josue Gonzalez Tom Green Aimee Howley Bill Hunter Noel Jantzie Greg Kirschner Jack Letarte Benjamin Levin CJB Macmillan John Nicholls Susan Nolen Alan Ogletree Hugh G. Petrie Thomas J. Pugh Louis Schmier Walter"Ev" Shepherd Leslie Wade

Date: Thu, 18 Nov 1993 12:25:17 EST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: ANYTHING but deregulation and privatization In-Reply-To: Message of 11/18/93 at 09:54:00 from ADAVIS@CUDENVER.BITNET

I bring to the attention of this list certain changes in West Virginia policy that strike me as troubling. I am aware that similar policies have been adopted elsewhere as well. In West Virginia these policies seem, however, to be directed toward the systematic destruction of teacher education. Interestingly, these policies spell out a sort of legislated deregulation of the teacher education process. In this state, policy makers seem to believe that teacher education programs are totally useless and that teacher education ought to take place after people are employed. These policy makers view colleges and departments of education as something worse than wasteful--sort of as parasites on the educational system, draining it of its vitality.

From this sentiment derive policies directed toward downsizing teacher education programs, providing various options for alternative certification, and linking salary increments to in-service rather than college credit. The rationale is that better teachers will be produced for less money if counties are permitted to take people with liberal arts degrees and give them on-the-job training. Moreover, the state has made a serious effort to destroy administrator-training programs, offering the option for anyone with an MA and administrative experience (including all teachers) to receive a 5-dollar certificate that permits them to serve as principals (all levels), supervisors, vocational administrators, or superintendents.

So my questions to the list are these:

1. Where else is this happening and why?

2. Are the policymakers correct in their judgment of teacher education programs?

3. How should colleges and departments of education respond to these policy initiatives?

--Aimee Howley; College of Education; Marshall University

Date: Sat, 20 Nov 1993 09:44:35 CST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Teacher education vs not

Aimee Howley asked a question that seems interesting to me, though no one has responded yet --about the effort of West Virginia to diminish or eradicate teacher education programs. I consider it to be primarily an empirical question as to whether graduates of teacher education programs make better teachers in general than people with other college training who have become teachers through various alternative means.

Since I believe some people with degrees outside of education make excellent teachers and that some education graduates don't make very good teachers, and that many education graduates do not know enough subject matter content to be able to teach as well as they should, I tend to be partial toward alternative teacher certification, based on demonstrable ability to teach, not on degree earned or knowledge of subject matter. With inservice or extra coursework as needed to learn or polish any missing skills.

I know many knowledgeable people cannot teach their knowledge to others very well, so I do not think a degree in chemistry will automatically make one a good chemistry teacher or that anyone with a degree can teach first grade, etc. But surely there must be some better way than what we have been doing to get knowledgeable, good teachers in more classrooms. Is there any research about any of this, or anecdotal evidence, or any theories..... Will the children of West Virginia end up in ignorance?

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Sat, 20 Nov 1993 12:33:57 EDT Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Greg <YL361C@GWUVM.BITNET> Subject: Re: Teacher education vs not In-Reply-To: Message of Sat, 20 Nov 1993 09:44:35 CST from <DEMS042@UABDPO>

The teacher education question is one that is very troubling to me. As Rick Garlikov said, I too believe that some people with little training can be very good teachers. I went to a Catholic school which did not require certification, and I had some very good teachers, who often didn't even have M.A.'s in their subject areas.

However, I think in general that teacher education is extremely important, especially considering the problems found in public schools today. Someone going into teaching without proper instruction and preparation may be simply over-whelmed.

More personally, I am concerned about my future. I have wanted to become a teacher for as long as I remember. So, I planned my education to get the best preparation as possible. At the end of it all, I should have M.A.'s in education and my subject area. Now I fear that, in light of all the alternative certification routes, I may be seen as over-qualified and too expensive.

Any thoughts on this? Will people who seek advanced teacher education be squeezed out of teaching? Will it knock wages down so that people with any college debt cannot afford to become teachers?

Greg Kirschner yl361c@gwuvm.gwu.edu

Date: Sat, 20 Nov 1993 14:04:18 EST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: Teacher education vs not In-Reply-To: Message of 11/20/93 at 09:44:35 from DEMS042@UABDPO.BITNET

I tend to share Rick's uncertainties about the best ways to attract (or perhaps cultivate) teachers who are both intellectually attentive and committed to the work. Coming to West Virginia twenty years ago with an East Coast liberal arts degree and the conviction that I could teach, I was convinced that any school system would want to hire me. But whether they wanted to or not was immaterial. I had two years of undergraduate work still ahead of me if I was to meet WV certification standards. Now--having found a way to be certified at the graduate level, having taught for a considerable time, I find myself deep in the midst of teacher education, mostly committed to a belief in its worth.

But it frustrates me that colleges of education haven't or can't find ways to accelerate or telescope instruction in pedagogy for those who have a good understanding of subject matter and good teaching "instincts." This type of alternative certification is far different from what our state department of education has in mind--but it is not an approach that I've seen advocated by teacher educators either. Are teacher educators simply protecting their turf? Or are they wedded to a socialization process that discourages certain sorts of talent? Or are they correctly upholding the benefits of an educational process that serves all prospective teachers well?

--Aimee Howley

Date: Sat, 20 Nov 1993 17:23:19 -0500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: PROHUGH@UBVMS.BITNET Subject: Re: Teacher education vs not

Rick Garlikov said:

****Aimee Howley asked a question that seems interesting to me, though no one
****has responded yet --about the effort of West Virginia to diminish or
****eradicate teacher education programs. I consider it to be primarily an
****empirical question as to whether graduates of teacher education programs
****make better teachers in general than people with other college training
****who have become teachers through various alternative means.

=======================

Yes and no. As Gene has recently said on this list, facts do not change beliefs (at least not very often), beliefs interpret facts. For a reasonably complete overview of the existing empirical evidence, see Linda Darling-Hammond's recent article in the Peabody Journal, I believe. (I don't have the exact citation here, but will post it later.) The major conclusion is that, on the whole, the evidence slightly favors graduates from approved teacher education programs over alternative route teachers (although it also depends on what the "alternate" route consists of).

However, other work shows that there is, as would be expected, a great deal of difference in quality among graduates of DIFFERENT teacher education programs. Shortly after A Nation at Risk some years ago, the "evidence" of SAT scores of high school students who indicated they INTENDED to go into teaching was that they were the worst and the dullest, not the best and the brightest. We did a study of the quality of actual teacher education students across the SUNY system. What we found was that by any measures we had, SAT scores, grade-point averages, grade point averages in general education, grade point averages in the major, graduation rates, etc., the teacher education majors were the equivalent of their counterparts IN THE SAME UNIVERSITY OR COLLEGE. There was, however, a great deal of difference across institutions. Similar studies, I was told, indicated the same thing in California and Washington. All of this is perfectly compatible with the initial low SAT scores when it is combined with Schlecty and Vance's work in the early 80s in North Carolina, I believe. They showed that what had happened over a period of time was that the more selective institutions in that state had largely gotten out of teacher education, leaving the field primarily to the third and fourth rate institutions of higher education, who, in turn, attracted the least qualified students across the board, and prepared proportionately more teachers.

=========================

****Since I believe some people with degrees outside of education make excellent
****teachers and that some education graduates don't make very good teachers,

=========================

Of course. There are even cases of some people without medical degrees making good doctors and fooling lots of people for a good long while, and lots of MDs who make lousy doctors. We also sometimes have shortages of doctors. I wonder why we never hear calls for alternative routes to medicine that could short-circuit all that irrelevant training in parts of medicine that I will never use as a dermatologist?

==========================

****and that many education graduates do not know enough subject matter content
****to be able to teach as well as they should, I tend to be partial toward
****alternative teacher certification, based on demonstrable ability to teach,
****not on degree earned or knowledge of subject matter. With inservice or
****extra coursework as needed to learn or polish any missing skills.

==========================

Given the abdication of responsibility by higher education noted above and the well-known unwillingness of state education departments to take seriously their responsibility for approving only high quality teacher education programs, and coupled with the only recently emerging willingness of the National Council for Accreditation of Teacher Education (NCATE) to impose meaningful standards, there just might be another policy alternative to alternative certification. PUT THE POORLY PERFORMING SCHOOLS AND COLLEGES OF EDUCATION OUT OF THE BUSINESS.

=========================

****I know many knowledgeable people cannot teach their knowledge to others
****very well, so I do not think a degree in chemistry will automatically make
****one a good chemistry teacher or that anyone with a degree can teach first
****grade, etc. But surely there must be some better way than what we have
****been doing to get knowledgeable, good teachers in more classrooms.

=========================

The traditional way that most professions try to solve this problem is by relying on a, roughly, four-fold approach so that people who ought not be in the profession get weeded out under one component or another. These are 1) strong standards for entry into the professional preparation program, 2) a preparation program which has to meet high standards, e.g., state or NCATE approval, 3) a serious internship, 4) an examination of ability to practice.

It will not be surprising that each of these components of professional preparation is less than satisfactory in teacher education, but is it really a wise policy choice to, therefore, throw over the entire system? Actually, what is happening in most states is that 4) is being relied upon to do nearly the whole quality assurance business, with any occasional gesture toward some mentoring by a senior teacher in the first year of a new teacher's job. The problem is that the kind of examination that could actually determine good practice is nowhere in sight. Furthermore, in no other field do we rely solely on an examination of ability to practice without the other checks and balances, no matter how good that examination is. We could, logically, put all the burden of quality assurance on examinations in medicine, the law, accounting, architecture, (and even cosmetology in New York), but we don't. Only in education is such a possibility seriously entertained. Why don't we, instead, get serious about improving all four of the traditional methods for assuring quality in the preparation of professionals? Is it, as Gene says, that the different policy positions reflect different very basic orientations which only allow some folks to see what they want to see?

=======================

****Is there any research about any of this, or anecdotal evidence, or any
****theories..... Will the children of West Virginia end up in ignorance?

=======================

It will depend, won't it, on whether or not they are lucky enough to get an alternatively certified teacher who just happens to take his or her responsibility seriously enough to try to pick up the missing parts of her or his preparation on her or his own. We do, indeed, learn from experience, but if it's lousy experience, it will be lousy learning.

Hugh G. Petrie 716-645-2491 367 Baldy Hall FAX: 716-645-2479 University at Buffalo Buffalo, NY 14260 USA prohugh@ubvms.cc.buffalo.edu

Date: Sun, 21 Nov 1993 08:36:52 -0500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: "Thomas J. Pugh" <tjpugh@MAILBOX.SYR.EDU> Subject: Aimee's ? re teacher ed. In-Reply-To: <9311182318.AB26097@mailbox.syr.edu>

Tom Mauhs-Pugh Cultural Foundations of Education Syracuse University

TJPUGH@MAILBOX.SYR.EDU TJPUGH@SUVM.BITNET

On Thu, 18 Nov 1993, Aimee Howley wrote:

**** of the teacher education process. In this state, policy
**** makers seem to believe that teacher education programs
****are totally useless and that teacher education ought
**** to take place after people are employed. These policy
**** makers view colleges and departments of education as
**** something worse than wasteful--sort of as parasites
**** on the educational system, draining it of its vitality.
****From this sentiment derive policies directed toward
****downsizing teacher education programs, providing
****various options for alternative certification, and
**** linking salary increments to in-service rather than
**** college credit. The rationale is that better teachers
**** will be produced for less money if counties are permitted
**** to take people with liberal arts degrees and give them
**** on-the-job training. Moreover, the state has made a
**** serious effort to destroy administrator-training programs,
**** offering the option for anyone with an MA and administrative
**** experience (including all teachers) to receive a 5-dollar
**** certificate that permits them to serve as principals (all
**** levels), supervisors, vocational administrators, or
**** superintendents.
**** So my questions to the list are these:
****1. Where else is this happening and why?

I don't know where else this is happening, but the why probably has to do with (1) a mistrust of program-specific, as opposed to examination-specific credentialling, (2) a concern with the expense of publicly supported ed. programs, (3) a decade of attacks on education focused on low teacher quality combined with a decade of increasing demands from teacher's unions, (4) a pervasive love affair in this country with anti-intellectualism, business model emphasis on performance over qualifications, and the entrepreneurial spirit, and (5) a clear lack of programmatic defense by schools of education exacerbated by continuous disagreement over core knowledge and commonly accepted practice of the profession.

**** 2. Are the policymakers correct in their judgment of
**** teacher education programs?

I certainly think we could do a lot better in preparing teachers to teach. From the standpoint of State educational policy needs and the staffing concerns of school districts, much of what the average (?) State college or university education program offers is inefficient at best, antagonistic to the State's interest at worst.

**** 3. How should colleges and departments of education respond
**** to these policy initiatives?

A good place to start might be to gain a coherent idea of State ed. policy and staffing concerns and address them explicitly and publicly.

Date: Sun, 21 Nov 1993 08:45:23 CST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Teacher education vs not In-Reply-To: Message of Sat, 20 Nov 1993 17:23:19 -0500 from <PROHUGH@UBVMS>

Hugh Petrie has provided a very extensive and well-thought-out reply to the issue raised by Aimee Howley, in his response to my seconding of her question. There are some residual questions I have.

First, however, I thought his presentation about the four step approach to trying to ensure only quality teachers reach, and remain, in classrooms -- along with the analogy he gives in that regard to other professions-- was extremely impressive and accurate. The topic of the (too often ignored) responsibility of schools of education to turn out qualified teachers, as opposed to certified teachers, has arisen here briefly before, with no real response to it when it came up a month or two ago. Hugh's comments are detailed enough that I hope there will be a response to that issue this time, and perhaps even some important policy recommendations coming out of the discussion.

But I do want to respond to the claim that other professions, such as medicine, do not offer alternative certification routes, particularly truncated ones. And I want to talk a bit about the difference between teacher training and medical or architecture training. First, medicine --though not medical schools-- do offer different, often shorter routes to practicing health care delivery. Midwifery, psychology, nursing, EMT training, chiropractics, homeopathy, acupuncture, lab technology, physician assistants, etc. are various entry modes into health care delivery. They meet certain needs (or at least try to or purport to). Further, doctors are often taught by people without medical degrees. Biochemists teach medical biochemistry; anatomists teach anatomy; researchers of whatever sort may teach about their specialties; bioengineers and bioethicists have various roles in teaching medicine and assisting medical practitioners. I would think that teaching would offer a great many similar opportunities -- especially for bringing in specialists periodically to teach areas the teacher is not particularly good at teaching -- for example, many elementary teachers have difficulty teaching certain math concepts: place value, fractions, general sorts of math reasoning. Would it be wrong to have those math specialists who have some real "instincts" for teaching (Howlee's or someone's apt term) be responsible for teaching these kinds of things in those classrooms?

I also do not believe that most undergraduate degrees give enough training in content. A B.A. in math education or even in math, may be insufficient training to be able to really teach math well -- even elementary math. A good teacher is not simply generally just a step or two above their students, but is someone who understands both the subject matter, and students, well enough to be able to make the subject really meaningful and inspiring to them.

Finally, I would like to know what sorts of things you all think an education student with good teaching instincts, who wants to teach in, say grades 6-12, needs to know, and whether he/she could not be taught those in one or two or three courses, one term? A local college gives a "fifth year" certification program, but from what I have heard from students, it seems to be a worthless, self-study, more or less self-directed, literature research kind of program that seems to have little meaningful or practical help. Surely that year could be better spent learning real teaching skills or helpful practices, rationales, etc., no? The 5th year program is an alternative certification program for people with undergraduate degrees outside of education.

These are some of the questions I want to raise at this point. But I want to go back over Hugh's response some more. And I hope most of you will take another look at it and that it can be the genesis of some very useful educational policy recommendations. It is an excellent post that deserves further discussion and recognition, I think.

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Sun, 21 Nov 1993 12:32:58 EST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: Teacher education vs not In-Reply-To: Message of 11/21/93 at 08:45:23 from DEMS042@UABDPO.BITNET

Rick asked about the things that a knowledgeable college student with good "instincts" for teaching might need to know about teaching in order to perform competently. I would suggest that foremost among them would be an understanding of what schooling is (has been and could be) all about coupled with an understanding (however imperfect) of what learning might encompass. Focus on the technical skills of teaching strikes me as a distinctly subordinate enterprise.

--Aimee Howley

Date: Sun, 21 Nov 1993 21:02:03 -500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: Teacher education vs not In-Reply-To: <9311201055.aa04275@ssi.edc.org>

I tend to agree with Rick Garlikov that alternative routes are important and that good teachers can come from inside or outside of schools of education. I like the graduate schools of ed that are concentrating on school development and teachers in classrooms. When I ran a private school, I always chose BA, BS and Masters degree holders in content areas - people who had those degrees plus the experience or makings of good teachers. In Vermont, there is some discussion about levels of licensure, where one needs the education school more (I think) between level one (just out of college) and level two.

Date: Mon, 22 Nov 1993 11:58:22 -0500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Kevin Drumm <drummk@POLARIS.NOVA.EDU> Subject: Re: Teacher education vs not In-Reply-To: <9311220249.AA21371@polaris.nova.edu>

Just a couple of questions on this topic.

I'm all for closing down "weak" schools of education. By what standards do we determine who is to be closed down? Since standards are shunned at both the secondary and college levels, where would we start?

What's wrong with "an emphasis on performance over qualifications?" I think part of the reason we are in the fix we are in is that we WORSHIP the flip-flopped arrangement of priorities.

Cheers,

Kevin Drumm NOVA University 305-424-5758 drummk@Polaris.NOVA.edu

Date: Mon, 22 Nov 1993 10:53:11 -0500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: PROHUGH@UBVMS.BITNET Subject: Re: Teacher education vs not

I have now found the reference to the Linda Darling-Hammond literature review article I mentioned in my post of November 20. It is "Teaching and Knowledge: Policy Issues Posed by Alternate Certification for Teachers." _Peabody Journal of Education_ vol. 67(3), pp. 123 - 154.

Rick Garlikov raises some additional questions.

=========================

**** But I do want to respond to the claim that other professions, such as
****medicine, do not offer alternative certification routes, particularly
****truncated ones. And I want to talk a bit about the difference between
****teacher training and medical or architecture training.
**** First, medicine --though not medical schools-- do offer different, often
****shorter routes to practicing health care delivery. Midwifery, psychology,
****nursing, EMT training, chiropractics, homeopathy, acupuncture, lab technology,
****physician assistants, etc. are various entry modes into health care delivery.
****They meet certain needs (or at least try to or purport to).

====================

You point to an extremely important area of policy reform in the system of teaching, not so much teacher education, although it could be used there as well. Other professions have any number of auxiliary roles and specialties defined so that the central practitioner can devote his or her time primarily to what she or he knows best. In education, we have one teacher, one classroom, with little use of teacher aides, interns, or specialists. The few places we do have specialists, they tend to be folks who "pull out" the kids for remedial reading or LD classes, instead of working as part of a team of professionals. The problem here is that this system would likely require more of a differentiated staffing pattern in schools than traditionally teacher unions would like to see.

Where all of this might impact on teacher preparation would be if we did less overall preparation of teachers and did more of it in "professional development schools", the teaching hospitals of the teaching profession. In this way you could get more of a seamless web of expertise, from university professors and graduate students to mentoring from experienced practitioners, all working as a team, which, since we teach as we were taught, might do more for encouraging team teaching than any amount of exhortation.

================ Rick further says

****Further, doctors are often taught by people without medical degrees. Biochemists teach
****medical biochemistry; anatomists teach anatomy; researchers of whatever
****sort may teach about their specialties; bioengineers and bioethicists have
****various roles in teaching medicine and assisting medical practitioners. I would
****think that teaching would offer a great many similar opportunities -- especially
****for bringing in specialists periodically to teach areas the teacher is not particularly
****good at teaching -- for example, many elementary teachers have difficulty
****teaching certain math concepts: place value, fractions, general sorts of math
****reasoning. Would it be wrong to have those math specialists who have some real
****"instincts" for teaching (Howlee's or someone's apt term) be responsible for
****teaching these kinds of things in those classrooms?

================

Again, this is a very plausible suggestion. If we had teams of teachers and teacher aides and instructors and specialists responsible for different aspects of the educational experience, we could, in principle, make better use of their individual talents. However, one needs to realize that this would represent a MAJOR cultural change in our current egg-crate organization of schools.

=================

Rick goes on,
****I also do not believe that most undergraduate degrees give enough training
****in content. A B.A. in math education or even in math, may be insufficient
****training to be able to really teach math well -- even elementary math. A
****good teacher is not simply generally just a step or two above their students,
****but is someone who understands both the subject matter, and students, well
****enough to be able to make the subject really meaningful and inspiring to them.

===================

I would say, ESPECIALLY elementary math. Think for just a moment about the typical collegiate math degree. What parts of it are at all useful for teaching elementary mathematics? The things that might be, e.g., number theory, geometry, and statistics, are often not even part of the collegiate curriculum. We have to have geometry taught at another college here at UB since our math department seldom does so. However, as Denise says, the usual response by state legislators is

==================

****Virginia is trying to remedy the problem of how to educate teachers by
****requiring college students to major in a discipline (content area) and
****get a certification to teach. This is true no matter what grade the
****individual wants to teach (NK-12). There is no longer a major in education.
****What are your thoughts on this approach to teacher education?

==================

Without a major change in the ways in which the non-professional education portions of the major are offered in the typical institution of higher education, this cure will be worse than the problem. I think it was David Berliner about 7-8 years ago who studied the effects of majors on teaching ability and, essentially, found that it made no difference except, perhaps, for those who teach at the advanced placement level. Here, too, I would predict that there would be major differences across the majors and institutions.

There is no question in my mind that a careful approach to offering the major with attention to what Shulman calls pedagogical content knowledge would help a great deal, but, again, this will require a MAJOR change in how, and even whether, our arts and sciences colleagues see themselves as part of teacher education. As Pam Grossman has suggested in a recent _Teachers College Record_ case study on several alternate route teachers, they teach as they were taught. They also learn from the experience of their first job. Those who were concerned that the seminar, abstract styles of their college courses weren't getting across somehow managed to hook up with some people who helped them out. Those who didn't blamed the students for not learning, even though it was painfully apparent that the kind of instruction these alternate route teachers offered was wholly inappropriate.

I would go so far as to predict that the major problems in teaching in secondary schools arise from teachers modeling the teaching they saw in their arts and sciences courses in college. The second major problem in secondary schools probably comes from inadequate content teaching in the arts and sciences. The problem is NOT that secondary teachers don't have a major in their field. Almost all do. The problem is more likely to be that the person who majored in chemistry will also have to teach a biology section and a mathematics section as well as chemistry. The one bright spot here is that the public is beginning to demand more accountability on teaching undergraduates from our institutions of higher education. Maybe that will help.

That brings me to some comments on the value of an "academic" major for elementary education. Implemented mindlessly, as the Virginia system appears to be, this, too, would be a disaster. Think for just a moment about the typical majors in college and their possible usefulness in elementary education, given what we currently ask our elementary teachers to do. If we REALLY want deep understanding of subject matter, combined with a knowledge of the development of kids, then we are, if we don't change our systems drastically, basically asking elementary teachers to attain the level of understanding of FOUR OR FIVE major fields which we don't think even very many of our bachelors or masters students achieve in one field. And just how long is the preparation period for elementary teachers to be?

I would fully grant that if the major problem in secondary education is probably inadequate pedagogy, the major problem in elementary education is probably inadequate content knowledge (and the major problem in middle schools is raging hormones). However, for reasons like those noted above, the answer is unlikely to be to require an academic major of elementary teachers, especially as we currently conceive of academic majors. One solution, compatible with the notion of changing to more team-oriented approaches to schooling outlined above, would be to require elementary teachers to take a seriously and carefully designed minor in one of several areas, e.g., math, reading and literacy, science, sociology. Then that person could be the "expert" on the team in his or her area of concentration.

Rick Garlikov goes on to say

==============

****Finally, I would like to know what sorts of things you all think an education
****student with good teaching instincts, who wants to teach in, say grades 6-12,
****needs to know, and whether he/she could not be taught those in one or two or
****three courses, one term? A local college gives a "fifth year" certification
****program, but from what I have heard from students, it seems to be a worthless,
****self-study, more or less self-directed, literature research kind of program
****that seems to have little meaningful or practical help. Surely that year could
****be better spent learning real teaching skills or helpful practices, rationales,
****etc., no? The 5th year program is an alternative certification program for
****people with undergraduate degrees outside of education.

and Aimee Howley echoes this concern

****But it frustrates me that colleges of education haven't or can't
****find ways to accelerate or telescope instruction in pedagogy
****for those who have a good understanding of subject matter and
****good teaching "instincts." This type of alternative certification
****is far different from what our state department of education has
****in mind--but it is not an approach that I've seen advocated by
****teacher educators either. Are teacher educators simply protecting
****their turf? Or are they wedded to a socialization process that
****discourages certain sorts of talent? Or are they correctly
****upholding the benefits of an educational process that serves
****all prospective teachers well?

====================

I'm not at all sure what "good teaching instincts" are nor how we would tell if someone has them, but the notion of a serious alternative ROUTE or multiple ENTRY POINTS to certification is a good one. The problem is that state departments of education seem to equate "alternate" with emergency and lower standards of certification and colleges of ed seem, for the most part, to believe in "one best way". The challenge is, I think, to design serious fifth year and MAT programs with highly interactive and collaborative clinical work in professional development schools along with significantly new and improved work with our arts and sciences colleagues. Again, however, the problem is a MAJOR cultural difference between what we in higher education value in arts and sciences, in schools of education, and what most schools value. Nevertheless, we must, in my judgment, begin to bridge these cultural divides and think about what the Education Commission of the States is beginning to call the SIMULTANEOUS RENEWAL of schools and higher education.

Sorry to have gone on so long, but maybe this will be helpful.

Hugh G. Petrie 716-645-2491 367 Baldy Hall FAX: 716-645-2479 University at Buffalo Buffalo, NY 14260 USA prohugh@ubvms.cc.buffalo.edu

Date: Mon, 22 Nov 1993 15:00:31 MST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Education for classroom aides In-Reply-To: <9311222110.AA25882@acs1.acs.ucalgary.ca>; from "PROHUGH@UBVMS.BITN

Hugh Petrie has a wealth of ideas for teacher education and I plan to look back at them from time to time. The ones that stood out on this reading had to do with differential staffing. Some years ago, I was involved in an evaluation of a Follow-Through program that made extensive, but rather mechanical, use of teacher aides. At the same time, I evaluated an innovative program in a wealthy suburban school that was making really interesting use of parent volunteers. Some of us here are now thinking about what we would want to include in a short course for volunteers or aides. Anyone had experience along this line?

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Wed, 24 Nov 1993 22:34:56 MST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: "G. Noel Jantzie" <gnjantzi@ACS.UCALGARY.CA> Subject: Re: Teacher education vs not In-Reply-To: <9311222110.AA25882@acs1.acs.ucalgary.ca>; from "PROHUGH@UBVMS.BITN

Having just finished a rather hectic term instructing prospective Social Studies teachers on the intricacies of methodology and classroom practice (they're now out in the schools for their first practicum round of four weeks before returning on campus for more classes), I believe that I can shed some light on why at least a one or two year program from a school of education is necessary for the development of superior beginning teachers.

The g.p.a. for my methods class was probably in the area of 3.7 to 3.8 out of 4. B.Ed. students are required to have an average of at least 3.5 to enter the program at the end of their first year of general studies at this university. Students who have a first degree are required to have at least a 3.7 average before being admitted into the secondary Social Studies program. Most of my students (19/35) had a first degree in one of the social sciences (primarily history, secondarily sociology); three had M.A.'s; many of the other students are working on simultaneous degrees in the faculties of Arts and Education. Several of these students had extensive experience teaching and instructing in other countries or in fields which did not require formal certification. Most of them are highly motivated, despite the fact that job prospects in our province are worse than dismal (the Alberta government proposes to cut 20 to 30% of the Education budget over the next three years). Yet regardless of their motivation and ability these students have at least one major flaw that was evident as they entered the program: their conception of teaching and learning was based entirely on their experience as successful students in a system that depended primarily on lecture, the replication of textbook answers and the taking of notes as the dominant instructional methodology (I think I see a cross-connection to the "Why Don't Teachers Incorporate Research on Learning" thread here).

So one of the prime tasks that the methods instructors in Social Studies set ourselves was to model a variety of more effective classroom approaches as we persuaded these students that there is more to teaching than "stand and deliver". It would be nice to believe that in their eleven weeks of practicum in the classrooms of this province our students would get that message from practicing teachers, but....as I hear more and more stories coming back from the classrooms I begin to understand the despair of those Curriculum Supervisors who have been trying to improve practice in our jurisdiction over the past ten years. I just ran into one of my students in the library as he was pouring over the microfilmed back issues of newspapers preparing his lesson-plans for tomorrow. As he told me that his students were just finishing studying the issue of NAFTA and he was looking for a concluding activity I blurted out the idea of holding an informal horseshoe debate--allowing students to make use of the concepts and arguments they have been researching for the past week. "Well...I don't know if my co-operating teacher would like it," he replied. "He usually has them sit in straight rows and I don't think they are ready for this kind of group work yet."

Unfortunately that kind of a classroom appears to be the rule rather than the exception as reported by the students who have been bringing back their classroom observations to their practicum and methods instructors. I know that there are different schools out there, I've taught in two over the past twelve years, but it seems clear that the impetus for change and improvement that I have seen owes a great deal to some of the people working within the faculties of Education in this province. I believe this is particularly so in those instances where the instructors or faculty have close or recent connections to the classroom (three of the four Social Studies methods instructors here are graduate students just out of the classroom).

If Faculties and Schools of Education are not the people to provide leadership in education then who are? Some practicing teachers are burned out, some are mired in unproductive teaching methods, some are just focusing on raising their average on the provincial exam and some are counting down the days to early retirement. Principals are kept busy on the rodent-wheel of system and department administrative meetings. The general public seems wedded to the idea of school as it was experienced in the good-old-days of reading, 'riting, 'rithmetic and 25% graduation ratios. As for politicians....well this is a pg rated forum and I don't want to cross-post to alt.scatalogical.comments.

That's it for now, they're shutting down the microlab and I still have 25 unit plans to mark that my students left me before they went out to the schools.

--Noel Jantzie

Date: Fri, 26 Nov 1993 17:49:56 CST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: the nature of good teaching

There was a question about the nature of teaching and learning (Aimee Howley asked it, I believe, or Cindy Cotter). And there has been much comment about teacher directed classes versus student centered courses, along with comments about the problems of teaching the way one was taught, especially the way one was taught in college. (Hugh Petrie, for example, said "I would go so far as to predict that the major problems in teaching in secondary schools arise from teachers modeling the teaching they saw in their arts and science courses.") If Hugh and others mean something other than "lecture, memorize, regurgitate on the exam", please say so. Many of my liberal arts courses were not of that type, so I am wondering if there is something besides that which is also considered NOT good teaching. Hugh also said he was not sure what "good teaching instincts" meant. All this goes together I believe, and I would like to make some general comments for your considerations; and ask some questions.

What I mean by a good teacher is someone who can get, or help, (1) a student to learn something, (2) to understand it in those cases where understanding is appropriate, (3) in the easiest, sometimes most efficient way (as long as efficiency does not sacrifice learning or understanding), AND (4) make it interesting enough for the student to want to learn more and/or to use what is taught in some meaningful way. The better teacher is not always the more knowledgeable person. I once served as a graduate teaching assistant to a man who knows fifty times more information about the history of philosophy than I will ever know, but his lectures and his topics of course study for philosophy 101 killed the interest of hundreds of students that unknowingly signed up for his course when he was the lecturer. Though I knew far less, I could turn out students who knew far more philosophy, understood it better, liked it better, and wanted to go on in it. He was by far the better scholar, but for introductory philosophy, I was the better teacher, I would argue.

I would argue that a teacher is to be judged by how well they teach a given student or group of students -- where the person doing the judging needs to have a good idea of the degree of difficulty of the subject matter for a given group of students. Judgment is a somewhat subjective enterprise but hardly just a matter of whim. There is some inter-subjectivity to it. If I can teach almost anyone to ride a bicycle in thirty minutes, and they enjoy learning and want to ride all day after they have learned, and if they are not afraid with me teaching them, and don't cry, etc., then I am a better bicycle riding teacher than someone who makes kids cry, makes kids not want to learn, takes weeks to teach them, and makes the experience so terrifying and unhappy that they really don't much care to learn or to ride their bike once they have learned. Something similar could be said for any two teachers in any given subject.

And, I am with Hugh, though I may say it more strongly than he does, in believing it is reprehensible for ed schools to turn out bad teachers and for administrators to keep them on and in some cases give them to the most vulnerable children. And I do not think it is that difficult to tell whether somebody can teach something or not. Unless, of course, the judge does not have a clue how to tell whether a student or group of students has learned something, and learned it enthusiastically or not. In terms of POLICY RECOMMENDATIONS, I would love to see school administrators and college faculty and departments able to be fined or sued for teaching bad courses and/or for graduating and certifying bad teachers -- not just inexperienced teachers, but teachers with little or no ability to teach anyone anything. A little personal or professional responsibility with some personal consequences might do wonders for instituting a real four-point quality control program of the sort Hugh has described.

In regard to "teaching instincts", there are a number of things involved that all relate to the above. Does a perspective teacher understand that more than one method may be necessary to teach a given person a given skill or some understanding or knowledge? Does he/she understand that the way things are organized and presented can make a big difference in how or whether it is learned? Does the teacher understand that the failure to teach is not necessarily, or likely, the students' fault, and that it means a different teaching approach is necessary, or a different attempt, or some "block" needs to be overcome somehow, or that some other thing needs to be taught first in order to make the current subject accessible to the student? Is the teacher motivated to keep trying? Is the teacher excited by success of the student's learning, and is the teacher really interested in general in getting or helping students learn things? Does the teacher think knowledge and ability is great to achieve, and to help others achieve? Do they care more about students' learning than about getting a bell-curve grade distribution? Does the teacher understand nothing is taught until a student has learned it; that presentation is not teaching unless the presentation is meaningful to the student in the right way? Does the teacher see the student as a human being or just an empty vessel? Does the teacher try to find out what the student already knows, or is learning as the "lesson" progresses? Is there any dialogue, or any curiosity on the teacher's part as to what sort of impact he/she is actually having on the child? Is the teacher able to appreciate ambiguities, mistakes, and misunderstandings, and treat them as natural occurrences without making the student feel the student must be somehow stupid or inept, or that it is their own fault they cannot understand? Etc., etc., etc.

Aren't these the kinds of things that good teaching is about?

Now, I surely do not understand the stuff about teacher directness versus student centered learning, or however it is called. I do not see these as mutually exclusive nor jointly exhaustive. When Louis Schmier took his camera equipment to the classroom he tells about, he was directing the lesson and controlling its content within certain boundaries; it is just that he is doing it by letting students do certain things actively instead of just taking notes or "listening" while sitting passively. He has set up the environment, and probably says enough things and steers just enough so that the students' explorations are productive. Had he not set any direction or taken any equipment in, or answered any questions at all, little would have happened. I assume that we don't want students having to reinvent by themselves the history of civilization and its achievements. Some sort of "telling", guiding, steering, directing, or whatever has to go on or it will take them thousands of years to learn thousands of years worth of accumulated knowledge. Isn't "teaching" supposed to be a more or less shorter way for students to gain knowledge than by mere exploration and re-invention. Otherwise why have teachers and schools at all! Aren't we interested in the most effective ways to transfer knowledge and ability to the next generation (or whoever comes to us for learning). If a way is the most effective, does it matter whether it is lecture or not? Perhaps some people can lecture very well, like a good story teller. If it is not effective, does it matter whether it is student "driven" or not?

From what has been said, and the examples given, I assume YOU mean by "teacher directed" either of the following two things: 1) teachers just lecturing, especially droning on in ways that stifle children's learning, and 2) teachers not letting students explore and attempt to figure things out, or do, things that would be good learning experiences for them, and which they could do successfully in some fashion or other. Having kids paint by the numbers would be teacher directed, I presume, even though the kids are doing the work. Using the Socratic method to question and challenge students, and make them figure things out for themselves, would, I presume, be child directed even though the teacher would be guiding the discussion to keep it logical and sensible in case a student happened to start going too far astray. Is this a fair description of the dichotomy you have in mind? If so, I think the words are misleading that you use to express the dichotomy; misleading to ed students and to others. I would rather see the emphasis put on when "telling" might be most appropriate, when exploration, when questioning, what sorts of questions are the most productive, how to guide exploration without "killing it" or oversteering it, yet without letting it turn into fruitless, wasted effort.

Aren't there better ways to talk about good teaching than whether it is lecture or not, teacher directed or not, etc.?

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Sat, 27 Nov 1993 10:06:52 EST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Tom Green <TFGREEN@SUVM.BITNET> Subject: Things here and there on teaching ability 'n such

Some jottings: 1. Good teaching involves an ability to discern what is going on in the mind of the student (child, adult, conversation partner, author, composer, stage manager, musician etc) ) and even more than a mere ability to do this, it requires the actual exercise of discernment. This may explain why 'regurgitation,' rote, mindless mumbling 'talking at,' and suchlike are often represented as bad teaching even though each of these has a place in the total scheme. The fact is that none of these approaches would be pointed to as paradigmatic cases of 'discerning what is going on in the mind of the ....' It is OK for teachers to teach in ways that model the ways they were taught, but only if (or at least especially if) they were taught by (with) someone skilled at discerning what is going on in the mind of the -----. There seems to me, moreover, some interesting connections between this element of teaching (gift of teaching abilities) and such other things as a capacity to read, listen, write, notice, observe, and a thousand other verbs all of which are implicated in what teachers (or at least, teaching) aim to do.

Well several thoughts turned into one. That's it. Hugh Petrie suggested there were 'teaching abilities' and Rick Garlikov asked what they were. This is my suggestion. I have others -- I THINK. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + THOMAS F. GREEN (TFGREEN@SUVM.BITNET) + + EMERITUS FROM SYRACUSE UNIVERSITY + + PHILOSOPHER IN RESIDENCE ON POMPEY HILL + + Box 100 Pompey, NY 13138 (315) 677-9935 + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Date: Sat, 27 Nov 1993 10:41:49 MST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Walter Shepherd <ASWES@ASUACAD.BITNET> Subject: Re: the nature of good teaching In-Reply-To: Message of Fri, 26 Nov 1993 17:49:56 CST from <DEMS042@UABDPO>

On the subject of excellent teaching, here is an essay I wrote some time ago for another reason. If any of you have time to read it, I would appreciate your reaction.

Ev

===================================

Who Are the Excellent Teachers? W. E. Shepherd

The State of Arizona has a new career ladder program for teachers. That's a merit pay plan which says basically that everyone who jumps through a certain set of hoops--more college classes, professional organizations, workshops, that sort of thing--is an excellent teacher who deserves more money than his colleagues who lack either time or inclination for hoops. It's another in a long succession of attempts to reward the best teachers, in spite of the fact that few agree on how to identify them. Since everyone who ever sat in a class considers himself an expert on excellence in teaching, I feel completely justified in putting forth my own criteria. This is, of course, a response to the title's question which I shall interpret as an attempt to identify the outstanding teachers; to that end, I wish to propose four criteria.

First, teachers must know their subject well. There is a myth out there that a good teacher can teach while staying a few pages ahead of the students. Nonsense! A clever student could do as much! We need teachers who are able to set long- term (five- or ten-year) goals for students in their field. Anything less is the blind leading the blind. It is, by the way, fairly easy to verify mastery of a subject--except for those "subjects" in which the purpose is to instill attitudes or feelings, of course, where it is impossible. (Such courses have proliferated in response to social problems abandoned at the doorsteps of our schools. Things like drop-outs, drug abuse, sex, poverty, disease, single parent families, minority/cultural conflict.) Please note that nobody in the education business is talking about mastery of subjects and our current Teacher Proficiency Tests, which ask calculus teachers to demonstrate their ability to do fractions, are a pathetic embarrassment.

If my teachers are masters of their subject, they nevertheless would not be teachers who "cover material." We all know the type; they are in a relationship with a textbook rather than with students, so caught up in their subject that students come and go almost unnoticed. Excellent teachers care about children. They are acutely aware of students' learning styles, frustrations and needs, but are not so involved in relationships that it never occurs to them that there are dangers in altruistic indulgence, nor that the discipline of good, hard work can be just the thing to help a child rise above very real personal tragedies. Still, a mastery of the subject and an acute, sensitive understanding of children are only fundamental tools to a teacher--a carpenter's hammer and saw. They do not make him excellent. Excellent teachers succeed in bringing students and subject together where others fail. How do they do it? Here's where it starts to get sticky.

Many teachers act as if they carry a coin in their pocket; on one side is bribe and on the other is threat. Motivation is a toss of the coin where the teacher tries to find what the student does (or does not) want. Usually such teachers will concentrate on the obvious--health, wealth, popularity, success, good grades and feeling good, versus disapproval, failure, poverty, disease, bad complexion, etc., some or all of which will come to students if they do (or do not do) as the teacher requires. Actually, these teachers are often very subtle in their application of this method and some of them may seem quite successful, especially those whom the students genuinely admire, and who use that admiration like the Pied Piper used his pipe. Those are not excellent teachers! Theirs is a process of seduction that may lead to good grades and even to happy students, but not necessarily to significant learning. When their students won't cooperate, when they just don't care about either the bribe or the threat, such teachers are quickly confounded and dump the blame squarely on someone else. I've heard them to say:

--I taught that to you kids last week. Why didn't you learn it?

--Kids just don't care about anything anymore.

--We just can't get any support from parents nowadays.

--Thirty years ago kids didn't have jobs, cars, Nintendo, easy sex, drugs...etc., etc.

--I just don't know how they expect me to teach so many kids in one room!

--They won't do the work. If they choose to fail, it's their choice. I'm here to teach them if they want to learn.

I didn't make those up. They are all real quotes! The names have been omitted to protect the guilty!

The best teachers I have ever seen don't think in terms of motivational techniques at all. They have an infectious enthusiasm for their subject and are truly surprised at the occasional student who doesn't catch it too. Like the teachers above, they have a coin in their pocket while they are teaching, but their coin has encouragement on one side and challenge on the other. They challenge children to take on the toughest task they can handle and then give them the equipment and encouragement to do it. They set a mountain in front of their students (the more worthy the subject, the higher the mountain) and then get them to climb it because it's there! It may seem that I am making a difficult task even harder by limiting motivational gimmicks to enthusiasm alone, but that's not their only tool! There is another which, though largely ignored in evaluations, is probably the most important of all, since none of the others matters if it is missing.

An excellent teacher is able, in his own mind, to separate children from the work they do, so that he can tell one child that his work is unsatisfactory, even worthless, without telling the child that he is unsatisfactory or worthless. He is able to tell another that his work is outstanding without the child believing that his work makes him an outstanding person. The latter is just as important as the former since the quiet student, waiting unnoticed in the back of the class to decide what school has to offer him, will learn from what the teacher says to others.

--The teacher likes Mary. Mary's smart. I'm dumb, so that means the teacher doesn't like me. Get me out of here!

I know an excellent French teacher who tells of a student who kept coming back year after year. Four years of very hard work, and he never earned a grade higher than a C! Another student had an absolutely impenetrable Texas accent and, although his grades were rather good, his French would never rise to the rigorous standards of the sensitive Gallic ear. The puzzlement in her voice as she tells of them makes it clear that she does not see anything in herself that would explain their enrollment each year in the next higher level with little hope of outstanding success. I've seen her teach and I know what it is. Every student from the brightest to the slowest is challenged to do his very best with no fear at all of failure. Here's an example of her criticism as she returns papers.

--Jimmy, could you see me about this? It looks to me like you're a little vague on which tense to use in these situations. Don't worry, we can fix it.

--Eric, I can see that the imperfect tense has no more mysteries for you. Now what would happen to those sentences if the imperfect tenses were all changed to the pluperfect?

It is impossible to tell from the teacher's conversation with her students which ones she likes or dislikes, or even who has the best grades. Certainly there are students she doesn't like--some are obnoxious, manipulative, spoiled brats--but her judgment of their progress in French is unaffected by her feelings. In her mind there is absolutely no connection whatsoever between the quality of the work her students do and the value she places upon them as individuals. Such a teacher frees her students to plunge into more and greater challenges without the fear of failure because they know that failure tells nothing about their personal worth.

There are, after all, four fairly simple things we should ask of our teachers: they should be masters of what they propose to teach, they should understand and care about children, they should manage relationships with their students using encouragement and challenge rather than bribery or threat, and their students should reflect the security and freedom that come from a teacher who judges work on its merits and not people. Such people should teach for their own sake and for the sake of children. Now who believes a Career Ladder merit pay plan will encourage them in these things?

============================== | Ev Shepherd | | Scottsdale, Arizona | | ASWES@ASUACAD | | ASWES@ASUVM.INRE.ASU.EDU |

Date: Sun, 28 Nov 1993 22:34:40 EST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: "John V. Gallagher" <gallagherj@SATURN.ROWAN.EDU> Subject: Re: On the Nature of Teaching

Does this mean that we should let anyone without an education in the fundamentals of pedagogy enter our nation's schools as teachers, much as is touted by the proponents of the "alternate route?" All they need is a degree and expertise in a subject discipline.

Or

Is this a statement in which one is not sure if there is a science to education and that maybe it is an art?

Or

Maybe all this stuff we try to fill kids heads with is not relevant. Maybe we should spend our time teaching students the skills of life and the skills of living. Of course, that flies in the face of the so-called national standards. Who says that the national standards are right?

Interesting -- we get back to the three fundamental questions we learned in our first research course:

1) What to teach

2) How to teach it

3) How do we know we taught it? ( and how do we know the students learned it?)

John V. Gallagher Associate Professor Rowan College of New Jersey School of Education and Relate Professional Studies

gallagherj@saturn.rowan.edu

Date: Sun, 28 Nov 1993 22:02:09 CST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: On the Nature of Teaching In-Reply-To: Message of Sun, 28 Nov 1993 22:34:40 EST from <gallagherj@SATURN.ROWAN.EDU>

John V. Gallagher said:
****
****Does this mean that we should let anyone without an education in the
****
****fundamentals of pedagogy enter our nation's schools as teachers, much as is
****
****touted by the proponents of the "alternate route?" All they need is a degree
****
****and expertise in a subject discipline.
****
**** Only if it is demonstrated that they make as good or better teachers than those with training in the fundamentals of pedagogy. I was asking whether there was any evidence either way.

****
****Or
****
****
****
****Is this a statement in which one is not sure if there is a science to education
****
****and that maybe it is an art?
****
**** I am certain there is science, art, and attitude involved. I am not particularly convinced schools of pedagogy appreciate the latter two or that they get the science part of it right anyway. Or that they impart any of the three aspects to their ed students, or care whether they have or not. Nor am I convinced education administrators understand or care as much as would be reasonable.

****
****Or
****
****
****
****Maybe all this stuff we try to fill kids heads with is not relevant. Maybe we
****
****should spend our time teaching students the skills of life and the skills of
****
****living. Of course, that flies in the face of the so-called national standards.
****
****Who says that the national standards are right?
****
**** I am sorry, but I don't know to what this refers. I can't think of what was written that prompted this question in this particular instance.

****
****Interesting -- we get back to the three fundamental questions we learned in our
****
****first research course:
****
****
****
****1) What to teach
****
****
****
****2) How to teach it
****
****
****
****3) How do we know we taught it? ( and how do we know the students learned it?)
****
****

The second example is one common to every student who ever had difficulty with first year algebra. A kid does not understand how to work any problems; so what do all the lousy algebra teachers say? Everybody can answer this one: "You need to work more problems. You just aren't working enough problems. If you work more problems, it will become clear to you." Bulloney. If a student cannot work any problems, he/she can't work many of them. There is a more basic difficulty than mere lack of practice or diligence in this sort of situation. Do pedagogical theories not understand or teach that? Do they not teach it in ways ed students can understand it or remember it? Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Wed, 1 Dec 1993 09:48:40 LCL Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Tom Green <TFGREEN@SUVM.BITNET> Subject: Re: On the Nature of Teaching In-Reply-To: Message of Tue, 30 Nov 1993 19:19:51 CST from <DEMS042@UABDPO>

On Tue, 30 Nov 1993 19:19:51 CST Rick Garlikov said:

****What I am really getting at is that it seems to me that schools of education
****should, above all, graduate or not graduate their own students on the basis of
****how well those students can teach. =================================================================
Rick: What would you think of the proposition that we should judge the performance of teachers of ethics by determining the moral qualities of their students? Tom Green

Date: Wed, 1 Dec 1993 14:53:04 -0500 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Greg Camilli <CAMILLI@ZODIAC.BITNET> Subject: Re: On the Nature of Teaching

I had have been following the discussion on good teaching with a moderate amount of interest until today, when a light bulb went on. As I understand it, part of what makes a good teacher is the ability to listen, to understand students' difficulties, and to use flexibly and creatively any number of techniques to promote learning including: lecturing, discussion, novel classroom arrangements, drawing students out, and leaving them alone. I don't quite remember if there was a consensus on whether these techniques could be taught in teacher education programs or not. However, it strikes me that these are not qualities that teachers alone need, rather they central to both self-awareness (with one's self as the student as well as teacher) and satisfying social relations. These are qualities we seek through disciplining ourselves (guidance), pursuing our own interests (discovery), having a philosophy and a code of ethics, and being spiritual (or at least recognizing such needs).

Instilling these qualities into teachers can be called "teacher education," but outside the classroom this state of evolution is perhaps just short of enlightenment. There are two directions that this line of reasoning can pursue. It first comes to mind that such expectations are way too high, and out of line with expectations in other professions (with exceptions for priests, sages, gurus, etc.). How is a teacher taught to be sensitive to the needs and to the nuances in behavior of every student? Introduction to Nuances for Teachers 101? (This brings up a secondary issue. Since when did teachers have to be taught these qualities? Were our educations defective, or was the population of teachers 35 years ago more gifted?)

The second line of reasoning is that these expectations are not too high, and that they can be taught. However, the type of instruction that would be required would be a more direct form of training in personal strength, courage, sensitivity and competence. These qualities would be the foundation to which more specific teaching skills and subject matter knowledge are added. It isn't clear how this might be accomplished, but should this training take a more specific form for teachers given that everyone could benefit, especially students?

Gregory Camilli Tracy Lien

Date: Wed, 1 Dec 1993 23:28:22 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

When I was a principal, I had to make decisions about granting or denying tenure. THAT was certifying that I thought someone was a good teacher, or not. What I expected the colleges to provide me in a first year teacher was someone they had reason to believe might someday BECOME a good teacher. I am now teaching in a teacher preparation program. There is no way I can know if my students will be good teachers (though I do, of course, have my suspicions.) But ultimately, the only way we will know if any individual is going to be a good teacher or not is to let them teach, to work with them early in their careers, and see if their potential is both sufficient and realized.

There is much that I can teach that I hope will be helpful in shaping the attitudes and developing the skills that make a good teacher. But as to whether the individual is capable of creating and sustaining the relationships that are at the heart of good teaching, their early supervisors and colleagues will have a great deal to do with whether a "good teacher" results.

* * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET

Date: Wed, 1 Dec 1993 19:40:53 CST Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: On the Nature of Teaching In-Reply-To: Message of Wed, 1 Dec 1993 09:48:40 LCL from <TFGREEN@SUVM>

Tom Green said:
****On Tue, 30 Nov 1993 19:19:51 CST Rick Garlikov said:
********What I am really getting at is that it seems to me that schools of education
********should, above all, graduate or not graduate their own students on the basis of
********how well those students can teach.
**** Rick: What would you think of the proposition that we should judge the performance of teachers of ****ethics by determining the moral qualities of their students?

I am not asking that education professors be judged on how well their students teach. I am asking that professors and schools of education not certify people to be teachers who cannot teach. Obviously not everyone learns what they are supposed to in various courses, and that is not always the fault of the teacher; but it is the fault of the teacher to pass students who do not learn what they are taught. By the way, an ethics course is not a socialization course, at least as I teach it. What I try to do in part is to help students be able to decide what is right or wrong, good or bad. I cannot make them choose the right, though I believe, as Socrates did, that people with any ethical understanding and sensitivity at all will choose the right. By passing a student I am in part certifying that he has learned to be reasonable about determining what is right or wrong, good or bad. I am not certifying he/she will always be right or will choose the socially acceptable or fashionable course. Nor am I certifying that he/she is a good person.

I don't hold schools of education accountable for teaching all their students how to be good teachers; I do hold them accountable for graduating those that are not good teachers and that the schools ought to know are not good teachers.

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 08:28:35 EST From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: On the Nature of Teaching To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of 12/01/93 at 19:40:53 from DEMS042@UABDPO.BITNET

Rick--

Teaching depends a great deal on context, which makes it difficult for teacher education programs to assure that their graduates can "actually" teach. Moreover, the climate of schools often discourages thoughtful teaching, so some of the very best candidates (by which I mean the ones who understand kids, subjects, and how to bring the two together meaningfully) have the worst time demonstrating their ability to teach (as that role is defined by the real schools in which they do their student teaching).

I worked with a number of student teachers whose performance in one setting was exemplary and whose performance in another was poor. Since we can't put student teachers in all the sorts of settings that they might actually find themselves, we have to rely on a few placements. Our judgments about teaching potential are, thus, quite speculative.

--Aimee Howley

Date: Wed, 1 Dec 1993 22:15:24 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: On the Nature of Teaching To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Wed, 1 Dec 1993 14:53:04 -0500 from <CAMILLI@ZODIAC>

I believe that Greg Camilli and Tracy Lien overstate the kinds of under- standing I argue that teachers need. I think it does not take the training of a Shaulin (sp?) priest to ask students to explain why they give the answers they do on an assignment or test. When the whole class gives a wrong answer on an exam, I don't think it takes a psychologist/priest to think it might be helpful to ask some or all of the students WHY they gave that answer -- at least as a starting point. And I do not think it would be that difficult to teach teachers, for example, to ask students to explain how they are trying to work algebra problems they say they cannot solve.

Nor do I believe it is difficult to explain to ed students the typical kinds of mistakes students tend to make in various disciplines, and some kinds of things they might try, to correct those mistakes. I am not saying teaching is easy or that one can find methods that will teach anything to anyone. Nor am I saying teachers ought to be able to get into every students' "head" in order to find out what sorts of family/social/etc. problems he/she has that impede academic progress, let alone be able to solve all those problems. That WOULD take superhuman skills and knowledge.

I am saying that too many teachers I have seen tend to never think to ask students what they think or why, about a given academic topic, for there not to be some sort of general pedagogical flaw in this regard in a number of education programs. Those of you interested in educational research might want to try to design a way to ask the question of schools of education that Bill Hunter suggests -- in ways that don't simply get a perfunctory "Of course we teach that!" when perhaps they do not teach it at all but only assume they do.

I also have indirect evidence to support the claim that few teachers ask students what they think about things and why -- I do that quite a bit in my classes, and most students say I am the only teacher, or one of just a few teachers, they have had that asked them what they thought and why, and paid attention to their answers. I always find that surprising, though I should know better by now, and sad. When I talk to my department chairman about some of the topics we discuss in class, which I assume are reasonable to discuss in a course whose title throughout the entire state is "Ethics and Society", she always gives a startled laugh and says something like "Rick, you are the only person I know who would talk about that kind of thing in class." I really don't understand why it is so difficult to have honest discussion with students. And if you don't want to do it with issues of social controversy, why is it difficult to do it with algebra or English grammar, as in the examples I gave previously? "Why did you subtract X from both sides in this step?" See what the student says, and then pursue the response in some reasonable way. This does not take the kind of sensitivity training Greg and Tracy seem to infer. And, in fact, Bill Hunter thinks schools of education teach, and expect their students to do, this. I just don't think they do teach it, though I think they can and should.

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 09:00:54 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Wed, 1 Dec 1993 23:28:22 EST from <FACV@NMUMUS>

John Covaleskie said:
****
****There is much that I can teach that I hope will be helpful in shaping
****the attitudes and developing the skills that make a good teacher. But
****as to whether the individual is capable of creating and sustaining the
****relationships that are at the heart of good teaching, their early
****supervisors and colleagues will have a great deal to do with whether a
****"good teacher" results.
****

1) Not all teachers will get jobs with supervisors and colleagues that help them. If they are not better prepared, and rehearsed, to teach than you describe, you are sending them into their profession "rudely stamped", "unfinished, sent before \their| time/Into this breathing world, scarce half made up." \with apologies to Richard III|

2) Even if you are correct that this is the job, and the understood job, of schools of education, it promotes one more of those miasmas in society where "operations are a success but the patient died" because the linking of the links in a chain are nobody's responsibility. Everybody just provides the links, and if they get hooked together for some good result, great, but if they don't "it's not my responsibility". So, as caring and competent as John may be about teaching what he perceives to be his subject matter, this still provides an institutional cop-out for the educational "establishment" or of the educational training/development process. Especially since schools of education also train, certify, and supply the administrators who do not do a good job too.

3) I believe it should be easy to tell whether a person can teach to another a particular topic or subject in a way that helps the student learn the subject and appreciate having learned it or inspired to learn more about it (with the caveats I gave earlier about this not meaning that to teach well one has to be able to teach every subject to every person, etc.). I think it should not be as impossible as it seems to be for schools of education to weed out those people who cannot teach anything to others very well. I don't think this necessary aspect of good teaching is dependent upon waiting to see whether some one can "create and sustain the kinds of relationships that are at the heart of good teaching." I am not convinced those kinds of relationships are necessary for good teaching, though they may be necessary for someone to be a good teacher, colleague, social human being. The kinds of things Louis talks about and the kinds of things Greg and Tracey discussed are wonderful things, but not necessary in many cases for teaching particular material well. It appears to me from observations of teachers teaching in classrooms and in one-on-one teaching situations, all under fairly ideal types of situations with regard to socio-economic conditions, that there are many people with education degrees and many with teaching positions, who simply cannot teach material. They have no idea what the students know or don't know to begin with, or as they proceed. They have no idea how to make the material relevant or "alive" for students. They sometimes stifle a student's line of pursuit that may have become productive for learning, but which they don't know how to develop or let the student develop. Etc. This kind of thing should be observable somewhere in education school, and corrected or the student should be steered out of the profession.

4) Those with a natural gift for teaching (i.e., those who are the opposite of the above descriptions) ought to be able, it seems, to be taught whatever other aspects of the profession that teachers need to know in a shorter time period, since they have the, or one of the essential, ingredient(s) already. That is why some have asked for a shorter certification process for those with a degree in a content area and "good teaching instincts."

John, and others in education, does this just make no sense to you? Or does it seem totally wrong in some way? Or do you see a kernel of something good that I am just not saying quite right or that is missing some things about the whole educational training enterprise that I just am not seeing?

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 11:37:21 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: In reply to your message of THU 02 DEC 1993 10:00:54 EST

I think Rick's extraction of my comment left out a critical part: that when I was a principal, I indeed felt it WAS my responsibility to certify that X was a good teacher. Though a kangaroo Joey comes but half formed and ill-made into the world, it is not before its time. Your expectation that a school of education will produce a fully formed teacher is not, in my opinion, a reflection of the requirements of being a good teacher.

Two of which are wisdom and judgement, the products of both knowledge and experience. While we can certify the former, the latter takes time. No matter how I teach or what I teach, the new teacher is a new teacher, neither fully formed nor before his or her time.

And while it is true that we can identify those who are sure to be bad teachers (and not all those who enter teacher prep programs graduate out of them), that is not the same as being able to identify the ones who will be good, much less merely average.

Further, the culture of the school into which the new teacher goes determines a great deal. Both how the teacher develops and how the teacher is perceived are functions of variable environments. Good teachers in one school are considered to be bad in another context with different students, expectation, and demands. Likewise, teacher placed in one environment will grow and blossom, becoming a good, or even a great teacher. Placed in another environment, that same graduate will whither and spoil. (Enough metaphors).

I think, Rick, you underestimate the complexity concealed by the simple term, "Good teaching."

* * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET

Date: Thu, 2 Dec 1993 14:11:52 U From: Cotter_Cindy Subject: Re: On the Nature of Teaching X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

In the discussion of student-directed vs. teacher-directed learning, I argued the importance of learning with no teacher at all.

Eugene Bartoo said, " But bashing teachers in general is neither the answer, nor accurate."

I did not bash teachers. I LIKE teachers. I said I thought teacher-directed learning was necessary and important. It's a matter of emphasis. I think the emphasis is wrong in the system as structured, and that changing it will require tremendous work and imagination.

Steve Wright, in the middle of arguing that the emphasis should be placed on learning rather than teaching, said, "I'm not sure if I am making any sense and I have to go to a meeting. The bell just rang. I hate bells! "

See what I mean? John Gatto argues that schools don't teach what they think they're teaching. One of the things they DO teach is that no intellectual endeavor is more important than the bell schedule. Bell-centered learning, I guess.

Cindy Cotter

Date: Thu, 2 Dec 1993 17:03:37 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312021642.AA76059@acs1.acs.ucalgary.ca>; from "Covaleskie, John"

From John Covaleskie:
****
**** I think, Rick, you underestimate the complexity concealed by the simple
**** term, "Good teaching."

If so, you are not alone. Your own experience persuades you that YOU are a good teacher and so it is reasonable for you to think you understand the concept. Yet Schon has shown us that expert practitioners in any field often have only very rudimentary notions of what in fact constitutes their expertise--it is knowledge in doing or what Ryle called "knowing how" as opposed to "knowing that."

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Thu, 2 Dec 1993 17:44:46 U From: Cotter_Cindy Subject: Re: On the Nature of Teacher X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I worked as an aide in a junior high school for a few years when I was a student. Several teachers told me I ought to become a teacher. I did love the kids and I loved trying to find ways to present information that would work. I put in lots of extra unpaid hours, enjoyed them all, and learned a lot too. But I never was inclined to teach for real, and I've never regretted that decision.

For one thing, I didn't want to have to deal with all the non-teaching responsibilities, squelching food fights in the cafeteria, making sure Maria has a hall pass. And my heart wasn't in the prescribed curriculum. I often thought a kid might be better off flying a kite than studying some of the stuff in the math book. My job was to teach computer programming which at that age I still say is pure gravy. If a kid wants to learn it, fine, but if he doesn't, that's fine with me too.

And I had trouble with those bells, too. Once a boy was late for his next class because he and I were talking. The teacher laid all the blame on the boy -- he should have known better. I should have known better too, but the conversation seemed like a really valuable one.

And there were nearly 40 kids in this supposedly gifted class. I say supposedly because they dumped in extra kids when there was no room anywhere else. Then there was the new computer, unused, stored in the teacher's lounge. No one knew what to do with it. Why did you buy it, I asked the math chairman, when we're short of paper? Well, the computer came out of a special fund that couldn't be used for paper, and if we didn't spend the money on something this year, we wouldn't get it again next year.

So, suppose the teachers were right, and I could do a great job of teaching computer programming. I think I might have fit Rick Garlikov's definition of a good teacher -- that is, able to teach the material, more sensitive than the actual teacher to where the kids were at -- but I don't know that I'd last two weeks in a real school.

Cindy Cotter

Date: Thu, 2 Dec 1993 08:58:59 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: On the Nature of Teaching X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312020525.AA01394@tolstoy.u.washington.edu>

Rick Garlikov asks why we can't teach our teacher ed students to ask THEIR students why they answered a question a certain way, chose a specific strategy, etc.

I think we can (and many do) teach this--in fact in our program we have our students do a project where they observe two consecutive lessons, then interview two students (same ones after each lesson)--in part, asking them to use the knowledge they were supposed to have gained from the lesson, and to "think aloud" or explain their approach or understanding as they do it. This is teaching them to "do educational psychology" in the classroom, or at least one small example of it.

However, if one is to be able to really make sense of such information from one's students, one must know the discipline well. I can learn something from listening to a physics student think-aloud through a problem we both understand. But for me to capitalize on that explanation, know what's behind it (in terms of physics understandings) and where we might profitably go from there, I must have a deep understanding of the discipline of physics, as well as an understanding of the teaching-learning process. In our program, which requires an undergraduate major, we trust the bachelor's degree to teach this disciplinary understanding. But is this realistic? And what about the poor elementary certification students, who must know five or six different disciplines?

If any of you would like a really fascinating and informative look at how different levels and types of disciplinary understanding affect beginning teachers' planning, teaching, and assessing, I highly recommend Pam Grossman's book, "The Making of a Teacher." The seminal article we require our ed psych students to read is by J. J. Schwab "Education and the structure of the disciplines," in I. Westbury & N. J. Wilkof (Eds.) Science, curriculum, and liberal education (1978).

Susan B. Nolen 322 Miller Hall DQ-12 University of Washington

Date: Thu, 2 Dec 1993 16:09:29 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Thu, 2 Dec 1993 11:37:21 EST from <FACV@NMUMUS>

So what I wanted to ask John is: when you were a principal, did you ever have to fire or non-renew a teacher who simply could not teach, and who you felt could not learn to teach? Many? How do you explain their getting teaching degrees?

Were most of the teachers you hired as first year teachers as good as you thought reasonable for them to be, given four years of college and little teaching practice on their own?

Would you comment on the cases I described, and how you would analyze or treat situations where teachers prescribed practice where there was no understanding, and where teachers do not ask students why they give the answers, etc. they do? Is this rare in schools you have observed. It seems quite common where I live; and it is what I call not just "not good teaching" but really bad teaching. We can talk about supervisors later.

Rick

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 09:09:14 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: certifying teachers X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312021542.AA28239@tolstoy.u.washington.edu>

Rick stated:
****
**** 3) I believe it should be easy to tell whether a person can teach to another
**** a particular topic or subject in a way that helps the student learn the subject
**** be able to teach every subject to every person, etc.).

Rick, if you know some easy, valid, and reliable way to tell this, please pass it along to those of us who must make these decisions. Anyone who has evaluated teachers (preservice or practicing) knows that there is a difference between showing one can teach a particular topic to a certain kind of student and "being a good teacher," if that means one who on a regular basis with a variety of students and a normal range of topics can effectively plan, teach, and assess student learning. I'm not trying to say we shouldn't try; indeed we must try to do the best job we can in schools of education to send out students who are well-prepared to become good teachers. I'm merely saying it's NOT easy, and that teaching (and therefore the assessment of teaching) is much more complex than many give it credit for.

**** 4) Those with a natural gift for teaching (i.e., those who are the
**** opposite of the above descriptions) ought to be able, it seems, to be
**** taught whatever other aspects of the profession that teachers need to
**** know in a shorter time period, since they have the, or one of the essential,
**** ingredient(s) already. That is why some have asked for a shorter
**** certification process for those with a degree in a content area and
**** "good teaching instincts."

Great. I think you're absolutely right. Now explain to me how I can tell, upon entry to a teaching program, who has this "natural gift" and who doesn't. I'd just like to be able to tell, upon application, who will be harmful to students and who won't, so I can keep the harmful ones out of even student teaching.

Susan B. Nolen University of Washington.

Date: Thu, 2 Dec 1993 16:55:41 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: On the Nature of Teaching and Questioning To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312020537.AA87111@acs1.acs.ucalgary.ca>; from "Rick Garlikov" at

re: Rick's concern about asking why everyone in a class missed a particular question and the presence or absence of instruction toward that end in teacher preparation.

Darn it, Rick, you are forcing me to be specific. OK. Most teacher preparation programs have a course (or at least a part of a course) on educational testing. Such courses usually include, as one of the purposes of testing, the idea that tests are a means of finding out how effective instruction has been. As I tell my students: "If the class does poorly on a test, there are several possible reasons: the class is dumb the students didn't do the work (I point out that these conclusions tend to be very popular and that they may _occasionally_ be true, but that they represent lazy thinking on the part of the teacher and should be accepted reluctantly after other possible explanations have been considered and found wanting.) the test is not very good the instruction was not good enough"

Of course there are others, but the above are the more likely problems, along with the possibility that although the test was good and the instruction was superb, the test and instruction did not match one another (this may or may not be reason for saying that the test was bad--an extreme example would be that the teacher gave the wrong test).

To deal with understanding test results, *one* of the elements of an educational testing course usually is test analysis or item analysis which includes a variety of statistical techniques to determine which questions are not working well and why not. The "why not" part would ordinarily include the procedure "ask the students what they understood by the question and why they answered the way they did." It is partly because of this process that Jim Popham has been known to say "the best way to become a good teacher is to learn to develop good tests."

Now, that is _one_ course. There are often entire courses or units of other courses devoted to the topic "classroom questions" that attempt to get students to understand the difference between asking "Did you think question 4 was referring to the _reign_ of a ruler or the _reins_ on a horse?" vs. "What did you think question 4 was asking you about?"

Again, we can introduce these ideas, we can give tests to see that they understand, we can advise them when we see them in student teaching, but though item analysis has been part of teacher preparation for 40 or more years, you would be hard pressed to find many teachers who have ever used the technique in their own classes. This is partly because they often believe that it takes time away from "teaching," partly because it is cumbersome (software solves that problem, and largely for reasons I don't understand.

Please understand, Rick, I am not being defensive, I am trying to respond to your concern.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Thu, 2 Dec 1993 18:59:33 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Thu, 2 Dec 1993 17:03:37 MST from <hunter@ACS.UCALGARY.CA>

Bill Hunter said:
****From John Covaleskie:
****
**** I think, Rick, you underestimate the complexity concealed by the simple
**** term, "Good teaching."
****
****If so, you are not alone. Your own experience persuades you that
****YOU are a good teacher and so it is reasonable for you to think
****you understand the concept.

Actually, I think I understand the concept better than I can teach. I think the concept is not all that complex, though I think what it takes to be a good teacher is complex. I can teach some things well; some things not well; some things not at all. I could not be a good elementary school teacher although I sometimes teach certain topics as a visiting resource person in elementary schools and seem to have good success teaching those topics.

Yet Schon has shown us that expert
****practitioners in any field often have only very rudimentary
****notions of what in fact constitutes their expertise--it is
****knowledge in doing or what Ryle called "knowing how" as opposed
****to "knowing that."

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 22:04:56 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: On the Nature of Teaching X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312020229.aa13729@ssi.edc.org>

I agree with Rick Garlikov on the relative simplicity of a questioning attitude, and the need for teachers to have the habit of asking their students what they think and why. There is evidence that implies that neither education schools nor the K-12 systems do much of that sort of thing:

The rise of the "thinking curriculum" in policy debate. The organization of curriculum around knowledge, not questions. The poor abilities of students to apply or transfer what they know. The segregation of low achieving students into less rigorous tracks. The assumptions of curriculum that higher order thinking must be "put off" until one has mastered "some basics."

I like Rick's research challenge.

On Wed, 1 Dec 1993, Rick Garlikov wrote:

**** Those of you interested in educational research might
**** want to try to design a way to ask the question of schools of education that
**** Bill Hunter suggests -- in ways that don't simply get a perfunctory "Of
**** course we teach that!" when perhaps they do not teach it at all but only
**** assume they do.

What kinds of evidence would tell us that students are being asked about their thinking and assumptions, their propositions and elaborations on ideas? What evidence would there be to follow such probing with new, unplanned adjustments in teaching to deepen and extend the learner's thinking? Does existence of this evidence in ed schools predict higher instances of the same practices in K-12 classrooms?

Date: Thu, 2 Dec 1993 20:45:28 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Thu, 2 Dec 1993 09:09:14 -0800 from <sunolen@U.WASHINGTON.EDU>

Susan Nolen said:
****Rick stated:
****
**** 3) I believe it should be easy to tell whether a person can teach to another
**** a particular topic or subject in a way that helps the student learn the
**** subject
**** be able to teach every subject to every person, etc.).
****
****Rick, if you know some easy, valid, and reliable way to tell this, please
****pass it along to those of us who must make these decisions. Anyone who
****has evaluated teachers (preservice or practicing) knows that there is a
****difference between showing one can teach a particular topic to a certain
****kind of student and "being a good teacher," if that means one who on a
****regular basis with a variety of students and a normal range of topics can
****effectively plan, teach, and assess student learning. I'm not trying to
****say we shouldn't try; indeed we must try to do the best job we can in
****schools of education to send out students who are well-prepared to become
****good teachers. I'm merely saying it's NOT easy, and that teaching (and
****therefore the assessment of teaching) is much more complex than many give
****it credit for.

Sue, your previous post pointed out a particular thing I left out, in order for my claim to be true: the person doing the judging has to know enough about the topic (usually) in order to be able to ferret out whether the learner has learned from the student teacher. I am assuming, however, that for the most part that is true. It is not necessary in those cases where the learner clearly says he/she does not understand what the student teacher is getting at; it is necessary where the learner thinks he/she understands and the judge has to determine whether the learner really does understand. I think it may be difficult or impossible in some cases for a judge who does not understand the topic to judge whether it was taught or not. And that does happen in some cases -- judges get fooled. (I have just done a theoretical review of the literature on teaching numerical "place-value", and I argue there are a number of cases where the researcher does not properly understand the concept of place-value and therefore is incorrectly judging the effective- ness of teaching it.) The cases I was thinking of when I made the claim, were cases where the judge understood the topic and could fairly easily "pre-test" and "post-test", or could diagnose student learning at the same time the student teacher did, by observing learner's answers or responses to questions and comments.

Moreover, I have only been talking about teaching a topic under reasonable conditions. Reasonable conditions are not always existent. Both you and Cindy Cotter point out some of the things in schools that make teaching almost impossible for even the otherwise best of teachers. Having to teach topics you don't know well; having to teach with interruptions by intercoms and bells; having to write out lesson plans; monitor lunch; deal with parents and unhelpful administrators; not having sufficient materials; etc., etc., etc. But these are all kinds of things that need to be changed wherever possible, not things that good teachers can simply work around. Nobody can work around many of those kinds of things; and I am not faulting people who cannot teach well under those sorts of conditions; nor do I have a cure that does not involve first removing those conditions.

**** > 4) Those with a natural gift for teaching (i.e.,
****those who are the
**** opposite of the above descriptions) ought to be able, it seems, to be
**** taught whatever other aspects of the profession that teachers need to
**** know in a shorter time period, since they have the, or one of the essential,
**** ingredient(s) already. That is why some have asked for a shorter
**** certification process for those with a degree in a content area and
**** "good teaching instincts."
****
****Great. I think you're absolutely right. Now explain to me how I can
****tell, upon entry to a teaching program, who has this "natural gift" and who
****doesn't. I'd just like to be able to tell, upon application, who will be
****harmful to students and who won't, so I can keep the harmful ones out of
****even student teaching.
**** Since I think people can learn how to teach, I do not mean for those without natural ability to be kept away from student teaching. I want them kept away from student teaching if they have not learned how to teach (or had natural ability to teach) by the time they are supposed to be ready to student teach. That being said, at whatever point you want to test YOUR students: 1) Have them teach you something you do not know -- maybe a card game or a board game or some such; or a physics or math principle, if that is their field. See how well they teach it to you. Or 2) have them teach another adult. See how well they teach a math principle to someone whose field is English, or vice verse. Or 3) test them out on a small group of students in a serene environment where the students realize their possible failure to learn in this case is no reflection on them, but that you are testing to see how well this person can teach them something.

Remember, all this is designed to do is to see whether somebody is egregiously bad enough to keep out of a classroom type situation at the present time, or is good enough to skip ahead faster into a classroom situation once they have learned whatever other things are necessary to know about being in a classroom. Success at this is not a guarantee of success in the classroom; but failure at this would, I would think, be virtually a guarantee of failure in the classroom. (I am talking about severe failure here, not just an approach that fails but could be made to succeed with a chance to reflect on the failure and modify the approach. Even REALLY GOOD teachers sometimes come up against a brick wall and need to reflect and revamp in order to achieve success.) If a student of yours cannot teach you or some other adult, or some small "control" group of children something he/she is assigned to teach -- and demonstrates real inability to teach -- I would think you know they are definitely not ready for the classroom yet. Is this a feasible way of looking at this? Is it pragmatically feasible given your workload, etc. Remember, the point is simply to discover those people who really are not ready (yet) to teach, before inflicting them on a bunch of kids and the classroom teacher who is responsible for them. I think there are many people who simply cannot teach (perhaps anything) very well. Those are the ones we need to keep away from kids whom they would intellectually harm. And, at least in Alabama, many of those people are not kept away from classrooms. If you find a number of these kinds of student teachers once you have turned them loose in a classroom, couldn't you just as easily, more quickly, and less detrimentally find them beforehand using something like the ways I just described? Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 22:47:12 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: On the Nature of Teacher X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312022139.aa12301@ssi.edc.org>

Cindy's comment about being a good teacher who could last two weeks in a real school reminds me of an earlier conversation we had about whether schools were really about learning or not. If we were to scrap the current model altogether and start from fundamental principles, would it be realistic to ask of almost all policy questions and issues whether they promoted learning in the most effective and pragmatic way possible?

23:08:48 -0500 Date: Thu, 2 Dec 1993 21:47:38 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: On the Nature of Teaching and Questioning To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Thu, 2 Dec 1993 16:55:41 MST from <hunter@ACS.UCALGARY.CA>

re: Bill Hunter's explanation about teaching education students about class questions and testing items.

I did not find that defensive. That is very informative and interesting. And I would love to know if anyone has some sense of why, as Bill says, we "would be hard-pressed to find teachers who use the techniques" he explained so well. If someone could fill in the "largely for reasons I don't understand" that Bill mentioned. I suspect one of his reasons is a primary culprit -- asking questions takes time away from lecturing or "telling"; and I suspect it is a natural tendency to think telling it once or twice will be faster than asking students about things. When I tried to get some teachers to do things that would make issues more "live" for students, so that "answers" would be meaningful to them the first time, they balked at the suggestion EVEN THOUGH they admitted that just lecturing the way they usually did meant that they had to lecture two or three times and that much of the class still would not get it. Yet, it seemed to them that trying to do something to indirectly make lecture unnecessary or more effective was somehow not what teaching was about, and would take time away from teaching. Maybe a better way needs to be figured out to teach the ideas Bill described. Interesting challenge. Thanks, Bill. By the way, I assume some students learn and use these techniques though, at least in some (perhaps rough) form or other, yes? Would liberal arts grads be more likely to do this? Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 2 Dec 1993 23:09:40 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: On the Nature of Teaching X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312022147.aa12595@ssi.edc.org>

Susan Nolan asks if we can expect undergraduate school to give teachers a deep enough understanding in a field of knowledge to enable them to capitalize on questions which probe learners' knowledge. I agree with those who say a maturing in the real world is a part of becoming a teacher, and at the same time, I share Rick's worry about the potential for bad teaching to then become "someone else's problem."

But apart from both of those propositions, I think even very experienced and quite mature teachers need help from their peers when it comes to seeing the potential inherent in "next teaching steps" based on questions and probes about student thinking. For me, this is the strongest argument in favor of getting more than one teacher in a room with a group of learners.

Date: Thu, 2 Dec 1993 23:26:33 -0500 From: Josue Gonzalez <jg124@COLUMBIA.EDU> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <199312030022.AA22995@mailhub.cc.columbia.edu>

On Thu, 2 Dec 1993, Bill Hunter wrote

**** YOU are a good teacher and so it is reasonable for you to think
**** you understand the concept. Yet Schon has shown us that expert
**** practitioners in any field often have only very rudimentary
**** notions of what in fact constitutes their expertise--it is
**** knowledge in doing or what Ryle called "knowing how" as opposed
**** to "knowing that."
**** -- Yes! It's that special feeling you get when you know you've connected well with another human brain and that brain will, henceforth, go in a slightly different direction from where it was going before, at least until the bell rings.

Schon also says, as I recall, that one of the trademarks of the reflective practitioner is that he/she knows much more than he/she ever needs to use to solve a problem. One of my students -- a former EE major -- says it's sort of like the wattage on your stereo: you need to have a lot more watts of power than you will ever use. The windows would break and the neighbors would scream if you ever turned it full tilt, but the presence of those extra watts makes the lower volume sound great! Huh? * . . . . . . . . . . . . . . . . . . . . . . . . . * . Josue M. Gonzalez . Teachers College . . 212-678-3746 - voice . Columbia University . . josue@panix.com . New York, NY 10027 . * . . . . . . . . . . . . . . . . . . . . . . . . . . *

Date: Thu, 2 Dec 1993 23:27:48 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: On the Nature of Teaching and Questioning X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312022155.aa12717@ssi.edc.org>

On Thu, 2 Dec 1993, Bill Hunter wrote: "If the class does poorly on a test, there are
**** several possible reasons:
**** the class is dumb

I am truly surprised that this attitude still exists. What are the assumptions about learning involved in this piece of evidence?

**** the students didn't do the work
**** (I point out that these conclusions tend to be very
**** popular and that they may _occasionally_ be true, but
**** that they represent lazy thinking on the part of the
**** teacher and should be accepted reluctantly after other
**** possible explanations have been considered and found
**** wanting.)
**** the test is not very good

I bet this is true 90% of the time when measured in terms of old paradigms of teachers filling the heads of students. And I bet its true 95% of the time or more when measured by criteria of authenticity.

**** the instruction was not good enough"

...not good enough for this kind of testing.

I would start the list differently: What about the goals of embedded assessment in letting the student make a judgment for him or herself? Or the possibility that the teacher needs to learn something about the profile of the class? What about being able to see how far one (both the learner and their teacher) needs to go to meet high standards?

The course you describe on testing sounds like it might be of interest to someone who would like to work for the College Board someday. Or perhaps it would be of vital interest to policy makers attempting to analyze and reform large scale testing and assessment practices. It does not surprise me in the least that very little of it gets into a classroom, and I'd recommend reconsidering it for all but highly specialized students focusing on evaluation methods.

David Gibson dgibson@ssi.edc.org

Date: Fri, 3 Dec 1993 07:48:30 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: In reply to your message of THU 02 DEC 1993 17:09:29 EST

Rick:

Fair enough questions. I will do my best to answer them, but the issues are more complex I think. (See Sue Nolen's post).

Yes, I had to deny tenure to new teachers who I did not feel were going to ever be what I consider to be good teachers (note the qualifier here: "What I consider to be good teachers." I am quite certain that at least one of those new teachers would have met the expectations of many other principals, who would define REALLY GOOD teachers differently from me.)

But yes, most rookies with whom I dealt were as good as I could have expected a new teacher to be, nor am I alone in this. In fact, many of my colleagues would much prefer to hire new teachers than more experienced teachers, quite apart from the salary savings. (Other present or former administrators on this list might want to comment on this???)

And this last point begins to illumine part of what I think your questions miss, Rick. Your questions presume that teachers are made in schools of education, and I submit that teachers are made in their schools of hire, where they actually practice their craft. What schools of education do is identify those prospective who we believe have the knowledge and skills to become good teachers, or at least not to be really bad teachers -- and remember, despite the implications of your questioning, we DO refuse to graduate some candidates.

There is a great deal of research that suggests that the social forces of a school community to a great deal to shape the professional practice of new teachers. More than schools of education do. In many ways, I was more influential on any individual career as a principal than I am as a professor. And this is why I and many of my former colleagues, would prefer to hire brand new teachers -- we get to make them what we define as good teachers.

Sue pointed out that her students do an exercise in follow-up of the sort you define -- rightly -- as good teaching. She can now certify that those students CAN do such things. But can she certify that they WILL? Not likely. And whether they will is, according to research (and common sense, it also seems to me) largely determined by whether such actions are valued by that person's new peers after graduation. Assume that the disciplinary knowledge exists (which we do NOT certify) and assume that the initial sense of good teaching exists, the climate in which the practitioner begins practice is terribly important to the development of good habits -- and that is one of the things that you ignore. You mention Louis's practice as an example -- and yet look at the pressures to be otherwise that Louis keeps telling us about. Not all brand new teachers have the self-assurance that Louis has.

Nor do they necessarily have the confidence in their professors to withstand the barrage of stories from their more experienced peers when they start being told, "Yes, I know what you learned in school, kid, but now you're in the real world." New teachers rapidly become normed to the culture of their schools, and new teachers rapidly usually become just like old teachers, and that is often in quite clear contradiction to what they are taught in their teacher prep program.

We cannot certify that will not happen.

But let me return you to Tom Green's question, which I think you missed the force of. Living ethically is a practice. You, as an ethics prof, can reasonably be expected to certify that the student who passed your course have the knowledge and skills to do moral reasoning, and that they may be presumed then to know how to behave ethically. But you cannot be expected to certify that they will do so. That it is the way it is with any practice. Though you might comment that you are a failure when a student of your gets caught hacking, no one who thinks seriously about the nature of ethics would suggest that you deserve blame for bad teaching because one of your students does anything wrong; nor would you get the credit for having been Mother Theresa's ethics prof.

likewise, if we pay attention, we can certify that our students have certain skills that can make them potentially good teachers in at least some circumstances (and please try to remember that "good teaching" is not ONE thing -- some chalk and talk teachers, though not to my taste, are very good and effective, and exactly what the principal wants. They would not make tenure in another school). We cannot guarantee that they will be inclined in fact to apply those skills and that knowledge in their real situations. Nor can we certify that they will not unlearn what we have taught and replace it with the teachings of their colleagues in practice.

And I think most principals realize that, however much we might grumble about schools of education. As I said, most of my former colleagues liked hiring new teachers just because they could be shaped.

* * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET

Date: Fri, 3 Dec 1993 09:30:31 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: certifying teachers X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312030346.AA13744@tolstoy.u.washington.edu>

Rick--

I agree with your remarks about the judges needing to know the subjects the student teachers are teaching, though this is often not the case in my current institution. The worst story I heard was when one of my students, teaching French using immersion techniques, was observed by the "foreign language" supervisor--who of course spoke no French. But this person was able to "evaluate" my student based on the behavioral checklist used for the purpose. This list focuses on "teaching behaviors" and "student behaviors," such as making sure the teacher has the students' attention. There is no place on the evaluation form to judge how well the student teacher grasped the topic, nor how well the students learned. (We're actually working on better ways to evaluate student teachers, but this one's been in place for nearly 20 years.)

However, what I was referring to in my earlier note was the necessity for the student teachers themselves to have disciplinary knowledge.

On testing students in the way you suggest, that's what the above-described checklist procedure is supposed to do. Our students go out in the field right away (actually, they have to have 60 hours of experience with children prior to admittance, but this can be volunteer work or observation only.) They have field experiences all the way through their teacher ed program, as is the case with probably the majority of programs at the moment. This reflects the fact that we can't teach them how to teach (even at a rudimentary level) without letting them practice on live kids.

So although what you describe is necessary, it's not sufficient either to screen out the really bad ones before or even after they get into our program. In fact, where we often really start to worry about the likelihood that new teachers will actually cause harm to students is when they are in our assessment course. Here they have to design an assessment plan for a unit of instruction, and design a grading policy for the grading period in which that unit fits. This process brings a lot of scary beliefs out in the open (e.g., the threat of poor grades will motivate poor students to try harder.) And I think we need some way to defensibly (legally I mean) weed out student teachers on their potential to cause harm through what I would consider unethical practices, many of which are common throughout the school system (e.g., grading on a curve, marking down for late papers, grading on attendance, use of invalid assessments, etc.)

Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195

Date: Fri, 3 Dec 1993 10:56:46 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312030315.AA27340@acs1.acs.ucalgary.ca>; from "Rick Garlikov" at

Rick's analysis of good teaching seems to suffer from too much attention to the extremes--yes, most of us can probably agree that if all 200 students walk out of a room invigorated by the ideas they heard and able to use those ideas constructively that something called good teaching took place in there. Likewise, if all 200 walk out grousing, the odds that good teaching occurred are low (but I would argue that it is not certain that they were exposed to poor teaching--sometimes innovative approaches to teaching that place heavy responsibility on the students' shoulders are INITIALLY perceived as bad by students who later understand better how much they learned--Rick, you may well appreciate this point since you clearly learned early to take responsibility for your own learning). But what about the far more abundant cases in between. What about the instructor who gives a brilliantly entertaining talk (day after day) and has 98% of the students charmed, but is in fact giving them outdated information, wrong information and no reason or opportunity to apply what they learn? Do we believe the 4 out of 200 who happen to see through this? Chances are they dropped out if they could. What about the teacher who struggles mightily to prepare good learning materials, to insure that students are exposed to quality information, to give them experiences that encourage them to work with that material constructively (and maybe even asks them why they answered incorrectly) but who is perceived as boring and monotonic in class and too demanding with regard to marks?

Rick's personal comments may shed some light on his reasoning about this matter. You indicated that you work hard to learn the things you know (and do) and that a consequence of this hard work is an understanding of the kinds of difficulties others may encounter in learning those things. Some of the teachers you have encountered may not have had this experience. For them, the things they teach "come naturally" and the fact that others find them difficult is a puzzle. This does not relieve them of the responsibility for finding out why others (their students) encounter difficulty and how to enable them to overcome those difficulties, but it might contribute to their slowness in questioning in the way you advocate. Please note that what my previous post was suggesting is that perhaps teaching itself has come rather easily to you (and I think this is an implication of what you said--you work hard to learn something, but then you "know" how to teach it). Thus, you have difficulty understanding how it can be that teaching is not so simple and self-evident to others. Does this make sense to you?

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Fri, 3 Dec 1993 11:07:52 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312030356.AA43828@acs1.acs.ucalgary.ca>; from "Rick Garlikov" at

Sorry to keep adding on here. Rick's response to Sue Nolen suggested having students attempt teaching something to see if they are any good. This too is part of most teacher prep programs. We usually take it a bit further, having them teach to a small group of peers and videotaping it for later analysis and feedback--this is called micro-teaching and it has a fairly long and honorable history in teacher education. Very often, it is not necessary to tell people they cannot teach--this experience tells them--at the University of Calgary, we have a very high proportion of students who do not pursue teaching degrees after having tested themselves in this way.

Still, I feel a need to point to a problem in Rick's suggestion; one that we deal with fairly often. There are lots of excellent teachers, especially in the early grades, who are absolutely wonderful in working with children but who become completely and totally inept when asked to work with peers (e.g., to teach a professional development workshop or to conduct a seminar in a graduate course). Some of them manage to overcome this problem with experience (for example, to get through the teacher education program), but if we were to completely rule out people because they had difficulty teaching adults, we might eliminate some outstanding teachers.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca
Date: Fri, 3 Dec 93 11:20:37 MST From: Bill Hunter <hunter@acs.ucalgary.ca> X-Mailer: ELM [version 2.3 PL11v]

Right. For all intents and purposes, ALL of our graduates (at U. of C.) learn HOW to use those techniques, but few actually do. Even those of us who teach them tend to use them only when there is an evident need to (e.g., teaching a course for the first time, suspecting that the material is not getting across).

Roughly half of our teacher ed students are people who already possess liberal arts degrees. I do not think that they are noticeably better at this stuff, but we have not researched the question. They might be a little better, but they are also, on average, quite a bit more mature, which I would assume to be a more important variable. Those liberal arts grads who are planning to be secondary teachers, however, would seem to me to be slightly WORSE (on average) in this regard (I was a liberal arts grad and a secondary teacher myself) in that they tend to be more interested in there subject matter than in pedagogy--their successful experience as learners of history, chemistry or whatever has often persuaded them that all they need do is tell people. While they may manage to do better than that for the purpose of succeeding in the program, I think many leave unconvinced and rapidly return to telling when they find that to be rewarded and rewarding in many schools (One of the things I frequently here from students is "why don't you (or doesn't Prof. So-and-so) just TELL us what we need to know.)

And while I have your undivided attention, thanks for raising a non-fiduciary question that is generating a lot of interest (at least for me).

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Fri, 3 Dec 1993 11:08:57 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Student Assessment X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

David Gibson wrote: "The course you describe on testing sounds like it might be of interest to someone who would like to work for the College Board someday. Or perhaps it would be of vital interest to policy makers attempting to analyze and reform large scale testing and assessment practices. It does not surprise me in the least that very little of it gets into a classroom, and I'd recommend reconsidering it for all but highly specialized students focusing on evaluation methods."

Susan Nolen wrote: "And I think we need some way to defensibly (legally I mean) weed out student teachers on their potential to cause harm through what I would consider unethical practices, many of which are common throughout the school system (e.g., grading on a curve, marking down for late papers, grading on attendance, use of invalid assessments, etc.)"

Perhaps the two of you could discuss what appear to be widely divergent views about the importance of assessment. I think assessment is essential to guide teaching and that it's often done very badly. I was quite excited by hearing Susan list those unethical practices.

The issue of grading on curve is an interesting one (to me). Presumably Susan is offended that a student who has not mastered the material will do well if everyone else understands it even less, or that a student who has completely absorbed all the major concepts may receive a low grade if his peers aced him on the details.

Assuming a valid test, how SHOULD it be scored? I've been taught by teachers who hate curves. 90% and above is an A. 80% and above is a B, etc. But what is the basis of this method? Has someone determined that any student who understands the core of the material well enough to proceed with new material will answer 70% of the questions correctly? How did they determine that? I can imagine a test where 70% of the items tested the core material, another 10% were a bit more sophisticated, and the last 10% even more sophisticated. That's not an easy test to construct.

We take up that assumption of validity later.

Cindy Cotter

Date: Fri, 3 Dec 1993 13:41:54 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: Student Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312031924.AA10168@tolstoy.u.washington.edu>

Cindy Cotter wrote:
****
**** Perhaps the two of you could discuss what appear to be widely divergent views
**** about the importance of assessment. I think assessment is essential to guide
**** teaching and that it's often done very badly.

I agree.
****
****
**** Assuming a valid test, how SHOULD it be scored? I've been taught by teachers
**** who hate curves. 90% and above is an A. 80% and above is a B, etc. But what
**** is the basis of this method? Has someone determined that any student who
**** understands the core of the material well enough to proceed with new material
**** will answer 70% of the questions correctly? How did they determine that? I
**** can imagine a test where 70% of the items tested the core material, another 10%
**** were a bit more sophisticated, and the last 10% even more sophisticated.
**** That's not an easy test to construct.
****
**** We take up that assumption of validity later.

Actually, I think the scoring and use of the assessment are part of validity. I use scoring rubrics a lot in my teaching, and I try to think about what I want students to understand and be able to do, then try to set the rubric (and scores) to correspond to different levels of competence. For example, with our assessment portfolio rubric, we decided that "just passing" marks on all parts of the portfolio stood for accepting uniformly mediocre performance. So we set a passing grade somewhat above that--they'd have to get full credit for at least some parts of the portfolio to pass. We also weight the different aspects of the portfolio by their importance to our course goals (which reflect our views of what a competent beginning teacher should know about doing ethical assessment).

The 90%, 80% etc algorithm doesn't have much meaning unless you have determined that these levels of performance on the specific assessment represent clearly and meaningfully different levels of skill, expertise, or knowledge. I remember asking back in the 70s why we used 75% competency in our "criterion referenced" tests, and was told that it was just an arbitrary decision that worked well enough most of the time.

We require our students to do what we would want them to do in the classroom: design and use assessments that allow them to judge how well their students have learned important "learnings," and justify their assessment decisions (planning, construction, scoring, and use for grading and further planning), including how their assessments fit with their overall goals and objectives and how they represent the discipline (math, science, history, etc) to their students.

Item analysis and other forms of critiquing one's assessment tools are part of the design process. I think in more traditional courses the amount of time spent on various aspects of educational assessment don't match what teachers need to do very well. For those who are interested in this topic, Bob Linn wrote an article in Teachers College Record a few years ago called something like "what we should teach in ed. assessment courses for preservice teachers." I based my course on his recommendations.

Sorry to be so long-winded.

Sue Nolen U. of Washington

Date: Fri, 3 Dec 1993 18:08:00 EST From: Leslie Wade <LWADE@NAS.BITNET> Subject: odd thoughts on nature of teaching... X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Cindy said a couple of things that helped me clarify some thoughts that had been wandering around my mind these last two days--thanks, Cindy...

One has to do with someone else's comments about the bell and how we're teaching something different than we think we are. I've been monstrously frustrated in a class this semester because the eminently regarded professor has been more concerned with the process of the class than with the content. Now usually I'm big on process, but in this case it has meant that he is more interested in making sure each and every one of us comments on the current discussion, whether or not we have any particular opinion to express. This results in 17 members of class repeating in their own words the opinions of 6 students. If we don't do so voluntarily, we are called upon specifically to add our thoughts (or someone else's).

The other manifestation is his insistence that we write about his choice of topics in a strict number of pages. He says we need to learn to express ourselves concisely. I'm all for good writing, but that is not the purpose of this class. I don't disallow the possibility that the class could have more than one purpose (improved writing AND comparative educ theory), but insisting we formulate a personal philosophy of education in 3-5 double-spaced pages seems at cross purposes--at least if he wants any real depth of thought. And the improvement of writing skills is not listed as an objective in either the syllabus or the course description.

In this case, the proverbial bell has been welcome but I resent that it's cost me 1500 and that I've been brought to the point that I actually LIKE that bell. I'm not sure what the point of this was--probably that I'm appalled that this should happen in a department of Educational Leadership. On an unrelated subject... Dec 3 Washington Post had a short article about a local school district reevaluating how they determine gifted/talented status. It wasn't the point of the article, but it mentioned that 84% of the student population had shown enough promise to be formally tested for IQ (I think). Why is it that there is such pressure to be gifted/talented? Somehow we've made being ordinarily good a BAD thing. And if three-quarters of any school is gifted/talented and the bell curve takes care of part of the rest and calls them in need of remediation, that leaves the "ordinary" students in a very small minority. Am I the only one who thinks this is a little strange? Date: Fri, 3 Dec 1993 20:45:05 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: teaching and teacher ed programs X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> I very much appreciate in particular Sue Nolen, John Covaleskie, and Bill Hunter's taking the time, effort, and patience to answer me and argue with me when necessary. I have felt that what I was arguing was important, but also that it was presumptuous in some way for me to keep on. I am especially glad that Bill Hunter said he has really enjoyed this; because a number of times I almost quit because I felt like I was being a nuisance in persisting. Bill Hunter's point that teaching comes naturally to me because learning does not come easily, and that my learning methods make teaching easier for me than for many other people, was an eye-opening statement. I had not thought about it in that way. And I am guilty often of thinking that things which seem easy to me should be easy and obvious to others. It is the things that are/were difficult for me that I tend to expect others to have difficulty with and that I try to use patience and understanding in order to help them overcome those difficulties. I also forget that people don't tend to see bad examples as bad examples. I would say I learned as much, if not more, about teaching from bad teachers (like Leslie Wade described, and worse) as I did from good teachers -- swearing never to teach like them. That is why the statement always seems strange and incredible to me that "people tend to teach like they were taught", even when that was bad. I may just have to learn to accept that it is true though regrettable. Bill Hunter and Sue Nolen have reminded me that for many people teaching just is not that easy an enterprise; and teaching teachers to overcome the difficulties is not that easy an enterprise. Intellectually I may have to accept that, though "intuitively" it seems false, and emotionally it seems not "the way the world should be". (If wishes were fishes.....) Sue and Bill also helped me to see that (1) they and other educators recognize some of the problems I have mentioned, (2) believe some of the things I believe about teaching ed students, and (3) have been, or are doing things to try to improve ed programs in ways I was not aware occurred from the things I had read and from the people I get to talk with locally. I am glad to see those kinds of things recognized as important. The story about the French teacher was a great story, but one which most of the people I talk with in education probably would have seen nothing wrong with. I raised some of the issues I did in hopes that what I see locally is not universal or the norm. At least now I see it is not universal; that is heartening. The Internet at least has given me a wider perspective of things going on in education. Local environments can distort that if that is all you see. (One of the horror stories I had related previously but left out here, was the case of the urban school district that divides the pages in the math book by the days in the school year and teaches those number of pages (or "mixed fractions" of pages) per day, come hell or high water. When you see stuff like that, you wonder if that is something they are trained to do, and whether everybody is trained to do that. At least now I know that such teaching methods are not universally espoused in schools of education. As Bill says, this is one of my extreme examples, however, and I am only being hyperbolic about worrying about ed schools teaching this poorly. But there are some apparently fashionable education practices that I do worry about, that we can discuss at some later time.) I also had not thought about Sue's point that someone might not be able to teach adults yet who could teach kids quite well. That was interesting. I tend to think "teaching is teaching"; kids, adults, teens, no difference other than the kinds of information, examples, experiences you can meaningfully use. Some of my best philosophy students were fifth and sixth graders, and I taught them the same way I did adults. It never occurred to me that some people might not be able to teach the same way around adults as children. All of this brings me to John Covaleski's latest post, about which I am still troubled. I understand and am familiar with the points he makes that the school environment in which one teaches has tremendous power to shape teachers' practices, moreso than ed school. I have seen that happen in schools locally; and I have even seen how one "model school" summer practicum program for established teachers painstakingly wins them over during the summer to better practices, only to watch them revert to their old ways when they get back among colleagues that don't respect those practices. I simply draw different conclusions from John about that fact. First, since one of the issues at the beginning of this discussion was whether there could not be truncated alternative teacher certification programs for people with degrees outside education, it seems this might be evidence for that being reasonable, since the "real" teacher education would come on the job anyway. John may have understated more than he meant to the role of the normal teacher ed program. Although Bill and Sue recognize their efforts are not always successful, still they seem to think that their efforts can and do make a difference in how their students will teach, or try to teach. I don't think Bill and Sue see their roles as just trying to get their students up to the threshold of learning how to teach. The conclusion I would reach from the kinds of evidence that John gives, and that Sue and Bill add to, is that ed programs need to be bolstered in such a way as to help prevent on-the-job bad influences from swaying new teachers; and that ed schools should take a more aggressive approach at informing the public about the harms of at least some of the most egregious practices they know of in schools. I would still maintain that if ed school faculty understand that their efforts in current programs will be "at risk" in the the "real world" of actual schools, they need to try to figure out ways to minimize that risk. Teaching ed students case studies of the occurrence might be one such approach, if to be forewarned is to be forearmed. A mandatory seminar after a teacher's first or second year of "real" work, dealing with the problems of putting "theory" into practice, and "reminding" them of the importance of what they have been taught, with suggestions that might be workable and meaningful, might be another approach. A longer program to begin with might be necessary. (Relating it to my experience, I did not really get the kind of feel for understanding philosophy until somewhere in my first year of grad school. I am told by one older engineer that most engineering grads are worthless to an engineering firm till after some number of years with them of on-the-job experience. Perhaps four years of undergrad work in a number of areas is just not sufficient preparation.) To use the example John gave about teaching ethics, that Tom Green originated, IF what I had to do would be to teach people to BE ethical, not just understand moral reasoning, I would teach more than what I teach in my ethics course. There would be other courses, ones that involve case studies, for example, where people really fouled up by being unethical. There would perhaps be out of class entrapment type situations to show students how easily they can be pressured or persuaded to do the wrong thing; and how they can get caught when they "go along". I don't know what all would be involved, but I know that if I were supposed to instill ethical behavior in my students I would want to do all I could to do it in such a way that it "held" not just in the artificial environment of the classroom, but in as many realistic sorts of situations as I could figure out -- perhaps the university version of army war games, or some such. I find it hard to accept John's view that seems to say the ed school's role is just to teach theory and skill, etc., but not to teach it in such a way as to reasonably withstand those bad influences which he knows to exist. Of course, good influences also exist "out there" and you want students to be open to those, but you can perhaps do that and still not leave them as susceptible to "bad influences". Finally, some of this comes back to Tom Green's points about educational policy's not really being made by anyone or by a forum such as this, or school board, or state department of education, etc., and to someone's point that the articulation of ideas is not sufficient to bring about change. Although the articulation of ideas, especially the first time, is usually not SUFFICIENT to bring about desirable change, I believe the articulation of ideas is NECESSARY in many cases to bring about change; and that it is important in most cases. Further, I believe that the frequent articulation of important ideas in as many psychologically persuasive ways as one can muster does have an effect on practice, at least over time. I believe, as the economist Keynes did, that ideas effect practices far more than do the actions of leaders or of governments. But those ideas have to be articulated, and they have to be repeated, made public, and expressed in as many different and persuasive ways as possible. At some point they will sift into people's consciousness and into practice. I think that ed schools have a responsibility to articulate and publicize ideas about good, and about egregious, teaching practices in order to help change bad policies and bring about better policies, so that what they teach their students is not "undone" in the first year or two or three of their students' professional careers. It may be that no one forum or person can actually make policy (as opposed to just STATING a policy), but I think the expression of ideas tends to help make policy, and can be instrumental in making policy, if done well. And insofar as EDPOLYAN provides a forum for such expressions, and insofar as school officials, and ed school leaders have various sorts of public and private forums to express ideas, these all have the opportunity to significantly contribute to shaping the kinds of practices that constitute "real policy". Rick RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU) Date: Sat, 4 Dec 1993 07:22:11 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: appearances vs understanding X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> During breakfast this morning, Sue's story about the assessment of the teaching of a French lesson by someone who did not speak French was still entertaining me, and it reminded me of a story my dad told about his army days. His best buddy in the army was a fellow whose family was from Hungary. I think the fellow was Catholic. The Jewish High Holiday, Yom Kippur, had arrived and my father was to be given a pass to go off base to worship. His friend said "Get one for me too." And my dad had said "They won't give you one; you're not Jewish." His friend said "They won't know that. Look, when we go into the office, I'll tell them I'm Jewish, and I'll 'prove it' to them. I'll tell them I speak Yiddish just like you do. Then, when I start talking animatedly in Hungarian to you, you respond the same way in Yiddish; and they won't know the difference." It worked! Sometimes there is nothing like animation and good eye contact, I guess. Rick RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU) Date: Sat, 4 Dec 1993 10:24:45 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: Teacher Prep, PS To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> Rick comments on the factor of time, and I suspect that has a lot to do with this discussion. Students spend a few hours a week for a few years in a teacher preparation program. They then spend forty hours a week for a many years in a school. Which can we reasonably expect to have more effect on the new teacher? * * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET Date: Sat, 4 Dec 1993 10:09:45 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: Re: teaching and teacher ed programs To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: In reply to your message of FRI 03 DEC 1993 21:45:05 EST ****Nice post. ****Rick. Though I fear my most recent post may have been misinterpreted. So let me try to clarify further. Schools of ed do have a purpose; if I did not believe that I would be back as a principal, or (more likely) back teaching kindergarten. What I am saying is that that job is not to make prospective teachers into good teachers. Rather, it is to make prospective teachers into POTENTIALLY good teachers. To switch back to ethics, it is not just that your job is not to make students into ethical people; it is that such a task is conceptually incoherent. Likewise, though perhaps not quite so definitively, that is the case with making good teachers. They are not MADE; one chooses to be a good teacher. And the questions of ethics are not really all that different from questions of good teaching. Tom Green, in a superbly insightful article called "The Conscience of Leadership" (1987 ASCD Yearbook titled _Leadership: Examining the Elusive_ edited by Linda T. Sheive and Marian Schoenheit) makes the comment that "...the term 'professional ethics' is a redundancy.'" The article was specifically about educational administration, but the comment, I think, has broader application. Schools of education do a good job of teaching the sort of skills Rick accuses us of not teaching. Yet our graduates do indeed easily mold themselves and learn new techniques, quite in contrast to what they had learned in our classes. What I think Tom is reminding us of is the fact that technique is insufficient for good professional practice; what is needed is a strong sense of purpose and a clear understanding of what Goodlad, Sirotnik, and Soder refer to as the _Moral Dimensions of Teaching_. That is, we are better, I think, at teaching technique than purpose. That is, when we teach a technique, we teach ITS purpose, but what is often not done as well is teaching why THAT PURPOSE matters. I would certainly not say that shortening the certification process for the "natural teacher" is a reasonable response to this problem. Rather, I think we need to beef up substantially the "foundational" aspects of teacher preparation. It is not enough that students have clear understandings of what to do; they need a much deeper understanding of WHY these things matter. BTW, I strongly suspect that those of us on this list are already aware of the importance of this aspect of teaching prep, but that many of our colleagues are not exactly sure why we spend so much time here. Further, it seems important to me to keep a rein on what we believe we can accomplish. There are things that schools of ed can indeed do to help students be prepared for the challenges of teaching. But it is beyond our scope or ability, and it makes no sense to me, to say that I ought to be able to assure Rick that X will be a good teacher. I can certify that my methods students can plan instruction, and I can certify that they can organize that instruction, and use a variety of techniques, etc, etc, etc. But I can also certify that any principle worth his or her salt could undo all of that in a week. And I can also testify that if that is not the way school is "kept" on his or her first job, it will be an unusual person who can resist peer pressure. So, I am not saying that schools of education have no responsibility for preparing good teachers. But I will repeat what I said in a different post. I think, Rick, that you seriously underestimate the complexity both of good teaching, and of preparing young people to engage in that activity. The chain of events that leads to that outcome is long; to point at schools of ed as the weak one is to scapegoat. And for us to accept the challenge, and take on ourselves the task of "making" good teachers would be an act of both extraordinary hubris and foolishness at the same time. Our best and most successful efforts should make more likely the thoughtful practitioners we want. But we can never do more certify their potential. There is much yet to do after teachers graduate. And I also want to add my thanks to Bill's. Your comments are provocative in the best sense of that word. And I think they do in fact raise a different question: why are our students so willing to abandon what we teach them in favor of the standards of practice of whatever school they get their first job? And is there anything we can do to make our teaching "stick" longer?? * * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET Date: Sat, 4 Dec 1993 17:07:36 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: Teacher Prep, PS X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312041820.AA15758@tolstoy.u.washington.edu> To piggyback on John's comments: There is also the influence of the field placement school(s) during teacher prep itself. One of the things we're wrestling with now is how to work with field folks to help students make sense of the different district and building cultures and ways of working. It's not just that they go from the university setting ready to be wonderful (or to learn to be wonderful) and that they are then remolded by their first teaching position. There is from the beginning a tension between what they see and hear in the field and what they see and hear at the university. (Not to mention the diversity they see in the university itself!) ______________________________________________________________________________ Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195 Date: Sat, 4 Dec 1993 21:30:19 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teaching and teacher ed programs To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Sat, 4 Dec 1993 10:09:45 EST from <FACV@NMUMUS> I accept John Covaleskie's description of the question I was asking: why is what ed students are taught so readily abandoned? and what, if anything, might ed schools do to make them "stick" better? I still disagree about the ethics stuff, because I think there are effective ways to help people be ethical, and to tell they are. But arguing that sort of gets us off the track a bit, though I think there are some relevant parallels to the education case. I think real understanding is not as easily abandoned under adverse pressures as agreement for the sake of a grade, or as "surface" understanding. One form of surface understanding is being able to, say, solve all problems that are set up a certain way, but not to be able to see that problems set up a bit differently are really the same sorts of problems. So, for example in ethics, one might argue, that ethical values are all relative and that a teacher has no right to mark you down for holding that just because he disagrees with you. In education, one might perfectly well hold that tracking is a bad thing, but when faced with divergent abilities among many of your students, group them into 'manageable' groups that you can teach more efficiently. A typical engineering example is taking engineering students through a refrigerator testing area where many refrigerators are run with their doors off, until they breakdown. The rooms are heavily air conditioned, and engineering students invariably ask why air conditioning is needed if the refrigerators are putting out all this cold air. Yet these people know (on another level, somewhere in their brain) that no machine is 100% efficient, that refrigerators are machines that only move heat (from the inside to the outside air around the coils), that in doing so they create heat as well as transferring it, and that both the heat they are transferring and the heat they are creating is going into the room. The heat created adds heat to the room since the heat transferred simply balances the cold left behind. Without the air conditioners it would get very hot in the room with all those refrigerators running. The engineers in some sense know all the principles but I would say they don't really 'understand' the principles except in a surface way. John talks about this with regard to teaching the techniques and principles of education as needing to teach not only the techniques and their purposes but why the purposes are important. I would say teaching why the purpose matters is part of teaching the purpose; but if John sees a distinction and wants to teach both, that comes out to the same practical end as far as I am concerned. At any rate, I think there are techniques, such as discussing case studies of pressures that influence teachers on the job, doing certain kinds of role playing, and giving many, many examples and explanations of why the principles they have been taught matter, that can help these principles "stick", or, as I would say, help teach them more fully to begin with. I think real or full knowledge does give power to do what is right as well as to know it, in ethics and in education. Former students have told me they perceive and respond to ethical issues differently after my course; they try to use what they have learned, and sometimes they cannot help using what they have learned. It would be interesting to see if there might not be techniques to make what you teach ed students "stick" to them better in that same way, so those committed to educating children will also be committed to doing it in ways that research and reflection show most reasonable, even in the face of great pressures to the contrary. In ethics and in education we cannot make people be committed to caring, but I think there are ways to help those who are committed to caring more likely to do the right thing. Finally for now, I did not mean to imply schools of education have the chief responsibility for the remedying the problems of education or that they are the weakest link. I think schools of education, however, can have greater influence than they perhaps realize; and I would wish them to use that influence -- so that there will be more public knowledge about teaching, in the hopes that such public knowledge will put pressure on schools and principals to do the right things in those school districts that might otherwise be less prone to doing the right things. Rick RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU) Date: Sun, 5 Dec 1993 13:13:36 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: teaching and teacher ed programs X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312050742.AA11779@tolstoy.u.washington.edu> It's not so much a matter of sticking or falling off, or even of learning to the point of being able to recognize when the knowledge is applicable in the field. In an article soon to appear in Teaching & Teacher Ed, John Nicholls & I report a survey of teachers' beliefs about effective motivation strategies. Our conclusions: they already know (at least their group means indicate that most know) much of what we'd teach them about effective ways to increase students' motivation to learn. Other research reports that the strategies most often used, however, (extrinsic rewards and punishments )are not the ones that teachers in our survey (data from four states) report as most effective. We suggest that it may be a case of goal differences rather than lack of knowledge. Given a goal of increasing students' motivation to learn, without competing goals (like raising standardized test scores, "covering" material, etc.) they might indeed use these effective strategies. But in real life there are many goals that compete with increasing students' motivation. I think this could be true of many of the practices we try to teach in schools of ed. Given a context in which the goals of the school match the strategies we teach, I think beginning teachers are much more likely to use the strategies. Further, if this is true in the field placement schools as well, they are more likely to want to deeply learn these strategies because they see them as useful. ______________________________________________________________________________ Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195 Date: Mon, 6 Dec 1993 08:28:05 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: The Teachers Colleges and the Field X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312060151.AA08993@tolstoy.u.washington.edu> To the good points made by Gene & Tom G.--I agree that the tension is natural, good, and necessary. Even students tell me that they appreciate hearing the different views, not only between ed school and field, but also within the ed school itself. Others, though, tend to get overwhelmed by the contradictions. I say don't try to cover up or reduce the differences, but do try to help students (1) make sense of the differences and get beyond "everyone's point of view is equally valid," (2) get a grip on their own beliefs and assumptions about learning, teaching, etc., and (3) begin to learn how one can hang on to one's integrity and perhaps idealistic goals within a system that may actively work against them. ______________________________________________________________________________ Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195 Date: Mon, 6 Dec 1993 11:45:28 -0500 From: Kevin Drumm <drummk@POLARIS.NOVA.EDU> Subject: Re: certifying teachers X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312031530.AA25823@polaris.nova.edu> John Covaleskie's post on superintendents and principals wanting new teachers so they can be molded to those peoples' view of "the real world" of teaching absolutely frightened me. I was shocked! I think it was on another List where someone asked about the resurgence of fascism taking place here in the good ole U.S. of A. Well, John's given an example of it, albeit with an insidiously democratic temper. John, you yourself note the lack of confidence in new teachers and how quickly they assimilate in their new culture. Why not hire a few experienced teachers who might just have the self-confidence to resist the almighty status quo you and your colleagues are trying to protect by hiring and molding new teachers in your image of good teaching. Who appointed you and your colleagues to engage in such brainwashing...? I know--your school boards who didn't know any better. I learn more and more from this list every day about why our students are not learning what they should and it saddens me no end. No cheers this time. Kevin Drumm NOVA University 305-424-5758 drummk@Polaris.NOVA.edu Date: Mon, 6 Dec 1993 10:42:39 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: teaching and teacher ed programs To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312040512.AA36863@acs1.acs.ucalgary.ca>; from "Rick Garlikov" at I was really impressed by Rick Garlikov's recent post (and not only because it treated me well). We all write here with the hope that we might make some difference by doing so. Thanks, Rick, for taking the discussion so much to heart. Contrary to your expectation, I DO agree with John Covaleskie that the job of teacher preparation is to get people to the point where they are ready to begin learning to be teachers. Indeed, I tell my students: "When you are finished with this course, you should be prepared to begin a lifetime of learning about X--that is my aim." I came to realize that this was true when I was coordinator of a graduate program in school psychology. Students entered that program with background in both education and psychology (which usually meant a degree plus additional course work) and some related work experience. They spent two years in the program including a rather intensive practicum. At the end, every one of them would say "I don't think I am ready." This reminded me that when I finished the Ph.D., I did not feel ready to teach teachers so I looked for positions that would give me a lot of opportunities to observe others teach. Rick's post mentioned that he has known engineers who report the same feeling of unpreparedness. The thing is, we never really know what a job requires until we take full responsibility for doing the job. Then we are in prime condition for learning. All too often, building administrators are swamped with administrivia (I believe Berliner has pointed out that schools are extraordinarily thin on management-to-worker ratios) and tyros at teaching are left to their own devices. A lot of schools are now developing mentorship programs to address this problem. None of this is meant to relieve teacher educators of their responsibilities. As I have indicated earlier, the teacher preparation program may serve as a self-screening opportunity for some and an elimination process for others (who are flunked out or advised out). While Rick marvels at the number of less-than-excellent teachers he has encountered, I sometimes marvel at the number of totally and completely inappropriate people we end up discouraging--in particular, people who suffer mental illnesses or who are verbally and (rarely) physically abusive of others. There are also people whose motivations seem to have more to do with power than with learning and some who are responding to external pressures ("my whole family is made up of teachers, I just HAVE to be a teacher). Such people reveal themselves in papers and in classroom interactions. If we succeeded only in significantly reducing the numbers of such people who actually enter the profession, that would be worthwhile, but I think we do more. People leave us with a sense that there are choices to be made about approaches to instruction, with an awareness of a variety of such approaches, with and increased awareness and appreciation of the complexity of teaching, with enthusiasm about the promises of diversity as well as an awareness of the challenges it poses, and with some specific skills in such things as testing, using computers and/or other technological aids to teaching, planning instruction, giving oral presentations, etc. I like to think that they leave with their ideals intact and that the flow of new teachers through the system is a constant challenge to the momentum of mediocrity that can easily develop as classroom challenges exhaust the energies of experienced teachers. bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca Date: Mon, 6 Dec 1993 13:06:00 EST From: Leslie Wade <LWADE@NAS.BITNET> Subject: On asking questions X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> ****And I would love to know if anyone has some sense ****of why, as Bill says, we "would be hard-pressed to ****find teachers who use the techniques" he ****explained so well. If someone could fill in the ****"largely for reasons I don't understand" that Bill mentioned. ****I suspect one of his reasons is a primary culprit -- ****asking questions takes time away from lecturing or ****"telling" I wonder if some teachers don't ask if their students understand them or the material they teach because hearing that the student didn't get what the teacher intended would somehow imply that the teacher is incompetent. It's not easy to hear something you don't want to hear, and most of us avoid it when possible. If our aim is to communicate something and we think we're doing it very well, it would be difficult to hear that we weren't. It takes a teacher willing to learn from his/her students to be challenged like that--as well as someone willing to appear less than the perfect authority figure. What do you think? Leslie Wade Date: Sun, 5 Dec 1993 18:44:38 MST From: Gene Glass <ATGVG@ASUACAD.BITNET> Subject: The Teachers Colleges and the Field To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> Rick and John and Sue and Bill and David and many others have now said so many things about good teaching and the training of teachers and the place of the pre-service training vs continued growth as a teacher that one hesitates to intrude into the discussion unless one has followed it religiously. I don't want to be redundant with what others have said and I haven't gone back to study the record carefully. With contingent apologies for saying what might have been said, I offer this one simple thought: The tighter alignment of the college preservice curriculum and training with what "the field" (by which I guess I mean administrators, boards and experienced teachers) wants that training to be is not necessarily a thing to be sought after. Indeed I would argue that in many instances it should be resisted--for at least two reasons. To the extent that the field (the receiving institutions) define the good teacher and the skills and knowledge and understandings that the good teacher should have, to that extent the question naturally arises why the field doesn't train them themselves. Don't tell me that the field is indeed asking that question because I know they are. There are many schools--perhaps the majority--that define the good teacher and the good teacher's role in ways that the college would regard as anathema. "Teach these children so that they will score high on the state test; enforce this discipline program; cover the adopted text." It is the college's responsibility to question many conceptions of education held by the field and to try to advance a better conception. Small wonder that educators in schools shake their heads and marvel at what ivory tower idiots those professors in the college are, and how it would do them good to come out here in the field for even just a week and see what life is really like on the firing line. Well, I submit that it would be a loss to the entire enterprise to disabuse the professors of their idealistic and grand views of the good teacher. I am arguing that the perennial tension felt between the college and the filed is a good thing. It may be a sign that the college is doing its job. If I heard from a number of school districts that the teachers college was doing a fantastic job and delivering them first-year teachers that looked exactly like what they wanted, I would be suspicious that the college had lost its bearings and no longer was a place where the conventional view was critiqued and extended. GENE V GLASS Glass@ASU.BITNET College of Education Glass@ASU.EDU Arizona State University 602-965-2692 Tempe, AZ 85287-2411 Date: Sun, 5 Dec 1993 21:53:31 EST From: Tom Green <TFGREEN@SUVM.BITNET> Subject: Teachers Colleges, the Field and Irrelevance X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> I was delighted to see Gene Glass' recent entry on this topic. It ways what I have thought of contributing many times, but with a slightly different twist. To wit.... In virtually all of the professional schools a certain detachment from practice is highly desirable and probably absolutely essential if practice is to every be reformed, changed, or improved. The problem is never just relevance and irrelevance, attachment and detachment. It is always in what respects and when and for what purposes and within what limits. Tgreen Date: Thu, 2 Dec 1993 09:02:23 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Aimee's note on contexts X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312021351.AA20083@tolstoy.u.washington.edu> Critical point! Both you and John C. have noted very important constraints on our ability to predict who will be a "good teacher." There is also the role of subject matter knowledge. When many middle school principals prefer to hire elementary certified teachers because it gives them more "flexibility" in assigning teachers to subjects, a student teacher might student teach in social studies and science and find themselves teaching algebra the next year. They might be a dynamite U. S. History teacher, but be a very unsatisfactory guide to the intricacies of quadratic equations. Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195 Date: Mon, 6 Dec 1993 13:45:27 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Re: On asking questions X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312061847.AA01591@tolstoy.u.washington.edu> Leslie writes: **** I wonder if some teachers don't ask if their students **** understand them or the material they teach because **** hearing that the student didn't get what the teacher **** intended would somehow imply that the teacher is **** incompetent. **** Possibly. There is some interesting research that finds that the more teachers know about the topic they're teaching, the more open they tend to be to student questions and discussion. I suspect that a good part of not encouraging student questions may be a "don't look back, someone may be gaining on you" kind of defense mechanism. What if you can't answer? What if the question itself sounds relevant, but you can't even really make sense of it? It can be pretty scary stuff. Sue Nolen U. of Washington Date: Mon, 6 Dec 1993 17:09:25 EST From: CJB Macmillan <cmacmill@MAILER.FSU.EDU> Subject: Re: On asking questions To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <199312062149.AA11692@mailer.fsu.edu>; from "Susan Nolen" at Dec 6, 93 1:45 pm Questioning can be scary - and we should remember that to ask a question is to determine what the responder must formally say. You cannot answer "What is the sum of two and two?" by saying "purple". In a way it's merely impolite for young people to ask older ones questions -- for it's a way that they can control their elders. Are teachers afraid of losing control in this as in other ways? C. J. B. Macmillan (cmacmill@mailer.fsu.edu) Date: Mon, 6 Dec 1993 16:56:03 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: certifying teachers AND asking questions To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312062043.AA81817@acs1.acs.ucalgary.ca>; from "Leslie Wade" at De Leslie Wade quotes my question: **** >>What about the teacher who struggles mightily to **** >>prepare good learning materials, to insure that **** >>students are exposed to quality information, to give **** >>them experiences that encourage them **** >>to work with that material constructively (and maybe **** >>even asks them why they answered incorrectly) but **** >>who is perceived as boring and monotonic in class **** >>and too demanding with regard to marks? and wonders: **** I wonder WHY such a teacher would be considered **** boring and monotonic--I get the part about too demanding. What I had in mind is someone who is simply not an effective public speaker--someone whose speech is without variations in tone or texture, who fails to make eye-contact with the class, who may have some slight impediment or accent that makes it difficult to understand all that they say, who speaks too softly, etc. etc. I know a few such people who put great effort into their teaching but who are systematically rated poorly on those teaching evaluations you referred to. I had at least one such instructor myself--he would develop brilliant lectures that told rich and exciting stories about the ways in which research studies emerged in his field to add nuances of clarification to the development of theory. As far as I know, I was the only student who really appreciated that--everyone else said he was "boring," I assume because his voice did not convey the excitement that was inherent in his material. bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca Date: Mon, 6 Dec 1993 11:29:48 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Re: Teaching and Learning X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> I gather from the recent discussion (which I have found VERY interesting) that we can't rely on revamping teacher education programs to reform the schools. I would enjoy hearing more about the socialization process of new teachers that thwarts such reform efforts. I'd also like to ask Gene Glass whether he would consider it healthy for schools of education and public schools to be in closer alignment if that meant the public schools were moving closer to the ideals of the university. Date: Mon, 6 Dec 1993 11:34:40 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Re: On Asking Questions X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> Leslie Wade asks: "I wonder if some teachers don't ask if their students understand them or the material they teach because hearing that the student didn't get what the teacher intended would somehow imply that the teacher is incompetent." I think part of the problem is that teachers are on a schedule. Whether it's as horrible as Rick Garlikov's example or not, they are still responsible for covering a certain amount of material in a specified amount of time. It's very discouraging when you're ready to move on and it turns out the kids aren't. There are some beautiful examples from a math class described in a fairly recent article in the Kappan. I'll try to find it. Cindy Cotter Date: Tue, 7 Dec 1993 08:27:39 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Not asking is not teaching X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312070707.aa02763@ssi.edc.org> On the issue of teachers not asking whether students understand them or the material they teach... On Mon, 6 Dec 1993, Cotter_Cindy wrote: **** I think part of the problem is that teachers are on a schedule. Whether it's **** as horrible as Rick Garlikov's example or not, they are still responsible for **** covering a certain amount of material in a specified amount of time. It's very **** discouraging when you're ready to move on and it turns out the kids aren't. I think that teachers who are driven by the curriculum are not teaching kids. We are working very hard at both state and local levels in Vermont to help define how to do more with less "coverage" of the type mentioned above. In some sense, a teacher shouldn't be "ready to move on" if the student isn't. Sometimes the student's lack of readiness is so apparent that few would disagree with this axiom; for example, when a student can't play a scale on the piano, no good teacher would try to introduce a Mozart piano sonata. Why should we allow an analogous situation to occur in other fields of knowledge? Date: Mon, 6 Dec 1993 21:26:01 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teaching and teacher ed programs To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Mon, 6 Dec 1993 10:42:39 MST from <hunter@ACS.UCALGARY.CA> I have been having some difficulty trying to put together some things that Sue, John, and Bill have been saying, but I think I may have resolved some of my difficulty, so I want to try to express the problem and the resolution. First, I liked Bill's last post, which said in a nice way something like:"Rick, if you don't like some of whom you see passing an education program, you ought to see what we are able to weed out." That reassures me a bit, I think. But I still feel like a few boulders are getting through the sieve. However, on to the serious stuff. There seems to be a tension or tight balance between new teachers moving into a school bringing with them enthusiasm, new ideas, and fresh ideals to infuse into the school, versus new teachers who move into a school with all the above, but whom the staff and reality rob of all that as they force the new teachers to conform to their "old" methods, etc. In many cases perhaps there is a little fresh infusion and a little bit of harsh acclimation, rather than one side's totally transforming the other. And, of course, there are new teachers who move into pretty good, nurturing, receptive schools where their energies and enthusiasm and skills blend nicely with the experience and wisdom of the faculty, so that both the new teacher and the experienced staff enrich each other. I assume this latter is an ideal situation, and that it even happens some times. What I have seen in the recent past locally, however, is where the new teachers pretty much are dominated by the old, with the principals being ignorant of good teaching practices and/or simply siding with or allowing the old teachers to dominate the new. It is that experience that tends to color my perspective when I write. I see, in some other threads on this list, and by some of what Sue writes, that one way ed schools try to change this sort of situation is by forming partnerships with schools or school districts. I know that is difficult for a number of reasons, but apparently there are some successful endeavors of this sort. There is one endeavor in our district, but it is at a difficult stage right now, because the university people are not having the kind of input, etc. they thought they would; but they are keeping at trying to make it work. At any rate, I applaud this effort, as a way of trying to help children, and also trying to set up an environment that nurtures new teachers in the right way rather than just letting them go into an environment where they will "lose" everything they have been taught. So, on the one hand I see John, Bill, and Sue sending out people ready to become successful in the right environment, but knowing that not all environments are right. Since I see many bad environments, I tend to see many ed graduates being thrown to the wolves. But apparently, even if that is not the general case, Sue, Bill, and John know that it happens sometimes, perhaps even frequently. My concern is that if it happens a lot, are ed schools then not simply shaping their students in the same way that some chefs prepare beautiful ice sculptures that take a day to create and then begin to melt in minutes, once the banquet begins, and which, in a few hours, will be totally ruined. I suppose you all see more successes, with a bit more permanence, yes? If not, how can you reconcile preparing your students and then having to send them out as sacrificial lambs to the slaughter in ways that don't benefit children anyway? Now Sue points out that it is not that teachers forget or grow to disbelieve what they have been taught, but rather that they get put in environments where they are not able to use what they believe and have been taught. I would think that whatever the cause, it must be somewhat disheartening for you as you have to watch your hard work go for nothing, and your lovelycreations melt before your eyes. Is that not difficult for you? Or is it just so rare (except where I live) that it is not a significant problem? Do most of your grads go on to first jobs where they can mature, blossom, and thrive at least enough for you to feel satisfied about your efforts? It seems to me that you must see enough success of this sort or you would not have the strong views that you all hold that what you do is worthwhile. For, unlike the ice carvers, whose creations serve a purpose, although a brief one, if most of your graduates are ruined before they get to teach much, your work will have served no real purpose at all, not even a transitory one. Further, I think my problem with "non-sticking" of what you teach applies, even if Sue's theory is correct that teachers don't lose their knowledge but just cannot use it, because the goals that are set for them are not furthered by what you have taught them. I would want your students taught well enough that they are a force to resist such bad goals, just as they would resist influences that serve to dissuade them of your teachings. I am not talking about setting them up to be martyrs, but helping them become strong forces of change who are flexible in accepting good influences from old staff but who have a fighting chance at successfully resisting bad influences; and who have skills and initiative to help them convert others to their methods AND to the proper goals of children's educations. I would like to suggest one partial way of helping students be more reflective in their teaching practices and more articulately assertive in their resisting pressures to do what they rationally believe to be in children's best interests. Sue first mentioned faculty pedagogic diversity as a difficulty for developing strong students, since students see no agreement or consistency and perhaps therefore no wisdom or meaning in what they are taught, but then in a later post saw it as a potential tool for developing reflective, assertive students, since students would have to sift and choose among competing philosophies. I think the former case is more likely to prevail unless an ed school has an active program for bringing diversity of staff to bear properly on students. I wish to suggest such a way here. It is what I meant by role playing in previous posts about ways to develop independent, strong, assertive, reflective teachers. Students are often able to figure out how to give teachers what the teacher wants. They learn to play the game, and can often do, write, and say things they do not believe or believe in, but they do it for the grade or the degree. That may please teachers who mistakenly think student answers and behavior reflect student convictions, but it does not really produce students with convictions, or ones they can effectively express or promote in the face of pressure. Since there are pressures on new teachers involving their fear of keeping their jobs, fitting in, being liked and respected, etc., it seems to me that in order to help students learn to resist those kinds of pressures, you have to set up situations while they are under your guidance that teach them how and when to do this. One way is to give them case studies of situations that occur in schools, so they have a better chance of recognizing and dealing with job pressures. But it seems to me that what I call role-playing may be an important teaching tool also -- not role-playing in the sense of the students' acting out roles, but role playing in the sense of the faculty putting the kinds of pressures on students that may happen to them on their jobs. Turn students hunger for grades and their degree into a cause of their being reflective and assertive instead of being sycophants. The way to do this is to (1) warn them they will face problems at the school of education just like they will face on the job, and they will face some of these problems outside of class. Their job is to deal properly with such pressures, for their responses will have an effect on their grades. (2) Set up collaborative faculty efforts by which some faculty will exert pressure on students to do things that go against what other faculty will be teaching them. (3) When students do not respond to such pressures properly, let them know it was a situation that was meant to show them the kinds of forces they will possibly face on the job, and give them examples of how they might have better responded. Since you will have made the situation 'real' for them, your advice will probably have more meaning to them. For example, have one teacher give an assignment that the students know may or may not be graded by a teacher the students know has a different view from the teacher giving the assignment. Or have one teacher pressure students to make teaching demonstrations they know an assessing teacher will not think good. Do this periodically and repeatedly, and let students know that what is required is for them to (1) do the assignment the way they really believe it should be done, so that if they are going to get into trouble, it will at least be for something they believe in, rather than for simply guessing wrong about whom to try to please, (2) have good reasons to justify their actions, and (3) be able to articulate their reasons well enough to be convincing to the person putting the pressure on them to do what they believe is wrong. One of the ways I try to get my students to argue what they really believe is to tell them the first day of class I will ask them sometimes to justify a belief which is actually unjustifiable, and that sometimes I will challenge them and try to back them down even though I believe they are right. Since they will not know when I am doing this, they need to argue only things they believe, not give arguments they simply think I want to hear. Sometimes when I do this, I try not just to be a rational 'devil's advocate', but to act temperamentally and forcefully and authoritatively, so that there are signs to them that they may risk upsetting me if they persist. Still, they had better persist. I let them know immediately after they respond, what I have done and why. I do not think it is fair to play devil's advocate as a teacher without letting them know afterward that is what you were doing. I do this kind of thing early in the course and periodically. And I do it in a way that kids them rather than chastises them for their mistakes. It does not take long before they start to be quite honest in class. I also try very hard to be appreciative of their efforts even though I may vehemently disagree with their ideas, so they can see I am not penalizing them for honesty. It is this sort of thing that I think a faculty could do in concert, so that they are exerting pressures on students in ways similar to the ways pressures will be exerted on them in their profession. (I warned a new principal coming into our school system after our first superintendent forced out or fired --by the end of the first year-- five of seven school principals, three of which he had hired himself, that since this superintendent seemed to manage by whim, it was impossible to know what would please him, and so the more satisfying way to proceed would be to probably do what he thought right, rather than to do what he thought expedient -- since there would be no telling what was expedient anyway. If you are going to get in trouble no matter what you do, it may as well be over the thing you thought was right.) I think it takes setting up, entrapping, and then explaining these sorts of situations to students in order to make them be reflective and responsible. I think ed school faculty could do such things in partnership with each other, where one faculty member pressures students into having to choose between doing something his/her way or the way a different faculty member is known to want and where the student is not able to ascertain who is "in charge" or who will be assigning the grade, assessment or whatever. I think you would see students start to more quickly be assertive, rational, and independent. It might also be fun for everybody. And I think if you don't do these sorts of things, students will just "kiss up to" whichever teacherthey have and go through the motions of giving that teacher what will get the student a good grade from him/her. But the student will often then not acquire any genuine convictions or have the courage, or the practice, to articulate and champion them. Then, when placed in similar "real" situations on their initial placement, they will bow to whatever pressures come their way. Finally, Gene, Tom Green, and I think someone else seemed to think that if ed schools taught the same sorts of practices that 'real schools' did, that would be a real loss, since ed schools should have higher ideals and aim for better practices. Gene, argued and they concurred, that ed schools needed to be different from working schools. Why assume that if both are alike, they will be like the 'real' schools. Why not assume ed schools can bring real schools up to their standards instead of having to "lower" themselves to working school practices and values? (You are acting like God in the old sick joke about the person with a severely deformed hand who goes to church to pray that God would make both his hands alike, after waving in God's face the deformed hand and telling Him to look at what He has done to him. So God obliges the man by deforming the man's good hand too.) Rick RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU) Date: Mon, 6 Dec 1993 22:00:39 -0500 From: JE01@SWTEXAS.BITNET Organization: Southwest Texas State University Subject: Re: certifying teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Your message dated "Mon, 06 Dec 1993 11:45:28 -0500" <01H661PLI17299DGMW@academia.swt.edu> Now here is a scary story folks. When I was a school board member some while back, our elementary principal *always* wanted to hire brand new teachers so he could mold them into the school culture. He was deathly afraid of more experienced teachers who would (gasp) have ideas for change and improvement. He seriously wanted to keep the lid on. It was always a struggle at hiring time, since the board said that it found his personnel practices a bit difficult, but when push came to shove, the board as a whole was unwilling to confront his practices. I do not think that this is unusual either -- new hires are cheaper (), and more easily intimidated -- less likely to rock the boat. It is usually not as obvious as this, but methinks that it is a more pervasive practice than it appears.

- Jill

-=--------------------------------=- Dr. J. Ellsworth N5XUF -= je01@academia.swt.edu =- Southwest Texas State University -=--------------------------------=-

Date: Tue, 7 Dec 1993 08:47:14 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: New directions in induction X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312070410.aa29825@ssi.edc.org>

On Mon, 6 Dec 1993, Cotter_Cindy wrote:

**** I gather from the recent discussion (which I have found
**** VERY interesting) that we can't rely on revamping teacher
**** education programs to reform the schools.

I have really enjoyed this conversation too. I'm saving it to share with a lot of people. I think we can't rely, but we can celebrate the innovation and find much support for school reform in higher education. I am curious, positive, and expectant about the kind of partnerships being formed today, where future teachers work as co-researchers in classrooms and where higher education teachers are organizational consultants and co-teachers in those same schools and classrooms. I also think there are higher education institutions that are seriously re-examining the nature and role of the undergraduate experience, who are developing a sense of the continuum of learning, and who have an interest in helping create new environments for the induction of new teachers.

As someone who is in a high school on a regular basis, helping direct curriculum and instruction, I see the new teachers coming out of higher ed as some of the most positive forces for change.

I would enjoy
**** hearing more about the socialization process of new
**** teachers that thwarts such reform efforts.

Ah, but you are right here, Cindy. The system into which they are inducted can often thwart their idealism, energy, and vision of reform...as it does to many other more experienced teachers. We are working with a consortium of colleges to establish an induction and support system, and I would enjoy hearing from others who are thinking about this issue. I don't know if it has been published yet, but soon to be available is a book by the Northeast Regional Lab on this subject. One of its authors is working with us Vermont to think through some of the challenges.

David Gibson dgibson@ssi.edc.org

Date: Tue, 7 Dec 1993 08:43:20 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: On asking questions To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Mon, 6 Dec 1993 13:06:00 EST from <LWADE@NAS>

****
****I wonder if some teachers don't ask if their students
****understand them or the material they teach because
****hearing that the student didn't get what the teacher
****intended would somehow imply that the teacher is
****incompetent.
****
****It's not easy to hear something you don't want to hear,
****and most of us avoid it when possible. If our aim is to
****communicate something and we think we're doing it
****very well, it would be difficult to hear that we weren't.
****It takes a teacher willing to learn from his/her students
****to be challenged like that--as well as someone willing to
****appear less than the perfect authority figure.
****

I agree with part of this, though I would not be so much concerned with the authority figure aspect. Some teachers may seek power of authority, but not all do; and perhaps not most that I have seen. But I think it is difficult to "check your work" -- to go back after putting in lots of time and effort to see whether you have done it successfully or not. It is easier to say, "Well, I'm sure I have done it right, so I'll just go on." You don't want to find out it was wasted effort AND that you need to do it again in some different way to get stuff across.

However, that does not explain why teachers cannot ask questions BEFORE they get into material, to see where students are "coming from", or why they cannot ask questions AS they go along, to see what students are getting. By the way, however, that does not mean asking "DO YOU UNDERSTAND?", since the answer to that will usually be "Yes" even when the student knows he does not under- stand, and especially when he thinks he does, though he really does not. You have to ask other kinds of questions to see whether the students can use the material correctly in slightly different contexts, etc. Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Tue, 7 Dec 1993 09:15:04 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: Student Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312031442.aa01463@ssi.edc.org>

I agree with you that assessment is vital, and that it is often done badly...sometimes it is not done at all and instead, kids are simply tested. Although the terms might be synonymous, I'll reserve "assessment" for meaning that the learner actually learns something about him or herself and what they know and can do, and "testing" for meaning that he or she has been measured for some other reason - such as comparing to others, checking to see if the teacher has "succeeded" in instruction, checking a whole system to see if it has "covered" some material, etc.

On Fri, 3 Dec 1993, Cotter_Cindy wrote:

**** David Gibson wrote: "The course you describe on testing sounds like it might
**** be of interest to someone who would like to work for the College Board someday.
**** Or perhaps it would be of vital interest to policy makers attempting to analyze
**** and reform large scale testing and assessment practices. It does not surprise
**** me in the least that very little of it gets into a classroom, and I'd recommend
**** reconsidering it for all but highly specialized students focusing on
**** evaluation methods."
****
**** Susan Nolen wrote: "And I think we need some way to defensibly (legally I
**** mean) weed out student teachers on their potential to cause harm through what I
**** would consider unethical practices, many of which are common throughout the
**** school system (e.g., grading on a curve, marking down for late papers, grading
**** on attendance, use of invalid assessments, etc.)"
****
**** Perhaps the two of you could discuss what appear to be widely divergent views
**** about the importance of assessment. I think assessment is essential to guide
**** teaching and that it's often done very badly. I was quite excited by hearing
**** Susan list those unethical practices.

I don't think Susan and I are far apart. I was questioning whether ALL future teachers need a 40 hour course on testing methods, or whether that way of teaching them about testing is effective for future classroom practice. I agree that the list Susan has begun are examples of testing practices that have little or nothing to do with what I think should be the primary purpose of testing...to give honest and instructionally helpful information back to a learner about their performance. I disagree with those who have other purposes for testing, or with those people who would put other purposes above or in the way of learning. I'm not an ethics professor, but I'll accept a definition of those kind of practices as unethical if we hold that teaching has a moral dimension.

I commented about not being surprised that so little of what is taught in the testing methods course gets into the classroom. It would have perhaps been more polite if I had stated my personal preference for teachers who learn about and improve their assessment skills in situ, in real learning situations, with learners and classroom experts helping them gain deeper understandings of the inseparability of instruction and assessment. I think teachers need to acquire assessment skills and habits that foster learning. My reservations about the intensive course, are that the focus might be on validity, reliability, statistics, etc...all good and necessary to large system testing processes where individual learning is not the primary concern and the information being sought is directed to policy makers, law makers, textbook makers, test makers,...almost everyone except a learner for the purpose of improving performance.

**** The issue of grading on curve is an interesting one (to me). Presumably Susan
**** is offended that a student who has not mastered the material will do well if
**** everyone else understands it even less, or that a student who has completely
**** absorbed all the major concepts may receive a low grade if his peers aced him
**** on the details.
****
**** Assuming a valid test, how SHOULD it be scored? I've been taught by teachers
**** who hate curves. 90% and above is an A. 80% and above is a B, etc. But what
**** is the basis of this method? Has someone determined that any student who
**** understands the core of the material well enough to proceed with new material
**** will answer 70% of the questions correctly? How did they determine that? I
**** can imagine a test where 70% of the items tested the core material, another 10%
**** were a bit more sophisticated, and the last 10% even more sophisticated.
**** That's not an easy test to construct.

These are very important questions. I'm learning more all the time and would appreciate others views. The "curve" will probably exist in any sufficiently large measure of randomness. I've heard Grant Wiggins then assert that people who use it to grade are admitting that their teaching job is roughly equivalent to random behavior. As a curve basher, I rather like this answer. On the other hand, I've heard serious teachers with strong physical science backgrounds say that is faulty reasoning. That the curve exists because there is a random distribution of the learner responses to instruction. I think this just means that our instruction style has been incapable of dealing with individuals.

In either case, "fitting" grades to a curve is not helping a student with instructionally relevant information. The information is about how one compares to others, not about what one has learned. I suppose a curve can give a teacher some useful information, so perhaps teachers should just keep that information to themselves and concentrate on telling and showing learners what they can do to improve performance.

Your question about absolute scales is interesting. If the student is an open heart surgeon and she is performing on you, what percentage would you be interested in? Would a curved grading system give you more or less confidence? Maybe some of our opinions about assessment have to do with the values we place on the content we are teaching. I think we should examine our values, teach what we value, and help learners with the best possible information we can so they can measure themselves in the light of those values.

David Gibson dgibson@ssi.edc.org

Date: Tue, 7 Dec 1993 12:14:47 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: learning to teach X-To: Multiple recipients of list EDPOLYAN <EDPOLYAN@asuvm.inre.asu.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Rick has pulled together a number of different conversations in his recent post. It really began to focus on how one learns to teach--the component that comes from a teacher preparation school or college and the component that comes in practice. The note concluded with a reference to Gene G's earlier note about tension between schools of ed and k-12 schools. Rick's comments reminded me of the question from Cindy_Cotter (I think) to Gene: would he still think that tension was good if it pulled mainly in the direction of current school practice?

Nobody asked me, but I think it is vital to insert a point here in defense of the schools (Alan Ogletree almost did that and I'll bet he's starting to worry about multiple personality syndrome:-)). It is simply not the case that all the white hats are on teacher educators. We have our share of misguided folks and retrograde thinkers--there is no need to create the conflict Rick suggested--it is there and very real, though not often in a single course. And schools have their share of far-sighted progressive thinkers and innovative teachers whose practices often lead the way for research and teacher education. Nobody in this game has a monopoly on wisdom or virtue--an assertion that is supported daily in this forum where a home schooler and a software developer and a philosophy teacher and graduate students and iconoclasts of every shape and kind contribute to the richness of the dialogue.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca
Date: Tue, 7 Dec 1993 11:35:20 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: On the Nature of Teaching and Questioning To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312070524.AA37725@acs1.acs.ucalgary.ca>; from "David Gibson" at D

Perhaps I should clarify further what I was getting at in the post about possible interpretations of poor performance on a test by a class. "The class is dumb" is included in my list primarily because it is a logical possibility. It is first in my list because both logically and empirically, it is unlikely to be the actual reason for poor performance and therefore can be quickly eliminated. Indeed, if a class consisted of mostly, or exclusively, students with academic difficulty, the test ought to be geared to their abilities and to the material taught, so even in this unlikely event, one would still want to go on to consider better reasons for the poor performance.

In relation to this question, David Gibson asks:
**** If (assessing the effectiveness of instruction) is one of the
**** purposes (of testing), does that role of serving the pedagogical
**** needs of the teacher interfere with any role testing may have for the

It may. One must remember that in a classroom many tests are administered and that their collective value is greater than their individual values (as, for example, your running speed might be better measured by taking the average of performances over six or seven races rather than relying on a single observation). In my own practice, I use the item analysis technique described earlier along with the kind of questioning that Rick Garlikov has called for to determine whether some items on a test may have been poorly constructed or whether my instruction may have been less than adequate in some area. If I can determine that these were serious possibilities, I can usually find some way to make an appropriate adjustment so that the test scores I end up using reflect the student's achievements more accurately. Such adjustments have less to do with good test design than with fairness and the perception of fairness.

and:
**** I wondered if the lack of translation from an intensive course on
**** testing to the classroom might be related to the relevance of the
**** content. If the content of a course on testing is not that relevant to
**** classroom practice, then I don't think it is bold to suggest a review of
**** the "fit" of the course with the needs of future teachers.

Reviewing the suitability of course content is always appropriate and should be an ongoing exercise. "Relevance" of the content to classroom practice is only one of the desirable criteria for such reviews. As Gene G. recently noted, a certain tension between what is done in teacher preparation institutions and what is done in schools is probably a healthy thing. In this specific case, for example, I have continued to include item analysis in that course because it is a relatively simple technique that, used even on rare occasions, can contribute to refining one's test development skills and one's understanding of the possible ways in which a test can go bad. So, while I realized that my students would use it little, I taught them ways to simplify and approximate and I encouraged them to make SOME use, but even the use they made of it on a class assignment would have, I think, contributed to their development. Now, we distribute software that will do most of the drudgery and all of the calculations and I am hopeful that the technique will be a little more palatable to the current generation of teacher candidates. For the course as a whole, I can understand that people might anticipate that a course on testing might focus on abstractions and applications that would be of little value or interest to a classroom teacher, but it is certainly possible (and I would say highly desirable) to structure such a course so that it focuses more on techniques that are useful in the construction and evaluation of classroom tests and other classroom measurement and observation tools. Our students do not say this course is irrelevant; indeed, it is very positively evaluated by students and more than a few will come back each year and say that it was the only course in which they learned things that they actually use in their first year of teaching.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Tue, 7 Dec 1993 10:29:03 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Re: On Asking Questions X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I found the Kappan article I mentioned in an earlier post on the difficulties of teachers asking their students questions. It's "Talking Mathematics: 'Going Slow' and 'Letting Go'" by Susan Jo Russell and Rebecca B. Corwin, and appears in the March '93 issue. The article describes a project to encourage teachers to have students talk out loud about what they're doing and discuss their thinking with the teacher and with other students.

"This process was accompanied by a loss of the comfortable feeling of closure and tidiness that mathematics once seemed to embody. As teachers spent more time listening to their students, they were shocked to find that their students did not understand ideas that the teachers had though were straightforward." p.557

"For example, Martha, a fourth-grade teacher, noticed as she watched her students count by twos on a hundred board that many of them did not seem to be using the terms even and odd comfortably. Taken by surprise that her fourth-graders might not have a thorough understanding of evenness and oddness, she asked, 'What do you mean by even?'

[A discussion among the teacher and six students followed, in which one student suggested you could split 5 apples evenly because you could cut one in half.]

"Martha let go of both her previous plan for the lesson and her assumptions about her students' knowledge. However, this letting go was difficult. ...While it is easy for researchers to be delighted and intrigued by the diversity of children's understanding, it may not be so easy for teachers, who feel responsible for their students' learning, to react with the same kind of delight when they begin to let go and uncover the complexity of apparently simple ideas and their students' confusion about them." p 558

This example was followed by another in which a teacher tried several times to bring closure to a subject and couldn't. Finally she "completely let go of control of the discussion by turning it over to the class" and also let go of "time frame and lesson plan."

This same issue of the Kappan also contains a great article called "How Old Is the Shepherd" by Katherine K. Merseth.

"In order to focus discussion, consider the following nonsensical problem:

"There are 125 sheep and 5 dogs in a flock. How old in the shepherd?

"Researchers report that three out of four schoolchildren will produce a numerical answer to this problem."

I think I'll stay off the soapbox for once and let these excerpts stand on their own.

Cindy Cotter

Date: Tue, 7 Dec 1993 12:31:50 CST From: Alan Ogletree <alano@VNET.IBM.COM> X-To: edpolyan%asuacad.BITNET@RICEVM1.RICE.EDU To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Jill Ellsworth wrote ===>
**** Now here is a scary story folks. When I was a school board member
**** some while back, our elementary principal *always* wanted to hire
**** brand new teachers so he could mold them into the school culture. He
**** was deathly afraid of more experienced teachers who would (gasp) have
**** ideas for change and improvement.

Before we get too far into this righteous indignation stuff, just remember that schools of ed do the exact same thing. They take mostly "new hires" (i.e., young students with "minds full of mush", as a certain rotund radio commentator is wont to say) and then mold them into the type of teacher the particular school of ed is designed to generate. The way I understand the system, you can't teach without a certificate, and you can't get a certificate without going through an accredited program, and a program can't get accredited unless it meets this or that criteria. To me, this seems to indicate an even greater fear of "ideas for change and improvement" than this principal exhibited. Remember, when you point your finger at that principal for wanting to "mold people into the school culture", three fingers are pointing straight back at you....

Alan Ogletree Houston, TX

Date: Tue, 7 Dec 1993 10:34:26 -500 From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Preparation for teaching/leadership X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312070811.aa04084@ssi.edc.org>
Rick,
I just wanted to pick up on two things you said which reminded me that we have been talking a lot about preparation for teaching, and I would also like to learn about preparation for organizational leadership. In speaking of preparing students to resist pressures on their ideals and what they know to be good teaching practice...

On Mon, 6 Dec 1993, Rick Garlikov wrote:

I am not talking
**** about setting them up to be martyrs, but helping them become strong forces of
**** change who are flexible in accepting good influences from old staff but who
**** have a fighting chance at successfully resisting bad influences; and who have
**** skills and initiative to help them convert others to their methods AND to the
**** proper goals of children's educations.

Schools seem to need several people around and in them who know how change occurs in organizations - not only for the benefit of resisting pressures to conform to institutionalized mediocrity, but to help propel the organization into a new future. I'd be interested in learning how ed schools prepare students for organizational life, organizational behavior and patterns, and for understanding how change occurs in organizations. In Vermont, there is some talk about setting up a "critical friends" network so that schools will have continuous organizational development support. What can new teachers bring to this kind of network? In particular, in what ways are they prepared to advance this alternative version that Rick proposes...

Why not assume ed schools can bring real
**** schools up to their standards instead of having to "lower" themselves to working school practices and values?

David Gibson dgibson@ssi.edc.org

Date: Tue, 7 Dec 1993 13:19:25 -0800 From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Molding new teachers X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312072013.AA11740@tolstoy.u.washington.edu>

I agree with the characterization of "the system" as including both the college of ed and other places our students learn and teach. But the idea that most beginning teachers can be "molded" by a college of ed or even a principal denies the student much of a role. In my experience, the students who are most likely to leave sharing my views of teaching & learning are those who pretty much felt that way when they came in. They just leave (I hope) with better strategies to use in pursuit of those goals. Those who have vastly different views of teaching and learning pretty successfully resist most attempts to change their minds. There are many in between too--and some who have mind-changing and view-broadening experiences while they are with us (often occurring in the field) that start them thinking in new directions.

But make no mistake, it is the students themselves who make sense of the experiences they have before, during, and after ed school. They may get a prod or a poke from us, but they "mold" themselves.

Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195

Date: Tue, 7 Dec 1993 18:02:50 -0500 From: JE01@SWTEXAS.BITNET Organization: Southwest Texas State University Subject: uh, say what?? To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Your message dated "Tue, 07 Dec 1993 12:31:50 -0600 (CST)" <01H676SGGTQ699DKAR@academia.swt.edu>

Uh Alan,

I think you have mistaken me for someone else - I don't train teachers, so the finger pointing analogy breaks down pretty far. I don't perceive that I was engaged in righteous indignation - to do that, you have to be thinking that you are in some way better than someone else - I don't, although you seem to for some reason.

Although pushing your buttons has been fun, you need to get a grip -- never assume that I support the singularity of state certification, I don't. Indeed, like Tom Green suggested, I think that schools of education need to find some distance from practice in order to improve it....and I strongly support alternative ways of welcoming a broader variety of people into school teaching.... your assumptions are way out of line.

My point in relating the story is that I found it frustrating to see the hiring process going awry, and to point out that these practices go on - not that they are good practices.

-Jill

Date: Tue, 7 Dec 1993 17:47:25 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Re: Student Assessment X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

On the subject of testing classes for education students, David Gibson said, "My reservations about the intensive course, are that the focus might be on validity, reliability, statistics, etc...all good and necessary to large system testing processes where individual learning is not the primary concern and the information being sought is directed to policy makers, law makers, textbook makers, test makers,...almost everyone except a learner for the purpose of improving performance."

I suppose it's fair to warn you that I love statistics. It was the first math course I had that really grabbed my attention. And years later I became a statistician for a while. I mention this not to claim any great expertise (wish I could) but to explain a certain bent, a way of thinking. I find statistics extremely helpful in understanding the real world. I have come to understand that a great many people don't feel the same way. How odd!

Having said that, how can you dismiss validity and reliability of testing as not being the concern of the teacher and the student? Validity simply means testing what you meant to test. I have seen this simple concept violated often. What did it mean when the kid flunked this math test? Is he poor at math, at reading word problems, at understanding test instructions? Perhaps he didn't answer a question because he didn't understand the situation in which the problem was embedded. What was the test actually testing?

And reliability just means that the scores will not be thrown off by irrelevant factors like whether the teacher is in a good mood when he scores the test or whether it's scored by the teacher or the aide.

At the time I took my course in testing I'd already been working as an aide for a while and had already begun struggling with these concepts on my own. I didn't know there was an academic discipline devoted to them. I just knew I needed this stuff to teach 7th graders. How could I tell whether they'd mastered the material well enough to move on? So many things I tried didn't work, but they were the things the teachers around me, and the teachers I'd had in school were all doing, and probably still are. The testing course was amazingly practical and useful. No, I didn't start producing nationally normed tests to administer in the junior high, and I didn't use all those statistical techniques in the junior high classroom, but the basic principles were invaluable.

I've also been startled by the lack of sophistication teachers show in using and interpreting commercial tests. They LOVE grade equivalent scores without at all grasping how often they are misusing them. For example, I worked with a teacher who was in charge of a bilingual program. Her high school students couldn't cope with the grade level English tests, so she gave them tests normed at the eighth grade level, then used grade equivalent scores to make placement decisions (instructionally relevant, I'd say).

David also mentioned that "fitting grades to a curve is not helping a student with instructionally relevant information." Well, I suppose it might be helpful sometimes to discover that you don't understand something when you thought you did. I remember the horror of being completely lost on my first computer programming assignment when I'd thought I'd understood all the lectures. It could also be helpful to know, if you're considering career choices or college choices, how you stand in relation to your competitors.

But I think the real issue here is why grades are assigned at all. In what way is ANY grade instructionally helpful, regardless of scoring method? Suppose students were given feedback galore and never graded. Would you be happy? Would it work for the school? Would the parents be satisfied? Would the students? (By the way, I would be satisfied, but I think I might be in a small minority, and I'm not concerned with making an entire school system work, just with educating myself and my daughter, a different proposition altogether.)

I don't want to be misinterpreted here as meaning that I think grading on a curve is a good idea. I don't. It's just that I've seen all sorts of absolute grading methods that didn't make sense either. I had a statistics teacher (who taught testing) decide retroactively to count the final exam only one half it's usual value in determining the course grade when the scores were too low, for example. An art teacher refused to use curves, thought they were horrible, insisted everyone meet high standards. Then at the end of the semester, he gave us all extra points on the grounds that the room had been too cold in the early mornings. This is objectivity?

The basic problem isn't curves per se, it's the scary willingness of teachers to construct tests, score them, and assign grades without really knowing what they're doing. Some good testing courses couldn't hurt.

Cindy Cotter

Date: Tue, 7 Dec 1993 21:58:19 MST From: Gene Glass <ATGVG@ASUACAD.BITNET> Subject: Re: Teaching and Learning To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Mon, 6 Dec 1993 11:29:48 U from <cotter_cindy@MSSMTP.LACOE.EDU>

On Mon, 6 Dec 1993 11:29:48 U Cotter_Cindy said:
****I'd also like to ask Gene Glass whether he would consider
****it healthy for schools of education and public schools to
**** be in closer alignment if that meant the public schools
****were moving closer to the ideals of the university.

I suspect that in some respect I deserve this question (which has its own answer buried shallow within it) since I parodied the "field's" demands on the university with words like "Send us teachers who can make our kids score high on the state test, enforce our discipline program and cover all the pages in the adopted text." My implying that schools ask that or only that of the university preservice training program is unfair. Some are capable of asking for more enlightenment than the teachers colleges are capable of producing. Cindy Cotter wonders if I think that the field should be moving in the direction pointed by the college. Not always; not in every way. Education looks one way when viewed from the college; it looks different from the firing line. Both views are too complex to characterize as simply right or wrong. They need to rub against each other.

GENE V GLASS Glass@ASU.BITNET College of Education Glass@ASU.EDU Arizona State University 602-965-2692 Tempe, AZ 85287-2411

Date: Tue, 7 Dec 1993 22:13:22 MST From: Gene Glass <ATGVG@ASUACAD.BITNET> Subject: Re: Student Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 7 Dec 1993 17:47:25 U from <cotter_cindy@MSSMTP.LACOE.EDU>

I guess this is not my day to agree with Cindy Cotter.

I think that:

a) schools have far too many tests of all kinds

b) teachers don't need courses on testing (surely elem teachers don't and sec teachers hardly do)

c) reliability is irrelevant (nobody really knows what the heck it is anyway)

d) validity is not what the psychometricians think it is

e) ALL grading is curve grading; it could be no other way Corollary: "mastery" is a chimera; it doesn't exist

f) schools don't need grades (A, B, C etc)

g) teachers and students would be better off if teachers wrote narrative evaluations instead of giving grades

h) school administrators and doctors give so many tests for the same reason--to protect against law suits.

GENE V GLASS Glass@ASU.BITNET College of Education Glass@ASU.EDU Arizona State University 602-965-2692 Tempe, AZ 85287-2411

Date: Tue, 7 Dec 1993 23:18:25 CST From: "<Benjamin Levin>" <levin@CC.UMANITOBA.CA> Subject: Goals and activities To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Date: Tue, 7 Dec 1993 22:43:05 MST From: Walter Shepherd <ASWES@ASUACAD.BITNET> Subject: Re: On Asking Questions To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 7 Dec 1993 10:29:03 U from <cotter_cindy@MSSMTP.LACOE.EDU>

In response to Cindy Cotter's post of Tue, 7 Dec 1993, (appended):

It seems to me that teachers are shocked because, all rhetoric aside, they cling tenaciously to faulty learning theory. If they would relinquish the notion that learning is something that they create--or even manage--in the child's brain, they could see that the behavior that Cindy describes can be explained. Let's do it this way; let's say that meaning is created by the child as he interacts with the world around him and he learns it if he finds a hook to hang it on. Notice that I said nothing about accuracy or about how well his meaning or learning correspond to objective reality. (I think this even relates to the problem that the student in Rick's story had last week. It certainly relates to what I understand in first-language acquisition; children begin at a remarkably young age to create a workable grammar of their language--a process we are only beginning to understand.) The problem for the teacher in this situation is to assure connectivity--what he is teaching must connect with something that the child already understands; and he must find the means to discover what the child did with it--how he actually made sense out of it on his own terms.

As for the question about the age of the Shepherd (Who wants to know!?!), the children who answered may simply have believed that the teacher expected an answer; they aren't expected to really think about those things. (What is this some kind of trick question? Ok, 39!)

Ev (Shepherd)

Date: Wed, 8 Dec 1993 10:56:36 -0500 From: Kevin Drumm <drummk@POLARIS.NOVA.EDU> Subject: Re: Power and influence X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312080523.AA00166@polaris.nova.edu>

In response to Benjamin Levin's question:

Hmmmmm! (As Arsenio would say.) What would be the implications if in schools and colleges the least powerful all of a sudden had as much power as the most powerful...?

1) My biggest fear would be more consumerism--and not due to students' demands for watered down curricula and relevant course work, but due to faculty and administrator's lack of conviction and courage to advocate for rigor at the risk of the newly empowered students canning their butts.

2) Tenure would have to go or we could never have a balance of power. Graduated. multi-years contracts would replace this archaic system.

3) We would find out that students are as demanding and as discriminating as WE hoped they would be but under the current system expect they won't be.

4) Classes would be MUCH more interesting with all those teachers teaching with one foot in a wastebasket.

5) Administrators would be forced to pay attention to services, thus TQM initiatives would be entirely unnecessary.

6) Teachers would collaborate with their students in the first class meeting and adjust SOME of their teaching to the desires of the students in the class. This would be based on serious negotiations via an agreed upon format.

7) Classes would be offered 24 hours a day with many more in self-directed formats.

8) Undergraduate education would look more like what Jeffrey Alexander (p. B3) describes in last week's (Dec. 1) "Chronicle". (BTW, a suggestion I made some time ago on this List. Maybe I should join Louis and try to publish some of my stuff since some of it seems to show up in The Chronicle a few weeks later by another author...)

9) We wouldn't have grades any longer, as Gene suggests we should not now.

10) All of us would be in fear for our jobs like everybody else in the country and thus we would have a much more competitive and quality system.

I'll stop here, but I could go on and on. This is someone's cue to Perrotize this discussion and interject the fear tactics demonstrating why we should not consider balancing the power scales in schools and colleges.

Cheers,

Kevin Drumm NOVA University 305-424-5758 drummk@Polaris.NOVA.edu

Date: Wed, 8 Dec 1993 11:32:30 -0500 From: Kevin Drumm <drummk@POLARIS.NOVA.EDU> Subject: Re: Student Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312081341.AA06683@polaris.nova.edu>

I disagree with one of Josue's statements, that is "that every minute spent testing is a minute not spent teaching." A good test teaches, and teaches a lot. I would agree with Josue if he inserted "badly" or "poorly" after "testing."

Cheers,

Kevin Drumm NOVA University 305-424-5758 drummk@Polaris.NOVA.edu

Date: Wed, 8 Dec 1993 13:16:10 EST From: "Covaleskie, John" <FACV@NMUMUS.BITNET> Subject: Re[2]: Power and influence To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: In reply to your message of WED 08 DEC 1993 00:56:36 EST

Just a question: if teaching is a legitimate activity to be going on in school, and if some teachers know more about some things of importance than the students do, is total equality the best system? Is it possible (however unlikely and/or rare) that teachers actually do know that the best way to proceed is to do X, even if the students do not wish to do so? If so, equality might actually thwart the purpose of schools.

It seems at least conceptually possible to say that students should have the rights appropriate to the responsibilities they incur as students, while teachers should have those rights appropriate to fulfilling their responsibilities as teachers. In many, though not necessarily all, respects, these can be expected to overlap.

* * * John F. Covaleskie * * Assistant Professor of Education 801 Summit Street * * 113 Magers Hall Apt 7 * * Northern Michigan University Marquette, MI 49855 * * Marquette, MI 49855 * * 906/227-2768 906/227-5742 * * * * FACV@NMUMUS.BITNET

Date: Wed, 8 Dec 1993 09:29:49 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Re: On Student Assessment X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

In response to Gene Glass who guessed today was not his day to agree with Cindy Cotter. I'd said teachers should learn more about testing, he said no, testing is overdone in the schools and grading is a bad idea.

Well, we're not really that far off. My argument in favor of testing classes for teachers was based on the assumption that they will continue to test. And I think they will. If they do, and if they make decisions about how to teach and how to place students on the basis of test results, I'd like to see them do a better job of it.

In that same post I said I was not in favor of grading but suspected I was in a minority. I must be at least in public school circles, because they all grade. But remember me? I'm the one who argued against public schools in the first place. I see testing and grading as two of the mechanisms more or less inherent in a large, government-run, bureaucratic, over-controlled -- but Ev Shepard will tell me I'm a jingo if I go on.

So, I don't like testing any more, and perhaps even less, than most. I like it even less when done badly, in a hurtful way.

Cindy Cotter

Date: Wed, 8 Dec 1993 12:15:47 PDT From: Mark Fetler <MFETLER%CC1@TS9.TEALE.CA.GOV> Subject: Ritual Assessment X-To: edpolyan%asuacad.bitnet@cunyvm.cuny.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Perhaps it is commonly assumed that assessment is a rational activity undertaken for rational purposes -- something along the lines of gathering information in order to make better decisions. Gene points out another motive, that of defense against lawsuits, which boils down to schools defending their interests against political opponents. Benjamin Levin raises the specter of organizational behavior and notes that much of what goes on in many organizations, including schools, contributes little to the "bottom line," whatever that might be.

Expanding on that point, I suggest that much assessment has primarily to do with ritual purposes and only secondarily with rational purposes. People just expect schools to be places where people are tested. Its part of the culture. You're tested to get in. Progress is assessed. And at the end you are certified to be done. Sure, sometimes the information is actually used to make placement or instructional decisions. And a few teachers do make a serious point about rational assessment procedures. But the main thing is that you have been tested. Its a rite, a rite of acceptance, passage, or certification of status. Society expects testing, and schools provide testing. Many people think the whole point of school is to pass the test. How perverse! What happened to teaching and learning?

Our emphasis on assessment may be just an historical quirk. Certainly other cultures of education are thinkable. Portfolio/performance approaches represent an alternative, as does a one-on-one tutorial approach, or the methods of the Greek sophists. However, to note the ritual or political purposes of assessment only diminishes its importance to us if you insist on judging schools literally on purely rational criteria. Our culture demands assessment as a trapping of schools. Reliability and validity studies themselves function as ceremonies to shore up the credibility of assessment. Not infrequently have I seen assessments adopted following such studies, even though the statistics looked pretty poor. Its not the first time that politics and culture have beaten rationality. ....................................................................... Mark E. Fetler, Ph.D. - California Community Colleges - 1107 Ninth St. Sacramento, CA 95814 - (916) 327-5910 - mfetler%cc1

Date: Wed, 8 Dec 1993 15:51:07 -0500 From: Josue Gonzalez <jg124@COLUMBIA.EDU> Subject: Re: Student Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <199312081637.AA02762@mailhub.cc.columbia.edu>

On Wed, 8 Dec 1993, Kevin Drumm wrote:

**** I disagree with one of Josue's statements, that is "that every
**** minute spent testing is a minute not spent teaching." A good test
**** teaches, and teaches a lot. I would agree with Josue if he inserted
**** "badly" or "poorly" after "testing."
**** No, no, no. I was talking about testing with a big T, as in The Testing Industry, as in ETS, SAT's, IQ tests, etc. There's just too much of that stuff going on and we don't need more.

jg124@columbia.edu

Date: Wed, 8 Dec 1993 16:03:08 EST From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: Ritual Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of 12/08/93 at 12:15:47 from MFETLER%CC1@TS9.TEALE.CA.GOV

The rituals of assessment do seem to support a certain fetishism of rational decision-making whether or not they actually support rational decisions. Perhaps more importantly, they serve as mechanisms of surveillance and sorting--mechanisms needed by a pervasive (if not fully totalizing) capitalist state. It is noteworthy that many supposed educational innovations merely aim to perform this same function better than the prevailing practice. "Authentic" assessment is considered better than multiple choice testing and grading on the curve because it gives more and better information about individuals. The question of what we do with that information and who benefits from its accumulation are seldom asked.

--Aimee Howley

Date: Wed, 8 Dec 1993 14:25:45 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: testing, 1, 2, 3, testing X-To: Multiple recipients of list EDPOLYAN <EDPOLYAN@asuvm.inre.asu.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I have lived in places where teachers were required to take testing courses and in places where they weren't. I much prefer the former. At least in those places, when I complained about a teacher's testing procedures, they seemed to have an understanding that it was possible to do better. In the others, all I ever encountered was an arrogant assertion that "I know my material and it is the students' responsibility to show me they know it too," a complete and ill-informed reliance on essay questions, and a refusal to acknowledge that there might be some merit in considering how well a question was worded.

I don't expect to persuade Gene, but I felt it essential to provide a little opposition on this point.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Wed, 8 Dec 1993 12:26:58 -0500 From: "Thomas J. Pugh" <tjpugh@MAILBOX.SYR.EDU> Subject: teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

If there is a point to this story, it might be something along these lines. The public schools are often an embarrassingly accurate reflection of the larger society and the community in which they are imbedded. The anti-intellectualism, ideological feuding, petty bickering, political posturing, and resource scarcities that surround us are part of the environment in which "our" teachers teach. There are roughly 2.5 million teachers working in the U.S.; there are roughly 80,000 schools. There is a tremendous diversity among those schools and those teachers. There are chaotic schools and great teachers. There are wonderful schools and lousy teachers. Single causal theories about improving education or teachers are doomed to fail.

Tom Mauhs-Pugh Cultural Foundations of Education Syracuse University

TJPUGH@MAILBOX.SYR.EDU

Date: Thu, 9 Dec 1993 07:51:12 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: learning to teach To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 7 Dec 1993 12:14:47 MST from <hunter@ACS.UCALGARY.CA>

Bill Hunter:
**** It is simply not the case that all the white hats
****are on teacher educators. We have our share of misguided folks
****and retrograde thinkers--there is no need to create the conflict
****Rick suggested--it is there and very real, though not often in a
****single course. I know it is there; I want this conflict not to be "created" but to be utilized! My recommendations were to create situations that used this conflict to make ed students HAVE to reflect on the choices and make decisions about teaching they could support and defend. As it stands now, ed students who are not prone to reflection can work around each teacher one at a time. They would not find this so easy, if even possible, if conflicting teachers worked together to make the students THINK.

And schools have their share of far-sighted
****progressive thinkers and innovative teachers whose practices
****often lead the way for research and teacher education. Nobody in
****this game has a monopoly on wisdom or virtue--an assertion that
****is supported daily in this forum where a home schooler and a
****software developer and a philosophy teacher and graduate students
****and iconoclasts of every shape and kind contribute to the richness
****of the dialogue.

But what we all have in common is that we think about teaching and education -- its purposes, its possibilities, its actual consequences, how to improve it, what counts as improvement, what the flaws are, what techniques seem to work and why and when, etc., etc., etc. I think that if ed students and working teachers could simply be coaxed into thinking and reflecting more about their practices and the results they are actually getting (or not getting), it would make all the ideas and views that are taught in ed schools, and the practices that are urged by colleagues and administrators, better able to be appreciated, evaluated, discussed, and accepted or ably argued against. And that would help improve teaching. My suggestion about USING the conflicts present in ed schools was one way of possibly coaxing such "thinking". Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 9 Dec 1993 08:12:29 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Molding new teachers To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 7 Dec 1993 13:19:25 -0800 from <sunolen@U.WASHINGTON.EDU>

Susan Nolen:
****
****But make no mistake, it is the students themselves who make sense of the
****experiences they have before, during, and after ed school. They may get
****a prod or a poke from us, but they "mold" themselves.
****

That is why the experiences, prods, and pokes ed schools provide is so important. That is why I suggested utilizing conflicting views in a single course to prod students into thinking more than many of them do now. I suspect most don't think beyond how to "do what this teacher wants well enough to get the grade." I am not talking about molding students into a particular teacher, but to mold them into thinking teachers with skills and tools to improve their teaching by self-reflections and dialogue with others as they go along and gain classroom experience. Rick

RICK GARLIKOV (DEMS042@UAB.DPO.UAB.EDU)

Date: Thu, 9 Dec 1993 10:12:00 EST From: Jack Letarte <JCLETARTE@TAYLORU.EDU> Subject: Re: testing, 1, 2, 3, testing To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I agree with Bill Hunter's recent post on the importance of teachers learning about tests - if for no other reason, just so that they can be a "first line of defense". I was fortunate to have an outstanding first course in test and measurement during my first year in grad school. Since then, I have always thought of testing as something which can take many forms, and which tells me "how am I doing?" The problem, of course comes when it's time to turn in grades - what can we do if we don't test? Don't students expect tests, etc., etc.? Someone else on the list commented that testing is a part of the ritual. I'd agree with that also; we make progress when we are able to show students how many different outcomes of a course we are interested in, and how few of those outcomes can be represented on a machine-scored multiple choice test. Some knowledge of testing can help a teacher convince others (students, parents, supervisors) that the "fairness" of other assessment methods may be equal to the "fairness" of the multiple choice test, especially when we are clear about the expected outcomes of the course.

Jack Letarte Taylor University Upland, Indiana 46989 317-998-5153 Internet: jcletarte@tayloru.edu

Date: Thu, 9 Dec 1993 19:08:01 CST Comments: Resent-From: Rick Garlikov <DEMS042@UABDPO> Comments: Originally-From: Susan Nolen <sunolen@u.washington.edu> From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Molding new teachers X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312091421.AA10999@tolstoy.u.washington.edu>

Sue Nolen sent me this message which I thought was interesting. When I asked, she gave me permission to forward it to the list.

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

----------------------------Original message----------------------------

I have some problems with your way of helping students deal with the controversy, \in ed schools between different theories taught by different profs-RG| but I surely agree with you that we need to do this. Largely it is ducked, I think. A way we have tried here with some success is to team instructors that have different perspectives, and have discussions in the classroom with our students in which the profs also take different stances. (We teamed a C&I, a Spec. Ed., and an Ed Psych prof with a classroom teacher in a middle school seminar for prospective teachers and had some very interesting debates and discussions.)

Another idea a group of us just came up with is to have students design a basic approach to organizing their classroom, using theories of development and individual difference, but within the school they are doing field work in. Sort of: imagine you are in an interview with teachers and the principal of your current school. They ask you to describe how you would set up and manage your classroom, if hired. Given what you know of theory and what you know of the teachers and principals at your school, how would you describe your approach? How can you convince them to accept your ideas? Or something like that. We're still working on it.

Hasn't this been a fun discussion? SN

Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195

Date: Thu, 9 Dec 1993 19:22:54 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teachers To: Multiple recipientsof list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Wed, 8 Dec 1993 12:26:58 -0500 from <tjpugh@MAILBOX.SYR.EDU>

Tom Mauhs-Pugh recounts a horror story that surpasses most of our worst nightmares about schools, even though most people see some aspects of his experience in schools from time to time. But to have lived all of them at once..... From this example he says "Single causal theories about improving education or teachers are doomed to failure." I suspect that is true, and that his example demonstrates that pretty well. However, I think it is fair to say that our discussions about teacher certification and testing have not been efforts to determine single causal theories about educational problems. I think we are pointing out a number of specific problems, some of which are exemplified in his account. I don't remember anyone saying that if any one thing was "fixed" all would be well with schools.

I may have come closest to doing that, but I was talking about a particular, fairly well-to-do district where materials, supplies, "good" students from relatively advantaged backgrounds, etc. abounded, and where principals tend to be non-interfering (to a fault in some cases), and where teachers still taught poorly in ways that seemed to miss some real "basics". I pushed in this discussion for why THEY couldn't teach -- weren't they taught certain things about teaching? Is anybody? Etc. From that modest beginning, started if I remember, by Aimee Howley, the discussion has grown fairly logically and well-structured, I think for a discussion of this sort, into the diagnosis of a number of related ills.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Thu, 9 Dec 1993 19:55:27 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: teaching without testing, sort of X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I quit giving tests many years ago, and I'll describe here why, and how, I can teach without them. It may focus some of your ideas about testing for this particular case.

I teach philosophy. I believe that what I am going to say can be generalized for courses that primarily involve 'theorizing' of all sorts and may be irrelevant for courses that require straightforward memorization and recall of information -- and where it is important to know who has learned that information at least for the duration of the examinations. I was influence by a talk about testing from a professor at Michigan's Center for Research on Learning and Teaching (if I remember the center's name properly) in which he said essentially that 'you need to design tests that will show what students really know about a subject'. I then found that that was just about impossible with regard to doing philosophy for a number of reasons. And I found it was unnecessary in order to assess student ability; and that in some cases it even gave a wrong assessment. So I have given up tests, except in one kind of situation, and have used other ways to assess student abilities, not in order to grade them, but in order to try to better teach them.

Grading is itself a problem, since it is meaningless to integrate the assessment of a large number of skills into one single grade. For instance, in an English class, suppose a student knows grammar quite well and is quite good at writing essays for which he has any information at all; but suppose he does not spell very well or cannot write fiction very well. What single grade is fair to give the student? Suppose another student has a wonderfully creative imagination and is very articulate in his choice of words, but uses grammar that is most charitably perhaps avant garde. What grade shows this? Suppose in a philosophy course, a student is brilliant in one of the topics, and rather lame in another topic that makes little sense or holds no interest for him/her? Does a "B" or a "C" show that? Of course not.

If you only lecture, and you only give tests, I think, besides its being ineffective, you rob your students and yourself of some interesting human verbal exchanges that can be more meaningful to the students and that can be fun for all in a way that does encourage thinking. I know that when students challenge me, I learn more by having to figure out answers for them. One student and I argued to an impasse one day and it was not until a year later that I thought of something I thought would clinch my view for him. So I tracked him down and called him to tell him. I think it is important for students to see you are more interested in ideas than in simply testing and grading them. The students I had this past term all were black and we were discussing racism and "being black" in America. We were into a pretty intense discussion with me challenging some of their conclusions, when one of them said (and the others agreed) that "one of the problems with being black is that you are always outnumbered by white people if you are around any white people at all. You NEVER see a white person outnumbered by black people, so that the white person is the 'odd' person." I looked at them for a minute and then looked at the backs of my hands, held out my hands and said "What do you call this class?" Their mouths dropped, and one of them said "But you're not white...you're Rick". I don't think they would have felt that way if I had only lectured and tested. Then I would have been very white. And I don't think they would have learned as much about ethics.
Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Fri, 10 Dec 1993 09:27:40 MST Reply-To: Sender: From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: teaching without testing, sort of To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312100445.AA33936@acs1.acs.ucalgary.ca>; from "Rick Garlikov"

I would love to engage in a lengthy dialogue with Rick Garlikov on any topic and especially on the merits/problems of testing. However, for the moment, I'll settle for this: Rick--you say you don't do any testing; I say you are testing constantly. At the very least, you are measuring constantly. I share your concerns (and have others) about grades, but apparently you and I are both constrained to give them.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Fri, 10 Dec 1993 15:45:55 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teaching without testing, sort of To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Fri, 10 Dec 1993 09:27:40 MST from <hunter@ACS.UCALGARY.CA>

Bill Hunter:
****Rick -- you say you
****don't do any testing; I say you are testing constantly. At the
****very least, you are measuring constantly.

TRUE; hence, the "sort of" in the topic title "teaching without testing, sort of". It just is not testing in the usual sense for the usual reasons and in the usual way. Good observation.

Rick
Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Fri, 10 Dec 1993 19:36:03 -500 Reply-To: Sender: From: David Gibson <dgibson@SSI.EDC.ORG> Subject: Re: Student Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.bitnet@arizvm1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312080017.aa11882@ssi.edc.org>

In a recent series of exchanges stemming from Cindy Cotter's request that Susan Nolen and I share our views about assessment, the following discussion ensued. I believe my comment was that the content of a course on testing which might cover concepts of validity and reliability might not be so relevant to a classroom teacher. To which Cindy Cotter asked"...how can you dismiss validity and reliability of testing as not being the concern of the teacher and the student?

Validity simply means testing what you meant to test." I did not mean to imply that a classroom teacher would have no use for these concepts at all, but only those concepts as applied in large scale testing as we have known it up to this point. In addition, the terms might mean one thing to a classroom teacher and something to a large scale test maker. Gene Glass wrote "validity is not what the psychometricians think it is" I probably agree. I at least would like to expand the definition to: How well a series of assessments (more broadly than a test or set of tests) measures what students are expected to have learned. Cindy Cotter "...And reliability just means that the scores will not be thrown off by irrelevant factors like whether the teacher is in a good mood when he scores the test or whether it's scored by the teacher or the aide." In Vermont's large scale program of writing and math portfolios, "reliability" has been difficult to define and achieve. It seems that when the issue of scoring assessment changes from "answers" being "right or wrong" to "exhibiting" various degrees of agreed up on qualities, teachers cannot reliably put a work into the same bin as other teachers...even when there are only a few criteria.

The statewide experiment and adjustment is continuing. In the first year, changing the scoring process during "moderation" was supposed to improve things for this year. Next year, crisper definitions of the bins may help improve the process further. In the state's rush to get large scale reliability, some teachers have lost the sense that the project is about classroom improvement, because the "testing" issues seem to drive some of the state's processes of training. Others just see those folks as whiners who don't understand the importance of proving that a portfolio assessment system can have the same testing statistics as the methods it is seeking to replace. I'm not sure that we WANT it to. Assessment as it is practiced in the ArtsPROPEL project in Pittsburgh apparently has no raters other than the teacher and student, and differences in ratings are considered opportunities for reflection and teaching. I suppose one could say that at a statewide level, the same thing is happening, the absence of sufficient reliability is an opportunity for the entire profession to think about what it is doing. Gene Glass "reliability is irrelevant (nobody really knows what the heck it is anyway)." I'll pass along your comment to the Commissioner. If this could only be entirely true at the statewide level, I know a few people who could take a vacation. It may indeed be true in the classroom. I think the entire role of these terms changes when we think of the scoring of assessments as highly specific indications for next steps in instruction, including self-instruction.

That is how I think assessment looks when it is fully embedded in teaching-as-coaching. Reliability might then be thought of as a measure of how useful a feedback/scoring was to continued instruction. In large scale assessment, where the criteria is clumped into a very few bins, reliability would be expected to be low, because the feedback to the learner is relatively uninformative of "next steps." In other words, by the time a state system forms a highly reliable performance-based assessment, even a relatively uninformed person might be able to score pieces of work as well as an expert. We will have achieved reliability, but what will be the qualities of that information?

Cindy Cotter: "Well, I suppose [fitting grades to a curve] might be helpful sometimes to discover that you don't understand something when you thought you did. I remember the horror of being completely lost on my first computer programming assignment when I'd thought I'd understood all the lectures. It could also be helpful to know, if you're considering career choices or college choices, how you stand in relation to your competitors."

The interesting issue in your second comment, Cindy, is that you may very well have understood all the lectures, and even scored well on valid and reliable tests, and STILL not had a relevant, authentic, learning experience that prepared you for programming. I think this is similar to my critical reflection of the course on testing. Future teachers may take the courses, and even pay attention, like them and do well on their tests...and still be completely lost in their first real world teaching/assessing assignment. On your second point, I am personally not interested in testing or assessing for social/competitive comparison purposes. I suppose there may even be a link to motivation (albeit a selfish, "get ahead" goal) for school achievement...but I think of learning as something apart from school achievement in that sense, and I would rather have assessment serve the purposes I have in mind, simply getting better at knowing and doing things.

Gene Glass: "ALL grading is curve grading; it could be no other way Corollary: "mastery" is a chimera; it doesn't exist."

I'd be interested in Gene's reasons why absolute grading schemes also curve things. I fully agree that summative scores don't say much and do collapse most truly useful information. Perhaps that is why I think statewide performance assessment reliability mentioned above may also be gained with a loss of information. The "mastery" ideas seem to be like the benchmarks of performance rating systems. They represent steps toward expertise, not "mastery" in any global sense. And if learners are like the future teachers we've discussing elsewhere on EDPOLYAN, we should expect that they are not "done" when they leave the K-12 system either. So, in that sense "mastery" does not exist. Perhaps we could talk about a system of standards however, that contains a vision of mastery which is never fully reached. The image itself serves as the stars did to a sea going traveler of past days. In Vermont discussions, we've been going around on whether the word "standards" means something that can be reached or not. Perhaps they are like Gene's "mastery."

Bill Hunter said, "I not only share Cindy's assumption that teachers are likely to continue to use tests and should therefore know some things about them, I also think the potential abuses of tests are so enormous that it is essential that teachers be the first line of (critical) defense against testing abuses....To some extent, one might well argue that the continued use of grade equivalent scores is prima facie evidence of the failure of courses in educational testing, but I don't think that battle will be won by taking teachers off the battlefield." I absolutely agree, and meant to originally recommend a rethinking of HOW teacher s learn about testing, not WHETHER they do learn something or not. Aimee Howley then posted that ""Authentic" assessment is considered better than multiple choice testing and grading on the curve because it gives more and better information about individuals. The question of what we do with that information and who benefits from its accumulation are seldom asked." I think Aimee has a key idea of the discussion so far. No single system of assessment seems likely to serve all of the purposes of education. And no individual system of assessment will serve education at all if we don't know what we would LIKE it to do, and what it is SUPPOSED to do. I would just like to vote for putting most of our energy into systems that improve learning and learners first...and make teaching and the other parts of the education system accountable on that basis.

David Gibson dgibson@ssi.edc.org

Date: Sat, 11 Dec 1993 11:25:49 EST Reply-To: Sender: From: Aimee Howley <U176C@WVNVM.BITNET> Subject: Re: Student Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of 12/10/93 at 19:36:03 from dgibson@SSI.EDC.ORG

David Gibson generously read my comments on assessment as reflecting something like hope; I intended them (much less generously) as conveying something more like despair. Assessment is a technology whose purposes determine the form it takes. If the purpose of education really were to help children learn a lot of things that we and they thought were important, assessment would be indistinguishable from the dialogue and demonstration that constitutes teaching. The extent to which it is construed apart from and above the occasions of learning and teaching reflects the extent to which it is a technology directed at a much different--and much less worthy-- purpose. My despair comes from hearing in much of the public discussion about assessment themes and variations of this less worthy-- indeed fully suspect--purpose. --Aimee Howley

Date: Sun, 12 Dec 1993 11:03:38 MST Reply-To: Sender: From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: teaching without testing, sort of To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312102153.AA49807@acs1.acs.ucalgary.ca>; from "Rick Garlikov"

Rick G. responded to my comment:
****  Rick -- you say you
****  don't do any testing; I say you are testing constantly. At the
****  very least, you are measuring constantly. by answering:
**** "TRUE; hence, the 'sort of' in the topic title 'teaching without
**** testing, sort of'. It just is not testing in the usual sense for
**** the usual reasons and in the usual way. Good observation."

Good. Let's see if we can advance this a bit. In the testing embodied in your ongoing give-and-take with students, what are you measuring against? You talked about the students' levels of understanding, but can you say what constitutes evidence of understanding (as opposed to, say, parroting). Since you do indeed give marks that, I presume, differ from one student to another, do you have reason for having confidence that the better marks given to one student over another indeed represent "better" levels of understanding or would it be "more" improvement from some initial state? If the initial states vary, as I am sure they must, do you have some rationale that allows you to feel comfortable with the fact that students who start high and gain little may end up with higher marks (under the "better level" model) or with the fact that students who start low, gain large, yet finish with lesser understandings than some others may indeed receive better marks (under the "more improvement" model)? I expect you have good answers to these questions.

If not, you could probably not ethically continue to teach as you do (let me reiterate that it sounds as if you are a dynamic teacher who really is a catalyst for growth in reasoning for your students). My point is not to get you to defend your manner of teaching and testing, but rather to call attention to the kinds of questions that more formalized approaches to testing attempt to raise. I do not expect that a testing course will cause teachers to become test construction geniuses (I doubt that they exist); I expect only that it will provide them with tools for thinking critically about how they assess student progress. Concepts like reliability, validity, utility, etc., etc., provide some common ground for discussing assessment issues.

If Gene G. is arguing that the rigid application of measures of reliability and validity has little merit in the classroom, I am in full agreement; however, I believe strongly that there is merit in having prospective teachers think about how to make judgements about the kinds of assessments that are appropriate for their aims and of what qualities are desirable in classroom measurements. My earlier reference to item analysis is a case in point--you engage in assessing the quality of your questions as you hear them answered, you may rephrase them or abandon them altogether in favor of new questions. In a context in which a teacher uses a paper and pencil measure, item analysis may help them to make these same judgements (based on responses from many students rather than one--a characteristic that has both virtues and faults).

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Sun, 12 Dec 1993 14:58:22 -800 Reply-To: Sender: From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: teaching without testing, sort of X-To: X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.bitnet> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312121751.AA13790@fallbrook.csusm.edu>

As a newcomer to the list (and the Net, for that matter) I am delighted to find this discussion in progress and hope that the following comments do not re-plow old ground. Unfortunately I missed the initial remarks.

If a recommendation was made that teachers receive formal training in testing, I agree and would extend the recommendation to include all college graduates (and all state and federal legislators, if that were possible). Unfortunately our expectations of tests tend to derive from our personal experiences as test subjects and from our attitudes toward the claims of test publishers and wishful thinkers who promote tests as quick fixes for all sorts of threatened or perceived lapses in standards.

We would do better to draw our concept of testing from the realm of sports where a public demand for fairness has led to more reasonable con*tests*: Individuals or teams typically have repeated opportunities to show their stuff under uniform (in testing we say standardized) rules and criteria for winning (or passing to the next level). When judgment is involved, the really serious contests use a panel of judges to counterbalance inevitable personal bias. Furthermore, the athletes know the expectations and criteria for scoring in advance and have plenty of opportunities for practice. Athletic contests, then, have something in common with "authentic assessments," where subjects are evaluated upon their demonstration of important knowledge or performance at meaningful tasks. The limitations of traditional standardized multiple choice tests are intuitively clear if we think about using a multiple choice test ABOUT football to determine who will go the superbowl or who will be selected for a team. The limitations of free form assessment are also clear if we think about attempting to judge an athletic contest without clear rules about playing, scoring, or winning. Efforts to avoid the pitfalls of multiple choice tests will simply result in other, perhaps more insidious, injustices if alternative assessments are not carefully structured. Standardized paper and pencil tests at least have the value of recorded questions and objective rules for scoring. The move to authentic assessment should continue to allow scrutiny and reflection upon what is expected and how performance is evaluated. Your comments would be most welcome.

Joan Gipson-Fredin joangf@fallbrook.csusm.edu

Date: Sun, 12 Dec 1993 18:55:33 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teaching without testing, sort of To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Sun, 12 Dec 1993 11:03:38 MST from <hunter@ACS.UCALGARY.CA>

Bill Hunter:
****Good. Let's see if we can advance this a bit. In the testing
****embodied in your ongoing give-and-take with students, what are
****you measuring against?

What I am "measuring against" for grading is how clear, precise, and reasonable their comments are, and --as the term goes on-- their not saying things we have already shown inadequate, unless they have additional support we have not considered. But grading is not my normal consideration in any of this. I am trying to get them to derive for themselves much of the history of ethics by asking them questions and then pursuing what they say -- and having them pursue what each other says, and what I say. And I particularly am trying to get them to be able to be much more precise, specific, and reflective as we go.

**** You talked about the students' levels of
****understanding, but can you say what constitutes evidence of
****understanding (as opposed to, say, parroting).

As in real life, this is never a sure thing, but the point is to ask questions that require answers that are DERIVED from material that is taught, rather than in asking questions whose specific answers have been given already. This rules out parroting though it does not rule out a lucky guess or an answer that comes from something else than what I have "taught" or drawn out of them. For example, in teaching young children how to do representational "place-value" "regrouping" subtraction using poker chips, a question that tests for understanding is to ask them to take one red chip (which represents '100') and subtract '37' from it -- when you have not worked with hundreds before and when you have not done anything with 2 zeroes in the number from which you are subtracting. If they can extrapolate from the 10's and 1's work we have done, I take that as a sign of understanding UNTIL AND UNLESS some other evidence arises to show me otherwise. In philosophy classes I try to find questions or comments that evoke responses which require deductions from previous material. And I particularly relish trying to trick them into making faulty deductions, to see if they can avoid the tricks.

For example, I explain to them during the course why the normal understanding of the "Golden Rule" is a faulty ethical principle. (In short, other people don't always want or need what you want; and sometimes you and they may both want things which are bad or wrong anyway.) To see how well they understand this after I have given a series of memorable examples, I don't just ask them whether the Golden Rule is a good rule or not, because they will say it is not and give me back the examples. Instead I say something like "I know the Golden Rule does not work in some cases; but in what cases DOES it work as a good principle for deciding what is the right way to treat someone else?" Well, the answer is NEVER; but they don't usually see that because they are trying to think of cases where other people want what you want and where what you each want is a good (right) thing. But this latter part begs the question, so the Golden Rule always fails to be the guide, by itself, to what is right. If they can explain this, and why, then they have not parroted but have had to understand.

Or I can give them cases we have not discussed and see how they can use what we have discussed to elucidate these cases. I doubt I ever directly ask anything we have discussed in the way we discussed it. So there is nothing for them to parrot or merely repeat. Not answering satisfactorily does not show lack of understanding, and answering successfully does not guarantee understanding. But answering successfully gives evidence that 'confirms' understanding in the scientific sense of 'confirms'. (I can explain that if I have to, but I assume most of you have some familiarity with the notion of scientific confirmation and the difference between it and 'proof'. If not, just tell me.)

****Since you do
****indeed give marks that, I presume, differ from one student to
****another, do you have reason for having confidence that the better
****marks given to one student over another indeed represent "better"
****levels of understanding or would it be "more" improvement from
****some initial state? If the initial states vary, as I am sure they must,

Actually they vary very little, if at all. I have taught senior citizens and fifth graders, blacks, whites, wealthy, and poor, and sometimes people with very good educations outside of philosophy; and almost all start out making the same kinds of errors on the same issues, though the specific answers tend to vary a bit. The problem I have with grades is not distinguishing between students' abilities, but trying to figure out how to correlate different abilities with some particular single letter grade. I don't know how to do that, not with only three real grades to choose from (nobody who shows up and who gives this any effort at all is likely to fail; and I have no earthly idea what a "D" can possibly mean in a course. I once saw a student paper which a teaching assistant had marked with a "D+". I figured the "+" was sarcastically gratuitous; what the devil could a D+ mean!). I give A's to students who demonstrate generally the ability to make conceptual and logical distinctions and to be clear, precise, and logical about the subject, and who show a good grasp of the material. I give C's to those who have some understanding but who have not learned as much as I think they should have -- for reasons I give them -- in all these areas. And I give B's to those in between somewhere. My B range tends to be widest and I sometimes have students in it I wish I could separate somehow grade-wise, but not at the expense of giving some of them C's -- which I consider a fairly low grade in a course like this, where no one is a philosophy major and many are only taking it either out of curiosity or as a way to meet some sort of humanities requirement. I am not going to punish people or mess up their college or professional careers because they may not yet have become as facile at making distinctions, etc. as others. I don't want someone kept out of med school because he doesn't quite see why the Golden Rule doesn't ever work, though he sees why it sometimes doesn't work and knows what at least those times are.

**** do you have some rationale that allows you to feel
****comfortable with the fact that students who start high and gain
****little may end up with higher marks (under the "better level" model)
****or with the fact that students who start low, gain large, yet
****finish with lesser understandings than some others may indeed
****receive better marks (under the "more improvement" model)?

No. I have no rationale that really lets me feel comfortable with grades at all, in part because grades do not by themselves reflect specific knowledge, ability, or amount of improvement, or percentage of suspected potential actualized. I had a really bright student this past term who had an "attitude" which softened somewhat by the end of the term, but who missed some classes (for no good reason) that might have helped her learn some things that were important. In terms of logic and insight, she was better than almost everyone else in the class; and she had a better grasp of most of the material than most of the students. I wanted to give her an A for the things she did well and a B for the things SHE should have been able to do but couldn't do because of her absences. I finally just essentially threw up my hands and gave her the B. I told her why.

****I expect you have good answers to these questions. If not, you
****could probably not ethically continue to teach as you do

ONLY IF YOU ASSUME THAT GRADING --in the sense of giving A, B, C, D, F (or E) letter grades -- HAS ANYTHING EVEN REMOTELY TO DO WITH TEACHING! Or even remotely to do with assessing a student's learning. Until you can standardize teachers' expectations and assessment methods and the standards by which they are judging, you cannot tell me comparing a junior college student's B in a course with a Harvard student's B in a course has any significance for comparing either student with the other. I had a student last year who was really good in certain ways. He wanted to be a teacher. I thought he would make a great teacher. I sent him over to a local highly prestigious, expensive, private university for an interview. The ed prof that interviewed him called me back and said he was the most impressive student she had ever met. Yet he had a C average from this community college because he was not good at learning things that made no sense just so he could get a good grade. He despised many of the teachers he had had.

He got in to the university I sent him to, with a scholarship; and I have reports he is doing well, particularly given his academically disadvantaged background and the fact that he has a grueling job outside of school. A prof who I spoke with says he is a tremendous student and a tremendous asset to the class, though he is carrying a low B in her course --based on her exams-- for reasons she perfectly well understands. I don't know if she will make a non-exam kind of adjustment in his grade or not. I didn't even ask her. What do you think his grade will mean -- whatever it is-- if it is based primarily on exams and other formal means of grading?!

When I was in school at Michigan, I invariably opted for every honors course I could take. You had at least a B in most honors courses just for doing normal work. They were often better taught. It was easier to get an A or B in an honors course or section of a course than in a non-honors one.

When I taught as a grad student at Michigan, one girl brought in her boyfriend to visit the class. He was from Princeton and he had taken intro philosophy there; and he had "know-it-all, holier-than-my-students" written all over him. I invited him in and said he could participate. He started with some smug comments that my students pointed out to him why he could not even begin to justify. He ended up quite self-chastened. What did his Princeton A mean in his philosophy class? And one last example in this regard, I had a cousin at the U. of Oklahoma who transferred to Illinois after her freshmen year because she had a 4.0 at Oklahoma and did not feel like she was learning anything. She was majoring in chemistry at Illinois. She knew I loved philosophy so she took an intro course in it. She got a young guy who had a year ago received his PhD from somewhere based on his dissertation on St. Anselm, an eleventh or twelfth century (I for- get which) monk who has one famous strange argument that periodically shows up in intro philo courses for one class period or so. This guy taught the whole intro philo course using only Anselm's works. My cousin (and God knows who else) got a C, and an enduring dislike for "philosophy". What did her C mean? Maybe his tests were quite valid, reliable, etc. for testing what HE wanted to test.

****My point is not to get you to defend your manner of teaching and
****testing, but rather to call attention to the kinds of questions
****that more formalized approaches to testing attempt to raise.

****I do not expect that a testing course will cause teachers to become
****test construction geniuses (I doubt that they exist); I expect
****only that it will provide them with tools for thinking critically
****about how they assess student progress. Concepts like
****reliability, validity, utility, etc., etc., provide some common
****ground for discussing assessment issues.
****
****If Gene G. is arguing that the rigid application of measures of
****reliability and validity has little merit in the classroom, I am
****in full agreement; however, I believe strongly that there is
****merit in having prospective teachers think about how to make
****judgements about the kinds of assessments that are appropriate for
****their aims and of what qualities are desirable in classroom
****measurements.

I might agree with this, IF they can really learn this by studying FORMAL methods about this sort of thing. I have strong suspicions, however, against formal methods being able to teach this properly. And, I am also worried about what appropriate aims are to be tested -- e.g., the Anselm teacher probably did not have appropriate aims to begin with.

**** My earlier reference to item analysis is a case in
****point--you engage in assessing the quality of your questions as
****you hear them answered, you may rephrase them or abandon them
****altogether in favor of new questions. In a context in which a
****teacher uses a paper and pencil measure, item analysis may help
****them to make these same judgements (based on responses from many
****students rather than one--a characteristic that has both virtues
****and faults).

I don't think item analysis by itself can be anywhere near as powerful as "follow-up" questioning, for particular students. I am not content to "flush" students who give unique apparently bad answers to questions everyone else does ok on, just because that student has an insight they don't express very well in their "initial" written answer -- or does not express it in a way I am wise enough to understand it (or know I don't understand it) on his written answer.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Sun, 12 Dec 1993 22:48:03 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: teaching without testing, sort of To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Sun, 12 Dec 1993 14:58:22 -800 from <joangf@FALLBROOK.CSUSM.EDU>

Even though it is big business, sports is just winning and losing games played in as fair a way as we know how to play and officiate them. Nobody involved thinks it is perfect or that there are not problems. Sports is an artificial, contrived contest. Judging students is not a game, should not be a sport, and does not need to have artificial, contrived single-letter results, such as A, B, C, D, or F, that represents 10 to 15 weeks of work and effort, and learning. As long as classroom testing boils down to giving a single grade, it is a bad enterprise that cannot be done well, at least not in certain kinds of courses. Bad assessments should be more tolerable in sports than in academics; bad breaks may sometimes even out in sports; and if they don't, that is just too bad, and often cannot be helped. We can do better in regard to genuinely assessing students.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Mon, 13 Dec 1993 09:43:53 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: bad side-effects of some exams X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

In further response to Bill Hunter's questions about assessing students in the way I do: apart from issues he raises of validity, reliability, etc. of testing (i.e., giving specific exams), there is often a "hidden" problem with tests in that they encourage anti-intellectualism under certain conditions. I do not know exactly what those conditions have to be, but I can describe the phenomenon. The phenomenon is simply that getting a good grade on the test becomes the chief priority of (a) many, (b) most, (c) all students; and learning is only a means to that end. And as someone posted somewhere a few months back, this makes learning more than you have to for the exam, or working harder than you need to for doing satisfactorily on the exam, a wasted effort, somewhat tantamount to paying more than you need to for a purchase; and learning something after the exam is really terrible because then it 'doesn't count'.

One classic, though fictitious case, is in the movie "Dead Poets' Society", the Carpe Diem speech scene, where after Robin Williams has gathered the boys around the photos and trophies portraying students, long dead, in the glory of their vitality and youth, and articulately and passionately explaining to his boys how they too, just like these lads, will finish their existence as worm food and so they need to live life to the fullest, etc., the first question one kid asks is something like "do we need to know this for the exam?"

I don't know what gives students that mindset to begin with, since many students will enjoy discussion and learning. And my explanation does not explain why many people will be passive in classes, like adult Sunday school classes, where there are no exams anyway, except that they don't want to "waste" people's time with their views instead of hearing the teacher's (i.e., the supposed expert's supposed correct) views. But the point is that whatever promotes or causes such lack of student effort and lack of intellectually active participation in the first place, lecturing and testing, even in the way she did it, by beginning with questions, do not serve in any way to counteract the phenomenon. You have to do something to (1) more aggressively provoke excited response (usually by asking more interesting and challenging questions, I suspect, or by setting up the questions to make them have a dilemma or psychologically problematic "bite" to them), (2) get their minds off the tests somehow, either by not giving tests or by not letting them know what will be tested or by demonstrating to them that it is their active intellectual efforts that will stand them in much better stead on the exams than will anything they memorize or merely repeat.

Rick
Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Mon, 13 Dec 1993 10:38:47 EST Reply-To: Sender: From: Eugene Bartoo <EBARTOO@UTCVM.BITNET> Subject: Re: Student Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Sat, 11 Dec 1993 11:25:49 EST from <U176C@WVNVM> On Sat, 11 Dec 1993 11:25:49 EST Aimee Howley said:
****something like hope; I intended them (much less generously) as
****conveying something more like despair. Assessment is a technology
****whose purposes determine the form it takes. If the purpose of education
****really were to help children learn a lot of things that we and
****they thought were important, assessment would be indistinguishable
****from the dialogue and demonstration that constitutes teaching.
****The extent to which it is construed apart from and above the
****occasions of learning and teaching reflects the extent to which it
****is a technology directed at a much different--and much less worthy--
****purpose. My despair comes from hearing in much of the public discussion
****about assessment themes and variations of this less worthy--
****indeed fully suspect--purpose.

Right you are Aimee. Last year sometime the more, or less regular newsletter from ETS featured an article on portfolio assessment. In the article the woman in charge of the ETS project stated that one major problem with portfolio assessment was that it did not allow cross school comparisons important for policymakers. I did not save the newsletter so I can not attribute accurately, but that was the sense of the expression. This makes David Gibson's perceptive comment about losing information by establishing reliability for portfolio assessment also poignant. Its a shame that the concerns of ETS seem to lie more with making school-wide comparisons than with helping teachers make better judgments of student learning.

Date: Mon, 13 Dec 1993 16:33:18 LCL Reply-To: Sender: From: Tom Green <TFGREEN@SUVM.BITNET> Subject: Re: Student Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Mon, 13 Dec 1993 10:38:47 EST from <EBARTOO@UTCVM> On Mon, 13 Dec 1993 10:38:47 EST

Eugene Bartoo said:
****On Sat, 11 Dec 1993 11:25:49 EST Aimee Howley said:
**** ****something like hope; I intended them (much less generously) as
**** ****conveying something more like despair. Assessment is a technology
**** ****whose purposes determine the form it takes. If the purpose of education
**** ****really were to help children learn a lot of things that we and
**** ****they thought were important, assessment would be indistinguishable
**** ****from the dialogue and demonstration that constitutes teaching.
**** ****The extent to which it is construed apart from and above the
**** ****occasions of learning and teaching reflects the extent to which it
**** ****is a technology directed at a much different--and much less worthy--
**** ****purpose. My despair comes from hearing in much of the public discussion
**** ****about assessment themes and variations of this less worthy--
**** ****indeed fully suspect--purpose.
**** Right you are Aimee. Last year sometime the more, or less regular newsletter
****from ETS featured an article on portfolio assessment. In the article the
****woman in charge of the ETS project stated that one major problem with portfolio
****assessment was that it did not allow cross school comparisons important for
****policymakers. I did not save the newsletter so I can not attribute accurately,
****but that was the sense of the expression. This makes David Gibson's perceptive
****comment about losing information by establishing reliability for portfolio
****assessment also poignant. Its a shame that the concerns of ETS seem to lie
****more with making school-wide comparisons than with helping teachers make better
****judgments of student learning.

This is a beautiful illustration of a problem I mentioned earlier -- one in which we are all caught, but seldom make explicit. It is that education necessarily is something that occurs at a very low level of aggregation, but that policy necessarily is something that appears, and can appear only at a very high level of aggregation, and further that assessment [as Rick Garlikov uses it] is something that is engaged IN teaching, whereas assessment [as it occurs at ETS] is something that necessarily is bent to the problems of policy.

THOMAS F. GREEN (TFGREEN@SUVM.BITNET) + + EMERITUS FROM SYRACUSE UNIVERSITY PHILOSOPHER IN RESIDENCE ON POMPEY HILL + + Box 100 Pompey, NY 13138 (315) 677-9935

Date: Mon, 13 Dec 1993 17:10:00 EST Reply-To: Sender: From: Jack Letarte <JCLETARTE@TAYLORU.EDU> Subject: Re: bad side-effects of some exams To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

In response to Rick's post on why students do not respond in class, I have not kept up with research in this area, but do remember some findings from several years back: 1) Students establish pretty early which of their classmates can be depended upon to carry the load of class discussion. After a bit, they relax and let things continue that way. 2) There's some fear of being "put down" by the instructor with his superior knowledge. 3) Students see themselves as consumers of knowledge, and the teacher as a dispenser. 4) Teachers see the classroom as a "focused interaction", where responding to one another is called for; students see it as any other public gathering where one may choose to be anonymous. Just found the citation for this, and it is old: Karp and Yoels, The College Classroom: Some Observations on the Meanings of Student Participation, in *Sociology and Social Research, Vol. 60-4, July 1976, pp. 421-437.

Jack Letarte Taylor University Upland, IN 46989 317-9985153 Internet: jcletarte@tayloru.edu

Date: Tue, 14 Dec 1993 09:29:10 -0800 Reply-To: Sender: From: Susan Nolen <sunolen@U.WASHINGTON.EDU> Subject: Teaching about assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@uwavm.u.washington.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312131411.AA21791@tolstoy.u.washington.edu>

Here are Bob Linn's recommendations, followed by the citation. (He refers also to a survey in J. Ed. Msmt. (1986) by Arlen Gullickson which shows a mismatch between what is taught in ed msmt classes for teachers and what teachers think needs to be taught. Some of you may be interested in that as well.) Linn's recommendations for assessment classes for teachers & teachers-to-be: 1. Planning & constructing classroom tests. (Gullickson found this was the sole area of match between teachers' perceived needs and profs reports of instructional content, by the way.) 2. The use of non-test evaluation procedures. (Currently tends to be called "alternative assessment.") "Knowledge about assessment principles needs to be integrated with an understanding of the subject matter, pedagogy, principles of learning, and children." 3. Use of assessment results for instructional planning and formative evaluation. 4. Use of assessment results for summative evaluation. (Including fair grading practices, and how grading communicates what is important to learn within a discipline.) 5. Administration and scoring of tests. (Linn actually discusses non-test evaluation here as well.) 6. General assessment information regarding the selection and use of tests. Includes ethical issues of assessment. 7. Principles of measurement. Linn recommends reliability and validity, taught as guiding principles. Linn, R. L. (1990). Essentials of student assessment: From accountability to instructional aid. *Teachers College Record*, 91(3), 422-436. The text we use in our preservice assessment class is the only one I've seen that is built around the kinds of assessment teachers actually need to do. Those of you who teach such courses or are involved in teacher ed reform may want to check it out: Peter Airasian (1993) Classroom Assessment, 2nd Ed. McGraw Hill.

Susan B. Nolen 322 Miller Hall DQ-12 University of Washington sunolen@u.washington.edu Seattle, WA 98195

Date: Tue, 14 Dec 1993 11:07:05 PST Reply-To: Sender: From: "Andrew Coulson (Redmond)" <andrewco@MICROSOFT.COM> Subject: Re: bad side-effects of some exams X-To: edpolyan@asuvm.inre.asu.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Rick Garlikov wrote, "I do not know exactly what those conditions have to be, but I can describe the phenomenon. The phenomenon is simply that getting a good grade on the test becomes the chief priority of (a) many, (b) most, (c) all students; and learning is only a means to that end."

Schooling is not currently a voluntary activity. It is made clear to virtually all children that they are expected to attend school for the prescribed number of hours, days, and weeks each year. Students often fail to see the benefit of learning the subject matter in many of their courses. Even at the college level, when a course is required of the students' program they will take it, because they have to, even if they do not see what it will gain them. Though the value of the material taught in a given class may escape a student, the value of the grade rarely does. The importance of grades is readily apparent to anyone wishing to move higher up the academic ladder. We thus have a situation in which people are forced to do something they do not necessarily value, in order to gain something they do.

It seems clear that within the bounds of their own ethics they will try to subvert the system as necessary to get what they want. This might just mean studying only for tests, but it often means copying off other students or stealing the test in advance. This is not to say that if schools were entirely voluntary they would immediately empty out. It seems that people, particularly children, have a habit of balking at things they are forced to do even if they don't really object to the things themselves. There are a number of approaches to easing this problem: children could be given more flexibility in what they learn but, being children, they might not think far enough ahead to what they will want or need in the years ahead; children could be offered more freedom as to how and when they learn, though this would be difficult to orchestrate within our current system; a greater effort could be made to explain the value of the various fields of study (answering the "what do I need this for", "why do I have to study this" questions), so that they would not seem so distant from the children's lives; evaluations could be done, as has been suggested by Rick and others, in such a way that it is not so easy to cheat or to do well by simply memorizing a tiny portion of the material, though this will make the job of college admissions more complex.

I would love to hear other suggestions, and I apologize if these repeat some of the last month's discussions as I've been too busy to follow them. In studying some of the great figures of the Renaissance I've found, unsurprisingly, that they drove their own course of study from a very young age. Rubens, for instance, was forced to leave school and go into the service of a wealthy lady as her page, because his mother was unable to support him. He soon left this service, however, at the age of thirteen, after convincing his mother that he would quickly be able to earn his keep as an artist. This he did, and despite his lack of formal education he went on to negotiate several peace treaties between Western European nations (in addition to painting a lot of fleshy bodies). This is not to say that all children would benefit from self-directed study, but simply that there are many precedents, Rubens among them, for a freer approach to education than the one that has grown up in the recent history of the United States.

Andrew Coulson

Date: Tue, 14 Dec 1993 15:41:11 CST From: "John Nicholls " <U36927@UICVM> To: <dems042@uabdpo.dpo.uab.edu>

I much appreciate your emphasis on teaching rather than testing. When I was 'trained' as an elementary school teacher in New Zealand we were often told, "testing is not teaching." I think much of my life has been an attempt to interpret this. I now commonly ask students in my educational psychology courses to evaluate their own work and grade it. I only felt able to do this when I became very clear that what they should be doing was developing their own ethical stances on what type of teachers they hoped to be, and not collecting truths delivered by researchers.

I can, I think, only do this when I establish that this is what they are doing or should be doing. I do this by pointing out that the experts disagree with one another and that much more than facts are involved in these disagreements. I also have, or feel I have to directly assault student's notions that they are here to collect noncontroversial facts or truths which I can grade fairly. While they think this way, they destroy my chances of having discussions with them that might teach me something about their world views and that will get them to elaborate and challenge their own views. So I give then material to read that describes the deadly life in classrooms where students seek such answers and, yet, see it as fair.

Sometimes the first assignment, on day two of class, is to discuss such an article and, if bothered by it suggest remedies that will make sure it doesn't happen in this class. This barely gets the process of re-construing class as a collaborative exploration of different possible ethics of teaching in which it is assumed that we will leave disagreeing with one another but doing one another a lot of good by listening carefully and challenging openly and respectfully. Along the way the criteria for grading have to be discussed. Trust is essential and I think partly established by showing that people can, indeed must disagree with the instructor and survive with colors flying. But more positively, I generally advance the notion that every significant paper will be unique and be of quality if it involves some significant dialogue between personal and school knowledge. These concepts are discussed as part of the course -- they come up in what I use as first reading -- so a course on education offers special advantages I think.

With grad classes, which are usually intro classes, I am ready to allow that everyone can get As. But I find students generally very tough on themselves and I more often raise than lower grades. But grades are a small part of the process. they have to write detailed evaluations of their papers and I write evaluations of these, often elaborating in detail why I disagree with the substance of their paper or their evaluation, but still allowing an A if they have evidence of a significant dialogue between school and personal knowledge. When this comes of, I find it much more liberating for me than teaching used to be. I used to delight in setting tricky questions that would reveal flaws, if they existed, in students' knowledge of majortexts. Real thinking was, I used to think, what I was testing. I now see this as a trivial exercise. All knowledge of consequence in this business is controversial and it is so because it involves ethics and politics (not science that is distinct from ethics). My role is to keep a lively, strenuous, conversation going on the topics. I am delighted when students who disagree with my positions later drop by to talk and ask advice on careers or ask me to write on their behalf. These things suggest to me that, for these students, I succeeded. It is a more satisfying vision of success than my old one wherein the successful student was the one who got the various texts right and applied or used them "intelligently."

John Nicholls.

Date: Tue, 14 Dec 1993 14:15:41 MST Reply-To: Sender: From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: good teaching and testing To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I'll take this off list else I will seem to be flattering you.

[But he sent it accidentally to the list, and so it appears here.]

I agree with nearly all of what you are saying. I personally like to get test data for moderate sized classes (20-50) as a perception check, especially if I consider that part of the purpose of the course has involved mastery of some body of information (not a profound objective, but a common one--suppose, e.g., you were teaching the history of philosophy--with a large class, would you not want some assurance that people completing the course could associate some important names with their achievements?). For large classes, I cannot imagine getting to know students as well as you describe (here I am talking about 100 or more students). For teachers in schools, neither of these concerns is likely to hold since they work with smaller numbers for longer time periods. They do, however, bear a burden of documentation for both administration and parents--e.g., are you prepared to accept the judgement of the teacher you described without some evidence of the basis of her judgement (clearly, you do not trust her questioning skills). Anyhow, what I wanted to get at it is this: again, I think the extent to which what you do is natural for you makes it difficult to see how unnatural it may be for others. Specifically, the formal nature of learning about testing _may be_ a reasonable way of getting folk toward what you do with such facility. We also teach people about lesson plans because some of them would not have another way of thinking ahead. Remember that we are not expecting to produce polished teachers. In truth, we have to recognize that "average teachers" is what we can hope for, but we can keep trying to push up the average (and I think we are). You seem to be a gifted teacher and we do get some of those through our doors to, but we can't take credit for them. We can only hope to inspire them and support them.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Wed, 15 Dec 1993 05:05:30 CST Reply-To: Sender: From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: good teaching and testing To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 14 Dec 1993 14:15:41 MST from <hunter@ACS.UCALGARY.CA>

Bill Hunter:
****I personally like
****to get test data for moderate sized classes (20-50) as a
****perception check, especially if I consider that part of the
****purpose of the course has involved mastery of some body of
****information (not a profound objective, but a common one I think this is right.

I only worry that course material is more often thought to be of this sort than it ought to be. E.g., where history is thought to be primarily facts or math thought to be primarily algorithms. And as you describe these, you are using it more for seeing what you need to teach ("over again") than just for 'grading' students. That is quite fair and important, I think.

****For large classes, I cannot imagine getting to
****know students as well as you describe (here I am talking about
****100 or more students).

It can be done if you don't have too many such classes at one time -- though it is done better for teaching purposes than for grading purposes. It is difficult to have much confidence in grading this many, particularly some of the ones who are reticent much of the time, or who, when called on, seem to have a hard time (psychologically) saying what they think.

****They [school teachers] do, however, bear a burden of
****documentation for both administration and parents--e.g., are you
****prepared to accept the judgement of the teacher you described
****without some evidence of the basis of her judgement (clearly, you
****do not trust her questioning skills).

The problem is making the documentation be meaningful, which, I guess encompasses validity, etc., but also significance of material tested, etc.

****Anyhow, what I wanted to get at it is this: again, I think the
****extent to which what you do is natural for you makes it difficult
****to see how unnatural it may be for others.

Except that it is 'natural' for me only after getting certain perspectives, perspectives which I think could be taught to others perhaps without too much difficulty. In fact, perhaps, taught more easily than the 'hard' way I got them. When I ask for certain techniques to be used in schools, or certain things taught, teachers and principles tell me they won't work with kids; and I say they worked with my kids; and they say 'that is because your kids are so bright'. I honestly reply 'They just seem bright because they learned these things taught this way; if you taught this way to all kids, most of them would seem bright too. I don't think my kids have any more than average intellectual ability -- my wife and I have just been trying to develop what they have.' I think perhaps there are techniques -- some of which you and Sue and John and others may use (or you may have more and better ones-- that can be taught to ed students to make them also seem to be natural or to have teaching come easily.

****Specifically, the
****formal nature of learning about testing _may be_ a reasonable way
****of getting folk toward what you do with such facility. We also
****teach people about lesson plans because some of them would not
****have another way of thinking ahead.

My concern about these sorts of things was (is) that (1) they become ends for teachers instead of means -- e.g., teachers may feel bound to stick to lesson plans regardless of student academic needs and forget their lesson plan was only a "plan"; and more importantly (2) that ed students, as any students, won't see the real significance behind the formal descriptions -- so that they develop a kind of surface understanding of validity, reliability, etc., but don't have it in some sense 'internalized' -- like the engineers who understood physics principles in some sense but did not understand about the need to air condition refrigerated testing rooms. Perhaps this is a distinction between 'book knowledge' and 'practical understanding'. It took me a long time before I realized that what philosophers call 'validity' and 'soundness' have a meaningfulness outside of intellectual debate or doing philosophy. And it is still very difficult to translate their formal meanings into ordinary or useful explanations without having to ignore some difficulties, though these difficulties may be only of an esoteric nature if they are difficulties at all. But even when teaching these concepts in a less formal way, it is really difficult to have discussions with students and at the same time point out where these things come into play. For every time I do it, I feel there may three or five or ten times I don't because (1) it would disrupt the flow or take too long, or (2) because even I don't view what I am doing in those terms at the time I am doing it. Since, in a sense, the formal description is only a characterization of an idea or process (or whatever), it often is not the way we think about what we are doing -- perhaps just as basketball players are not thinking about geometric angles or the coaches explanations about a given passing pattern when they shoot off the glass or break down the floor.

****Remember that we are not
****expecting to produce polished teachers. In truth, we have to
****recognize that "average teachers" is what we can hope for, but we
****can keep trying to push up the average (and I think we are).

That is really what I wanted.

****You
****seem to be a gifted teacher

Of course I have only been describing my 'successes', and those as seen through my own eyes. Many days I don't feel I did very well.

****and we do get some of those through
****our doors to, but we can't take credit for them. We can only
****hope to inspire them and support them.

That is an extremely important and worthy goal in itself.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Wed, 15 Dec 93 18:21:42 MST From: Bill Hunter <hunter@acs.ucalgary.ca> In-Reply-To: <9312151224.AA50298@acs1.acs.ucalgary.ca>; from "Rick Garlikov" at Dec 15, 93 6:14 am X-Mailer: ELM [version 2.3 PL11v]

I am not embarrassed about the content, but about the error itself. However, it is small potatoes. Tom Green sent me a little note saying he was glad it appeared on the list and asking if he could be included in some invitation he thought I was issuing to you. I don't know what he meant, so I've told him to let me know what it was and I'll deal him in.

I am not responding to your last post because it left me with nothing to say. I have saved much of this conversation and hope to use it in my testing course next term. I really appreciate the thought you have given these issues and the time you have put into writing for the list. I am quite certain that I would have seriously considered a philosophy major had you been one of my instructors. My only undergrad course in philosophy was taught as a lecture to 1,500 students. Needless to say, there was not much interaction.

One small comment. You acknowledge the difficulty of reticent students. My wife was very gifted student and I am sure that she would have been both vocal and articulate in your class, but she took it as a matter of principle in most courses to stay quiet (for some of the reasons you mentioned). A teacher less gifted in framing evocative questions and responses could easily underestimate someone like her. There WERE courses in which she would talk frequently (part of our courtship took place in a Grade 12 English class which had weekly debate/discussion days--we were so often obviously and vocally on opposite sides of issues that only the two of us could understand how we managed to stay together and I am sure some folks still wonder nearly 30 years later).

You may understand better than I why I always feel compelled to go on and explain my point--here it is: I share your perception of the dangers of teachers accepting as normal, expected, or desirable, those techniques that we encourage them to use as heuristics in becoming skilled teachers, but I think that is balanced by the dangers of people THINKING they can do what you do when they are not really able to adequately inspire participation (your recent story was a good case in point).

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Wed, 15 Dec 1993 10:56:44 PDT Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Mark Fetler <MFETLER%CC1@TS9.TEALE.CA.GOV> Subject: Ritual Assessment

"Does Spending Money on Education Help?" asks Howard Wainer in the 12/93 Educational Researcher. He's talking about a table prepared by the Heritage Foundation and published last June by the Wall Street Journal that compares state SAT rank with average per pupil expenditure. Wainer observes, "It appears that a state's SAT ranking worsens by almost 6 places for every thousand dollars that was spent on the average pupil." More interesting than Wainer's refutation of the inference (the sample of SAT takers is nowhere nearly random or representative) is the fact that that the Heritage Foundation does this kind of study and the Wall Street Journal publishes it. And, it seems that some prominent group or person resurrects similar SAT findings almost every year. In many schools, districts, and state agencies there is a palpable tension just before the College Board releases the test results. Jobs, elections, and funding often depend in part on relatively insignificant swings in the average. Why?

Considered rationally, it is easy to understand how self-selection makes the SAT a wildly inaccurate barometer of school, district, state, or national achievement. We're talking common sense, pre introductory statistics. Of course, the Heritage Foundation senses a political lever, an argument for schools to "do more with less." On the other hand, more liberal groups manage to find evidence of gender or ethnic bias in the results, in order to support equally political agendas, i.e., to do more with more. I don't say this to diminish the value of political dialogue, but only to point out that the espoused rational claims for the SAT don't really support the actual political agendas of these groups. Interesting is that the political claims made with the SAT generally do carry weight with the public. Why?

The answer, I think, is that the SAT has become a symbol of the condition of public education. The success of the College Board and ETS has made the SAT highly visible. Its tough to make any simple generalizations about public education. Governance, funding, mission, structure, etc. vary enormously within and across states. The SAT score or rank is one apparently simple number that sums it all up and helps people to make sense of schools. The credibility of the SAT is boosted by the seeming rationality of using a test score, a number, to describe performance.

This wave of enthusiasm in testing probably had its origins in federal evaluation requirements for categorical programs. Many states followed suit. At the same time individual decisions about selection and placement of individuals needed to be defended against possible litigation. What could be more fair and objective than a test? While the quality of individual assessments and program evaluations varies enormously, the overall result has been to heighten the visibility and importance of test scores to the general public.

Now testing and program evaluation are the expected routine in public education. The catch is that good assessment tends to be expensive. So we find expedients that have strengths and weaknesses. Good commercial tests may meet technical standards, but aren't relevant to the curriculum. Teacher devised tests may be relevant, but are technically weak. Mandated assessments are irrelevant to the classroom and therefore may be poorly administered, scored, and reported. There are many ways that the quality of a score can be degraded. And yet, it is almost unthinkable that schools reduce the amount of testing. Likely, portfolios will be an add on, and then we'll see numerical scores assigned to the portfolios!? Tests have become symbols and part of the ritual of public education. And as most leaders know, one does not casually dispense with established symbols and rituals.

Date: Thu, 16 Dec 1993 11:25:26 EST From: TFGREEN@SUVM To: "Richard (Rick) Garlikov" <dems042@uabdpo.dpo.uab.edu>

Rick: Thought this might amuse in re your reference to that scene from Dead Poets in which the same question is raised.
Tom Green

A colleague gave me the following item; I pass it on to you now, hoping you'll enjoy it.

THE LESSON

Then Jesus took His disciples up the mountain, and gathering them around Him, He taught them saying: "Blessed are the poor in spirit, for theirs is the kingdom of heaven. Blessed are the meek. Blessed are they that mourn. Blessed are the merciful. Blessed are the they who search for justice. Blessed are you who suffer. Be glad and rejoice, for your reward is great in heaven." "Remember what I am telling you."

Then Simon Peter said, "Do we have to write this down?"

And Andrew said, "Are we supposed to know this stuff?"

And James said, "Will we have a test on it?"

And Phillip said, "Can I borrow a pencil?"

And Bartholomew said, "Do we have to turn this in?"

And John said, "The other disciples didn't have to learn this!"

And Matthew said, "When do we get out of here?"

And Judas said, "What does this have to do with the real world?"

And the other disciples likewise. Then one of the pharisees who were present asked to see Jesus' lesson plan and inquired of Jesus His terminal objectives in the cognitive domain.

And Jesus wept.

Date: Thu, 16 Dec 1993 14:49:51 -0500 Reply-To: Sender: From: Greg Camilli <CAMILLI@ZODIAC.BITNET> Subject: Re: testing, 1, 2, 3, testing To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

My students in an introductory measurement class have just completed an assignment in constructing a short multiple choice test. This consists of identifying a construct; developing a blueprint; writing, critiquing, and piloting items; administering the test and performing an item analysis; revisiting the test items and options; and finally determining whether anything has been learned. Why do this kind of thing? It's almost impossible to understand the strengths and weaknesses of multiple choice tests unless you've gone to the trouble of constructing one. The same is true for constructed response and essay exams. What my students learn is that individual questions, no matter how carefully crafted, sometimes fail to be good indicators of a student's knowledge. In fact, that is why a test is composed of multiple questions whether it is written, oral or observational; hence the concern with reliability. A really bad test, reliable or not, is one in which most or all of the items miss the mark. We enter the realm of validity (when we speak of a test missing the mark) which a lot of psychometricians have worried about including Anastasi, Cronbach and Messick.

Cronbach in the 1988 book TEST VALIDITY described five perspectives on validity: functional, political, operationist, economic and explanatory. Further, he wrote that "Validation speaks to a diverse and potentially critical audience; therefore, the [validity] argument must link concepts, evidence, social and personal consequences, and values." It seems like he could have been addressing (broadly) many of the concerns with testing that have recently been posted. My point here is that many persons in the field of educational measurement are not ignorant of issues concerning test validation. Rick has advocated the idea that whether a student demonstrates competence can only assessed through verbal "give-and-take" interaction. While this may be true of philosophy, it is clearly not true for other subjects such as writing and math classes. This is not to say that classroom discussion is not important, only that it cannot be central to assessment. Classroom tests, if well-made, can be useful for assessing particular skills which can involve higher order reasoning. I note that it is possible for a teacher to make a mistake by giving too much credence to a test score. However, to argue that tests should not be used because they are fallible implies that only infallible methods should be used for assessment. This is the force of many of Rick's arguments; they are emotional appeals to anomalous decisions based on various types of tests.

In the hands of a less dedicated teacher than Rick, I dare say that verbal give-and-take is as equally fallible as a classroom test. Other arguments against testing have been based on the premise that any valid method of assessment must directly concern instruction, that is, must be indistinguishable from classroom instruction. If it isn't, this just shows that we don't really care about children or students. I don't understand what makes this assertion obviously or necessarily true. I held a discussion on grading during my last class session, but first I had students give assessments of all member of their small groups (including themselves). First I had students give a global rating on a scale from 1 (low) to 5 (high) of each members contribution. Then they turned over this sheet and ranked each member in term of their contribution. I then allowed them time to give written comments, including a critique of this assessment method. As you might imagine some students bristled at the activity of "assessment," AKA grading.

After this activity, to make a long story short, I asked them whether they would prefer an educational system in which grades were not assigned. The result wasn't even close: for many different reasons, most preferred a graded system. One student remarked that in college it should be the professor's responsibility to establish, maintain and communicate standards, while it should be the student's prerogative to take the professor's class, i.e., to accept the standards.

Date: Thu, 16 Dec 1993 23:51:04 -0500 From: Josh Barbanel <jobarb@PANIX.COM> Subject: Re: Ritual Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.BITNET@CUNYVM.CUNY.EDU> X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN%ASUACAD.BITNET@CUNYVM.CUNY.EDU> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <199312151928.AA07632@panix.com>

Testing is only a symbol and ritual of public education in the mythical school house depicted in many posts on this lists. Teaching in this mythical school house is a holy calling carried out by high priests, the professional teachers, who always knows best, and are constantly struggling to break through the bonds of conventionality that stand in the way of brilliant communication with students. The reality at least in public schools in New York City, is far more complicated, and sometimes bleak. While there are many solid old fashioned schools, and every teaching innovation ever dreamed up by man has found a home in one school or another, there are scores or perhaps hundreds of schools that are failing. For whatever reason, poor local leadership, political corruption, bureaucratic bungling, administrative grid lock, inferior professional development, counterproductive contract requirements, demoralized teachers, some of these schools have been failing for years -- and blaming their students for the failures. (And these schools do poorly not only compared to suburban schools, but also compared to other high poverty schools.) It may be that it is only through the use of valid, credible and easily understood measures of school performance, from dropout rates, attendance, to standardized testing, that these schools be identified, and public pressure can begin to make a case for change.

We are far from the day when a portfolio evaluation will convince parents in an impoverished neighborhood that it was time to try to depose a failed tenured principal and reorganize the school. Last week, the New York City Board of Education reported that one out of four intermediate schools, had failed to meet any one of nine city and state performance standards, from MINIMUM competency tests in math and reading, to attendance standards, and measures of student gains. These schools are undoubtedly stocked with qualified, and often caring teachers and administrators. Yet for whatever reason, many school people say, these enterprises are failing, and need to be retooled. (And by the way, while school spending in New York City, is below average for the state, It spends upwards of \$7,200 a student, more than the median spending in all but a handful of states.) As a reporter who writes about schools in New York City, but does not work in a school, testing -- and you are not going to get me to defend bad testing, or misinterpreted tests -- is one of the few tools capable of providing an independent view, even if a distorted one, of what goes on in a school.

Josh Barbanel The New York Times jobarb@panix.com

On Wed, 15 Dec 1993, Mark Fetler wrote:

**** ... it is almost unthinkable
**** that schools reduce the amount of testing. Likely,
**** portfolios will be an add on, and then we'll see numerical
**** scores assigned to the portfolios!? Tests have become
**** symbols and part of the ritual of public education. And as
**** most leaders know, one does not casually dispense with
**** established symbols and rituals.
****

Date: Fri, 17 Dec 1993 08:02:33 -800 From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: TEACHING WITHOUT TESTING, SORT OF X-To: EDPOLYAN@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <01H6HK582GCI8WXDTY@asu.edu>

Commenting on the need for better professional(and public) development in what constitutes good assessment, I suggested that some of the more important assessment concepts are already known to sports fans, and further that this informed public helped not only to monitor their application but to improve fairness in that domain. (Legislators seeking to improve education, in contrast, often want to make comparisons based upon a single, sometimes largely irrelevant, measure.> > >
**** On Sun, 12 Dec 1993, Rick Garlikov wrote:
****
**** > unfortunately I think if we follow your sports analogy with testing, sports
**** will be as problematic as testing students is.
****
**** Rick, I don't believe I understand you here. Could you please explain?
**** Changing sports was not part of my recommendation.
****
**** My point is that concepts of making fair judgments translate well from
**** sports to other arenas of assessment.
****
**** Some of the more important ones are that the rules be known and
**** consistently applied, that the playing field be level, and that
**** participants be given broad and sufficient opportunities to show their
**** abilities.
****
**** The questions you raised about fair judgment in the sports world apply
**** to educational evaluation as well.
****
**** For example, you point out the difficulty of comparatively evaluating
**** players who performed under different circumstances (changes in equipment,
**** differences in rules, unequal stresses). How about the differences in effort required between the kid who has a computer in his house and the one who doesn't even own a typewriter? Should their portfolios be judged the same?
**** We need to think critically about our evaluation systems (especially
**** selection processes that determine educational opportunity) in the same way.
**** How about the differences in effort required for the kid who has a computer in his room and the one who doesn't even own a typewriter? Should their portfolios be judged the same? >
**** > Even though it is big business, sports is just winning and losing
**** > games played in as fair a way as we know how to play and officiate them.
**** > Nobody involved thinks it is perfect or that there are not problems.
****
**** Agreed! If rules of educational evaluation were as broadly and vigorously
**** debated, there would be less arbitrary assessment, less overreliance on
**** imperfect measures.
****
**** Sports is an artificial, contrived contest.
****
**** It seems to me that by definition all tests (whether by opposing teams
**** or competing individuals) are artificial and contrived. The challenge is
**** to make an academic test as representative as possible of important real-life expectations, to treat test subjects equitably, and to judge them fairly. >

**** Judging students is not a game, should not be a sport, and does not need
**** to have artificial, contrived single-letter results, such as A, B, C, D, or F,
**** that represents 10 to 15 weeks of work and effort, and learning.
****
**** Absolutely! The judgment process is not the same as the
**** playing or learning process but must be a part of either of system. Your
**** main concern seems to be the outcome measure (the grade) rather than the
**** evaluation methods that lead up to it.
****
****
**** As long as classroom testing boils down to giving a
**** > single grade, it is a bad enterprise that cannot be done well, at least not
**** > in certain kinds of courses.
****
**** Unfairness can be a problem both in assessment methods and in overreliance
**** on the results. I'm sure that you would agree that concern about the
**** artificiality of grades does not excuse sloppy assessment.
****
****
**** breaks may sometimes even out in sports; and if they don't, that is just too
**** bad, and often cannot be helped. We can do better in regard to genuinely
**** assessing students.
**** >
**** largely because of the secrecy that surrounds it and the consequent
**** unreasoned faith in its infallibility.

****
**** Joan Gipson-Fredin
****
**** joangf@fallbrook.csusm.edu
****

PS If the sports analogy does not work for you, can you suggest one that does? I can't think of any assessment model considered to be fair where a lone individual's intuitive judgment determines the outcome.

Date: Fri, 17 Dec 1993 10:50:25 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: response to G. Camilli's objections X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Greg Camilli: "Rick has advocated the idea that whether a student demonstrates competence can only be assessed through verbal "give-and-take" interaction. While this may be true of philosophy, it is clearly not true for other subjects such as writing and math classes."

Rick: This is not quite what I advocated. I agreed that for evaluating memorization of material, exams may serve if the tests and testing conditions are proper. And I would add, since Greg mentions 'writing', that the learning of certain techniques may be demonstrated by exams. What I contend is that tests don't tend to (1) demonstrate students' states of understanding of material very well; and (2) they don't tend to show what specifically the problem is when students don't do well. Not all material involves understanding, but much material does, including writing and doing math. Further, I have seen cases, though admittedly perhaps anomalous cases, where students get the right answers to a large number of different, say math, problems though they do not really understand what they are doing. This may not be serious if they have learned, say a good algorithm, and just don't understand it and never need to. (Though I am not sure this is really teaching them math, or giving them a chance to become really competent in math, but....) But sometimes students learn faulty algorithms which work in many cases. So a student might get an 85 on a math exam but really not have 85% mastery or competence. I have recently met a third grade girl who has some really screwy ways of doing subtraction -- that give the right answer in a great many cases. But (1) it does not give the right answer all the time (though you have to get into 'higher level' problems to see that if you are testing her), and more importantly (2) she doesn't have a clue as to why her method works or doesn't, and she cannot tell when it works and when it fails. On the basis of math tests she seems better than all the other students in her class; but on the basis of talking with her about how she does subtraction problems, terrible problems with her understanding emerge; and she will be at a severe disadvantage in some cases as she progresses through math, unless her understanding can be enhanced or corrected. And, I would argue that writing well involves understanding that is different from just parroting a given writing technique or knowing the rules of grammar, for example the famous Churchill response about the "situation up with which I will not put".

Greg: This is not to say that classroom discussion is not important, only that it cannot be central to assessment.

Rick: I didn't mean that discussion had to be classroom discussion or that it had to be strictly oral and verbal. One can discuss doing math or writing (using examples put on the board or overhead projector, etc.) and can do it with individuals in a classroom or with individuals during office hours or in the cafeteria.

Greg: Classroom tests, if well-made, can be useful for assessing particular skills which can involve higher order reasoning.

Rick: The "if well-made" is very important. I maintain that it may take discussion to figure out whether the test really was made well or not. Plus, *I* could never design one I was really happy with. Discussion always showed any exam I came up with to be problematic in some way I would not have suspected without the discussion.

Greg: I note that it is possible for a teacher to make a mistake by giving too much credence to a test score.

Rick: This seems very important! And how much credence should be given to test scores? And what exactly is 'giving credence', or too much credence?

Greg: However, to argue that tests should not be used because they are fallible implies that only infallible methods should be used for assessment.

Rick: You wouldn't mind being assessed (say for the need for open heart surgery) by methods known to be fallible? You wouldn't mind a teacher determining your future by using methods known to be fallible? It is one thing to use a method that you don't know to be fallible; quite another to use a method that is demonstrably fallible. And, if by ASSESSMENT you mean grading (as opposed to trying to figure out what you need to teach), then assessing on known fallible grounds is not right to do to another human being who does not deserve that grade, especially if the grade has important and irrelevant consequences.

Greg: This is the force of many of Rick's arguments; they are emotional appeals to anomalous decisions based on various types of tests. In the hands of a less dedicated teacher than Rick, I dare say that verbal give-and-take is as equally fallible as a classroom test.

Rick: I don't know that asking teachers to treat students right is simply an 'emotional appeal'. And, I believe it is much more difficult for a teacher to be UNdedicated and 'hide behind' verbal give-and-take in assessing students than to hide behind tests. Perhaps encouraging verbal give-and-take would foster greater dedication of teachers and students. We have ingrained students (and their parents) to accept the results of tests since tests are supposedly somehow objective. But tests are not necessarily objective in terms of demonstrating ability, understanding, mastery, etc.; and they are not objective enough in many cases to justify giving grades which affect students' lives in the way they do.

Greg: Other arguments against testing have been based on the premise that any valid method of assessment must directly concern instruction, that is, must be indistinguishable from classroom instruction. If it isn't, this just shows that we don't really care about children or students. I don't understand what makes this assertion obviously or necessarily true.

Rick: I don't recall anyone's arguing that in this discussion. My points were (1) that classroom instruction needs the kinds of assessments that help teachers figure out what they need to teach and how to teach it most effectively, (2) that assessments just for the purposes of giving a grade were in some cases not necessary, representative, or fair, since grades are often not necessary, representative, or fair, and (3) that tests often tended to promote classroom and course selection behaviors and attitudes that were antithetical to learning.

Greg: I held a discussion on grading during my last class session.... To make a long story short, I asked them whether they would prefer an educational system in which grades were not assigned. The result wasn't even close: for many different reasons, most preferred a graded system. One student remarked that in college it should be the professor's responsibility to establish, maintain and communicate standards, while it should be the student's prerogative to take the professor's class, i.e., to accept the standards.

Rick

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Fri, 17 Dec 1993 11:06:08 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: TEACHING WITHOUT TESTING, SORT OF To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Fri, 17 Dec 1993 08:02:33 -800 from <joangf@FALLBROOK.CSUSM.EDU>

In response to Joan Gipson-Fredin. Sorry, I didn't mean for us to get bogged down in the sports analogy, as I think you could tell, given your last question. I only meant that sports was not clear-cut objective assessment either in the way it often seems.

But in answer to your last question "...can you suggest an analogy that does \work|? I can't think of any assessment model considered to be fair where the a lone individual's intuitive judgment determines the outcome."

First, remember my grades are given in collaboration with the students individually. Each has to accept the grade on my or his/her own rationale or that grading method is NOT used. They can always opt for an exam or paper or some such if they think my judgment is incorrect. So the grades are not just based on my intuitions.

But isn't grading an essay a method that generally boils down to a teacher's lone intuitive judgment? Or a teacher's rejecting alternative 'objective' answers that a student might have a good argument for? A friend of mine even failed a state bar exam under suspicious grading circumstances, and the bar association of Alabama has a rule that no exam answers can be re-graded -- either by the same grader or by any other grader. Not only is the grader's judgment considered sacrosanct, but his first judgment is. The Alabama Supreme Court upheld this policy, as did the U.S. Supreme Court.

Also, most boss's lone judgments do not get a review process by employees.

These things don't seem fair to me; but they apparently seem fair to zillions of people.

I don't think lone intuitive judgment is a fair basis for GRADING, but I am not sure other assessment methods that seem objective don't often boil down to lone intuitive judgment, though not in an obvious way. Sports boils down to that though not in an obvious way. Much traditional grading also does....

Rick

Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Fri, 17 Dec 1993 09:53:37 U From: Cotter_Cindy <cotter_cindy@MSSMTP.LACOE.EDU> Subject: Infallible Assessment X-To: edpolyan%asuacad.bitnet@arizvm1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Rick Garlikov wrote in response to Greg Camilli: "You wouldn't mind being assessed (say for the need for open heart surgery) by methods known to be fallible? You wouldn't mind a teacher determining your future by using methods known to be fallible? It is one thing to use a method that you don't know to be fallible; quite another to use a method that is demonstrably fallible. And, if by ASSESSMENT you mean grading (as opposed to trying to figure out what you need to teach), then assessing on known fallible grounds is not right to do to another human being who does not deserve that grade, especially if the grade has important and irrelevant consequences."

Demonstrably fallible assessment methods are used all the time in many fields to make important decisions. We give police officers guns and allow them to shoot people, knowing police officers are fallible. Would you prefer NOT to have open heart surgery on the grounds that no infallible assessment device existed when in fact you needed the surgery to save your life?

Cindy Cotter Cotter_Cindy@lacoe.edu

Date: Fri, 17 Dec 1993 17:11:03 -0500 From: Greg Camilli <CAMILLI@ZODIAC.BITNET> Subject: Re: response to G. Camilli's objections To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Rick, I will share your comments with several students in my measurement class, including the the woman whose thoughts I conveyed in my last post. I'll post their comments, probably sometime in the new year. Also, let me add that when I speak of measurements as being fallible I mean that they contain some unknown degree of error. Of course no one would use a test that is demonstrable fallible. That would imply the "true" measurement is in hand to use as a standard. I don't reallt think you circumvent this problem by having your students agree with their grade. Or perhaps you do think this creates an infallible standard?

Date: Fri, 17 Dec 1993 19:30:11 -800 From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: TEACHING WITHOUT TESTING, SORT OF X-To: X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.bitnet> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9312171720.AA11804@fallbrook.csusm.edu>

On Fri, 17 Dec 1993, Rick Garlikov wrote:
**** In response to Joan Gipson-Fredin. I only meant that sports was not clear-cut objective assessment either in the way it often seems.

Alas. "Tis true. What we must ultimately discard is our notion that any judgment can be made without passing through a fallible filter. Because of that fallibility, at least for important decisions, we must have checks and balances. In assessment those ideas translate into terms such as validity, reliability, and "multiple measures."

**** But in answer to your last question "...can you suggest an analogy that
**** does work? I can't think of any assessment model considered to be fair
**** where a lone individual's intuitive judgment determines the outcome."
****
**** First, remember my grades are given in collaboration with the students
**** individually.

I did not know this, Rick, because I joined this list too late. I would like to read your original posting. The ideas you are grappling with strike me as very important.

Incorporating a second judge, a student who is likely to be well-informed about his or her own effort, does add another point of view, even if it isn't a disinterested one.

So the grades are not just based on my intuitions.

I see that you are a conscientious psychometrician at heart.
****
**** But isn't grading an essay a method that generally boils down to a teacher's
**** lone intuitive judgment?

Yes and no. In the classroom the teacher often is the sole decision-maker, and many meaningful indicators of progress gathered over the semester can provide needed balance.

But in cases where writing ability counts a great deal and the product of a sole test session is used in evaluation, the standard for assuring fairness differs. Often it is a team of judges who decide through a "holistic scoring" process. The rules of holistic scoring seek to counter the potentially unfair aspects of grading essays--among these, differences in topics, readers, readers' standards, and reactions to students' personalities and identities. You might enjoy reading Edward M. White's _Teaching and Assessing Writing_ Josey @ 1980, for an especially clear and lively look at the surrounding issues.

**** A friend of mine even failed a state bar exam under suspicious grading circumstances, and the bar association of Alabama has a rule that no exam answers can be re-graded -- either by the same grader or by any other grader. Not only is the grader's judgment considered sacrosanct, but his first judgment is. The Alabama Supreme Court upheld this policy, as did the U.S. Supreme Court.

Unfortunately, it is not unusual for lawmakers to set great store in what can turn out to be goofy testing practices. When the tests involve professional licensing, it's really tough for an individual to make a challenge. Generally speaking, either they don't want to be critical of the process because they would like to become members of the club or because they just have and would like to believe that the process is a model of fine discrimination. It takes courage to challenge these systems and an awareness of how such processes should work. Eventually the judges will catch on. Let's help 'em.

**** Also, most boss's lone judgments do not get a review process by employees.

This does not mean that employees find them fair. Because of the potential for inequities, most public agencies counter possibly arbitrary and capricious judgments with a review system to assure due process.

**** These things don't seem fair to me; but they apparently seem fair to zillions
**** of people.

It's good that they don't seem fair to you. They probably don't seem fair to many other people either, but there are many barriers to improving testing systems.

**** I don't think lone intuitive judgment is a fair basis for GRADING, but I am
**** not sure other assessment methods that seem objective don't often boil down
**** to lone intuitive judgment, though not in an obvious way.

Sports boils down to that though not in an obvious way. Much traditional grading also does....

**** An important point. Biases may not be at all obvious, but they are there. Stephen Jay Gould (_Mismeasure of Man_) would agree. So where does that leave us? Still with a responsibility to make decisions about who goes forward and who does not, still with a desire to do that as fairly as possible. We will never achieve a "once and for all" perfect standard, but we must be as fair as we can. The principles of fair academic assessment (like the principles for fair officiating in sports) have been hashed over, represent our current collective best effort to prevent injustice, and are nonetheless fallible. Our best recourse is to know them, apply them, and challenge them when needs be. Unfortunately many teachers, who could be among the most articulate spokespeople for improvement, know intuitively when something is awry, but leave college without the training to help correct the problems. The injustices in assessment systems will respond to public pressure when there are enough caring and well-educated people who are also well-equipped to take them on.

Joan Gipson-Fredin

joangf@fallbrook.csusm.edu

Date: Sat, 18 Dec 1993 03:29:34 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Fri, 17 Dec 1993 09:53:37 U from <cotter_cindy@MSSMTP.LACOE.EDU>

Cindy, Greg, and Joan all picked up on my response to Greg about whether he would want to be assessed on an important matter by a demonstrably fallible assessment procedure. They all pointed out in one way or another that since we generally did not have infallible procedures for much of anything, my standards for tests were a bit too high.

I think I tried to pack too much into the word "demonstrably", since that post was already very long, but for those still interested in this discussion, I will explain here what I was trying to get at. I think there is some kind of ambiguity in the fallible/infallible distinction, but I can't quite sort that out, so let me just try to explain in different words what I meant.

I believe that formal procedures never, or almost never, can replace judgment, though they enhance judgment in certain ways; and that what we need in any kind of situation, in education or anywhere, is to use the best judgment we can -- which may include formal judging procedures, but cannot be limited to them for the following reason: Formal judging procedures are attempts to eliminate the problems with intuitive judgments -- that is, the KNOWN problems with intuitive judgments. But new problems are discovered periodically. Discovering problems with formal judging procedures requires judgment and insight itself. It would be folly to ever assume we have perfected a judgment procedure that itself never needs to be monitored and judged in the light of new information, new circumstances, or new perspectives. Such folly is the secular equivalent of Pharisee-ism.

Judgments and judging processes always need to be monitored and, where necessary, refined. And reflective, reasoned judgment always, or almost always, needs to be the guide. If you notice that autopilot is flying you into the ground in a particular case, it is time to suspect there is something wrong with the autopilot, even if in general the autopilot flies much better than you do. I am taking formal assessment procedures to be something like autopilot assessment procedures. There are three problems with formal assessment procedures of the sort we have been discussing: (1) they are subject to new discoveries that might show them in need of amendment, (2) they may not be properly understood or used by practicing teachers (which may be what Gene meant when he said that 'nobody' understands validity (or whatever) anyway, and which is what I was worried about with regard to students trying to learn these concepts in ways that were meaningful to them outside an ed school assessments course, and (3) they are subject simply to accidental mistakes, which judgment is necessary to detect.

Examples of the latter case are recipes typed up with obviously wrong ingredients or quantities, or math calculations that give very erroneous answers. (In teaching math, one of the problems is getting kids to understand how to use 'rough judgment' to help them spot egregious mistakes they make in applying formal procedures. If you don't use judgment, there is no reason to suspect any answer you have double checked merely procedurally (making the same mistake each time) is incorrect. Formal procedures merely crank out answers, not necessarily right or reasonable answers.) At any rate, the point about fallibility I was trying to make is that it is more risky than necessary to relinquish reasoned judgment to merely formal or procedural judgment IN IMPORTANT KINDS OF CASES (like grading) because there are too many possibilities for error in merely following, or trying to follow, formal procedures.

My point is not that formal assessment procedures are wrong to use, but that they are wrong to be totally and blindly relied upon without continuous checking and monitoring by reflective and reasoned judgment in those kinds of cases where the extra effort is warranted by the importance of the consequences. (In cases where the consequences are not that important, the extra effort is not so necessary. E.g., our bathroom scale sometimes appears to be five pounds off. That is not worth getting a new scale to correct.)

The general point about not relying solely and blindly on formal procedures is important not only in education but in all other areas of life also. For example, law tends to be a fairly formal enterprise; and therefore it often leads to the obviously screwy sorts of judgments and results that either get us to change the law or to shake our heads at the lack of wisdom or concern of legislators and/or judges. Or following a doctor's orders blindly can lead to problems. An older man I know was released from the hospital after a bout with pneumonia, but at home was having really terrible difficulties which he and his wife thought were just the normal consequences from pneumonia coupled with age. The symptoms sounded suspiciously to me like he was having a medication reaction and I had them check with a pharmacist, who confirmed the symptoms were the likely result of a dangerous drug combination. When the medications were changed, the man recovered almost immediately. It was not easy talking the couple into calling the pharmacist; they assumed the doctor and hospital knew more about health than they or I or the pharmacist would. GENERALLY THAT WOULD BE TRUE, but you cannot let what is generally better than your own judgment simply totally replace your own judgment. (Autopilot into the ground....)

My favorite experience with formal procedure replacing judgment is the case where I bought a chocolate shake at a brand new McDonald's. The shake was slightly off-white in color, and it tasted like a not very good vanilla shake. I thought the counter high school girl had made a mistake. She and another girl with her explained that is just the way their chocolate shakes are. I figured the manager needed to know there was a problem with the shake machine, and asked to speak to the manager. She came out and assured me the shake machine had just last week been installed by McDonald's experts to McDonald's specifications and there was no problem with my shake. "That is the way a McDonald's chocolate shake is supposed to be!" I said I had had many different McDonald's chocolate shakes from all over, and this was not like any of them ever. She said "Well, this machine is the way it is supposed to be so there is nothing wrong with that shake." After she went back to her office, I thought of what I should have said to her, and said to the girls, "Gee, I should have had her taste a vanilla shake and a chocolate shake and show her she wouldn't be able to tell the difference." The girls looked at me and said, "No, she would have been able to tell the difference easily. Our vanilla shakes taste like chalk." THEY knew something was wrong.

The point of all this is simply that neither initial "intuitive" judgment alone, nor formalized procedures alone, nor the two in combination give the most reasonable and reasonably justifiable assessment procedures. What is needed is reflective judgment that looks at both intuitive judgment and formalize refinements AND at whatever else seems relevant to make a determination that is the BEST one can do at the time. This judgment itself may lead to further refinements, and the process continues. Joan, in her post on this, spoke of checks and balances, but she was speaking of formal checks and balances --validity, reliability, etc. I am arguing that we also have to always allow "informal" checks and balances as well -- in this case, the kind of give-and-take discussions I previously described.

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

[At this point, with the discussion pretty much having run its course, another major discussion on a totally different topic began. Hence, the gap in dates before the next entry.]

Date: Wed, 5 Jan 1994 22:33:03 -800 Reply-To: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> Sender: Education Policy Analysis Forum <EDPOLYAN@ASUACAD.BITNET> From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: Infallible Assessment In-Reply-To: <9312181337.AA11585@fallbrook.csusm.edu>

Happy New Year, folks.

It was my pleasure to conclude the old year with an introduction to the internet and to an engaging conversation about "teaching without testing, sort of." While Rick Garlikov argued the pitfalls of formal assessment and the merits of his classroom assessment system for philosophy students, I argued that society would benefit if a larger segment had more instruction in formal assessment.

Rick has since kindly shared his original post with me and has commented further. I see that we are not so very far apart. . .

On Sat, 18 Dec 1993, Rick Garlikov wrote:

**** Formal judging procedures are attempts to eliminate the problems with
**** intuitive judgments -- that is, the KNOWN problems with intuitive judgments.
**** But new problems are discovered periodically. Discovering problems with
**** formal judging procedures requires judgment and insight itself. It would be
**** folly to ever assume we have perfected a judgment procedure that itself never
**** needs to be monitored and judged in the light of new information, new circum-
**** stances, or new perspectives.... Judgments and judging processes always
****need to be monitored and, where necessary, refined. And reflective, reasoned
**** judgment always, or almost always, needs to be the guide.

To this I say, "Amen." Oscar Krisen Buros would surely applaud. But Rick goes on to say:

**** If you notice that autopilot is flying you into the ground in a particular
****case, it is time to suspect there is something wrong with the autopilot, even
****if in general the autopilot flies much better than you do. I am taking
****formal assessment procedures to be something like autopilot assessment
****procedures.

This seems to be the crux of our differences and largely a matter of a need to define terms. To me "formal assessment" is a system properly *based upon* reflective, reasoned judgment and intended to assure reflective, reasoned monitoring. Tests of all sorts originate with reflectionupon what is worth assessing and at best conclude with an evaluation of how well the test served its intended purpose (followed, ideally, with improvements before the test is reused). Rick's objections seem to me to zero in on *violations* of good formal assessment--tests which miss the mark, fallacious numeration (remember his great example of the essay returned unmarked but mysteriously scored as an"86" with a score of "87" crossed out), and overreliance on the results.

But, Rick, let's not throw out the baby with the bathwater. Good formal assessment practices attempt, among other things, to ensure that all of the students get asked the most important questions (or are assigned the most telling tasks), that all students have a fair opportunity to show their stuff, and that they are impartially evaluated according to consistent standards. One of the more important features of formal written assessment is that all of its stages are usually recorded, allowing for analysis and reflective judgment about its value by independent judges.

As you have so far described your approach, Rick, it seems to rely very heavily on your skillful application of your good intentions to teach all of the main ideas, involve everyone to the best of their ability, and to remember how well they did. Keeping track of it all--while engaged in Socratic dialogue, no less :) -- sounds like a real challenge. Chances are there's some as yet unmentioned underlying formality to your method. (You seem to be bent on discovering all of the principles of fair assessment through philosophizing.) Please tell me more.

Joan

joangf@fallbrook.csusm.edu

Date: Sat, 8 Jan 1994 19:34:08 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Wed, 5 Jan 1994 22:33:03 -800 from <joangf@FALLBROOK.CSUSM.EDU>

Joan Gipson-Fredin:
**** Rick Garlikov wrote:
****
**** Formal judging procedures are attempts to eliminate the problems with
**** intuitive judgments -- that is, the KNOWN problems with intuitive judgments.
**** But new problems are discovered periodically. Discovering problems with
**** formal judging procedures requires judgment and insight itself. It would be
**** folly to ever assume we have perfected a judgment procedure that itself never
**** needs to be monitored and judged in the light of new information, new circum-
**** stances, or new perspectives.... Judgments and judging processes always
****need to be monitored and, where necessary, refined. And reflective, reasoned
**** judgment always, or almost always, needs to be the guide.
****
****To this I say, "Amen." But Rick
****goes on to say:
****
**** If you notice that autopilot is flying you into the ground in a particular
****case, it is time to suspect there is something wrong with the autopilot, even
****if in general the autopilot flies much better than you do. I am taking
****formal assessment procedures to be something like autopilot assessment
****procedures.
****
****This seems to be the crux of our differences and largely a matter of a
****need to define terms. To me "formal assessment" is a system properly *based
****upon* reflective, reasoned judgment and intended to assure reflective,
****reasoned monitoring. Tests of all sorts originate with reflection upon
****what is worth assessing and at best conclude with an evaluation of how well
****the test served its intended purpose (followed, ideally, with improvements
****before the test is reused). Rick's objections seem to me to zero in on
*****violations* of good formal assessment--tests which miss the mark,
****fallacious numeration (remember his great example of the essay returned
****unmarked but mysteriously scored as an"86" with a score of "87" crossed
****out), and overreliance on the results.
****

Now, I am not certain about how you or other test experts "evaluate how well the test served its intended purposes", but it is this aspect that I am most concerned with; and I would think there is no way to judge that totally by merely examining the tests through some sort of formal analysis. You cannot find that some people did not mispeak themselves or misunderstand the question or have assumptions you need to know, etc., all in quite rational, legitimate, intelligent ways. And my informal observations show me that teachers and students (like many other people) frequently misunderstand each other without realizing they are doing so, but which may be obvious to a third party. Or which may not be obvious to anyone until some "crucial" evidence is stumbled upon that demonstrates there is a problem or some unsuspected dichotomy. Look- ing at the tests will not demonstrate whether the tests are reliable each time; You still have to talk with the students.

Further, if you are teaching, and a student misses something on a test, I presume a good teacher wants to help the student understand what he/she missed. Now, it frequently does no good in that regard just to tell the student the correct answer. Suppose the teacher had merely told the students (as she did) that "Tom is sleeping" IS a sentence. They might get that right on a re-test, yet have no clue why something that is an "inaction" is an "action". (Remember, she had told the class sentences have to name a 'thing' and an 'action', and they had all thought sleeping was not an 'action'.) Often when students miss a test item, or when much of the class misses a test item, the teacher will launch into another (or repeat the same) lecture about what the answer ought to be -- again without trying to see how the students were thinking about the problem and how to solve it. So again the lecture will often be without relevance to how the students are thinking about the material.

But moreover, students can often get things right in some mechanical or lucky way without any real understanding of the process. That is not uncommon. And talking with them is the only way I know of to try to discover this. For example it was a common practice on college board exams for a long time (and may still be) to put a likely incorrect answer in front of the correct answer. So if a student wasn't sure whether to do a problem one way or the other, picking the second answer was more likely to be correct. Or, as I pointed out, the third grade girl who was able to do subtraction problems of certain sorts right by using unreliable methods that simply happened to work for the kinds of problems they were working. In calculus, we often had to use the chain rule to get certain kinds of derivatives. On one exam we had to use a "double" chain rule in an abstract problem. I not only missed it; I never understood how to that particular problem at all, and could not understand the explanation. I now believe I never really understood the chain rule at all, but could use my version of it to work "normal" problems. Yet, the missing of that one problem did not prevent me from getting A's in calculus, even though I think I had no real understanding of a very major, important element of calculus. I could work the 'normal' test problems the teachers thought showed a mastery of the subject. If they weren't going to talk with me, how would they determine otherwise?

****But, Rick, let's not throw out the baby with the bathwater. Good formal
****assessment practices attempt, among other things, to ensure that all of the
****students get asked the most important questions (or are assigned the most
****telling tasks), that all students have a fair opportunity to show their
****stuff, and that they are impartially evaluated according to consistent
****standards.

Ahhh, but there's the rub. How do you know which questions or tasks are the most "telling"? I agree that IF you knew what the definitive questions to ask were, that would be a good start. But I am saying you cannot always know you know them.

Further, you cannot tell whether a student is being given a fair opportunity "to show his/her stuff". You are seeking something analogous to definitive experiments in science. And those are hard to come by even in science, and are often hotly contested by those whose theories seem to "lose" out.

Then you think that standards' being consistent somehow makes them also accurate. That is hardly necessarily true. Teachers can have faulty standards and thus award the best grades to kids who parrot them, rather than who give actually better or more intelligent answers. Or a teacher can have a grade averaging system that messes up a good student who happened to have a bad day or who misunderstood one thing at test time that skewed his average badly. Yet in many other ways, it might be demonstrable that student was the 'best' student in the class.

****One of the more important features of formal written assessment is
****that all of its stages are usually recorded, allowing for analysis
****and reflective judgment about its value by independent judges.

Great except (1) that almost never happens anyway; other profs don't regrade student exams (even the BAR exam I told about); (2) the problems I have been discussing won't show up on re-grading any more than grading, and won't be any more apparent to other graders than the first.

****As you have so far described your approach, Rick, it seems to rely very
****heavily on your skillful application of your good intentions to teach all
****of the main ideas, involve everyone to the best of their ability, and to
****remember how well they did. Keeping track of it all--while engaged in
****Socratic dialogue, no less :) -- sounds like a real challenge.

Ahh, but this, and Alan Davis's comment show you both miss the point of what I was trying to say. 1) I am teaching to teach, not to grade; grading is an aftermath and is based on the best they can do, not some kind of average. I don't have to keep track of all the bad things they do or the wrong things they say -- as long as by the end of the term I know what the best they can do is. And they have zillions of opportunities to show me their best.

2) The Socratic method --where I really listen to their answers-- makes what they say tend to stick with me, because I have to think about it, and sometimes think up arguments against it, or think up ways to strengthen an idea that they only have the kernel of. Almost everything they say is interesting and memorable in some way or other. (I have written papers to them in response to class discussions when I felt the need. I even have written a 600 page manuscript that grew out of a three page paper I wrote a class in response to an issue where I disagreed with them -- and they still disagreed with me, for challenging reasons, and I had to write more. So I wrote six more pages and we discussed them. Then we moved on, but by then I was hooked on this topic, and kept writing long after the course was over. I take what they say seriously; so it is not hard to remember it.)

And I write down who said some things, if I think I will have difficulty remembering who said what. But the "what" is easy to remember. In order to teach them, I --as I have maintained all along anyone does-- need to know what they are thinking that I need to change or add to. I don't use the Socratic method as a game; I use it to find out what they think and why they think it. The whole course is a kind of dialogue back and forth between them and me, and between them and their classmates. We try to resolve every issue we discuss. We take each issue at least as far as we can. I am deeply involved in that discussion, so I know where we are and where most of them are in each issue. I am not having to keep track of every thing they say and then give them a 48/70 ratio or some such. I care where they end up; not how hard it was for them to get there or how many errors they made along the way. Same as when I teach a kid to ride a bike. I can remember most of the riding problems of most of the kids I taught (like when my sister hit a tree AFTER she realized she was riding by herself and started looking around for where I was), but I don't grade them on their problems. I just want them to learn to ride. They all do. What more do you want of them? (Besides safety or maybe bike maintenance, etc. -- but I mean only with regard to basic riding ability. They can start, stop, turn, go where they want. That is all that is required with regard to "learning to ride the bike".)

**** I suspect
****there's some as yet unmentioned underlying formality to your method. (You
****seem to be bent on discovering all of the principles of fair assessment
****through philosophizing.)

Maybe there is some formal mechanism to my methods. I don't know. But its being formal isn't what will make it right, even if it is right. And, no, I am not seeking the principles of fair assessment; I am only seeking to make particular arguably fair assessments. Before you can have principles of fair assessment, you have to be able to identify what your principles need to distinguish between. Otherwise you run the risk of having a formal structure that has no "real" significance for what it distinguishes. Trial by combat is a formal structure; but it does not likely decide criminal guilt or innocence, does it? Nor does looking at the procedure and making modifications in it, so as to include "stand-ins" or "handicaps", etc.

Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Mon, 10 Jan 1994 10:09:38 -800 From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: Infallible Assessment X-To: X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.bitnet> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9401090403.AA16420@fallbrook.csusm.edu>

It seems to me that we need to clarify terms, especially what we mean by "formal assessment procedures." My nutshell definition is that formality in assessment resides in a systematic attempt to counter known sources of probable error.

By this definition I see many elements of formality in Rick's method. For example, a clear statement of grading criteria counters possible student confusion about what performance counts in his class; ample opportunity to participate in class counters variability in individual performance; and student participation in their own evaluation counters the inevitable fallibility of a lone judge. Of course, in each case "counters" must be taken to mean "serves to counter" because, as far as I know, perfect assessment exists only in literature on the afterworld. If Rick and his students review the current practices from time to time, they probably come up with ideas for refinement.

My point is that there are many ways a teacher can formally enhance classroom assessment, and that those are worth pursuing. K. Patricia Cross's work on classroom assessment is pertinent. So are the _Standards for Educational and Psychological Testing_, prepared (and under continuing review) by the American Educational Research Assoc., the American Psychological Assoc., and the National Council on Measurement in Education.

Furthermore I agree with Rick that down and dirty multiple choice tests and one shot essays followed by summary judgment are not examples of good practice. I believe these remain in use because, generally speaking, 1) we tend to accept or ape the methods that were applied to us (especially when they ranked us highly); 2) we haven't given much thought to the matter and may even mistake their perpetuation as a contribution to "maintaining high standards;" and 3) we want to regard ourselves as fair and are reluctant to question our own practices.

I think that Rick has been an exceptionally good sport to share his approach with this list. While I understand that his goal is good teaching and that he would eschew his role as grader if he could, the question any teacher faces is how to employ assessment in the interests of instruction and fair treatment of students. Since Rick has genially continued to dialogue on this matter, perhaps he'll take a few more questions. These come from UC Santa Cruz and Berkeley juniors and seniors home for the holidays:

1) I would LOVE a system where I was graded on oral remarks. I am highly verbal, don't hesitate to speak out in class, and almost always hit it off with my teachers. But what about the students who are shy or who are reflective learners whose best ideas occur to them later, after the class discussion?

2) I'd be uncomfortable not doing any writing. I may sound pretty good, but often my ideas don't take shape till I write them down. Writing helps me learn, and a record helps me review my thinking. A record also helps if there's a disagreement with a teacher over a grade.

3) There's an imbalance in power which prevents students from challenging a prof's grading methods. I might not like a grading system but probably wouldn't say so if I needed the class. Also a written essay after a grade protest to be graded by the person whose grade you protested doesn't sound like much of a challenge procedure.

Because of the length of Rick's last post, I won't append it in its entirety but will gladly forward it upon request. I do, however, wish to respond here to a few remarks, and I'd be interested in hearing Alan's response, too.

**** It is almost impossible to say something in a way that cannot be
**** misunderstood.

Agreed. So we try hard and ask for feedback.

**** E.g., you and Alan Davis think I am somehow grading without
**** stated standards or just by intuition or whim.

Now, now. When did you begin to believe that I thought that? You have already told us that you explain your criteria to students and that you rely on *reflective* not *intuitive* judgment, and I have no reason to disbelieve you.

**** . . . if I were your student, you would have given me a low grade for my
****previous comments. Instead you are talking with me -- as I talk with my
****students.

Talking, si'. Grading, no. Certainly not in this rare forum for the relatively free exchange of ideas.

**** If Joan and Alan, say, give intricate multiple choice tests, and have
**** specific, uncompromising standards for each letter grade, say 92% for an A,
**** 85% for a B, etc., what do their grades mean about what students actually
**** think and can do? What if they teach poorly? What if they use a book that
**** doesn't explain things very well to a given group of students? They have set
**** standards and have "fairly" tested the students, but what if the standards are
**** meaningless in any real sense?

I wouldn't do that, Rick. I agree completely that such an approach is fallacious. Numbers do not add validity to a test that is fundamentally off target. (Stephen Jay Gould elaborates beautifully on this point in _The Mismeasure of Man_.) Once again, as a philosopher, you have zeroed in on a fundamental principle of good assessment and on an all-too-common belief that such numbers don't lie.

Date: Mon, 10 Jan 1994 18:46:29 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Mon, 10 Jan 1994 10:09:38 -800 from <joangf@FALLBROOK.CSUSM.EDU>

Joan Gipson-Fredin:
****It seems to me that we need to clarify terms, especially what we mean by
****"formal assessment procedures." My nutshell definition is that formality in
****assessment resides in a systematic attempt to counter known sources of
****probable error.
****
****By this definition I see many elements of formality in Rick's method. For
****example, a clear statement of grading criteria counters possible student
****confusion about what performance counts in his class; ample opportunity
****to participate in class counters variability in individual performance; and
****student participation in their own evaluation counters the inevitable
****fallibility of a lone judge. Of course, in each case "counters" must be
****taken to mean "serves to counter" because, as far as I know, perfect
****assessment exists only in literature on the afterworld.

Amen to that.... I can see a certain amount of formalization in the assigning of grades, though it is, of course, not by recipe or algorithm; but would you say there is any formalization in my analyzing or evaluating student comments and responding to them? I would think not. Unless you would count any sort of trial and error, poking and prodding, etc. as "systematic" attempts to counter sources of ... error. And, of course, I am more interested in the analyzing and using it to teach than in grading to give grades.

****I think that Rick has been an exceptionally good sport to share his
****approach with this list.

No; this group makes it very easy to share things with. Everyone who responds tends to have both curiosity and civility. This is probably the finest group of people to communicate with intellectually assembled (or unassembled, as it were) any place on the planet. And if anyone shows one of your beliefs to be wrong, they generally do it in such a way as to make you grateful to find out, and to have the new information or perspective. The amount of knowledge, experience, enthusiasm, and commitment to education and to reflection in general by people that contribute to this list is truly remarkable.

****While I understand that his goal is good teaching
****and that he would eschew his role as grader if he could, the question any
****teacher faces is how to employ assessment in the interests of instruction
****and fair treatment of students.

True!

****1) I would LOVE a system where I was graded on oral remarks. I am highly
****verbal, don't hesitate to speak out in class, and almost always hit it off
****with my teachers. But what about the students who are shy or who are
****reflective learners whose best ideas occur to them later, after the class
****discussion?

Since I grade on the QUALITY (i.e., truth, reasonableness, sign of having been attentive to previous discussion, etc.), quantity of comments is not necessarily in anyone's favor, and can be (and has been) a detriment if they talk without thinking. Also, these things and serious endeavor are what influence me more than "hitting it off with me". There are students I have genuinely liked as people who did not get A's from me, since I was not grading them on likability. There is no time limit on when a student can contribute, or when I can. As I said previously, I even called a student a year later when I thought up new grounds to convince him of something. Students can say or write anything they want at any time. It does not matter if we have gone "past" the subject. We don't really "go past" any subjects. Some things come up repeatedly as new questions or objections "dawn" on different students at different times. But also, I tend to phrase questions and comments, etc. in ways that evoke comment and reflection. I ask specific, pointed kinds of questions -- often with some introductory info first. Then I press students who seem to be thinking but not quite on the mark. The Socratic method (http://www.Garlikov.com/Soc_Meth.html) is a way of asking leading questions. And with the right attitude on the part of the teacher (which I hope to have) that indicates non-threatening intensity, even the most shy students participate. One woman last term told me I had drawn her out in a way that really pleased her because she never spoke up in other courses for fear she was not very good. She participated from the beginning in my class; and I had no idea it was not natural to her.

****2) I'd be uncomfortable not doing any writing. I may sound pretty good,
****but often my ideas don't take shape till I write them down. Writing helps
****me learn, and a record helps me review my thinking. A record also helps if
****there's a disagreement with a teacher over a grade.

So write to me or us then. I tell that to students at the beginning. Nothing says your comments to me have to be oral. Write me; I write back. Sometimes I read or mention what you wrote in class, and respond to it then. My own comments often don't take shape until I try to articulate them orally or in writing. Few people's do. It is difficult to know what you think sometimes until you get it out or down in words and see whether they make sense and seem to be true. Most papers I write come out quite different from the way I thought they would when I sat down to write them. Often when I am talking in class, I have to backtrack and say, "No, I take that back; I just thought of a problem with that." or "with the way I said that." I think it is extremely important to articulate your ideas in some way though, since almost no students who say or write nothing learn much in the course. When they finally have to say or write something, they sound just like people who were never there; pitiful; they have things all mixed up. Having stuff in writing does not generally help in a grade dispute. Half the time nobody else will bother to read it. If they do, it will probably seem worse to them than it did to the teacher. Generally, if you want a second opinion I can ask you the questions again in front of another person and let you state the answers -- or they can ask you the questions. After all, if you know the stuff, you should still know it. Right?

****3) There's an imbalance in power which prevents students from challenging
****a prof's grading methods. I might not like a grading system but probably
****wouldn't say so if I needed the class.

Finally, a great many Jews got in the trains the Nazis put them in because they were afraid to anger the soldiers they outnumbered and get themselves shot. So instead of a few dying, six million died. Jews today say "never again" will they be passive out of fear. I would hope you could speak up about a measly little case of unfair grading practice. If you are going to live your life in fear of consequences for stating your beliefs (even tactfully when tact is called for) then even if you have a longer life, it will seem much, much longer than it actually is.

****Also a written essay after a grade
****protest to be graded by the person whose grade you protested doesn't sound
****like much of a challenge procedure.

**** It is almost impossible to say something in a way that cannot be
**** misunderstood.

Agreed. So we try hard and ask for feedback.

**** **** E.g., you and Alan Davis think I am somehow grading without
**** ****stated standards or just by intuition or whim.

**** Now, now. When did you begin to believe that I thought that? You have
****already told us that you explain your criteria to students and that you
****rely on *reflective* not *intuitive* judgment, and I have no reason to
****disbelieve you.

I'll look up the passage from Alan and the one from you that made me think that, and post them in a minute. Sorry, if I misunderstood what you were getting at. Rick

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Mon, 10 Jan 1994 21:29:38 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Fri, 7 Jan 1994 15:33:00 MDT from <ADAVIS@CUDENVER>

Here are the passages Joan asked me about -- the ones that led me to think she and Alan thought I was grading by whim or intuition. I think Alan's may have misunderstood her, and fostered a mistaken interpretation in me of her remarks. I quote both passages, however, Alan's first.

Alan Davis said:
****It's hard to pick up the thread of the infallible assessment discussion
****after a hiatus of several days, but I found Joan's comments to Rick
****Garlikov right on the mark. Judgment in the interpretation of evidence
****for a particular purpose or inference is always necessary -- it is the
****essence of validity. But we fool ourselves that our "intuitive" assessments
****are somehow mysteriously valid, even when we cannot explain our criteria
****or the observations that led us to them. When we are evaluating someone's
****performance with consequences for them -- as when we assign them a grade --
****fair practice demands that we can describe and defend the procedure that
****led us to it. Otherwise we cannot defend the distinction between
****"informal assessment" and personal bias.
****
****Alan Davis

Joan had written, previous to that: "As you have so far described your approach, Rick, it seems to rely very heavily on your skillful application of your good intentions to teach..., involve everyone ..., and to remember how well they did. Keeping track of it all -- while engaged in Socratic dialogue, no less :) -- sounds like a real challenge." Particularly after reading Alan's post, I took this passage to be a mildly sarcastic claim that this could not be done, and that my grading was not based after all on what I thought it was, and was merely my intuitions rationalized and projected (if that is the right word) by this illusion.

By the way, I deleted a line in the last post that indicated the numbered questions in your previous post were from students of yours, not from you. I hope lurkers following this thread will not have been mislead, and will have remembered those were student questions. If not, this is a reminder.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Tue, 11 Jan 1994 07:01:57 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: assessment addendum X-To: edpolyan@asuacad.bitnet To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

I forgot to say one other thing in answer to the student's question about grading reticent students. One time I had a student who said only two things all term, but they were both such terrific things that demonstrated all I ever sought, and more, that I gave him an A. He argued that was too high, but I pointed out what he had said and why I thought it was so good, and told him just to take the A. At least one of the things he said was in comment to another student. He asked that other student a really pointed question that showed where the other student's thinking had been flawed; and then he pursued the point perfectly. I seek quality, not quantity.

Rick
Rick Garlikov (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Tue, 11 Jan 1994 08:22:34 CST From: "Bolland, Kathy" <kbolland@CCMAIL.BAMANET.UA.EDU> Subject: Re[2]: Infallible Assessment X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Those of you who have an internal or external need to use assessment techniques that bear a resemblance to multiple choice tests should become familiar with Michael Scriven's work on authentic, objective testing--or multiple rating. He states that it eliminates 6 of 9 problems with multiple choice assessment. I don't have a citation, but he talked about it at the most recent American Evaluation Association conference--November in Dallas.

As for me, I hope my 12 year old some day has a teacher/professor who teaches like Rick. If so, I have little worry about the kind of assessment system said person will use.

Kathy Bolland UA Eval & Assessment Lab Kbolland@ccmail.bamanet.ua.edu

Date: Tue, 11 Jan 1994 20:59:16 MST From: Bill Hunter <hunter@ACS.UCALGARY.CA> Subject: Re: Infallible Assessment X-To: EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9401120254.AA61012@acs1.acs.ucalgary.ca>; from "Joan Gipson-Fredin" at Jan 11, 94 7:57 pm

**** When you consider students entering your
**** classes, have you found that black students may be more reluctant (or less
**** practiced and consequently less skillful) than white students or that men
**** as a group are more likely than women as a group to assert themselves well
**** rhetorically? If so, what methods seem to work best for leveling the
**** playing field?

In context, it was clear that this question was not intended to imply unfairness on Rick's part, but it calls attention to the possibility of unfair teachers exercising their biases. Paper and pencil tests may also reflect the biases of their developers, but those biases can be more readily detected and can be challenged with more impunity (blaming the test for biases that reside in the test developer). Although it is now 25 years ago, I vividly recall student-teaching in a vocational high school in which young black women were not permitted to practice their craft on white students (blacks had only begun to be admitted to the program a year or two before) and their efforts to work on each other's hair were damned in advance by a teacher with the firm conviction that "you cain't do nothin' with their hair anyways." Likewise, a black student in the dental assistants program (the first admitted) brought me a very well written paper marked with a C- saying that ALL of the white girls got very high marks and that the teacher "hated" her. I promised to talk to the teacher and did. Her rationale: "If I don't treat this one good, then maybe I won't get anymore. Nobody wants them sticking their fingers in white mouths anyway." That wasn't the deep south, folks, that was Ohio. I know times have changed, but we are deceiving ourselves if we think every teacher at every level evaluates their students without regard to race, religion, ethnic origin, sexual orientation or gender. (Remember, we have people who teach that the Holocaust never happened.) Paper and pencil tests are certainly not a foolproof (pun richly deserved) solution to this problem, but I think they are often a contribution to greater equity.

I like what I hear from Rick. I'd want to trust his judgement. I was once the only student in a class of 30 who voted to have the mark based entirely on the judgement of the instructor (in fact, given some level of regard for the instructor, I always figured that the instructor's judgement was most of what I was paying for). Still, I would be unwilling to advocate a system in which this type of student evaluation was the norm. For those who may have thought me an apologist for the status quo, this may seem very cynical. It is.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691

Date: Tue, 11 Jan 1994 17:02:09 -800 From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: Infallible Assessment X-To: X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.bitnet> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9401110255.AA04701@fallbrook.csusm.edu>

On Mon, 10 Jan 1994, Rick Garlikov wrote:
**** Joan Gipson-Fredin:
**** It seems to me that we need to clarify terms, especially what we mean by
**** "formal assessment procedures." My nutshell definition is that formality
**** in assessment resides in a systematic attempt to counter known sources
**** of probable error.
****By this definition I see many elements of formality in Rick's method.
****For example, a clear statement of grading criteria counters possible
****student confusion about what performance counts in his class; ample
****opportunity to participate in class counters variability in individual
****performance; and student participation in their own evaluation counters
****the inevitable fallibility of a lone judge. Of course, in each case
**** "counters" must be taken to mean "serves to counter" because, as far as
**** I know, perfect assessment exists only in literature on the afterworld.

Rick responded:
**** Amen to that.... I can see a certain amount of formalization in
**** the assigning of grades, though it is, of course, not by recipe or
**** algorithm; but would you say there is any formalization in my analyzing
**** or evaluating student comments and responding to them? I would think not.
**** Unless you would count any sort of trial and error, poking and prodding,
**** etc. as "systematic"attempts to counter sources of ... error.

Actually, Rick, I do see formality in your analyzing, evaluating, and responding. Your goal is to elicit your students' best efforts. Recognizing that some students may be reticent, that people often need to clarify their thoughts, that learning continues in the exchange of information, etc.; you continuously prod, analyze, and give feedback. You also mention methodical employment of a certain intensity and style of questioning. Since all of this is purposeful, planned, and ongoing behavior, I would call it systematic.

Moreover, based upon all that you have said, it seems to me that you have a coherent formal system that serves your instructional purposes well. (We may be getting hung up on different definitions of "formal." I mean that you deliberately address essential elements of fairness, not that you are giving assessment "form" in rigid or superficial ways.)

Rick:
**** And, of course, I am more interested in the analyzing and using it to

Great. A useful assessment system serves to answer the question "How am I doing?" for both the teacher and the student.

I liked your remarks in response to student comments about the imbalance of power in the classroom. (Thank you, by the way, for clarifying in a subsequent post that those were student comments, not mine.) I would like to follow up on your comments because they relate to anther potential source of error in any assessment procedure, i.e. cultural bias.

Rick's remarks:
**** All of life makes it hard to speak up if you are always afraid of the
****consequences of someone's not liking what you say and having some power
****over you. . . .
**** Finally, a great many Jews got in the trains the Nazis put them in be-
****cause they were afraid to anger the soldiers they outnumbered and get
****themselves shot. So instead of a few dying, six million died. Jews
****today say "never again" will they be passive out of fear. I would
****hope you could speak up about a measly little case of unfair grading
****practice. . . .
**** If you are going to live your life in fear of consequences for
****stating your beliefs (even tactfully when tact is called for) then even
**** if you have a longer life, it will seem much, much longer than it
****actually is.

Well said, and I will relay your response to the students who raised this question. I share your point of view. Heck, my favorite movie is "Mr. Smith Goes to Washington." As a baby-boomer, part of my post-McCarthy-era education was to "think independently" and to "question authority." I even had a Rosie the Riveter for a mom. So I applaud you for continuing this tradition.

Nevertheless, some people learn not to speak up to authority (or at least not to do so as an individual) as a survival skill. This adaptive behavior may permeate an entire culture or segment of society. For example, Octavio Paz writes in the_Labyrinth of Solitude_about the characteristic masking of one's thoughts and feelings in Mexican culture. Although I don't recall Paz using these examples, I understand that it is better not to rile an arbitrary, capricious authority who carries a loaded weapon, or who can grab your daughter, or impound your goods, etc. with impunity. Rather than a risky off and on way to present oneself, the mask has become part of everyday expression. Arguably, the authoritarian tradition continues in Mexico partly because people have adapted so well to surviving in it.

Of course, students bring their cultural traditions with them to the classroom where they may conflict with the teacher's expectations. (It can easily happen that neither the student nor the teacher is fully aware of what's awry.) For example, to varying degrees Mexican-American students may "wear the mask"in US classrooms where it is likely to be counter- productive. Minimal class participation, down-cast eyes, and immobile faces aren't read in our culture as signs of "active learning," and the "reflective learning" that may be cooking away behind the mask often isn't recognized, encouraged, and rewarded.

Likewise, psychologist Claude Steele suggests that black students in our society may also shield themselves from authority and become "invisible" in the classroom where the behavior is misread as a lack of intellectual promise or activity.

There is also a body of literature on differences in rhetorical strategies used by (white) men and women which reveals that those who perceive themselves to be on the down-side of the power relationship show more hesitation, hedges, etc. in their speech. Arguing for the general restoration of rhetoric to the curriculum, linguist Suzette Haden Elgin claims that, in its absence, boys in our society are more likely to receive better, albeit informal, instruction in rhetoric than girls do.

I understand, Rick, that you go to great lengths to encourage participation in your class and that it is quality and not quantity of the contributions that count. When you consider students entering your classes, have you found that black students may be more reluctant (or less practiced and consequently less skillful) than white students or that men as a group are more likely than women as a group to assert themselves well rhetorically? If so, what methods seem to work best for leveling the playing field?

By the way, in community colleges here in California we are beginning to look routinely at "success rates" by race, ethnicity, and sex as a matter of public policy. The expectation is that the demographics of the body of students who transfer or graduate will mirror the demographics of the adult population in the surrounding community. When the proportions don't match, there is an expectation that the colleges will develop plans for improvement. It's a short leap from looking at overall college success rates to rates by course or teacher. Assessment at all institutional levels then becomes keenly important. The challenge remains to harness assessment to the aims of instruction in ways that are scrupulously fair to all students. It will take a lot of teachers like you, Rick, who are willing to turn their attention to these matters, to keep the tail from wagging the dog.

Thanks for continuing the dialogue.

Joan.

Date: Thu, 13 Jan 1994 00:51:30 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 11 Jan 1994 20:59:16 MST from <hunter@ACS.UCALGARY.CA>

RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Wed, 12 Jan 1994 22:58:04 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Tue, 11 Jan 1994 17:02:09 -800 from <joangf@FALLBROOK.CSUSM.EDU>

Joan, I did understand "formal" in a different way. I understood it to mean formally prescribed or algorithmic or automatic. That is why I was having so much trouble with some of the earlier comments. You are using it to mean any sort of deliberate and reflective, systematic or methodical, persistent attempt to evaluate fairly and as accurately as possible. In your sense, I plead guilty to formalism, though it strikes me as an odd sense of formalism, and perhaps not what Bill Hunter might call formal procedures for evaluation. Bill?

I agree with Bill that my grading system could possibly be unfair. I think that is true, however, in all courses that require some sort of judgment by teachers about student answers -- or even the questions to ask of students. Besides single letter grades not capturing students' myriad strengths and weaknesses, the bias of teachers is a real problem with grades. Including my own. Another reason why I think grades are really ridiculous for any meaningful purpose. But....

I have never noticed any difference in my courses between black participation and white, between male and female. As I said in one previous post, I have taught (as a grad student) at the U of Michigan, then in Alabama at UAB, and at a rural community college and a predominantly black urban community college. The only difference I notice in student answers is that it seems to me that students who have had meaningful life and work experiences seem to be better than students who have mostly been simply students. My UAB classes were night classes, and most of the students came after work. My community college classes were made up of a variety of ages, many people working, and many being parents. Most of these people have faced employers that were not ideal. And they have reflective ideas about fairness and right and wrong, etc.

It is not simply a question of age. Some of the best students I have taught were fifth and sixth graders, but that was with more selective topics about things they could deal with relevant to experiences or ideas they could have had. And many of them had definite reflections about things like school and about peer relationships and parents, etc. Some older students are very good; some not very good at all. You know the saying: there is a difference between learning from 20 years of experience and learning from one year of experience that you have 20 times. One of my more interesting encounters was in a demonstration workshop where an assistant superintendent sat in who never had much to say and who was always reserved and concise to the point of near unresponsiveness. I was trying to demonstrate the Socratic method, and he became so animated trying to respond and solve the questions I was asking, that I commented I had never seen him like that before. He just grinned from ear to ear and said he found it really interesting. So did one of the older female board members, but not the other. That group was mostly or all white, I think; don't really remember.

In terms of my own personality, until a year or two ago, I always found it much more difficult in real life to talk with men than with women; but that was not true in school -- either with teachers or students. As long as I am dealing with subject matter of interest to me, I seem able to converse equally comfortably with males and females. And they seem generally equally responsive. Individuals who are not responsive are just that -- individuals; and they can be white males as easily as anything else.

Louis seems to have some of the same approaches to teaching that I do, and his 'random thoughts' seem to contain comments by all different kinds of students. I would be surprised if he ends up with students whose creativity, participation, or ability are distinguishable along gender or racial or ethnic lines. Louis? I would guess that teachers who teach traditional subjects in traditional ways would get more the kinds of dichotomies Joan describes in the literature she mentioned. All the kinds of things that encourage "puppetry" and "parroting" tend to take their toll on a great many students. Perhaps even by gender and/or race. But I, and I am sure Louis, spend a good deal of time and effort trying to overcome all those kinds of inhibiting factors very early in our courses. Louis refers to it as engendering trust that is necessary for learning. I don't think of it so much as trust (though I see Louis' point in using it), as I see it as establishing intellectual comfort and freedom, and getting students to see that it is the pursuit of ideas that is important, not the pursuit of pleasing me or getting grades or trying to guess the answer I would think is right.

My black students see discrimination as a MAJOR ethical issue, whereas a white class won't even think about that much, other than in regard to affirmative action. Other than that, views about sex, guns, violence, honesty, fairness, learning, working, even abortion, tend to be pretty much the same from group to group and class to class. In some cases men's and women's ideas about sex and/or dating or marriage tend to differ, but when they do, nobody seems hesitant to speak up at all to make certain the other side hears what they need to know. Well, ALMOST nobody. There have been a few people who thought sex not a proper thing to be discussed in a classroom (even in a course called ETHICS AND SOCIETY), but they were relatively rare; and were of both genders, or as Gene would have it, of both sexes. Rick

Date: Thu, 13 Jan 1994 08:32:55 -800 From: Joan Gipson-Fredin <joangf@FALLBROOK.CSUSM.EDU> Subject: Re: Infallible Assessment X-To: X-cc: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.bitnet> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: <9401130711.AA09746@fallbrook.csusm.edu>

On Wed, 12 Jan 1994, Rick Garlikov wrote:
**** Joan,
**** I did understand "formal" in a different way. I understood it to mean
**** formally prescribed or algorithmic or automatic. That is why I was having
**** so much trouble with some of the earlier comments. You are using it to mean
**** any sort of deliberate and reflective, systematic or methodical, persistent
**** attempt to evaluate fairly and as accurately as possible.

Well said.

**** In your sense, I plead guilty to formalism, though it strikes me as an
**** odd sense of formalism, and perhaps not what Bill Hunter might call
**** formal procedures for evaluation. Bill?

I'm interested in hearing from Bill, too.

It seems to me that any "test" is mostly process, requiring reflection and judgment. Consider the design process, the decision process for selecting it, the processes of giving and taking a test, and the interpretation and use process that follows it. Principles of fair assessment apply all along the way, and as judgments are made, they give "shape" to an assessment system.

We tend, however, to think of tests in their most concrete manifestation as booklets of questions along with their administration scripts, scoring keys, and resulting numbers. Unfortunately, focusing on the concrete can lead to a very narrow basis for evaluating the "goodness" or "badness" of a test or for deciding, for example, that a test is or is not biased. All of the defining processes need to be considered in the context in which they will be applied.

Through this dialogue we have addressed fundamental assessment principles as they apply to your method. If one of us wrote it up, it's formality would be striking, even though little of it is concrete. Formality, of course, is a matter of degree. So I'm not saying you could not be more formal if needs be, but that the form of what you are now doing is apparent.

I would like to respond, by the way, to a couple of remarks from your earlier posts. First, to your reply to my question about when you began to believe that I thought you relied only on intuitive judgment and had no stated standards: what you read as a mildly sarcastic probe about your ability to recall student comments over the course of a semester (to identify their best effort) was indeed skeptical, but not sarcastic. It may have been the "no less" and the smily face that caused the problem.

And second, in one of your posts you referred to Alan Davis and me as "testing experts." I can't speak for Alan, but after thinking about this rather ambiguous term (which sounded a little like an epithet in context) and the nature of this forum, I thought I should clarify what I do. My role for the past several years has been to work with teachers and counselors (and others) to develop and administer systems for assessing academic achievement for placement in classes, admission to programs, and determination of minimum competency for graduation.

A changing cast of characters, new curriculum developments, changing accountability procedures, etc,. result in frequent dialogue about the application of assessment principles in the interest of learning, fair treatment, and administrative efficiency, and about keeping these interests in balance. I'm always looking for ways to improve the flow of information.

Under the California community colleges' broad definition, I am a member of the faculty, although I do not teach classes. I am, by the way, delighted not to be categorized as an administrator, since it has proven helpful to have as few barriers as possible between me and the professionals with whom I work. As Rick does with his students, I credit reason, effort, and good will (not the power to hurt) with eliciting our best efforts.

This dialogue with Rick and others has been very helpful to me, and I am grateful for the forum which allows it. In his glowing assessment of its membership, I think Rick once again shows good judgment.

Regards,
Joan Gipson-Fredin, joangf@fallbrook.csusm.edu Testing Services Coordinator MiraCosta College 1 Barnard Drive, Oceanside, CA 92056

Date: Thu, 13 Jan 94 12:25:25 MST From: Bill Hunter <hunter@acs.ucalgary.ca> X-Mailer: ELM [version 2.3 PL11v]

Clearly, you got the wisdom you paid for--I got to become judgemental.

There was also one time I didn't question--a C- on a paper in Comparative Education. I thought the professor was a hopeless jerk, but I could not afford to drop the course. In my paper for the course, there was one thing he found fault with--an incomplete citation (I had returned the book to the library without taking down adequate info. I could not then find it because I thought the author was Longmans--the publisher). I explained all this in a footnote and documented my efforts to retrieve the book. I knew this was shoddy, but I did not want to lose the reference and did want to deal with it as honestly as possible. When I got the C-, my response was "Jerk." No point in following it up.

What I am getting at is simply an elaboration of the point that I made in the last post--I like what you do, but would not advocate a system that depended solely on this sort of evaluation. Off-list, Kevin has characterized me as "suspicious" of others. Actually, people are more likely to accuse me of being too trusting than of being suspicious. However, in designing systems of education or instruction, I do think that we must begin with fair procedures rather than having faith that we will always find fair people. Tests are far from infallible and I would not insist on them, but I do think they are a part of insuring that fairness is at least an option.

bill hunter fax: (403) 282-0083 the university of calgary phone: (403) 220-5691 hunter@acs.ucalgary.ca

Date: Thu, 13 Jan 1994 07:40:30 -0500 From: Louis Schmier <lschmier@GRITS.VALDOSTA.PEACHNET.EDU> Subject: random thought--1/13/94 To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Hi there from dark, wet, cold "sunny, warm" south Georgia. Here I am in mourning (UNC lost to TECH last night), dripping wet from the blasted rain that started coming down half way through my walk. I felt like I was in an Iron Man marathon, walking one way and swimming back the other. Anyway, between the curses and the "why mes" I was thinking about an exciting and insightful discussion ensuing on one of the lists concerning classroom assessment. For those who aren't up on the politically or intellectually correct jargon, that means grades. It's like calling a janitor a custodian or a garbage man a sanitary engineer. Anyway, having been distracted by the to-do on some other lists and the demands of the quarter's beginning, I haven't been able to put in my two cents on a subject about which I have very strong feelings, even more so having just finished reading 120 first week wonderous student journals. But that *will* another matter. All I want you to do is read the enclosed holiday greetings.

I am sharing this letter with you to cause us to reflect on our craft and to applaud and celebrate a very courageous, yes couragerous, young lady:

Happy holidays, Dr. Schmier. You are a son of a bitch. You're putting me through hell in at home. Actually, I did it to myself. That C grade I got has made me do some talking with two very upset parents who aren't used to anything but As. Mom was upset that she couldn't brag on me anymore. I told she could, but in a different and better way. Dad called you a son of a bitch. I agreed, but I told him that you are a wonderful son of a bitch!!!

You put me through hell in your class. That wasn't supposed to happen. My parents couldn't understand why I had so much trouble. If was obvious to them that it was all your fault. After all, I was the valedictorian of my class, a straight A honors student. I was supposed to breeze through your class like I did in all my high school and my other college classes. All through school I was told that I was bright and smart and had a great future. I was told I was better than others. I was all this and all that. I really believed all that stuff and was really taken with myself and looked down on others. I thought I was really some hot stuff. All because of my grades.

Now I know that all my teachers taught me was how to pass their tests and those others that everyone took including the SATs. They taught me how to memorize. I call it tell-memorize-test-forget kind of teaching. The only thing I really learned was how to forget very quickly after a test. My last year or so, I cruised on my reputation. Teachers gave me "As" I can now honestly say I didn't deserve. I once handed in a paper with some blank pages inside and got it back with no comments marked an "A." He never even read it! I think some were afraid to be honest and give me less than an A because it would reflect badly on them. I sure learned, not memorized, a lot of history in this class, but not enough. Just enough to pass. I'd like to take the class again and really dig into it. I appreciate it now, but I think the most important thing I learned was humility. There were people in this class and in my triad who had lots lower high school grades but knew how to think better than I could, and I had to start learning from them! Now I know what you mean when you say grades are worth "shit." Everyone says we have to have them, and I guess I still have to play the stupid game, but now I know it's not the most important game in town because they don't say a thing about what I know and what I can know and what I am and what I can be and what I will become. I know now that what's important is that journey you always talked about in class, not the destination, that whatever I do, like that fourth boy, I do honestly and fairly and humbly while considering and helping others along the way just like you did for me.

Have a good one.

--Louis-

Louis Schmier (912-333-5947) lschmier@grits.valdosta.peachnet.edu Dept. of History Valdosta State University Valdosta, Ga. 31698

Date: Thu, 13 Jan 1994 16:15:13 CST From: "Bolland, Kathy" <kbolland@CCMAIL.BAMANET.UA.EDU> Subject: Re: random thought--1/13/94 X-To: Education Policy Analysis Forum <EDPOLYAN%ASUACAD.bitnet@ARIZVM1.ccit.arizona.edu> To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET>

Louis said: "concerning classroom assessment. For those who aren't up on the politically or intellectually correct jargon, that means grades."

I wonder how many people are going to jump on you for that one. Classroom assessment, when it is done well, is MUCH more than grades. The assessment is the judging, the weighing, measuring, evaluating, etc. The grade is just the label for the result of the assessment. Assessment happens all the time--formally and informally. Not just when grades result.

I've assessed you, just from your postings, as a very caring, very verbal kinda guy. No grade though.

Kathy Bolland Evaluation and Assessment Lab (see, I couldn't very well let your everyday talk get by) KBolland@ccmail.bamanet.ua.edu

Date: Fri, 14 Jan 1994 03:58:42 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: random thought--1/13/94 To: Multiple recipients of list EDPOLYAN <EDPOLYAN@ASUACAD.BITNET> In-Reply-To: Message of Thu, 13 Jan 1994 07:40:30 -0500 from <lschmier@GRITS.VALDOSTA.PEACHNET.EDU>

I was particularly touched by Louis's 'random thought' about Barbara, and her letter, and the help she has offered his current students. I will give that post to my 12 year old, who I am constantly working on to try to get her to distinguish between learning and getting good grades. In school, though not outside of school, she often seems to think that learning enough to get an A is simply all that is important. Maybe it would not be so bad if the standards for an A around here were higher, but they aren't. At any rate, she tends to judge herself by her grades, and in spite of what I say, seems to think I do too. She gets down on herself if less than an A (whereas I only care about whether she has learned after the test what she needed to know, and whether that knowledge is meaningful in any way or not), and thinks she is great if she gets an A, even if she does not understand the stuff all that well. Maybe Louis can help get through to her better than I. Thanks, Louis. And Barbara.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)

Date: Fri, 14 Jan 1994 03:26:39 CST From: Rick Garlikov <DEMS042@UABDPO.BITNET> Subject: Re: Infallible Assessment

In-Reply-To: Message of Thu, 13 Jan 1994 08:32:55 -800 from <joangf@FALLBROOK.CSUSM.EDU>

I appreciate Joan's explanation of what she does with regard to developing assessment procedures and how she goes about it. It seems she and I are not as different in our views as I, and perhaps she, both thought at first. I too have been grateful for the discussion and her participation and challenging questions. I like the way she describes how she sees her job, and think it is great her school or region has such a position available to help teachers and counselors make useful and helpful assessments for students.

I did not mean "testing expert" in a pejorative way or as an epithet. I meant simply someone who understood all the mathematical or statistical kinds of things about tests that I know very little about, except in general concept.

Rick
RICK GARLIKOV (DEMS042@UABDPO.DPO.UAB.EDU)