Teachers at the Crossroads:
Computers and Composition, Volume 17, Number 1 (2000) 31-40
Michael
Day
Note:
this article is adapted from a keynote address given at the Second Virtual
Conference of the Collaboration for the Advancement of College Teaching and
Learning, April 1999. The keynote
address is available at http://www.niu.edu/~tb0mxd1/cybercon2.html.
Technology and Evaluation
As
demands for technology use from state governments and local school
administrations have grown, more and more technology-using teachers have been
confronted with an important issue: how to maintain fairness in the evaluation
process of those who use technology for their classes. In the best of worlds,
we would be free to innovate with technology to our heart’s content, confident
that our innovations would be recognized at evaluation, tenure, and promotion
time as long as they were backed up by sound pedagogical theory. However, it is far more likely that we will
need, for the present, to explain not only what it is
that we do but why we feel compelled to do it.
Indeed,
I am sure that most readers of this journal have at least dabbled with computer
and Internet-based tools in their classes and scholarship, and many of us have
been asked by our schools to try out these tools for distance learning or to
enhance our regular classes. Most of us are probably in what Charlie Moran
(1992) calls an “amphibious condition,” (p. 14) finding our legs in cyberspace
with more and more confidence as we become accustomed to the new learning
environments of web pages, local area e-mail and chats, and the wider ranges of
Internet-based exchanges in our own work.
However,
many of us have come to realize that we take some substantial risks in bringing
technological innovation into our classes.
First, some teachers have found that students can be harsh critics when
evaluation time comes around. Because
computer technologies such as e-mail, synchronous chat, group discussions and
class web pages may be new and unfamiliar to some students, these students face
a steeper learning curve and may encounter more technological “glitches” (Palmquist etal. 1998) that leave
them frustrated at having spent as much or more time learning technology than
the subject of the class. Further, some
teachers are not as explicit as they could be about the rationale for using
computer technologies in the class, which often contributes to student
dissatisfaction. Indeed, James Morrison
(1997) identified just such student dissatisfaction in his case study on using
technology productivity tools in teaching.
He attributes the low student evaluations of his class to the fact that
he “had not ‘sold’” his objectives to the class and that he “had not
compensated for the fact that these added requirements increased their class
work substantially.”
Not
only does integrating computer technologies increase
student work, it also can lead to heavy faculty workloads that may hamper
teachers’ efforts to excel in the traditional evaluation categories of
teaching, research, and service (Kaplan 1991). For example, Elizabeth Sommers (1992) notes that some faculty even “steer clear”
of “such professional traps” because
they have seen colleagues become overloaded with the tasks of innovating with
technology. (p. 50) If
technological innovation takes time away from other important (not to mention
easily recognizable and justifiable professional activities, how can we account
for that innovation when it comes time for evaluation?
A
related issue involves the amount of “control” teachers appear to have over
their classes. Many computers and writing specialists tout the advantage
computer networks to foster what they call the “decentered”
or “student-centered” classroom, one which does not focus on the teacher and
uses individual student and student groups as multiple centers of expertise
(Barker and Kemp 1990; Hawisher and Selfe 1991). Outside
observers of these classes, be they online or within the more traditional four
walls, may misunderstand the kind of learning that is occurring in these
contexts, and be quick to judge the teacher as “not in control of the class”
and the students as out of control.
Another
risk we take is that some administrators and peers who evaluate our work may
not fully understand or may misunderstand our rationale and goals for using
computer technologies with our classes (Sommers
1992). Some of them do not use
electronic technologies in their own work, but far greater numbers have not
attempted to teach either undergraduate or graduate courses which incorporate
computer technology. For example, they
may be quite familiar with the process of conducting Internet-based research,
but have less understanding of how one teaches such processes, especially when classes on the
Internet yield up such a different looking “product” (or record of what went
on). As my colleague Janet Cross is fond of saying "How can they know the
dance without dancing?" We are open to other possible misunderstandings in
that because we are finding our way, charting new territory with these
technologies, we may be more apt to make mistakes or rethink our approaches. Further, class work (draft or discussion) on the Internet “can
depart dramatically from traditional conceptions of writing” (Herrmann 1991, p.
157). It may look
"messy" or disorganized to the inexperienced eye or when viewed out
of context, resulting in accusations of poor teaching.
It
is this final observation, the possibility that unsympathetic or inexperienced
eyes may be viewing our students’ online work in the context of evaluating us, that brings me to my central focus in this article. I
will discuss a phenomenon I call “the electronic panopticon”
and its role in faculty evaluations, then briefly present a recent example of a
case in which that electronic panoptic effect had an unfortunate result in a
faculty evaluation. Finally I will offer up a few guidelines which may help
teachers who use technology educate our colleagues about their work, provide
rationale for what they do, and document their work with technology.
About
250 years ago, Jeremy Bentham (1995) conceived of a
prison in which inmates, isolated from each other, could be seen by guards at
all times but not see others in the prison. As such, the inmates are the object
of surveillance, of information which can be gathered about them by watchers
who remain unseen. It is no accident that Foucault (1977) was so successful at
applying the idea of the panopticon to the hospital,
the workplace, and the educational institution, for here too can subjects be managed by a one-way flow of information. In the age of
information and the Internet, the concept of the panopticon
is vitally important in that it can help us understand the ways in which
computer and network technology allows some people to monitor every aspect of
the life and work of netizens (network citizens) who
spend a large part of their lives (online). As society becomes increasingly
dependent upon information collected by computer databases and shared at
lightning speeds, online people will also be subject to increased scrutiny by
observers unknown and undetected.
For
academics, the panopticon effect can work against
faculty members in the evaluation process in the following ways. We faculty
members may be very proud of our work on line and offer URLs and other online
addresses for our web pages and class work to those who evaluate us. However,
we need to recognize that evaluators may be watching us in ways of which we may
not be aware, and could be judging us by criteria that bear little relevance to
our pedagogical goals. Increasingly I hear of department heads and program
directors who routinely check the web pages of faculty to be sure that their
syllabi conform to departmental and program guidelines. Further, some faculty fear putting any student work on the web because the rough
draft quality of some work may reflect poorly on the teacher. Unfortunately,
the number of cases involving faculty members who are penalized and blamed
instead of rewarded for innovation is on the rise.
Granted,
we are at a crossroads with regard to notions of public and private on
the Internet and local area networks. Gone is the first blush of wonder that
students and teachers had at being able to share their work so effortlessly on
the web. In its place are rather practical questions about student privacy in
relation to the fantastic ability that students and teachers have to reach
larger audiences on the web. When we work on paper, we have the ability to
choose which papers we share with others, and which documents will form the
basis for evaluation. On the web, unless we password-protect some sites,
everything we do is subject to scrutiny from those who evaluate us. It is time
that those of us who generously share so much of our work with others on the
web recognize the possibility of outside observation over which we have little
control. Further, we should seek to limit our vulnerability by 1) specifying
which web pages are to be used in evaluating our work, and 2) providing as much
context and rationale as we can for the online work to be evaluated. But further specific guidelines come later;
let us first look at a situation in which the electronic panoptic effect
figured heavily in a faculty member’s evaluation.
An example situation[1]
Many
of us who participate in ACW-L, an Internet discussion group for the
My
colleague had built the online composition program at this university from the
ground up, and she had every reason to be proud of it. But because she had
limited publications (her CD ROM distance learning strategies was not
considered a publication because it was not picked up by a publisher), for
tenure she had to be rated as “excellent” in teaching, and a ninety minute meeting
was scheduled to discuss that teaching. As part of her tenure dossier, she had
included both printouts and URLs of her class web pages for evaluators to
review. These were the course web sites
for all of her distance learning classes, classes taught entirely on line. However,
before meeting to discuss her teaching, a few members of the committee did some
investigating of her other online class materials until they found examples of
student work which they felt was substandard, not up to the quality expected of
students at that school. These they printed and distributed to the whole
committee as evidence of lack of involved teaching, even though these were just
drafts of student papers, a record of process, not product. They also looked at the times (and intervals
between these times) that the teacher had logged on, and assumed that she had
not participated enough in the class.
They then used this combined evidence to sway the committee to conclude
that her teaching was capable plus,
but not excellent. On this basis, she
was denied tenure, but offered a chance to regroup and apply again.
This
committee may not have been terribly technologically literate, but the one
thing they thought they knew about was how to evaluate teaching and teacher
involvement. Few of them actually went to the class web sites to examine and
try to understand the context for the pages that were printed and brought in;
they merely looked at and dismissed the printed pages as evidence of inadequate
teaching. But the fact remains that members of the committee had judged the
pages my colleague had provided them using a print paradigm; their judgments
were based on the expectations one might have of the ecology of a face-to-face
class. This point is crucial: my
colleague insists that her committee was not equipped to evaluate her online
teaching. “There was a differential in the technological literacy of the people
on the committee. They had no clue that they were reading the record of a
process, and no clue what the process was,” she said. “They had never taught.(online)”
Furthermore,
the students in the class in question had never before taken an online class,
and they needed time to get used to the medium.
My colleague points out that online teaching has its own rhythm and
logic, fairly different from the traditional face-to-face class because of the
problems of time and distance. “They
can’t write as many polished papers, and the teacher cannot comment as much,”
she said. It does not seem fair that
committee members not well-versed in the problems of online teaching could
judge her teaching in that context.
But
more importantly, this case illustrates one of the pitfalls of Internet
technology. In addition to helping us in our research and classes, the wondrous
connection ability of the Internet can also be used as an instrument of
surveillance to allow others to peer into our academic lives. I do not have the space here to discuss all
the ramifications, but it might seem that my colleague’s privacy has been
violated, along with her right to control access to her own intellectual
property. Of course, the web pages in
question state that the copyright is held by her university, so we need to ask
1) whether anyone else at the university has the right to use those pages in
any context, and 2) whether my colleague should have allowed the university to
maintain copyright over pages that could be used against her, out of context.
When
I heard about my friend and colleague’s situation I had to stop and think. On
some level, this story clearly demonstrates a failure, perhaps on her part, to
communicate excellent academic performance, and illustrates the need for
technology-using educators to pay attention to the ways in which the current
evaluative structures may fail them when push comes to shove. In an academic
world which has up to now been dominated by paper — copies of publications,
syllabi, grants, reports, student work, letters of recommendation, etc. — what
happens when those of us pushing the envelope into new media try to present
evidence of valuable academic work in those very new media? The risk of
miscommunication is great, not only because our audience isn’t familiar with
new media, but also because we technology-using educators often have not specified
the criteria for evaluating our work. It also turns the very quality of
openness we prize in the technology back against us.
As
Janice Walker (1997) and others have argued, networking technology does change the rules. Is it desirable or even possible for us to
show how our work “fits” into pre-existing evaluation criteria when those
criteria do not adequately account for the new kinds of collaborative networked
teaching and research we do? However, since (as yet) we are not in a position
to make the rules,
First,
make our electronic work somehow "fit" into existing guidelines and
be able to justify it along traditional lines; second, do what we're doing now
and not have it count for purposes of tenure and promotion; or, third, change
the definitions of what is "valued" to fit what we're doing.
My
colleague tried the first option and failed, and I know several people who have
had to fall back on the second: be satisfied with their networked collaboration
considered more a hobby than a professional contribution. The third choice, though it may seem almost
unattainable, really shows the most promise.
We can start by asking the right questions of colleagues and
administrators, and by preparing ourselves with a strong rationale for the
professional value of our work with the electronic collaboration.
As
"early adopters" of educational technologies, most of us think that
we understand the pedagogical significance of what we do, and each of us knows
several colleagues around the country who approve of and applaud our work. And
yet our view is rather myopic, since our collegial communities, such as the
computers and writing community, constitute a rather small percentage of the
larger communities of our academic disciplines. Like many issues concerning
education and the greater public, evaluation too is about publicity, helping
others know and understand not only what we do, but also why we do it. One way of helping others know what we do is
to make it public on the World Wide Web or in public Internet discussion, or by
publishing it in print journals like this one (if the print medium will do it
justice). Another way is to take a
pro-active approach to the evaluation process and help each other prepare for
the situations and criteria we find at our individual institutions.
Over
the past few years I have been working with many others in the computers and
writing community to begin to define the questions we need to ask and the
guidelines we need in place in order to be fairly evaluated by our
administrators and colleagues. I suggest that all of us find out what our
schools and professional organizations are doing to adapt evaluation criteria
to faculty who teach with new technologies, or to draft criteria if they do not
exist. No, we cannot guarantee that these criteria will be used, but we can
work through our schools and professional organizations to explain the new
guidelines and recommend that they be followed. Overall, we need to PAY
ATTENTION to the structures of expectation evident in the evaluation criteria
at our schools. We need to do what we can to interpret what we do in terms of
those structures, as well as work to revise the criteria by which
technology-using educators are evaluated.
With
these goals in mind, I would like to offer a draft of some questions and
guidelines that technology-using educators (particularly those who use local or
wide area networks such as the Internet) and their administrators can use in
situations such as the job search, yearly evaluation, tenure, and promotion. Of
course, these guidelines will need revision and fine-tuning for the particular
situations in which we faculty find ourselves, but they can be used as a
jumping-off place. I should add that
there are excellent sets of professional guidelines ratified by the College
Conference on Composition and Communication and the Modern Language Association
(see Other Resources below) that interested teachers should also consult. These two documents differ from this one in
that they focus more on what departments and committees should do, rather than
the faculty member. In no way are these
guidelines meant to replace these other important resources; in most cases the
suggestions here overlap and supplement those guidelines listed above.
A. Following are some questions applicants
should ask in the campus interview or when negotiating an offer. Be sure to ask and negotiate these questions
in the interview stages before being hired.
Couch them in institutional contexts so you don’t appear to be assuming
that you are the only faculty member who will work with technology.
1. What role should technology play in my
teaching? If I use new tools such as the Internet in my classes, will my
innovations be rewarded in annual evaluations and at tenure and promotion
time? How are current faculty
members using technology and what reward structure currently exists?
2. What role will student evaluations play in
my annual evaluations and tenure and promotion decisions? If I use new
technologies and some students react negatively, will I be held accountable?
3. How will my online professional activities,
such as Internet discussions and web page hosting on the local and national
level be credited in terms of my service and scholarly activities?
4. Can peer-reviewed webbed publications be
used for evaluation, tenure, and promotion, and how much weight do they carry
as compared with print publications?
5. Can I be assured that those who evaluate me
will understand the technologies I am using as long as I explain the rationale
for my use of these technologies?
B.
Some guidelines for putting together an annual evaluation, tenure, or promotion
portfolio:
1. Make a regular and concerted effort to keep
your department chair informed about the scholarly projects you are undertaking
and their significance. Solicit feedback
from your chair about good directions to take in order to build a tenure or
promotion portfolio based on local expectations. No one understands such expectations better
than department chairs! (Selfe)
2. If possible without offending anyone, find
out who is on the committee and make your best effort to assess their knowledge
of and commitment to technological innovation. If possible, find out HOW they
intend to evaluate faculty work with technology. Also find out who actually makes the
decision. At some institutions, the
committee recommends, but the chair decides.
3. In your written portfolio, make every effort
to explain not only WHAT you do with technology in your classes, but WHY. Try
to give good reasons for using Internet and the web, even if you were requested
by superiors to use these technologies.
4. When in doubt, quantify. Some committees
need to see items such as numbers of e-mail messages sent and received, amount
of time spent on listservs and web pages, and numbers
of students affected. However, do check
with the committee on how much detail is appropriate in this area.
5. If the committee needs to stay paper-based,
print out examples of online class and scholarly work, but be sure to include
statements explaining the context and rationale for such work.
6. If you are reasonably sure that members of
the committee will accept and review your work online, include URLs of
exemplary pages in your portfolio, along with statements explaining the context
and rationale for such work.
7. Since student evaluations are often a vital
component to teaching portfolios, be sure to put your evaluations in context by
providing a narrative summary of those evaluations, especially for those
classes in which you were trying out new technologies.
8. Since institutionally produced evaluation
forms rarely cover the kinds of questions we need to answer about using
technology (such as those proposed by Steve Ehrmann
(1998) in the Flashlight Project[2]), create,
administer, and collect your own student evaluation forms, and give samples and
a summary in your portfolio.
9. You will almost always have students in your
classes who genuinely appreciate your efforts to use new technologies. Solicit
letters from those students and include them in your portfolio along with
letters from colleagues both on-campus and off who understand and appreciate
your work with technology.
C.
A few guidelines about what NOT to do:
1. Don’t assume that your administrators,
colleagues, and/or tenure and promotion committee will understand the
significance of your teaching, research, or service with technology unless you
explain the goals, rationale, context, and research that inform that work.
2. Don’t assume that your students will
understand why you are using certain technological tools without first
carefully explaining why the class will use these tools and what you hope to
accomplish.
3. Don’t rely on only the school-provided forms
for feedback from your students on your efforts to use technologies in class.
4. Don’t give URLs of online work without
defining precisely which pages are to be considered, and in what context.
5. Don’t forget to consult your chair, dean,
colleagues, faculty development committee, or anyone else who can help you
design the most effective teaching portfolio or tenure/promotion dossier. Keep them informed about what you are doing
well before evaluation time.
6. Don’t forget to carefully document every new
strategy or technology you use with your class, and keep detailed records of
what you did, how well it worked, and what results or feedback you received
from class members.
D.
Some guidelines for evaluators, administrators, and tenure and promotion
committees:
1. Make every effort to explain your
expectations, with regard to technology, to the faculty members you evaluate.
Preferably, these expectations should be published along with other guidelines
for evaluation, tenure and promotion in the faculty handbook. Guidelines should
include information on how and in what category faculty should document online
work, along with the relative weight of online scholarship and service,
including webbed publications and Internet discussion groups.
2. If online work is to be evaluated, follow
clear guidelines about the specific pages which are to be considered part of
the portfolio. Ask candidates to contextualize and provide a rationale for any
online student or scholarly work presented.
3. If you (evaluator) or the evaluating
committee lacks expertise in technology or online teaching, solicit the
opinions of outside evaluators who are knowledgeable with the technologies used
by the faculty member being evaluated.
This
is not a recipe for success, but a set of guidelines generated from the
experiences of many people. I hope that
it will help others who use technology in their writing classes evaluate
employment opportunities and prepare themselves for all kinds of evaluations,
not just tenure/promotion reviews.
Almost all of us at the very least face some sort of a yearly review in
which we must justify our work, and those of us who work with technology may
have a bit of extra work to do to explain the significance of that work. If we consult our colleagues and pay
attention to the local requirements, contextualize examples of our work, and
provide rationale for what we do, we stand a better chance of success in those
reviews.
CCCC
Promotion and Tenure Guidelines for Work with Technology:
http://www.ncte.org/positions/4c-tp-tech.html
Guidelines
for Evaluating Computer-Related Work in the Modern Languages:
http://www.mla.org/reports/ccet/ccet_index.htm
Kairos Coverweb: “Tenure and Technology: New Values, New
Guidelines.” http://english.ttu.edu/kairos/2.1/index_f.html
Sources
Barker, Thomas, &
Kemp, Fred. (1990).
Network theory: A post-modern pedagogy for the writing classroom. In Carolyn Handa
(Ed.), Computers and community: Teaching composition in the twenty-first century
(pp. 1-27).
Bentham, Jeremy. (1995). The panopticon and other prison writings. (Miran Bozovic, Ed.)
Conference on College
Composition and Communication Committee on Computers and Composition. (1999). Promotion and
tenure guidelines for work with technology. College Composition and Communication 51.1, 139-142. [Online, WWW]. Available: <
http://www.ncte.org/positions/4c-tp-tech.html> [November 1998]
Ehrmann, Stephen. (1998. The flashlight project:
Tools for monitoring the progress of our hopes and fears about technology in
education. In Horizon case studies. [Online, WWW]. Available: <
http://horizon.unc.edu/ts/cases/1998-07.asp> [July 1998]
Foucault, Michel. (1977). Discipline and punish: The birth of the prison.
(Alan Sheridan, trans).
Herrmann, Andrea. (1991). Evaluating
computer supported writing. In Gail Hawisher and
Cynthia Selfe (Eds.)
Evolving perspectives on computers and composition studies: Questions for the
1990s (pp. 150-170).
Hawisher, Gail & Selfe, Cynthia.
(Eds.) Evolving perspectives on computers
and composition studies: Questions for the 1990s (pp. 11-42).
Kaplan,
Modern Language
Association Committee on Computers and Emerging Technologies in Teaching and
Research. (1998). Guidelines for
evaluating computer-related work in the modern languages. [Online, WWW].
Available:<http://www.mla.org/reports/ccet/ccet_index.htm
Moran, Charles. (1992). Computers and the
writing classroom: A look to the future. In Gail Hawisher
& Paul LeBlanc (Eds.)
Re-Imagining Computers and Composition:
Teaching and Research in the Virtual Age (pp. 7-23).
Morrison, James (1997). Using technology
productivity tools in teaching: One professor’s odyssey, part II. In Horizon case studies. [Online,
WWW]. Available:<
http://horizon.unc.edu/ts/cases/1997-07.asp> [July 1997]
Palmquist, Mike, Kiefer, Kate, Hartvigsen, James, & Goodlew,
Barbara. (1998). Transitions: Teaching writing in
computer-supported and traditional classrooms.
Selfe, Cynthia. Personal E-mail to the author.
Sommers, Elizabeth. (1992). Political
impediments to virtual reality. In Gail Hawisher
& Paul LeBlanc (Eds.)
Re-Imagining Computers and Composition:
Teaching and Research in the Virtual Age (pp. 43-57).
Walker, Janice. (1997). Fanning the flames: Tenure and
promotion and other role- playing games. In Kairos: A journal for teachers of writing in webbed environments. [Online, WWW]. Available:
<http://english.ttu.edu/kairos/2.1/index_f.html> [Spring 1997]
[1] My colleague described her experience to me by telephone. She was unable to write for this special issue of Computers and Composition and gave me permission to report it here. She prefers that her name not be used. I recognize that there are two sides to this story, but I do want to provide her side by way of example.
[2] Questions that speak to the overall effect of using technology in the class on student learning. See the web site listed in the Sources section for more information on the Flashlight Project.