I have, in the past, threatened to blog on subjects of interest to computer scientists, graduate students, and graduate students in computer science. Something of a counterpoint to Uncertain Principles, Signal + Noise and some others, which tend to blog from the perspective of faculty members.
Since I have just taken a final exam in Concurrent Software Technique (in the obligatory Java) perhaps this is the right time to begin with an entry from the student's point fo view. And as Uncertain Principles kicked off with a post about student evaluations of professors, it seems only right that someone should write an article from the counterposition-- professorial evaluations of students... which is to say, course grades. (I suppose we'll handle letters of recommendation elsewhere.)
Grades are one of those all but necessary evils in this corner of academia-- almost everyone in the game needs grades to some extent. Some disciplines, like engineering or computer science, hold with them the assumption that you can stop your education after the bachelor's or master's level, put that relevant degree in a frame, and seek professional employment. And certainly, having been on recruiting missions for various engineering companies, yes, grades certainly matter for new graduates seeking employment.
Those students who desire or effectively require the full PhD (for instance, physics, where a master's degree is most often described to me as something between a booby prize and a door prize on the way out) are still dependent on them to a point, because it's relatively rare to stay at one institution from the cradle to the grave, and admission to other schools will be based, at least in part, on prior grades.
So unless an institution is so well-regarded, it cannot simply do away with the concept of grades, which invariably boil down to a single character (perhaps two, with a plus/minus scale) symbolizing your understanding of any given subject, and a three or four significant digit number characterizing your academic performance overall.
I don't really envy professors that particular job. Done honestly, and with an honest sample of students, it is not only time-consuming but it can turn into an indictment of your own teaching skills. This I say, having been on the grade-assigning side of the fence for long enough, when I held one of those almost-mythical teaching assistant positions where I held a larger say in grade assignments than the professors who were nominally teaching the course. (That's what happens when I grade each and every set of papers, and the break points are then determined by a conference between myself and the professors.)
Worse, though, the "standard" tools for assigning grades to us poor slobs, the students, are remarkably crude. Judging too heavily on homework is a mistake because while I don't cheat, copy, or suffer anyone to copy off my assignments, I know full well that there are enough students who do to render the exercise nigh pointless. The other common alternative, though, is to give one or two standard examinations and grade heavily on those.
That can be worse, though, because test-writing is an art form, and while I know that some professors spend considerable amounts of time worrying about the format and structure of their tests, it seems pretty apparent to me that many do not. For the former, there are Uncertain Principles and Signal + Noise, who obviously care about what they do. We gave oral exams in the class I TA'd, and I can personally attest to having given great thought beforehand to the questions I was going to ask.
On the other hand, though I have never heard a professor say outright, "Ah, I didn't give much thought to this exam!" I know to a certainty that some of them simply don't. Professors who might, unaccountably, be reading this, please be assured, I am not guessing at this. I, like any perspicacious graduate student (or even undergraduate, I think) can sometimes simply tell.
I recall far too many instances (many of these pertaining to the same professor) from my undergraduate days, giving microelectronics that were far more difficult than they intended to be, because the professor admitted that he hadn't bothered to solve the problem before he issued it to the class as a test. Joy. Sure, that balances out, because everyone is in the same boat, but it is certainly not fun while you're taking the exam, wondering if it's you who's not really cut out for this engineering stuff, or just the professor who's not really cut out for this teaching stuff. And it speaks terribly of the professor, in my opinion.
I've gotten exams which were obviously written only an hour before-- because they were fresh xerox copies of an exam handwritten in pencil, with the ink still smearing and the pages still warm. Thanks, professor, you're important to us, too. (It didn't help that that individual was clearly winging each and every lecture, using someone else's lecture notes from the web, that he had perused, but not thought deeply about. A shame, too, since he really knew his shit and really loved the material.)
And perhaps my favorite example of professors obviously not thinking about their exams is probably unique to computer science, especially programming courses. It is reasonable to put computer code on a piece of paper and ask questions about it as a form of testing. It is not reasonable, however, to put computer code which you have not compiled and whose bugs you have not eliminated on a piece of paper and ask questions about it as a form of testing. This happens far, far too often. Worse, it is almost completely avoidable, because often, all you need to do is compile and run the code (if it even compiles!) to determine that you've done something incredibly stupid.
So, yes professors, we often do know when you're winging the exams.
More subtle, though, are the exams which just make me tilt my head (and often, mutter dire imprecations) wondering, "What made you thought this was a suitable exam question, or exam style? What makes you think that this is really a good gauge of conceptual understanding?"
I'm sure every graduate of a technically-inclined program, be it in engineering or the sciences, has their own tales of professors who habitually gave exams where the averages were down in the 50's or lower, with no real point in mind for the difficulty level of the exam. (Only one professor has ever given me a reason for that practice which made any sense. All others I have asked merely shrugged.) I've even had classes where the exams were so ludicrously simple that I felt cheated.
(Yes, I am rather hard to please.)
And given the state of engineering today, there is a large and rapidly growing disconnect between engineering as it is practiced, and engineering as it is typically tested for in school. No one, for instance, sits down and consults Zverev and goes rooting through tables when they start to design a filter, anymore, unless there's something terribly exotic or troublesome, going on. Rather, we fire up our simulator/synthesizer software of choice and work in that environment.
And yet, those basics do need to be understood, and that understanding evaluated as part of the academic process; and far too often, the method of choice is still to park a poor student down with a pencil, a sheet of paper, a problem and (if he's lucky) a table of relevant values fro that type of problem, and tell him to come back in 120 minutes with an answer.
I would be shocked if other technical fields were much different in this regard. Electrical engineering is rife with this disjoint between pedagogy and practice, and it's to the point where I can no longer count on GPAs to tell me much of anything for new graduates, except "This is person is probably not a low grade moron or an egregious cheater." If I am lucky, the new graduate will come from my undergraduate engineering school, and I can ask him his grade in one or two particular lab courses, and form a good opinion based on my deep familiarity with those classes. I am often not so lucky.
And, once again, computer science has it's own particularly exaggerated form of this phenomenon. Where an engineering student might be told to solve a problem in pencil and paper, where he would ordinarily have a text or a computer in front of him, a computer science student is far, far too often given a sheet of paper and told, "Write a program in this language, that performs this function." (Or, slightly more common, given half a program and told to complete the other half.) This started out as a pet peeve of mine when I was taking elementary courses whose only purpose was to make sure I had mastered the language-- because, of course, they could not be certain I was not stealing the homework assignments from someone else. A necessary evil.
It is, however, becoming a larger and larger peeve, as I honestly cannot think of a larger disjoint between practice and pedagogy than that. I simply cannot. No sane person in industry sits down and writes code out by hand, without references. No sane person even types up code into their compiler, while looking at references, and expects it to be right the first time. I cannot, then, understand the purpose of administering an exam on that basis. Not only is it just plain silly, but unless the course is low level and about the particular language in the first place, when I am forced to do this, I am thinking particularly about the language in which I am (trying desperately) to write, not about the underlying problem that the code is supposed to solve.
I cannot understand what my limping through hand written, unpremeditated Java (for instance) code is supposed to tell a professor about my underlying knowledge of anything but that. I had opined recently that I despise pseudocode, but I am forced to admit that this is probably a reasonable application for it. Few things in academic computer science are as frustrated as knowing to a certainty that you could, if sat down at a compiler, write a program to answer the question, but just entirely blanking on a few key pieces of syntax or idiom.
I do know that this style of exam gives me the sweats every time I see it, and nightmares every time I take one. Invariably, I have to count on having done well enough on the homework to make up for the unimportant, but downgraded mistakes I will make on the exam. It could be just me, but I tend to think not, considering the savage bitchfest that the topic generates every time it comes up amongst my peers.
(For bonus points, I occasionally get half and half exams-- half questions about code that was clearly never compiled, and half questions telling me to write code with the expectation that it needs to be correct as it flows from my pencil. This, in short, is infuriating.)
All of which is convincing my, not very slowly, but quite surely, that many subjects at the graduate level and possibly senior graduate level in computer science and many engineering disciplines are simply not served by conventional pencil and paper examinations at all. Certainly, class projects, term papers, and presentations are time- honored solutions to some of these problems, but there are some problems involved here, as well. It's much harder to implement this in a quarter-system university than a semester system university, for instance. A quarter-system might leave you eleven or twelve weeks to teach a subject, and about half of that will be spent teaching enough that the students can start to understand a larger project. Also, especially for course-burdened undergraduates, no one wants to face four to six project deadlines at the same time. (Although it couldn't be much worse, from where I sit. I'd probably take it.)
Oral exams are an option as well, as long as class sizes are small enough to warrant the massive investment of time on the part of the professor. For that matter, learning to ask essay questions on exams would help, as well, but for some reason that isn't done (often enough.)
I've been coming to think that, if we only had enough hardware to make this practical at most universities, it would be interesting to give students, "Here, write me a program that does this...." style exams at computer terminals. Instead of writing the programs down on a piece of paper, by all means, give the students carefully personalized computers, with standard compilers, standard compiler and library documentation, and no internet access. That might still be brutal (now you have the damned compiler bitching at you, making you nervous!) but certainly closer to practice in the real world and not nearly as hideous as writing it all out in pencil.
The obvious drawbacks would be the amount of hardware and university computer labs required to make this feasible, as well as the stability of the hardware itself-- one shudders to imagine trying this on Windows machines.
I'm sure there are other innovative methods for testing material as well. Unfortunately, few professors seem to be interested in finding them.