Giving It All Away

A few weeks ago I reported that I gave away all the answers to my employment law class’ final exam in my last lecture to my students. My prediction was that this would have no appreciable effect on test scores.

I was wrong.

Test scores noticeably declined, with the highest score this semester coming in at roughly the same level as the lowest score the last time I taught the class. If the exam were to have been the sole basis upon which I based grades, two out of the three students in my class would have failed outright.

Now, three students this semester, and five the last time I taught it, is obviously not enough of a sample size to produce statistically significant information. But the results are still depressing. Out of eight students who have been given this question, the number of students who got this question right…

In the context of employment law, “discrimination” means:

a)      Evidence that suggests the accusations against a defendant are correct.
b)      Preferential treatment given to a member of a favored group.
c)      The belief that members of a particular religion are morally inferior.
d)      Taste, refinement, and discernment.

…is 25%, exactly the result one would expect if the students were using chance instead of actual knowledge. For the life of me I can’t figure out why this question is so difficult. I can see choosing answer “C” instead of the correct answer (answer “C” is indicative of one possible form of discrimination), but that one was picked only one time in eight. The most popular answer is still “D,” which irritates me to the point that I drop to my knees, shaking my fists at heaven in frustration as I beg a God whom I believe not to exist for a satisfactory explanation.

It’s night school, I know, but still, these are graduate students! They’re supposed to be smart! So apparently, understanding the basic definition of the single subject that constitutes two-thirds of a class is beyond the abilities of a graduate student.

So I’m kind of bummed out.
 
Burt LikkoBurt Likko is the pseudonym of an attorney in Southern California. His interests include Constitutional law with a special interest in law relating to the concept of separation of church and state, cooking, good wine, and bad science fiction movies. Follow his sporadic Tweets at @burtlikko, and his Flipboard at Burt Likko.

Please do be so kind as to share this post.
Share

31 thoughts on “Giving It All Away

  1. If you don’t mind me asking, why do you give multiple choice exams to grad students? And, perhaps a related question, in what subjects are these students grad students (MBA? Humanities? Social Sciences?….I had thought these were MBA’s, but I forget).

    I ask because in my grad programs (history, different schools for the MA and PHD), multiple choice exams, and mostly any type of exam, were never given. The exceptions were the qualifying exams or “practice” qualifying exams we did for some classes.

      Quote  Link

    Report

    • I do it to test the students’ abilities to apply the information the class is supposed to convey in ways that do not require me to also evaluate their composition skills, which experience has demonstrates are nauseatingly deficient. Although I’m not teaching composition, their writing skills are so poor, so often, that I simply cannot get past what appear to be random strings of words, sometimes lacking important components of communication. Like verbs.

      So I do it to give them a chance of demonstrating that they know the material without having to express themselves in writing, because if the only way they could get a grade was a paper, they’d all fail. (Actually, in this class, one student turned in a very good paper. But that’s one out of eight.)

        Quote  Link

      Report

      • Yeah….teaching can be disheartening in that way. If I ever teach again (and if I do, it will almost definitely NOT be at the grad level), I will probably introduce an objective component to evaluation (if not multiple choice, then at least short answer), in part because of the discouraging absence of writing skills. (I’ve also found, when I did teach, that writing skills weren’t always the issue. Sometimes students just don’t spend enough time learning facts. Alex Knapp, a long time ago, had a post on the importance of learning actual facts, and I largely with this.)

          Quote  Link

        Report

  2. Not surpising to me at all. Since you gave the context of employment law in the question, the diagnosis for anyone who got the item wrong has to be: IQ too low to become a lawyer. It’s stuff like this that drove me out of professoring.

      Quote  Link

    Report

  3. I realize you gave the answers ahead of time, and a little bit of deductive reasoning ought to lead one to the right answer….

    Still, I confess that although I chose B (it is the correct answer, right?), I was a little tripped up because the term “favored group” might to a layperson be interpreted to mean “favored [minority] group.” Not that that’s what you meant or that anti-discrimination law actually favors minority groups more than other groups. In fact, after reflection, I take you to mean “favored” in the sense of “favorable to people who already have unfair advantages.”

    But I can see that being a source of confusion.

      Quote  Link

    Report

    • The correct answer is indeed “B.”

      Yes, usually, the favored group is a majority group. White, male, under-40, Christian, heterosexual and cisgendered, natural-born American. Sometimes, though, there can be “reverse discrimination” or discrimination by one minority group against another. So what matters is that one group, any group, is favored over another.

      Still, I don’t think this is a particularly hard question. And I really don’t understand why half my answers are “D,” an answer that has nothing whatsoever to do with law.

        Quote  Link

      Report

      • Thanks, Burt.

        I had a high school history teacher once who gave multiple choice tests, but allowed us to make a case after the fact that an “incorrect” answer was actually right. I remember one question was something like “John Winthrop headed the Mass. Bay colony unchallenged for 30 years [or whatever],” and he kindly and graciously accepted my refutation that there was one time where he was relegated to deputy governor. (I had actually known about that tidbit…..I was that much of a nerd–and an annoying one–in high school.)

          Quote  Link

        Report

  4. Some people are just bad multiple choice test takers, flat out. They over think the concepts, look too hard for the “trick” part, forget that if you’re stumped choosing (a) is correct 92% of the time, that sort of thing.

    But this is horrifying.

      Quote  Link

    Report

    • I remember a true-false test I had on a US government class exam in 1994 that went something like this:

      With two women on the Supreme Court and one black person, the US Supreme Court is now perfectly representative of the US population.

      This is a paraphrase, but as I remember it, there really was something like “perfectly” in there to suggest absolute representativeness.

      Even though that ridiculously easy question is false and even though I knew at the time it was false, I had much difficulty answering it. I spent too much time rationalizing something like this (although this too is a paraphrase):

      “well, a 9-member court can never be ‘perfectly’ representative of the population, so maybe we need to think about how perfectly it can ever represent people. And there are two women now, and given the nominating process and the fact that there are fewer women lawyers [or so I believed] than male lawyers, maybe we need to consider the nomination process and the fact, for example, that blacks are about 10% of the population and Thomas is 1/9 of the court members and even he was nominated by a Republican and not a bleeding heart Democrat, and also, if Scalia is Italian and he can serve as a proxy for other immigrants, ergo….”

        Quote  Link

      Report

  5. Wow, that’s … I don’t even know what to say to that. Honestly, the best explanation I can come up with is that your students are trolling you. That doesn’t seem very likely, but I’m drawing a blank otherwise. They didn’t just get it wrong, but in such a bizarrely backwards way.

      Quote  Link

    Report

  6. Was this near the end of the test maybe? Too many questions, and they were filling in the last twenty answers in the final minute? Either that, or this wasn’t a grad school at all, but a grade school that you’re teaching a masters-level class at. Those are my two possible explanations.

      Quote  Link

    Report

  7. Better not fail anybody that got that question wrong. They might sue you for discrimination on the basis of disability- specifically learning disability.

      Quote  Link

    Report

  8. Okay, Burt, I have a challenge for you.

    The new format has a survey capability, IIRC. You need to put your exam up here at the League and see how we all do. I mean, come **on**.

      Quote  Link

    Report

  9. From the original post:
    The last time I gave this test, the average score was 74.8 out of 100 with a standard deviation of .978.

    The standard deviation was under a point? So all students got almost identical grades?

      Quote  Link

    Report

Leave a Reply

Your email address will not be published. Required fields are marked *