On November 19th 2020, I was given the privilege of participating in a panel from the International Center for Academic Integrity on the subject of e-proctoring, which I previously wrote about in my article titled “On Knuckle Scanners and Cheating – How to Bypass Proctortrack, Examity, and the Rest“. I learned a tremendous amount from that panel, and I believe the audience (over 150 participants!) did as well.
For part 2 of this discussion, please click here.
Moderator: Jake, over to you. What is your position on e-proctoring and what are the main issues?
Thank you very much for having me, Paul. My position is, as Sarah mentioned, trust is extremely important. And furthermore, I do not believe that e-proctoring works at a technical level. I think that the technology just is not there yet. The trade-offs that we make in terms of invasive watching of webcams, taking over a computer, shutting down all background processes – these are all things that we would consider to be a computer virus – if it were not being provided, and indeed required, by the university. These are all things that we accept because we say, “Well, this is what is needed in order to ensure the academic integrity of students.”
And at the same time, as Paul mentioned, I wrote an article that it can be very trivially bypassed. In this forum I’m not going to be giving exact instructions on how to bypass anything. I just wanted to point out that right now – maybe this is a closed book exam, maybe you’re looking at me through my webcam, maybe my whole computer is taken over – but just out of the view of the webcam, I have plenty of notes.
Even though I’m giving up so much of my privacy, I just defeated it. You can’t see my keyboard; you can’t see anyone behind my screen right now. And if a student really wanted to cheat, even with the expensive, invasive software, I believe they still could.
I think that there has to be a change in the way that we educate where we have to recognize that exams today are not like they were. Trying to make a one-to-one comparison between what we had in classrooms and what we have online just is not going to work with the software that we have today.
Moderator: Jake, I wanted to pose you the question: what are the obstacles and risks to consider before rolling e-proctoring?
Sure, thank you Paul. So, I want to speak quickly about these obstacles and risks, and I think that there are many. And in the interest of time I’m just going to breeze through these. If there any questions please feel free to contact me – I’m trying to answer questions via the Q&A section of Zoom.
But again, I’m just going to breeze through these. In terms of risk I think that there is a major risk in saying that if we don’t fully trust students, and we enable the software, and the software says that there was no cheating (you know, it didn’t come up with anything), then there was no cheating.
I think that there are a number of ways that you can cheat using the software, even trivially: like I said, I could have a sticky note on my screen right now. I could be using notes that are just out of view. It’s very, very difficult to really determine that there is no cheating. And so false negatives are a big issue.
False positives are also a big issue: if the algorithms or the artificial intelligence detects that your eyes are looking in a different direction, or that it says that you were cheating, perhaps there was some background noise, there are then two options. Is the professor going to go through and say, “Yes I’m going to make a formal complaint against the student, I’m going to accuse them of cheating”? Then it becomes up to the student prove a negative that there really was no cheating, and whatever it determined, it was an overeager algorithm.
Or alternatively the teacher can choose to ignore that pop up and it’ll slowly have a psychological effect that you ignore the software. “Yeah it always comes up with a dozen things that the student did wrong, but I’m just going to hit ignore on all those, it’s not really necessary.”
And so in all those cases, there’s a very slim number of cases where it A. finds cheating and B. there really is cheating. The majority of other cases could be problematic.
But in addition, I just want to go through really quickly a couple obstacles that I thought of. Specifically, that there are requirements which are now basically requirements for getting a degree, that are really just requirements for using the software.
One requirement that I just ran into and setting this up, I don’t know if you can see if I move my eyes around you can see the lights in the room reflected in my glasses. A number of the software packages will track your eyes to make sure that you’re looking at the screen. But if I position my eyes just right, you can’t see my pupils at all. And what happens then? Am I allowed to continue taking the exam, do I get marked as cheating? Can I trust that a professor will simply ignore it? I’m not really sure.
Furthermore someone mentioned as well that if it’s too dark in the room and the software can’t recognize my face because there’s not enough contrast between my face in the wall behind me, it may not let me take the exam because it can’t recognize someone sitting there. And we’ve seen news stories already that students with darker skin complexions have a lot of trouble maintaining the contrast between themselves and the wall behind them in a dark room.
An example of a dark room where students go regularly is a library. If you don’t have control over the lighting, you really can’t take am exam in a library anymore.
And other problems with the library might be that libraries are supposed to be quiet, but not necessarily silent. So if the if the software says that I can detect conversations, or even phone calls, near you – what if there is a phone call near you and it just has nothing to do with you? Am I going to get accused of cheating or not? I’m not sure what happens if someone walks behind me taking an exam and you can see two faces in the screen, and facial recognition software gives the professor pop up? Can I trust that the professor will know that that person is a stranger to me maybe they are cheating with me? It doesn’t necessarily determine the relationship between the people or whether or not they passed me a note or something like that.
Other issues are that in some cases, because there is such limitation of the view of the webcam, some software requires you to have multiple devices looking at your screen, so you would have to set up another device behind you, facing your laptop, to get a 360-degree view of the room. Not everybody has two devices, not everybody has an internet connection capable of doing multiple video streams. If you’re taking it in the library or a café, there’s not going to be a place to set that up.
So, I think that there are a number of risks in terms of really trusting the software, and the number of obstacles that are thrown in front of students to be able to actively use the software. Maybe they don’t have a room to themselves for 2 hours with high-speed internet connection and multiple devices, and these are not requirements for higher education.
Moderator: Jake, you had a comment?
Yes – I’m just a building off of what other people are saying, and again really quickly, I’ve already taken up my time.
One of the issues with bandwidth is that there’s so many people at home today because of COVID that if you are taking an exam and someone else is working from home, and then you have multiple students, even children, who are doing remote learning – that’s all eating up your bandwidth and you can’t necessarily guarantee it, especially non-techies.
Another issue is that the algorithms are effectively creating a baseline, or an average, for how students take an exam or even your specific exam. There have been reports of people saying things like, “I was reported for not using the backspace key as often as everyone else uses it.” What that is supposed to test is, if you are writing an essay live, you’ve just seen the short answer question for the first time. You’re most likely going to write something and backspace and start again a little bit. But if you’re the type of person who thinks first and then writes it down, you’ll hit the backspace key fewer times.
That’s one of a number of things that it can track to say, “This person is doing something anomalous.” And anybody who learns different is very likely to reach these kinds of anomalies. I personally read the questions out loud to myself, I learn better when I hear it, especially when I read it. At any time that I’m speaking that could come up as you know having a conversation. And in truth, I could be reading that out loud to someone standing just behind my computer, who then holds up a number of fingers for if it’s A, B, C, or D, because they’ve been looking it up. That is not what I, do but it very well could be, and I’m not sure that the software can stop me.
Moderator: Anybody else want to comment on alternatives to e-proctoring? Yes, Jake, please go ahead.
I personally would prefer as a student taking a course that test me in ways that are not straight exams, as Sarah mentioned. There are a number of other ways to test students: there are essays, there are in-person essays (just speaking with the teacher directly trying to describe your position). I think that exams are very much an in-person and analog-style method of testing.
And I think that there are a number of ways that you can innovate using the internet – even just having a Zoom call with a professor might test my knowledge better than multiple choice questions.
In addition, I also want to find out in terms of academic integrity, we really do trust students in many other forms of assessment. For example, if I’m assigned an essay, I write it on my own, and I hand it in – maybe I paid someone to write that essay for me. We really don’t know in any way except for the exams which are done live and in an invasive manner. We do trust students in a number of other aspects. We’re just talking about exams now when we want to make sure that we have a clear view of the students, which is interesting because at some level, we are trusting students regardless.
Moderator: Jake, what would you like to add, as a student still?
I’m actually taking a final exam that is e-proctored in about 1 hour after this finishes, so a little bit ironic that I’m talking about the subject right now!
But before I start, I just want to make a quick point to Alycia who mentioned posters on her wall. If the posters on your wall have faces on them, you can get in trouble for having multiple people in your stream because the facial recognition can detect multiple faces in order to make sure that you don’t have 4 people looking at your screen and trying to take your exam. That can actually also be a problem in multiple ways, not just the embarrassment about the poster but also because you can be banned from taking the exam for not having just one person taking it.
But that being said I wanted to provide a perspective of a student, not myself. But I wanted to give a hypothetical student who’s probably on the extreme, but some or all of this may be applicable to many students, especially undergraduate students.
This hypothetical student does not care about academic honesty. Probably you know anathema to this particular crowd. But for them, when you sign an academic pledge, it may just be the terms of service that you have to sign off on before you can start school. And you know that you can’t cheat, you know that it would devalue your education, you know you wouldn’t really learn anything, you also know that if everyone cheats, it would devalue the entire university.
That being said, you Google things a dozen times a day or more. If you don’t know the lyrics to a song or if you want to see if the celebrity is still alive, you’re just constantly Googling things. And then within maybe a one or two-hour block of your entire life, you can’t Google things. And for those students who say, “Yeah, but what if I could get away with Googling it? What if just outside of the view of the camera, I have my phone and could Google it and get away with it?” I’m not so sure that everyone takes that as a moral question. They should.
I’ve been to Rutgers as an undergrad and NJIT as grad school, and neither of them had any classes on academic Integrity. I signed one piece of paper and that was it. And in not caring about this, I also find that many of my classes that I’ve taken, and many that other students have taken, end up basically being copied and pasted online classes from previous semesters. The exams are exactly the same. The questions on the exam are from the textbook publishers. And if you were to Google the exact text of any of the questions, you’ll find that there are literally dozens of copies of it from all other schools where students have uploaded their past exams.
And so the end result is, if you really just need to get through school because you need a degree in order to get a job later, it may not be a moral question for you. Whether or not you’re just going to Google something and get the right answer to get an “A” and get out of there (and yes it means that you’re learning less).
At the same time how much do I really value the learning that could be done if I could just Google it and find out that the answers? Just check off Option A end and move on. And I don’t want to suggest that is extremely widespread. But I do want to say that students are under tremendous pressure both financially, in order paper school, as well as academically – if they don’t get through school, they’re not going to get a job. And the idea of failing an exam or failing a class means that you have to retake it – that’s going to take time and money. And if the alternative is just a Google it, you’re really juxtaposing someone’s professional future against a real quick lapse of academic integrity. I think it’s actually really common. To return to talking about alternatives to things – Todd was mentioning one way of viewing the question was “What about the classes that have final exams?” but I want to point out that I personally would really have preferred programs that ask me the kinds of questions that I can Google I can research, and that I have to provide my own answer to that Googling “A B C or D”, it wouldn’t have helped. I would like to do have given my own opinion as opposed to memorization a lot of the time.
I just want to say as a as an ending, I guess, challenge? To any of the schools that are looking to use this e-proctoring software, I have asked this of multiple proctoring vendors all have said no, but maybe you have a little bit more pull than I have. If you are of the belief that e-proctoring helps tremendously in cutting down cheating, I would like you to arrange with the with the e-proctoring software to give me a test.
I will set up a camera – the webcam obviously in order to take the test, as well as a camera facing me from a different angle so you can see exactly what I’m doing. I will cheat the entire way through the exam, and we’ll see how much the software actually catches. I can cheat in any number of ways: either the opportunistic method of cheating, or the actually complex software, setting up different computers, that sort of a thing. And we’ll see how much it how much it really catches.
And we can see whether or not the software is really detecting things that any student could do, if detecting something that a technologically advanced student could do, what a determined student could do. I just want to point out that everyone every one of the vendors that said no, they will not allow me to make any videos or post anything about how effective it actually is. Obviously, those vendors say that they are effective, but let’s use the scientific method, let’s actually test this out and see if this works.
And if they say no to the universities as well, that they don’t want their software tested – I think that’s kind of a black mark against them. You would certainly test drive a car before you sign a contract for it.