by discocrisco on 11/30/20, 12:55 PM with 306 comments
by ege_erdogan on 11/30/20, 3:18 PM
You don't succeed in remote teaching and learning by trying to make it as close as possible to the in-person setup. You have to treat it as an entirely different problem.
Consider the synchronous exam. It is a perfect method of grading in-person. It is hard to cheat and all people take the same test in the same place. It is as fair as it can get.
In an online setting however, people can face all sort of troubles in a few hour window in their home. Your internet might stop working, neighbor might be making too much noise etc. Everyone takes the exam in a different setting and it is as unequal as it can get. It is also practically impossible to prevent cheating.
by iandanforth on 11/30/20, 4:13 PM
Their "powerful AI engine" is almost certainly just humans. It might have a few off-the-shelf components like face detection but most of what they claim to do is just so easy to outsource that almost all companies do it. If there is any delay between the system observing a suspect behaviour and the student being told to correct it then they are definitely using humans.
An institution using a service like this is a huge red flag. You should take it as an indicator of a low quality administration if not a low quality institution.
As an engineering problem this task is hard. Ryan Calo (Prof of Law, UW) once presented a fascinating bit of research on trying to automate something as simple as fining someone for speeding. Given perfect information how do you build the system? If someone exceeds the speed limit for 1 second, do you fine them? If everyone around the person is exceeding the speed limit do you use the same rules? If someone oscillates between just above and just below the speed limit, how many times do you fine them? If someone exceeds the speed limit and stays there does this result in fewer fines? How do you square the code written with the law as written? The problems are so extensive it may be that application of rules like this require human level judgement. Proctoring an exam may turn out to be an AI-complete problem.
by low_key on 11/30/20, 11:24 PM
The software is a privacy disaster and any computer that has had any of this spyware installed should be considered compromised. I kept a separate hard drive and would swap it in to take tests.
To fix the problem, grading measures need to be changed to accommodate the new world of online classes rather than trying to shoehorn old test proctoring into a remote space. This software only stops bad cheaters anyway.
I'm already imagining the fights I'm going to have with my daughter's schools in the future when they ask us to install this malware.
by colechristensen on 12/1/20, 1:58 AM
They were already a problem in universities and this is their final and worst form.
University systems have the major problem that a significant portion of students are only there for a degree, and their participation is playing the game in order to get that piece of paper, and the GPA number rating them.
The core of this problem is the question “will this be on the exam?”
Testing of course can be an important part of learning, but making that the metric by which you decide to hand out degrees substantially damages the value of testing as a teaching tool, and damages the value of a university as a place for research and learning.
Another way needs to be found to sort students into degree worthiness.
Raising humans for the first quarter of their lives in a dystopian police state is not what anybody should strive for. We need to figure out how to measure people less.
by pdkl95 on 11/30/20, 4:18 PM
> "... people who have some sort of facial disfigurement have special challenges; they might get flagged because their face has an unexpected geometry.”
So this company is extrapolating "patterns ... associated with cheating" from facial geometry. This is just phrenology laundered through their "powerful artificial intelligence engine" black box. Predicting behavior from the shape of someone's skull is still bullshit pseudoscience, even if the calipers[1] are replaced with a bunch of linear algebra.
[1] http://antiquescientifica.com/phrenology_calipers_George_Com...
by parzivalm on 11/30/20, 3:48 PM
People that want to cheat or game the system will find ways, it isn't hard to do. In undergrad we setup a copy of the code checking software that our department used so that we could share code without it getting flagged as copies. I'm sure there are ways to game these systems too if you are motivated enough.
One of the issues is also just academia being so hyper-focused on thinking everyone is there to become an academic which is not the case for the vast majority of people. So these exams and the guidelines for student evaluation is grounded in that expectation instead of the reality.
by hahamrfunnyguy on 11/30/20, 4:05 PM
Great example of why our educational system isn't all that good.
I took a year of Japanese in college. A big part of the grade was memorizing and performing these dialogs. If you didn't remember the dialog word for word, you'd get points off. I wasted a lot of time trying to memorize those stupid things when I could have been acquiring new vocabulary words or studying Kanji. I took a trip to Japan a couple of years go, and did some brushing up for a couple of months before I left. I learned more in those two months than I did in my entire time at school.
by motohagiography on 11/30/20, 3:20 PM
Polygraphs are bullshit and basically select for submissive behavior, which I suppose is what these institutions are looking to reward, but it is conspicuous that nobody called these surveillance schemes what they truly are: degrading.
by ev1 on 11/30/20, 1:06 PM
How about writing open-book tests in ways that are impossible to cheat on if you don't understand the material? You've had almost a year to adapt.
Why outsource student PII to a developing country sweatshop? This all seems absurd to me, it's been much easier to just write a one-liner inline script to hook blur, focus, visibilitychange, and onkeydown and log the userid when the event happens.
by jccalhoun on 11/30/20, 3:36 PM
by NikolaeVarius on 12/1/20, 12:22 AM
Most of my engineering higher level classes were open everything not electronic. If you don't know how to tackle the problem, you're not going to finish on time.
by serjester on 11/30/20, 3:24 PM
This software isn't ideal but there's no good solutions here short of major rethink of how these classes, and possible all of the college, is structured.
by madhadron on 12/1/20, 1:41 AM
And then there were the take home, open book, open notes, you have three days to finish this exams...
by TrackerFF on 11/30/20, 4:01 PM
If you can't bother to make a proper exam, don't bother. Rather make the classes pass/fail on project work, or grade the classes on projects / home exams. Trying to bring a 100% replica of the physical exam space into the digital space is just a recipe for disaster.
by nlh on 11/30/20, 10:49 PM
In the real world of 2020+, almost everyone has a portable Hitchhiker's Guide to the Galaxy in their pocket. Let people use calculators. Heck, let people use Wikipedia! If they copy an incorrect fact from Wikipedia, well, that's their problem and penalize them for that.
Education needs to be considered in the context of the world we live in. If you want to test students, then develop ways of TESTING students - move to live oral exams. Written long-form essays. Develop education techniques that USE the tools we're blessed to have around us instead of fearing them.
I don't have all the answers as I haven't thought about this deeply enough. But I've got to imagine there are ways of testing learning & knowledge that aren't based on fact memorization & regurgitation or performing calculations in your head that can easily be done on a calculator.
by godelski on 12/1/20, 1:09 AM
As for tests, there is an easier solution to all this. Write your test as if it was a take home. Open notes. You can't stop people from communicating but often these types of exams/assignments it becomes clear who is doing it. In my undergrad all my upper division classes' exams were take home because "I can't test you on anything worthwhile in 2 hours." Honestly, most of us enjoyed these more. We often did the exams in the same room (we had the back of a building that was dedicated as a lounge to the physics students) and no one really cheated. The closest was "hey, I'm stuck on this problem and I know you are finished. Can I just use you as a rubber ducky?" (more like just explaining the problem to the person and the other person saying "uh huh" and no more) It also made me feel like an adult because our professors trusted us. As someone that frequently does poorly in a testing environment I was also surprised that I was able to get much higher scores on these tests despite the added complexity. The simple fact that I could "take a break and come back" was all the piece of mind I needed (or grabbing a beer when I felt frustrated). This also better reflected, in my opinion, what solving difficult problems were like in the real world. I could grab my book, go to the page that I know is helpful, sit and think, take of my shoes, pace, whatever. I was treated like an adult and it felt good.
by wccrawford on 11/30/20, 3:17 PM
The software itself shouldn't have any control over the student's grades. A person should have to review the flags and actually find some wrongdoing. Not just push 'yes' and walk away.
by kart23 on 11/30/20, 10:57 PM
Profs who use these things are examples of teachers who just don't care enough to revamp their courses to involve more project, critical-thinking assignments, or exams where cheating won't help. They don't want to do the work to adjust the course for remote learning, and just give the course that they always teach, and do whatever it takes to get as close to the in-person exam that they used to have.
by AbhyudayaSharma on 12/1/20, 5:06 AM
I'm wondering if it's possible to just do a custom build of Firefox/Chrome that does not trigger these out of focus events.
Coming to the video streams, just record a 1-2 minute video of yourself staring into your screen from your webcam and loop it through OBS. No one will notice anything. For MyPerfectice, it is even easier, they have their camera controls exposed as unobfuscated global objects. So you can essentially do something like `camera.stop()` and the webcam light turns off.
by swiley on 11/30/20, 3:19 PM
Or is this just another social pathology that we need to rethink?
by sershe on 12/1/20, 6:57 PM
So, if you copy a perfect solution and have no idea how what you copied works, you are likely to fail anyway; if you just happened to get unlucky with your problems you may still be able to talk your way into a B/C showing you know stuff.
This sounds like an absolutely perfect match for zoom, since the problem where there are a bunch of people in the audience constantly discussing some other problems (or problems similar enough you can get a hint) doesn't exist anymore, and you don't need to book a large room for 6 hours so the stragglers could all be talked to by professors.
by primroot on 11/30/20, 10:23 PM
by hacknat on 12/1/20, 12:35 AM
by sleepysysadmin on 11/30/20, 3:33 PM
I without question wrote every bit of this text. It's saying I plagiarized. I feel like plagiarism software isn't quite there yet.
1 thing is for sure though. I certainly don't use enough commas.
by bpodgursky on 12/1/20, 1:39 AM
- Finding an answer online (on Chegg), and copy-pasting it. For homework, this is super common. Unless you build every assignment from scratch each quarter, there will be an answer available online.
- Paying someone to take the exam for you.
"Let them use wikipedia" resolves like 0% of the problems people actually run into when teaching remote classes. The problems are when people cruise through a class with 0 actual effort, through a combination of looking up answers online, and getting someone to just sit exams in your place.
by souprock on 11/30/20, 11:20 PM
The whole system is put at risk by the fraud. Grades become less meaningful because we've added a randomish negative signal. That devalues everything.
by ilaksh on 12/1/20, 1:43 AM
It's clear that the proper execution of anti-cheating is critical to avoiding a situation where the cure is worse than the disease.
Having said that, I do think that it is needed and will improve college efficacy.
Cheating in college has been an epidemic, with a majority of students surveyed saying they have cheated, and a significant percentage of those saying that it's acceptable.
If the courses are too hard or curriculum irrelevant or course loads too high, then those problems should be fixed. Cheating is not the answer.
by lowbloodsugar on 11/30/20, 11:03 PM
What we need is a way to have every exam question be unique to each student. If a student google the "how to" and then does it, great!
Is it fitting that Vernor Vinge wrote about this in Rainbows End [1]? He was a professor at SDSU.
by chriskanan on 11/30/20, 3:32 PM
by caymanjim on 11/30/20, 4:40 PM
by mensetmanusman on 12/1/20, 3:33 AM
by searchableguy on 11/30/20, 10:53 PM
Any controlled studies?
Are college students more ethical and honest than their non college counterparts?
by dusted on 12/2/20, 9:46 AM
by heimatau on 12/1/20, 12:18 AM
by ashtonian on 12/1/20, 1:39 AM
by supportlocal4h on 11/30/20, 3:58 PM
by happynacho on 11/30/20, 3:29 PM
by qwerty456127 on 12/1/20, 1:56 AM
by MeinBlutIstBlau on 11/30/20, 11:07 PM
by Sparkyte on 11/30/20, 11:04 PM
by at-fates-hands on 11/30/20, 11:26 PM
Just a few highlights from the article that really stick out:
“You have to record your environment, you have to record the whole desk, under the desk, the whole room,” Molina recalled. “And you need to use a mirror to show that you don’t have anything on your keyboard.”
On top of that, if the wireless connection was disturbed during an exam, Molina said, students would receive an automatic zero — no excuses.
He said he didn’t realize he hadn’t sufficiently shown his notepaper to his webcam, or that his habit of talking through questions aloud would be considered suspicious.
“At the beginning of the exam, you leave the area for about one minute without explanation,” Merrill wrote in an email to Molina. She added that it looked like he was using his calculator for problems that did not require a calculation and that he solved certain problems too quickly. As a result, Molina was given an F in the course and his case was submitted to SDSU’s Center for Student Rights and Responsibilities, where he could appeal the decision.
Neekoly Solis, an SDSU junior and first-year transfer student, said each test-taker now has to verbally explain each of their calculations to their webcam every time they use their calculators during an exam.
Then, she had to show the camera her desk, and underneath her desk, with her bulky desktop computer. She realized she was in a pair of shorts, and her webcam was picking up — and recording — seconds of her bare legs that could be seen by her older male professor. She was creeped out.
“You have to do a crotch shot, basically,” said Jason Kelley, associate director of research at the Electronic Frontier Foundation, a digital privacy group based in San Francisco. He recalled watching a tutorial video from another proctoring system called HonorLock, horrified as he watched the video subject do a long pan of their body.
Some other unsettling parts about the data their collecting:
Respondus’s website states that the default data retention period for Respondus Monitor is five years, but the client can change that.
And worse yet, what about the appeal process? Not exactly in the students favor:
Molina appealed. But even well into the fall semester and over a month after the accusations were filed, the office had canceled his scheduled meetings twice due to coronavirus-related emergencies.
After the third rescheduling, Molina finally had the chance to explain himself. One week later, he received a letter of “no action,” meaning the university would not pursue disciplinary action against him. He forwarded the letter to his business administration professors and requested that he get the grade he deserved. He said he had already emailed the student ombudsman twice, and never received a reply. Merrill finally gave him his grade back, almost two months after he’d received an F in the class.
In conclusion, you have a dodgy software program, that's highly invasive to your privacy. It can take months to get your appeal figured out. In the meantime, you're left to twist in the wind. And worst of all, the company keeps your data for five years.
