Hacker News Reader

Comment by mkl | Hacker News Reader

Copy-pasting from my other comment:

> I see it as a currently-necessary annoyance, as the least bad option. The alternatives have greater deficiencies: human invigilators using Zoom etc. don't scale (institution experience); oral exams don't scale (my experience moderating such assessments); no invigilation leads to cheating (by few students but enough to be a real problem, especially for professional qualifications - my direct experience and institution experience); shutting down education until the pandemic's over is unfeasible.

Last year, we tried not monitoring, we tried Zoom invigilation, we tried orals. Lockdown rules prevented us from sending in-person invigilators in one semester (and we couldn't have hired even 10% of what that would take) and the other we invigilated in person like normal. We are spending more than ever before, so there are no savings. All of these methods failed. This year, grudgingly, we have moved to proctoring software.

View on HN

I find your statements in different parts of the thread quite contradictory. On one hand you say that things don't scale because of number of students, but then a bit down thread you mention it's not worth making a new course for 10 students (out of 640), but for 10 students in person proctoring is clearly not a problem. You also mention the issue of cheating for professional qualifications, but again, that's a tiny fraction of all exams.

Regarding scaling of oral exams, there's actually some interesting research/calculations (I try to find the reference later) and the cross-over is somewhere around 150 students when oral exams become slower (I do think this is quite teacher and subject dependent though).

Also about budgets, I believe that your department is spending more than ever, the issue is property services departments of the universities should be saving large amounts. Considering lock-downs and staff and students working from home, maintenance cost should be way down. I suspect though that money lands in completely different buckets (don't get me started on the business of university property services, we pay rent in our overhead cost when we get grants that are higher than renting office space on the main shopping street in our city).

Well I'm talking about decisions about different things by different people 1) running exams (institution level - run centrally), 2) creating bespoke purpose-designed online courses (mostly department level). The decision about exam invigilation was made at the institutional level, where 10 is a small number of online students for a course, and they have dozens of courses to worry about. For most of the institution's courses, other solutions don't scale, and for the institution as a whole they definitely don't. The course with 10 online students may only run online this year, and it takes hundreds of additional hours to make a purpose-designed online course (we're doing that for other courses where the material will be reused).

The oral exams I moderated didn't seem like they could scale past 20 students. The reason our online students are online is almost always because they are overseas, and the majority of them have limited English. In that context, orals are discriminatory, stressful, and very slow.

I'm in NZ. Our universities have mostly been operating in person as normal (i.e. costing just as much) with far fewer international students (who bring in money), except in addition we have comparatively small numbers of resource-intensive online students as well, and are attempting to bring parts of entire degree programs online for the first time.

Hah! I did my PhD in NZ and became a part kiwi (I still miss it to be honest). I'm back in Europe now, in Sweden and as you might know we have had a very relaxed (and very weird from a non-Swede perspective) approach to the pandemic. So we did have in-person and online exams, but all with proctering via zoom.

The bar to use such invasive, buggy, user hostile spyware should be a lot higher.

Just because this solves your problem (it clearly doesn't) doesn't mean it's a good idea. Killing half the students also solves your problem.

You need to live in a world where this is not an option, period. What's the next solution then? People forget there are always more manual solutions even if they take more time...

> human invigilators using Zoom etc. don't scale (institution experience)

Could you explain further why this did not scale?

I'm imagining an invigilator watching video feeds using an interface similar to, say, that used by security guards to monitor surveillance feeds. I would think that a single human invigilator can monitor more students using this system than an in-person setting. What am I missing?

In person, a single proctor can monitor 200+ students; our eyes capture far more detail and field of view than computer screens, and our peripheral vision is tuned for detecting unexpected motion. With Zoom, either you have one meeting with everyone in it, in which case students can look directly at each other, or you have a separate meeting for every student, in which case you need a large number of devices, all visible at the same time. I can't see the former scaling past 49, and I can't see the latter scaling past 20. If you had some software designed specifically for this and several big screens, it would still be pretty hard to pay anywhere near as close attention as you can in person.

On top of that, there's always a bunch of annoying mucking about getting set up for Zoom invigilation: angle of camera, light, checking environment, etc. All that needs communication to and fro, and it can take up to 5 minutes for a single student. Now multiply that by say 100 (a bit of parallelism is possible, but individual communication is needed with each student).

So get more proctors.

Given the cost of college courses, schools could easily afford $5 per student per course, and $100 for proctoring 20 students for a single exam is more than fair.

As both a TA and a student during the virus, we did better and so can you. Stop trying to shoehorn your in person lesson plans and exams into virtual and design new material and tests. Dont be a shitty teacher just because you're too lazy to change your now dated ways.

Please make your substantive points without swipes. Crossing into name-calling and personal attack is not cool.

https://news.ycombinator.com/newsguidelines.html

Please don't make assumptions or resort to baseless insults.

We have designed entirely new fully-online interactive courses in response to this.

In other courses, where we have e.g. 640 local students and 10 online, they need to sit the same test for it to be fair.

Sounds like you need to design an online version of that course if you're going to allow online students to take it.

While I disagree with the OP on the use of proctoring software saying something like "just design an online version of the course" displays a massive ignorance of academic workloads. Designing a new course, especially one which works online, is a huge amount of work, who is going to do all that? Most academics even in normal semesters work >50h (more likely >60h) per week.

Just adjusting your lecturing to online can easily take an additional 2h preparation per lecture, and often takes significantly more, because you have to also adjust all communications and how to make material available. That's even without redesigning the course for online learning. I know a lot of colleagues who got very overworked, because of the increased demand on teachers during the pandemic.

The expense isn't justified for 10 students, since this may well be the only year that course is offered online. They are not getting a bad deal out of it. The fully new online courses are first year, and the expectation is that students will come to study in person for the remaining years of their degree. The other course I'm talking about is second year, but those 10 students can't get into the country at present - a temporary obstacle.

> The expense isn't justified for 10 students, since this may well be the only year that course is offered online.

So instead you'd rather waste money on a barely-functional piece of software that actively invades your students' privacy?

It seems to function just fine (I'm not talking about Proctorio), and it doesn't invade privacy any more than the Zoom-based tutorials, which is barely at all. It is also better than any other alternative I know of, many of which have been tried already and found wanting. Privacy is not a god to whom all things must be sacrificed. All education involves losing a little privacy (in person people will know what you look like, where you're from, and many other things that are hidden for remote students; really they have more privacy than any others), and the fact that software is involved doesn't suddenly make things worse.

>the fact that software is involved doesn't suddenly make things worse.

When that software is required to be installed on private machines and is difficult to remove, yes, it absolutely does make things worse.

The software we use is easy to remove, and students have a lot of software installed on their private machines already, some of it more invasive in software terms (e.g. games with anti-cheating features). I have more qualms about the software and the privacy implications than the students do, and I'm resigned to it for now; the students just went ahead and installed it without comment.

There's two problems with this statement. First is the assumption that students don't care about privacy, second is the lack of discussion about consent.

I'm a student who takes special care about the software I install on my laptop. I use a Linux distro, run primarily open-source software, and sandbox every single proprietary app (limited access to files, no admin at all, no screen recording, disabled webcam, ...). I've also looked into several of these exam spyware tools (you really are forcing students to install spyware), and they're built with often hilariously poor security practices.

Which is to say nothing of the regularly stolen source code; If you held the exam spyware solutions to the same standards that you held students to, you would write up almost every single vendor to the Academic Integrity office. Another example of hypocrisy in academia from the perspective of a disgruntled student.

I deliberately do not install any video games with invasive anti-cheating functionality (and I regularly critique them, like I do for exam spyware); that is a false equivalence anyway, since they don't deal in the same breadth of personally identifiable information (like a permanently saved panorama of my bedroom).

Don't assume all students are the same.

Second, the consent dynamics are wildly different. For a game, its like "you trade this in for fun/relaxation" - and there's always other games that don't spy on you. I play those. With universities, many pulled a fast one and introduced the spyware to students after their tuition is already paid, and said "use it or drop the course". You can't switch universities because one university didn't consider the ethics of spyware; you can switch games much more easily.

Right, but that software was their choice to install, not forced upon them by their academic institution. It's good that it's easy to remove, but other people's computers are not your property.

Why should this be the only year; why not join the 2020s and do the later classes online in the future too? If the students did their first year remotely they've already learnt to learn that way, so it should work even better with second- and third-year students AFAICS.

Reply on news.ycombinator.com