Teaching and grading with Julia and Pluto (... and ChatGPT)

Hi everybody. We have used Julia and Pluto in an economics course over the last four years. Students use Pluto notebooks to solve numerical (and symbolic) problems, and the assessment moments (tests/exams) are also undertaken using Pluto notebooks in in-person evaluations (students use their personal computers). The assessment moments are open-book, with the possibility of accessing the internet.

We have around 300 students enrolled in our course, and until recently, we were not aware of any fraud at all. Pluto’s flexibility has allowed us to minimize the eventuality of fraud by truncating the test/notebook students receive by email. In essence, what we have done is this: (i) the test each student receives is randomly generated, (ii) students can not change the contents in the notebook’s cells, (iii) they can not open new cells nor delete existing ones, and (iv) students can not copy text from the Pluto window (browser).

Our teaching experience with Julia and Pluto has been overwhelmingly positive, benefiting both teachers and students. However, ChatGPT’s explosive impact has posed a significant challenge to all of us, particularly in the context of grading. ChatGPT can now answer any question, whether numerical, symbolic, graphical, or even a mixture of the formers, in the blink of an eye.

To avoid fraud, what have other universities done (or will do) to minimize the terrible problem of having students using ChatGPT in in-person evaluation assessments when the computer is used in that context? I would be very grateful if the members of this community could help with information about steps that have been (or will be) implemented to minimize fraud in such an environment (I am concerned with tests only, not with thesis or reports).

We have been discussing this issue at my university and we have one wish and a constraint:

  1. We want to avoid returning to a process where exercises and tests were solved using pencil, paper, and a rudimentary calculator.
  2. Switching off the internet during the evaluation moments is (apparently) not an option, given the large number of students involved (300) and the problems it would cause other students in that part of the campus. (not to mention that students may have other means to connect to the internet)

What are other universities doing to prevent fraud by students resorting to ChatGPT in in-person conducted tests/exams? Any help will be very much appreciated. Thanks.

4 Likes

Maybe this could be dealt partially with a proctoring system: on one Uni I know they use a proctoring system called Safe Exam Browser - News (which is also open source)

5 Likes

For computers provided by the university, I think that this a good idea. But, to me, requiring to install such a program on the students computer would be problematic from an ethical stand point (it is their computer after all).

The system mentioned above seems to support only Window, Mac OS and iOS. What would be done if a student has Linux or older OS which is not supported by the proctoring system.

Maybe an alternative approach would be to require that the users all connect to a WiFi access point that you monitor. (But they are also ways cheat in this case but at least it is less intrusive).

1 Like

@Tortar, thanks for the link. I think that software was used by many universities during COVID. I have never used it because we do not use Moodle or the like. We have used Pluto until now, and we have not had any problems with fraud because we customized Pluto to prevent that kind of problem. However, we are facing a severe problem because ChapGPT can now import screen captures and provide answers quickly and correctly in most cases.

A solution that blocks the screen capture functionality will greatly help, but given ChatGPT’s dramatic capabilities, something better would be advisable. Turning off WiFi access points can not be implemented because it would conflict with the needs of others on campus.

We do not impose a mandatory requirement for installing software on students’ computers. But I think that will not be a relevant problem. Those who do not accept it will have to take the exam in our computing labs, and I believe most students will choose their personal computers. Finally, the number of Linux users is residual, and we can easily deal with these cases.

I was expecting that given the daunting shadow of ChaptGPT over assessment moments, which are a critical part of university activity, there would be something better now than when COVID broke out. I am surprised that, apparently, we are still facing the same tools.