How to interact with student code

Community Teaching & Outreach
1

Hello ! I have kind of an awkward question : what would be the best way to interact with student code in a “competitive” form.

Let me explain : I plan on making a competition on optimal control. The idea is to ask the student to produce code that, given an argument x output a control u. I will then run a simulation with their code and output the quality of their solution. Then I want to have the code of 2 (or more) students interacting in my simulation.

What would be the best way to implement this ? Currently I’m thinking of the following :

  • give them an archive with a main.jl where I have implemented a dumb policy, and a simulator.jl to test any evolution they make
  • ask for an archive per student with main.jl as well as any other file they need. I will control the folder name, and each will have a main.jl with a function policy
  • have a file that call each student file in order and make the simulations

What I do not know yet

  • is there a good way to protect against name overloading between student code ?
  • is there a good way to give a local name (eg policy1) to a function defined in another file under another name (eg policy) ?
  • is there any good practice that I am ignoring to do this type of thing ?
2

You could ask the students to put their solutions into modules with student-specific names, e.g. module SolutionStudent1, module SolutionStudent2. Then you can call SolutionStudent1.policy etc.

1 Like
3

Yes, definitely: use a module (which is, incidentally, the right way to organize Julia code regardless). In particular, you can create an anonymous module for each student, which will guarantee that no student’s code can overwrite another.

Here’s a simple example, where your simulation is just:

function simulate(f)
  result = 0
  for i in 1:10 
    result += f(i)
  end
  result
end

Your student files might look like:

# student_1.jl

f(i) = 1

# student_2.jl

f(i) = i + 1

And your test harness might look like:

julia> for file in ["student_1.jl", "student_2.jl"]
         mod = Module()
         Base.include(mod, file)
         controller = mod.f
         @show simulate(controller)
       end
simulate(controller) = 10
simulate(controller) = 65

An even better approach would be to have each student create a working module on their own, something like:

Controller/
  Controller.jl
  whatever_other_files_they_need.jl

I would still do the Module() wrapper on your test code, though, to ensure that their codes don’t overwrite each other.

3 Likes
4

There is one downside of the approach I just proposed, which is that a student could (accidentally or maliciously) overwrite the behavior of a Base function and thus break the whole testing system.

You can reduce the risk of students interfering with each other by running each student’s code in a separate julia instance. Of course, it’s still possible that they could write something that would interfere with your actual test harness, so you may have to decide how much work you want to put in and whether there’s actually any danger of student’s being malicious. For a simple assignment where you are (presumably) looking at their code by hand, I wouldn’t be too worried.

2 Likes
5

Thanks for the pointers. I don’t think that ther’s actually any danger of student’s being malicious, but they could definitely use the same name and the module approach seems to be the right one.

I think I will ask them to define their own module (with given name, like group_1).

I have two more questions maybe (if you have time):

  • how can I loop over i to define the module group_i and call the module
  • how to use the Module() wrapper around another module

I am thinking something like

for group in [group_1, group_2]
         mod = Module()
         Base.include(mod, group)
         using group
         controller = group.f
         @show simulate(controller)
       end
6

If each student submits a module, there should be no reason to wrap that in yet another module. You should be able to simply do

for group in [group_1, group_2]
    using group
    controller = group.f
    @show simulate(controller)
end

For the record, when I have had to process student code submissions (when getting students to write their own modules was “out of scope” for the class) I include-d their submitted .jl files in a bare-bones module similar to the first suggestion. To answer you question, in case you go this route, you can give a module a name when you create it with mod = Module(:Name)

7

Does this “competition” considers the time taken by the students code? Because if it considers it, running them all in the same Julia process can be kinda unfair, as the first one to use some method will compile it, and even if they only call their own code, different length and structured codes can take different times to compile. I would run all codes two times and take just the second run info (and not count compilation time), or run each code in their own Julia process (and count compilation time).

8

Thanks for the point. I’ll keep it in mind. I am not planning on counting time taken by the code per say, but I’ll need to have some time limit to protect against loop running forever.

The competition part will be on the quality of the control obtained (optimal control being out of reach, approximation / heuristic will be required).

1 Like