This is a group assignment with an individual component. Assignment to groups is available on Moodle: Group allocation for the first coursework assignment 

Each group is assigned with the task of using an automated program repair tool to patch software bugs from the QuixBugs dataset: https://jkoppel.github.io/QuixBugs/. Note that the QuixBugs dataset contains examples written in Java and Python -- for this assignment you should target buggy programs written in Java. 

You only need to run one of the following two APR tools (your choice which one): 

- MAGPIE (https://github.com/bloa/magpie), with the genetic_programming setting 

- Astor (https://github.com/SpoonLabs/astor), in the jgenprog mode

Please make sure you can run the tools with the settings available on the CS lab machines, that is: Linux OS, Python-3.9 for MAGPIE or Java-11 for Astor. 

Hint: you can use master_legacy branch in MAGPIE and build Astor with Java-11. 

Please use Java-11 to compile the QuixBugs programs. 

You can run the code on your own machines, analyse results etc. to speed up the process. But please prepare instructions that would allow your fellow students to run your code within the environment specified as above. You can test the code by coming into the labs, when free, or test remotely, for instance, by connecting via ssh: https://tsg.cs.ucl.ac.uk/working-from-home/ 

Lab machine list is available here: https://tsg.cs.ucl.ac.uk/lab-status/

Since you might be running the tools on different machines (I encourage all members of the group to run at least one repair attempt), I recommend setting a common terminating condition that is not related to runtime: Please set generation size to 10 and population size to 40. For MAGPIE, the pop_size argument will set population size, the max_step the total number of evaluations, so please set max_step to 40*10=400.  Astor simply uses -generation and -population arguments, which you can set accordingly.

Each student in the group should try to fix one of the bugs, so a group of 5 students is required to try to repair 5 different bugs. Any buggy Java programs from QuixBugs can be used. Here is a list of buggy programs you might want to consider first: lis, rpn-eval, mergesort, levensthtein, knapsack, detect-cycle.

Note you are not required to fix these bugs, but to run the selected tool to try to find a correct patch, and be able to comment on the results.

Each student is required to submit 3 files: a group report, an individual report, and an artefact allowing replication.

The group report should be in pdf format, min. font size 11pt, min. margins 2cm, and no more than 2 pages long and include:

-the aim of the experiment

-formulation of the SBSE problem you are solving (hint: think about the key SBSE components)

-methodology used to obtain the results (tool used, its settings, etc.)

-list of buggy programs your group tried to repair

-result summary (how many bugs fixed/not fixed)

-critical analysis of the results, taking into account the technique used for repair (note there will be space for concrete patch examples in the individual reports)

Additionally, in the group report please state who contributed what, how you organised the team. This part will not be marked and is not counted towards the page limit. This section is to encourage co-operation within the group. It usually won't affect the group mark. Only in the case where a student might not have been engaged at all I reserve the right to apply a penalty.

The individual report should be composed of 1 pdf page, min. font size 11pt, min. margins 2cm. Each person should describe results obtained for one buggy program. Each person should describe results for different bugs, so please decide within your group who describes which bug. In particular, the report should include:

- the chosen buggy program name

- description of the bug

- information on results obtained for this bug

- critical analysis, including explanation why the bug was fixed or not, taking into account the technique used to repair the bug

The artefact should be in a form of a zip file which, at a minimum, contains a README.txt file that describes the steps required to reproduce your experiment on a Linux OS setup with Java-11 and Python-3.9 (command used to run the tool etc.). 

You can assume a jar file for Astor and MAGPIE folder are available. You cannot assume any other software installed and/or require other software to be installed. You can additionally add any other files needed for replication. Do not include tools.

Marking

You will get a percentage grade, as specified in the grading scheme presented in week 1. As a quick reminder, 70% and above represents distinction work, 60%+ good work, 50%+ satisfactory work. You need to get at least 50% to pass this assignment. 

This coursework constitutes 50% of your final mark. The grade you will receive will be the weighted grade of two components 0.6 * [group report + artefact] and 0.4 * [individual report]. For instance, if you get 70% for the group tasks and 80% for the individual report, your final grade for Coursework 1 will be 0.6 * 70% + 0.4 * 80% = 74%.

Both group and individual marks will be provided with textual feedback.

The group report is marked based on the following criteria:

- To what extent is all the required information provided?

- How well is the SBSE problem formulated?

- How well does the report demonstrate knowledge of SBSE techniques?

- How well does the report show how to apply SBSE in practice?

- How well is the report presented?

- Is the file in the right format?

The individual report is marked based on the following criteria:

- To what extent is all the required information provided?

- How well does the report demonstrate knowledge of SBSE techniques?

- How well is the report presented?

- Is the file in the right format?

The artefact is evaluated on the following criteria:

- Have the experiments been correctly conducted, following the specification in the coursework description?

- Is there enough information provided in the group report and the artefact to be able to run at least one repair attempt in the specified environment?

- How well is the artefact presented?

- Is the artefact in the correct format?