Autolab Public Documentation

Introduction

Autolab is an open-source automated code grading platform developed at Carnegie Mellon University and used by universities around the world. The UB CSE IT department maintains a customized variant of Autolab for use in our courses. It's hosted on premises at UB, and you can access it at https://autolab.cse.buffalo.edu.

This repository will serve as a guide, primarily for instructors, on how to use Autolab.

Table of Contents

Getting Started

• Your First Login
• Create a Course

Course Management

• Viewing Your Courses
• Course Page
• Manage Course
• Course Settings
• Enrolling Students
  • Using the CSE IT Classlists Tool
  • Uploading a CSV File
• Enrolling Teaching Assistants
• Additional Course Management Features

Create an Assessment

• Install Assessment
  • Import from Tarball
  • Create from Scratch
• Assessment Page
• Edit Assessment
  • Basic
  • Handin
  • Penalties
  • Problems
  • Advanced
• Add Problems
• Make a Submission
• View, Annotate, and Grade a Submission
  • View a Submission
  • Annotate a Submission
  • Assign a Grade by Annotation
  • Stacking Annotations

Create an Autograder

• Setup
  • Create an Assessment
  • Set the Handin Filename
  • Add some Problems
  • Add an Autograder
• A Minimal Autograder
  • grader.py
  • autograde.tar
  • Makefile
• Upload the Autograder to Autolab
• Test the Autograder
  • Upload a Correct Solution
  • Upload an Incorrect Solution
  • Upload a Problematic Solution
    • Crashing the Grader
    • Getting the Grader's Source Code
    • Setting an Arbitrary Score
• Formatted Feedback
• Embedded Forms
  • Create the HTML Form
  • Upload the HTML Form
  • Considerations for the Autograder
  • Embedded form with file submission (UB Feature)

Grader Assignment Tool

• Accessing the GAT
• Import Your Course from Autolab
• Manage People
• Manage Conflicts of Interest
• Create a Grading Assignment
• View a Grading Assignment
• Archive a Grading Assignment

UB Course Sections

• Configure your Course Sections
  • Magic Import
  • Create a Section Manually
• Configure an Assessment
• How it Looks to Students

Create an Autograding Image

• Creating your Own Image
• Testing your Image
• Add your Image to Autolab

The Autograding Process

• Sending the Job to Tango
• Tango Queues the Job
• Running the Job
• Autodriver
• After the Job is Finished

For IT staff, there's also a private internal documentation repository available here: https://github.com/UB-CSE-IT/Autolab-Internal-Docs.

Sample Autograders

Sample autograders are located in the sample_files directory. These are referenced throughout the documentation. Some autograder directories contain multiple correct and/or incorrect solutions to demonstrate how the grader works in different situations.

• autograder0 is the most minimal autograder just to give you an idea of the format.
• autograder1 is a more realistic autograder that grades 3 problems differently, but it's not robust enough for real use.
• autograder2 demonstrates how to use the settings.json file to get metadata about the submission.
• autograder3 demonstrates how to grade an embedded form submission.
• autograder4 demonstrates how to create an embedded form that allows submitting a file.

Navigation on GitHub

The individual pages don't have tables of contents. You can use the GitHub Outline feature to navigate between sections of a document.