cs24bt047 / assignment-precheck

Assignment Precheck

Students lose marks for things that have nothing to do with their logic: a prompt the autograder didn't expect,
a file named cs24b001_lab4.c instead of CS24B001_lab4.c, %f where the spec wanted two decimals, a main()
left in a file that the grader links with its own driver. This skill checks a submission against its own spec,
requirement by requirement, with evidence, before the student submits.

Ground rules

Read these before anything else. They apply to every step.

1. Check, don't solve. Never write, complete or rewrite code that implements what the assignment asks for:
   not as a patch, not as a "corrected version", not as pseudocode that maps onto it line by line. You may quote the
   spec, name the file and line, show expected against actual output, and describe the fix in words. Mechanical
   compliance fixes (rename a file, delete a prompt string, remove a banned #include, move top-level code under a
   main guard) may be spelled out exactly, because they are not the graded logic.

2. The spec is the only authority. Every FAIL quotes the spec line it breaks. A convention the spec does not
   state (comments, const, naming style, freeing memory, PEP 8) is at most an Advisory, never a FAIL. If the
   student's summary of the spec disagrees with the spec text, the text wins; mention the disagreement.

3. No PASS without evidence. A requirement is PASS only if you saw it hold: you read the code, ran it, or listed
   the files. If you could not build, run or see something, it is CANNOT VERIFY, with the command the student
   should run to check it themselves.

4. Leave the student's files alone. The bundled scripts build and run copies in a temporary directory. Do not
   reformat, rename or tidy anything in place unless the student asks for a specific mechanical fix.

5. Their code stays local. Do not paste the submission into web tools, pastebins or online compilers.

6. Only the conversation is evidence about the student. Check roll numbers and names against the spec's pattern
   and what the student tells you. Do not bring in account names, emails or usernames from your environment, and
   never quote them in the report.

What you need

• The spec: PDF, text, screenshot or pasted. Read all of it, including "Submission guidelines", "Note",
  "Important", the grading rubric, and any FAQ or clarification posts the student has. If there is no spec, ask for
  it; do not check against requirements you guessed.

• The submission exactly as it will be uploaded. If the student uploads a zip, check the zip, not the folder
  they made it from.

• Provided files, if any: starter code, headers, test drivers, sample input and output files.

If the spec points at something you do not have ("follow the style guide on Moodle", "usual submission rules"),
list it under CANNOT VERIFY instead of assuming what it says.

If the code is pasted into the chat rather than given as files, write it to a scratch directory with the exact
file names the student says they will submit, and say so in one short sentence. The scratch path itself means
nothing to the student; leave it out unless they will need it.

Workflow

Do the steps in order. Do not skip step 1 because the code looks fine.

1. Extract the requirements

Turn the spec into a numbered checklist, R1, R2, ..., each with the exact spec words it came from. Walk through
the categories in [references/requirement-checklist.md](references/requirement-checklist.md) so nothing hides in a
footnote:

• Deliverables: file names (case, roll-number pattern), zip name and layout, what must and must not be included

• Interface: function names, signatures, return types, provided files that must not change, whether main()
  is required or forbidden, whether the module must be importable

• I/O contract: stdin, file or command-line arguments; exact output format; prompts; decimal places; case of
  words; separators; final newline

• Constraints: banned headers, libraries or functions; required techniques (recursion, malloc, no globals);
  language standard or Python version; input limits

• Edge cases the spec names: n = 0, empty file, negatives, duplicates, invalid input
• Build and run: the grader's compile command or Python version, -Werror, "no warnings"
• Documentation, only if required: header comment with name and roll number, README

Two requirements are implied by every programming assignment and need no quote: the code builds (or imports)
cleanly, and it reproduces the spec's own sample outputs.

A spec line that can honestly be read two ways goes under Questions for your TA, not FAIL.

2. Inventory the submission

List what the grader will actually receive. For a zip, run python scripts/scan_source.py --zip FILE.zip, which lists
contents and layout without extracting anything. Compare with the deliverables: wrong names, wrong case, nested
folders, missing files, and stray files (a.out, *.exe, *.o, __pycache__/, .vscode/, test inputs). Stray
files are a FAIL only if the spec says "only" or lists the exact contents; otherwise they are an Advisory.

A file name the student states is evidence. If it fits the spec's pattern it is PASS; if it breaks it, FAIL. Do not
add a separate CANNOT VERIFY row for "the file you will actually upload". Use CANNOT VERIFY only when no name was
given at all. If the name reuses the spec's example roll number, one Advisory line is enough.

If the spec provides files that must not be modified (a header, a driver), compare the submitted copy with the
original byte for byte (git diff --no-index ORIGINAL SUBMITTED or fc /b on Windows).

3. Static scan

Run the scanner on the source files, passing anything the spec bans or requires:

python scripts/scan_source.py SRC_FILE_OR_FOLDER [...] [--header PROVIDED.h] [--proto "SIGNATURE"]
       [--ban-include HEADER] [--ban-call FUNC] [--ban-import MODULE] [--no-globals] [--require-recursion FUNC]

• --header stack.h checks that every prototype in a provided header is defined with the same signature.

• --proto takes a signature copied from the spec, once per function: --proto "double average(const int *arr, int n)"
  for C, --proto "letter_grade(marks)" for Python. C parameter names are ignored, and int a[] equals int *a;
  float against double or a missing const is a mismatch. Python parameter count must match; different
  parameter names are reported as a CHECK because they break keyword calls.

• The rest of the output lists includes and imports, every function defined, where the program writes output,
  fixed-size arrays and globals, and known autograder traps with line numbers: prompts in output, non-standard
  headers and functions (conio.h, getch, strrev, itoa), void main, Python code that runs on import,
  third-party imports, hard-coded paths, functions that print instead of returning.

TRAP and CHECK lines are leads, not verdicts: confirm each against the spec and the code before it goes in the
report. Then read the code yourself for the checks in [references/c-checks.md](references/c-checks.md) or
[references/python-checks.md](references/python-checks.md).

4. Build and run

Put each sample from the spec into a case directory as NAME.in (stdin) and NAME.out (expected output), copied
byte for byte. Add NAME.args if the program takes command-line arguments. Then:

python scripts/run_samples.py --lang c --src FILE.c [FILE2.c ...] --cases CASE_DIR [--compile "GRADER COMMAND"]
python scripts/run_samples.py --lang python --src main.py --cases CASE_DIR [--import-check]

• If the spec gives the grader's compile command, pass it with {src} and {out} where the file names go, for
  example --compile "gcc -std=c99 -Wall {src} -o {out}". Without one the default is
  gcc -std=c11 -Wall -Wextra {src} -o {out} -lm.

• --driver FILE.c compiles the student's files together with a provided test driver, for assignments where the
  grader links its own main. --extra FILE copies a provided header or data file into the build directory; a
  provided file replaces the student's copy of the same name, and the script says if the two differed.

• --import-check (Python) imports the module with no stdin and reports anything it prints, any attempt to read
  input, or any exception during import. Use it whenever the spec says functions are called or imported by a grader.

• Each case is reported as EXACT, WHITESPACE-ONLY (matches once trailing spaces and trailing blank lines are
  ignored), MISMATCH, CRASH or TIMEOUT, with a diff in which $ marks the end of each line. A non-zero exit
  code is reported even when the output matches.

• If gcc is missing on Windows, run the same script inside WSL (wsl python3 scripts/run_samples.py ...) before
  falling back to CANNOT VERIFY.

• If the spec names a Python version, list every installed interpreter before deciding it is missing:
  py -0p on Windows (it also shows versions installed by uv, pyenv or the Store), and ls /usr/bin/python3*
  plus uv python list --only-installed on Linux or macOS. Pass the matching path to --python PATH. Only when
  the version is absent from those lists is the requirement CANNOT VERIFY, and then name the versions you did find.

Then probe the edge cases the spec names, and the extremes of its stated input limits, by adding more .in files.
When the spec defines the expected output for such a case, add a .out and compare. When it does not, only crashes,
hangs and output that breaks the stated format are findings. You may compute expected values in a throwaway check,
but never show the student solution code.

How output results map to the report:

Line-ending differences (\r\n against \n) are normalised by the script and are never findings.

Samples copied from a PDF can carry curly quotes, non-breaking spaces, ligatures or a lost final newline that the
real grader does not have. If a mismatch is only on such characters, say so and ask the student to check the
original sample file instead of calling it a FAIL.

5. Report

Use this layout every time, in this order. Leave out any section that would be empty, except Must fix:

## Precheck: {assignment} | {F} to fix · {C} could not verify · {P} passed

### Must fix before submitting
| Breaks | The spec says | Problem and evidence | What to change |
|---|---|---|---|
| R3, R6, R7 | "Do not print prompts such as "Enter n"" | `CS24B001_lab4.c:7` prints `Enter n: `, so sample 1 starts `Enter n: MIN -1` and the n = 0 case prints `Enter n: EMPTY` | Delete the prompt text so only the required lines are printed |

### Could not verify
| # | The spec says | Why | How to check it yourself |
|---|---|---|---|

### Passed
R1 file name · R2 header comment · R5 `double average(const int *arr, int n)` · R8 n = 0 · ... (also exact: n = 1000 at ±10^7)

### Questions for your TA
- ...

### Advisory (not required by the spec)
- ...

Not checked: hidden test cases, correctness beyond the samples and edge cases above, plagiarism checks.

• One row per root cause. When one defect breaks several requirements (a prompt that spoils every sample, a
  crash that fails every case), report it once: list every requirement it breaks in the first column, quote the
  main spec line, and put the evidence for each in the same cell. {F} counts these rows, the separate things the
  student must change, not requirements.

• One status per requirement. Each requirement appears under exactly one heading. A FAIL is not repeated under
  Could not verify, and a requirement listed as passed is not questioned again under Advisory.

• Order the fixes by marks at risk: first anything that makes the grader score zero (wrong file name, won't
  compile, link error, crash or prompt on the samples), then wrong output, then documentation and style.

• "What to change" is words and precise locations. No replacement code for graded logic.
• If there are no FAILs, say so in the first line and still list what could not be verified.

• Keep it short. Passed is one line of requirement numbers and short labels, not a bullet or sentence per
  check. Extra inputs you tried beyond the samples go at the end of that same line as one parenthetical clause,
  never as a paragraph or list under it; the student needs to know they were tried, not how. Do not include
  scratch paths or the names of the bundled scripts; give a command only when the student should run it. At most
  three Advisories, and only ones that could cost marks or time.

Requests to handle carefully

• "Just fix it" or "give me the corrected code": say in one sentence that you will point to each problem but
  will not write the graded code, then give the location, the spec line it breaks, and the change in words.
  Mechanical compliance fixes can still be spelled out exactly.

• "Don't explain", "deadline in 10 minutes": send the short form only: the decline sentence if one is needed,
  the Must fix table, and one line naming anything important you could not verify. Leave out Passed, Questions and
  Advisory unless one of them would cost marks.

• "Is my logic right?" beyond the samples: run more inputs the spec allows and report what happens. Do not offer
  an alternative algorithm or a fixed version.

• "Edit my files for me": only mechanical compliance fixes the student names, never logic, and show exactly what
  changed.

• The spec contradicts itself or the course rules (C99 in one place, C11 in another): report both under
  Questions for your TA and check against the stricter reading.

Mistakes to avoid

• Saying the submission looks good after reading it, without running the samples.
• Failing a submission for something the spec never asked for.
• Normalising whitespace or letter case silently and calling the output a match.

• Testing on Windows and assuming a Linux grader behaves the same: conio.h, getch, strrev and itoa build
  with Turbo C or MSVC and fail with gcc; math.h functions need -lm on Linux.

• Stopping at the first FAIL. The student needs the whole list in one pass.
• Counting one defect as several FAILs, which makes a one-line fix look like three problems.
• Listing the same requirement under two headings, such as FAIL and Could not verify.
• Missing the requirements that sit in the submission section at the very end of the spec.
• Showing solution code "to demonstrate the expected output".