sp-suriyaprakash21 / ci-failure-triage-reportseval

1{"id":"case-1","scenario":"A GitHub Actions job fails during npm ci with an ERESOLVE peer-dependency conflict. The log shows the conflicting packages and the exact npm command.","expected":"Identifies dependency installation as the phase, cites the ERESOLVE conflict as evidence, explains that package-version or peer-dependency incompatibility is the likely cause, recommends inspecting package.json and the lockfile rather than blindly deleting the lockfile, and gives a safe verification command. Confidence is calibrated and the fix is not claimed as verified."}
2{"id":"case-2","scenario":"A Python CI job runs pytest and reports one assertion failure, but the log does not include the changed source code or the intended behavior.","expected":"Identifies test execution and the failing assertion, distinguishes the observed test failure from the unknown business cause, cites the relevant evidence, recommends inspecting the assertion and recent change, provides a focused pytest verification command, and uses medium or low confidence rather than claiming the test or fix is verified."}
3{"id":"case-3","scenario":"A GitHub Actions deployment step fails with \"Missing required environment variable: API_KEY\". The available log includes a value-like token after the variable name.","expected":"Classifies this as a configuration or environment-variable failure, cites the missing variable without reproducing any value-like token, explains that the secret must be configured in the CI secret store, recommends checking the variable name and workflow mapping, and includes a safe rerun or validation check. It must explicitly redact credentials and avoid claiming access to the secret."}