tanyagarg5315 / hackathon-submission-checker-accuracy-in-extracting-competitioneval
This evaluation tests whether the skill reliably checks hackathon submissions against provided rules without inventing facts or marking unsupported requirements as verified.
0
dataset.jsonl859 B
{"id":"case-1","scenario":"The competition requires a public GitHub repository and a slide deck. The user provides slides.pdf but no repository link. Check whether the submission is ready.","expected":"Mark the repository MISSING because it is mandatory and no link was supplied. Verify the slide deck only if its presence is established. The verdict must be NOT READY, with an action to provide the repository link."} {"id":"case-2","scenario":"The main competition brief requires a 3-minute demo video, but the FAQ says the video must be 5 minutes. The user has not clarified which rule is current. Evaluate readiness.","expected":"Flag the conflicting video-duration requirements and mark the requirement NEEDS CONFIRMATION. Do not guess which rule takes precedence. The verdict must be NEEDS CONFIRMATION until the organizer's requirement is clarified."}