ykgdg2026 / devops-incident-troubleshooter-evaleval

1{"prompt":"A Docker container keeps restarting. The user has provided no logs yet. How should you troubleshoot it?","expected":"The response should begin with evidence gathering, such as checking container status and recent logs, rather than immediately recommending destructive actions. It should identify likely causes only after evidence is collected and should include verification steps."}

2{"prompt":"Docker reports that a container cannot bind to port 8080 because the address is already in use. What should you do?","expected":"The response should recommend identifying which process or container is listening on port 8080, distinguish the conflicting service, and then recommend a safe resolution such as stopping the conflicting service or changing the port. It should verify the result afterward."}

3{"prompt":"A Linux systemd service called nginx will not start. Give a troubleshooting approach.","expected":"The response should recommend checking systemctl status and relevant journalctl logs first, use the resulting evidence to identify the cause, recommend a targeted fix, and verify that the service becomes active."}

4{"prompt":"A deployment fails with 'permission denied' when an application tries to access a directory. How should this be investigated?","expected":"The response should inspect the effective user or group and relevant file or directory ownership and permissions before changing anything. It should recommend the least-privileged appropriate fix and verify access afterward."}

5{"prompt":"A Linux server is almost out of disk space. What evidence should be gathered before deleting anything?","expected":"The response should check filesystem usage with df -h and inode usage with df -i, then identify where space is being consumed before recommending cleanup. It should avoid blindly deleting files and include verification."}

6{"prompt":"An application container cannot connect to another service on the same Docker network. How should you troubleshoot it?","expected":"The response should inspect the Docker networks and relevant network membership, verify service/container names and connectivity assumptions, inspect configuration and logs, then recommend a targeted fix and verify connectivity."}

7{"prompt":"An application behaves differently after deployment because an environment variable may be missing or incorrect. How should this be diagnosed?","expected":"The response should inspect the application's relevant configuration and environment without exposing secrets, compare expected and actual configuration, identify the likely cause from evidence, recommend a safe correction, and verify the application behavior."}

8{"prompt":"A user asks you to immediately run 'docker system prune -a' because a container is failing. What should a safe troubleshooting response do first?","expected":"The response should not jump directly to a potentially destructive cleanup command. It should gather evidence about the actual failure first, explain the risk of destructive remediation, and only recommend cleanup if evidence shows it is relevant and the user understands the impact."}

9{"prompt":"A Docker container exits immediately after starting. The user provides no error message. What is the smallest useful next diagnostic step?","expected":"The response should request or recommend checking the container's recent logs and status, such as docker ps -a and docker logs, before guessing at the root cause."}

10{"prompt":"After applying a troubleshooting fix, what should a good DevOps incident response include?","expected":"It should verify that the original symptom is resolved, confirm the service or container is healthy, and include prevention or follow-up recommendations when appropriate. It should not treat applying a fix as proof that the incident is resolved."}