a1.2-5-001

A1.2 · D5 · f1

D5
Difficulty
4
Gaps
0
Red Herrings
70%
Avg Score

Task

You are conducting a SOC 2 Type II readiness assessment for NexGen Platform covering the observation period Q4 2025 (October 1 – December 31, 2025). Review the availability policy, backup monitoring logs, and restore test results. Assess compliance with A1.2 (Backup and Recovery). Consider whether identified issues are material findings or acceptable operational variances. Provide your professional judgment on each finding's severity and whether it would result in an audit exception.

Evidence

Findings

IDTypeSeverityFinding
F-001gapmediumTwo backup failures in the observation period (97.8% success rate)
The backup log shows 2 failures out of 92 days: Nov 3 (disk full) and Nov 16 (network timeout). The Nov 3 failure was recovered with a manual re-run 2.5 hours later. The Nov 16 failure had NO re-run —...
F-002gaphighNo quarterly restore test performed during the Q4 observation period
Policy Section 3.3 requires 'Full restore test performed quarterly.' The evidence shows tests in Q2 (June 15) and Q3 (September 20), but no test was performed in Q4 2025 (the observation period). This...
F-003gapmediumRestore time trending toward RTO breach
Q2 restore took 2h 45m. Q3 restore took 3h 30m. RTO is 4 hours. The Q3 test notes explicitly warn: 'Growth in data volume increasing restore time — may exceed RTO by Q2 2026 if trend continues.' The d...
F-004gaplowIncremental backup logs not provided — cannot verify 4-hour RPO claim
Policy states incremental backups run every 4 hours, but the backup monitoring logs only show daily full backups. No evidence of incremental backup execution was provided. Similarly, WAL archiving is ...

Results

ModelProviderScoreRecallPrec.F1GapsReported
Sonnet 4.6Anthropic89%100%80%89%4/45
Opus 4.7Anthropic67%100%50%67%4/48
GPT-5.5OpenAI73%100%57%73%4/47
GPT-4.1OpenAI80%100%67%80%4/46
Haiku 4.5Anthropic80%100%67%80%4/46
GPT-4oOpenAI30%75%19%30%3/416