a1.2-4-001

A1.2 · D4 · f1

Difficulty

Gaps

Red Herrings

69%

Avg Score

Task

You are a SOC 2 auditor evaluating Cobalt Systems' backup and recovery controls under Trust Services Criteria A1.2. Review all provided evidence documents to assess whether backup processes, recovery infrastructure, and recovery testing meet the organization's stated objectives. Identify any gaps, deficiencies, or findings. Be careful to distinguish between genuine control gaps and issues that have been properly mitigated through compensating controls or formal exception processes.

Evidence

evidence/backup-policy.md — Cobalt Systems backup and recovery policy defining RPO/RTO targets, backup schedules, encryption requirements, and restore testing procedures
evidence/backup-log-q4.csv — Daily backup execution log for Q4 2025 (October-December) showing 92 days of backup operations including successes, failures, and re-runs
evidence/restore-test-q4.json — Q4 2025 quarterly restore test report with restore timeline, data integrity results, RTO compliance assessment, and quarter-over-quarter trend analysis
evidence/exception-register.csv — Exception register documenting the CISO-approved exception for the November 8 backup failure with compensating controls
evidence/disaster-recovery-site-info.md — Disaster recovery site infrastructure details and failover procedures -- related to DR but not directly relevant to backup control testing [noise]
evidence/vendor-sla-summary.csv — Cloud vendor SLA performance summary for Q4 2025 -- general vendor management data not specific to backup controls [noise]

Findings

ID	Type	Severity	Finding
F-001	red_herring	low	November 8 Backup Failure with No Re-Run The November 8 daily backup (BKP-2025-1108) failed due to a cross-region network timeout and no re-run was executed, creating an apparent 48-hour gap between successful full backups (November 7 to Nov...
F-002	gap	high	December 1 Backup Re-Run Exceeded 4-Hour RPO The December 1 backup failed at 02:31 UTC and the re-run did not begin until 08:15 UTC -- a 6-hour gap attributed to delayed on-call response. The re-run completed at 10:08 UTC. Combined with the orig...
F-003	gap	medium	Restore Time Trending Toward RTO Breach The Q4 restore test completed in 3 hours 50 minutes against a 4-hour RTO target, leaving only a 10-minute margin. The Q3 restore test completed in 3 hours 15 minutes with a 45-minute margin. This repr...
F-004	gap	medium	No Incremental Backup Logs to Verify 4-Hour RPO Claim The backup policy states that incremental file system backups run every 4 hours (at 06:00, 10:00, 14:00, 18:00, and 22:00 UTC) as part of the RPO strategy, and that WAL segments are archived continuou...
F-005	gap	low	Backup Storage Growth Without Documented Capacity Planning Analysis of the backup log shows consistent data growth from 142.3 GB on October 1 to 183.0 GB on December 31 -- a 28.6% increase over the quarter (approximately 1.5% per week). The backup policy requ...

Results

Model	Provider	Score	Recall	Prec.	F1	Gaps	Reported
Sonnet 4.6	Anthropic	100%	100%	100%	100%	4/4	4
Opus 4.7	Anthropic	62%	100%	44%	62%	4/4	9
GPT-5.5	OpenAI	80%	100%	67%	80%	4/4	6
GPT-4.1	OpenAI	80%	100%	67%	80%	4/4	6
Haiku 4.5	Anthropic	32%	100%	19%	32%	4/4	21
GPT-4o	OpenAI	60%	75%	50%	60%	3/4	5