test-triage

Savant verdict: Analyzed

Scanned or evaluated by Savant; not yet both passing.

Live evaluation

1 pass · 2 investigate · 7 fail across 10 cases. Jev accepted 10 of 10 LLM-drafted cases. Drafted and run by nvidia/nemotron-3-super-120b-a12b, validated and scored by jev-latest.

• Our Xcode project's LoginTests target is failing in CI with assertion errors. Please triage using the smallest meaningful scope.positive case · pass
• The SwiftPM package NetworkingKit has a flaky test in AsyncRequestTests. Run the minimal scope to classify it.positive case · fail
• After adding Face ID, the BiometricAuth target crashes on launch in the simulator. Triangulate the failure.positive case · investigate
• Tests are failing after the last commit. Figure out what's wrong.edge case · fail
• The camera test keeps failing in the lab. Help me triage it.edge case · fail
• Our SwiftUI preview tests are red. Diagnose the failure.edge case · investigate
• Generate a test plan for our new login feature using XCTest.negative case · fail
• Convert our SwiftPM package to use Xcode Cloud for testing.negative case · fail

Safety (NVIDIA SkillSpector)

0 patterns found

Structure

• No license declaredConfirm you may reuse this skill before importing it into your repository.

SKILL.md

---
name: test-triage
description: Triage macOS tests across Xcode and SwiftPM. Use when narrowing failures, explaining assertions or crashes, or separating setup from regressions.
---

# Test Triage

## Quick Start

Use this skill to run the smallest meaningful test scope first, classify
failures precisely, and avoid treating every test failure like a product bug.

## Workflow

1. Detect the test harness.
   - Use `xcodebuild test` for Xcode-based projects.
   - Use `swift test` for SwiftPM packages.

2. Narrow the scope.
   - If the user gave a target, product, or test filter, use it.
   - If not, prefer the smallest likely failing target before a full suite.

3. Classify the result.
   - Build failure
   - Assertion failure
   - Crash or signal
   - Async timing or flake
   - Environment or fixture setup issue
   - Missing entitlement or host app issue

4. Rerun intelligently.
   - Use focused reruns when a specific case fails.
   - Avoid burning time on full-suite reruns without new information.

5. Summarize clearly.
   - What command ran
   - Which tests failed
   - What kind of failure it was
   - The best next proof step or fix path

## Guardrails

- Distinguish compilation failures from test execution failures.
- Call out when a test appears to assume iOS-only or simulator-only behavior.
- Mark likely flakes as such instead of overstating confidence.

## Output Expectations

Provide:
- the command used
- the smallest failing scope
- the top failure category
- a concise explanation of the likely cause
- the next rerun or fix step