Chương 19 Nâng Cao
Test Reporting & Metrics
Tổng hợp kết quả testing, xây dựng dashboard và communicate hiệu quả với stakeholders về quality.
📄 Test Summary Report Required
Test Summary Report là tài liệu tổng kết sau mỗi testing cycle. Đây là deliverable chính của QA.
📋 Report Template — Sprint 14
Project: Sports Analytics Platform | Sprint: 14 | Period: Jun 2–15, 2025QA: [Name] | Environment: Staging (v2.5.1)
EXECUTIVE SUMMARY
Sprint 14 testing is complete. System is CONDITIONALLY READY for release pending fix of PROJ-445 (Major — payment discount calculation). All other features pass acceptance criteria.
TEST EXECUTION SUMMARY
• Total Test Cases: 156 | Executed: 153 | Pass: 147 (96.1%) | Fail: 6 (3.9%) | Blocked: 3
• Coverage: 96% of sprint stories tested (1 story blocked by environment issue)
DEFECT SUMMARY
• Total New Bugs: 12 | Critical: 0 | Major: 1 | Minor: 8 | Trivial: 3
• Open: 3 (1 Major, 2 Minor) | Fixed & Verified: 9 | Deferred: 0
RISK & RECOMMENDATION
PROJ-445 (checkout discount calculation) must be fixed before deployment. ETA from dev: June 16. Recommend deployment on June 17 post-fix verification.
Thành phần bắt buộc:
- Executive Summary1-3 câu cho PM/stakeholder đọc nhanh: đã xong chưa, go/no-go, vấn đề chính là gì.
- ScopeNhững gì được test, những gì không (in-scope/out-of-scope). Protect yourself.
- Test Execution NumbersTotal, Pass, Fail, Blocked với % — đây là "headline numbers" stakeholders focus vào.
- Defect SummaryCount theo severity, open vs closed. Major/Critical open = potential release blocker.
- Risk & RecommendationExplicit Go/No-Go với rationale. Nếu conditional: list conditions rõ ràng.
- Lessons LearnedSau mỗi release: gì went well, gì cần cải thiện, action items cho sprint tiếp theo.
📊 KPIs & Dashboard
| KPI | Công Thức | Target | Alert When |
|---|---|---|---|
| Test Coverage | (TC executed / Total TC) × 100 | ≥ 95% | < 85% |
| Pass Rate | (Pass / Executed) × 100 | ≥ 90% | < 80% |
| Defect Leakage | (Prod bugs / Total bugs) × 100 | < 5% | > 10% |
| Bug Fix Rate | (Closed / Reported) per sprint | > 80% | < 60% |
| Reopen Rate | (Reopened / Closed) × 100 | < 10% | > 20% |
| Automation Coverage | (Automated TC / Total regression TC) × 100 | Growing | Declining |
JIRA Dashboard Gadgets hữu ích cho QA:
- Bug count by severity/priority — donut chart
- Bug created vs resolved over time — bar/line chart (trend)
- Sprint burndown — line chart
- Bugs by component/module — identify problem areas
- My open bugs to retest — quick access filter
- Overdue issues — flags stale bugs needing attention
🚦 Go/No-Go Decision Framework
Go/No-Go không phải cảm tính — cần framework rõ ràng dựa trên data:
- Go ✅Pass rate ≥ 90%, 0 Critical open, 0 Major open (hoặc có workaround được PM approve), coverage ≥ 90%, performance tests pass, security scan clean.
- Conditional Go 🟡1-2 Minor bugs open được PM accept, tất cả Critical/Major đã fix và verified, coverage đạt với documented gaps, risk accepted in writing.
- No-Go 🔴Có Critical bug open, Major bug ảnh hưởng core flow, pass rate < 80%, coverage < 70%, performance fail SLA, data security risk.
⚠️ QA Role trong Go/No-Go
QA recommends, PM/business decides. QA cung cấp facts và risk assessment. Nếu PM chọn deploy với known risk → document decision in writing: "PM has accepted the risk of deploying with PROJ-445 open. QA recommends monitoring [specific metric] post-deployment."
👥 Stakeholder-specific Reports
| Audience | What They Care About | Report Format | Frequency |
|---|---|---|---|
| Dev Team | Specific bugs, reproduction steps, technical details | JIRA tickets, Slack updates | Real-time |
| PM | Go/No-Go, key risks, timeline impact | Summary email, 5-min standup | Sprint end + escalations |
| Executive/Client | Quality trend, % complete, major issues only | Dashboard, monthly report | Monthly / per release |
| QA Team | Detailed metrics, process improvements, lessons learned | Full test summary report, retro | Sprint end |
📈 Trend Analysis
- Defect Trend (bugs over sprints): Nếu bug count tăng dần → quality degrading. Investigate: new team members? Increased complexity? Reduced test time?
- Pass Rate Trend: Nếu giảm → build quality deteriorating. Nếu tăng → process improving. Share trend với team monthly.
- Bug by Component: Payment module có 40% tổng bugs → hot spot. Đề xuất: code review tập trung, thêm unit tests, pair programming cho dev ở area này.
- Defect Leakage Rate: Track production bugs per release. Tăng → testing gaps hoặc scope bị compress. Cần discuss với PM về test time allocation.
🏆 Senior Insight: Metrics as Conversation Starters
Metrics không phải để blame hay report — chúng là conversation starters. "Defect leakage tăng lên 8% sprint này" → hỏi: "Chúng ta có bị time pressure không? Area nào cần thêm automation?" Senior QA dùng metrics để drive process improvement, không chỉ để report.
✏️ Bài Tập
📝 Bài Tập: Viết Test Summary Report
Data từ Sprint 15 của bạn: 200 test cases thực hiện (180 pass, 15 fail, 5 blocked). Bugs: 18 mới, 2 Critical (1 fixed, 1 open), 3 Major (2 fixed, 1 deferred), 13 Minor (10 fixed, 3 open). Coverage: 92%. Performance tests: all pass. Deadline deploy: ngày mai.
- Viết Executive Summary (3 câu) cho PM
- Tính tất cả metrics: Pass Rate, Defect Fix Rate, số bugs còn open theo severity
- Đưa ra Go/No-Go recommendation với justification dựa trên data
- Nếu PM muốn deploy despite 1 Critical open, bạn sẽ document gì? Viết đoạn email.