-
Notifications
You must be signed in to change notification settings - Fork 34
Description
Problem
All 6 recent Copilot coding agent PRs are stuck in draft status with zero progression to "Ready for Review" or merge state. This represents a 100% stagnation rate and suggests a systematic barrier in the agent workflow completion process.
Affected PRs
| PR # | Title | Comments | Created | Status |
|---|---|---|---|---|
| #8567 | Verify YAML dependency migration | 0 | 2026-01-02 | Draft |
| #8566 | Add test coverage for CLI completion | 0 | 2026-01-02 | Draft |
| #8565 | Add completion descriptions | 0 | 2026-01-02 | Draft |
| #8564 | Add --completions flag to init | 0 | 2026-01-02 | Draft |
| #8563 | Fix awmg MCP config rewrite | 0 | 2026-01-02 | Draft |
| #8535 | Enable MCP gateway | 29 | 2026-01-01 | Draft |
Impact
Immediate:
- No agent outputs reaching production
- Agent effectiveness cannot be measured accurately
- Potential waste of computational resources
Systemic:
- Suggests gaps in agent workflow design
- May indicate missing "completion signals" in agent instructions
- Could affect user confidence in agent capabilities
Investigation Questions
-
Process Barriers:
- Are agents waiting for explicit human review signals?
- Do tests need to pass before draft → ready transition?
- Is there a missing step in agent completion instructions?
-
Technical Blockers:
- Are CI checks failing silently?
- Do agents lack permissions to change PR status?
- Is there a GitHub Actions workflow step missing?
-
Design Gaps:
- Do agent prompts define "done" criteria clearly?
- Should agents automatically transition to ready when criteria met?
- Is human review always intended, or should some PRs auto-merge?
Recommended Actions
Immediate (Next 24-48 Hours)
-
Review PR Enable MCP gateway (awmg) in smoke-copilot-no-firewall workflow with built-in mode, fix compile timestamp handling, and enhance health check logging #8535 (29 comments, most active):
- Analyze comment thread for completion signals
- Identify what's preventing ready transition
- Use as case study for systematic issue
-
Check Agent Instructions:
- Review Copilot coding agent prompts
- Verify if completion criteria are defined
- Add explicit "mark ready when done" instructions if missing
-
Verify CI/Test Status:
- Check if tests pass on these PRs
- Verify CI checks are green
- Identify any silent failures
Short-term (Next Week)
-
Add Completion Signals to agent prompts:
- Define "done" criteria explicitly
- Add instruction to transition draft → ready when criteria met
- Consider auto-merge for certain PR types
-
Create PR Status Dashboard:
- Track draft → ready → merged progression
- Alert on PRs stuck in draft >24 hours
- Measure agent completion rates
-
Document Best Practices:
- When should agents create draft PRs?
- What criteria mark a PR as "ready"?
- Which PRs should auto-merge vs. require review?
Long-term
-
Implement Automated Health Checks:
- Daily scan for stale draft PRs
- Automatic reminders to move to ready
- Metrics on draft dwell time
-
Agent Workflow Improvements:
- Add "mark ready" step to agent job template
- Implement progress checkpoints
- Create completion validation logic
Success Criteria
- Root cause identified for 100% draft rate
- At least 1 of 6 PRs moves to ready or merged
- Agent instructions updated with completion signals
- Monitoring system in place for future tracking
- Documentation created for PR status workflows
Related
- Discussion: Agent Performance Report (created this run)
- Workflow: Copilot coding agent
- Meta-orchestrator: Agent Performance Analyzer
Priority
High - This affects all agent outputs and ecosystem health measurement.
Discovered by Agent Performance Analyzer
AI generated by Agent Performance Analyzer - Meta-Orchestrator