-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Description
Describe the bug
The PR review generated by Claude is heavily biased by outdated and resolved previous conversation comments. Specifically, Claude includes suggestions or points out issues in its review that were already addressed and resolved in subsequent commits/updates to the Pull Request. This leads to noise and requires manual effort to ignore or dismiss irrelevant automated comments.
The two core problems are:
Stale Comments: The model is biased by previous comments, even if the corresponding code issues are resolved in the latest commit.
Bot Noise: Comments from automated users/bots (e.g., security scanners, dependency checkers) are included in the context, consuming valuable context space and potentially influencing Claude's review with irrelevant, non-human discussion points. We need a way to filter these bot users from the context input.
This bias increases with the length of the PR discussion.
To Reproduce
Steps to reproduce the behavior:
- Add a comment to a PR, for example: "Please rename temp_var to user_id in line 42."
- Push a new commit to the PR that resolves the issue (i.e., renames temp_var to user_id).
- Trigger Claude Code
- See error: Claude's review includes a comment reiterating the need to rename temp_var to user_id, despite the code already reflecting this change.
Expected behavior
Claude's review should focus only on the current diff and unresolved concerns. Specifically:
It should use the full PR conversation history to understand context, but it must prioritize the current state of the code.
It should not include comments regarding issues that have been addressed in the latest commit of the Pull Request.
The action should provide a configuration option to filter out comments made by specific GitHub users or bots (e.g., Dependabot, security scanners, or other automated tools).
Claude review after fix:
API Provider
[ ] Anthropic First-Party API (default)
[x] AWS Bedrock
[ ] GCP Vertex
