chore: Update XFails Report

[flashinfer-bot]

Summary

This PR updates the xfails report based on the current test suite.

Changes

• Updated reports/xfails_report.txt with current xfail markers from the test suite

How to Review

1. Review the changes to reports/xfails_report.txt
2. Check if the number of xfails has increased or decreased
3. Review the reasons for any new xfails
4. Consider addressing xfails that may no longer be needed

Notes

• This is an automated PR generated weekly
• The report shows all pytest.mark.xfail markers in the test suite
• Grouped by reason for easy analysis

---

🤖 This PR was automatically generated by the update-xfails-report workflow

Summary by CodeRabbit

• Documentation
  • Added a comprehensive xfails report that aggregates and presents test failure data, including total failure counts, detailed categorizations by failure reason, specific test file references for each failure type, and relevant condition notes.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[gemini-code-assist]

Summary of Changes

Hello @flashinfer-bot, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request automatically updates the xfails_report.txt to provide an accurate and current overview of all expected test failures within the project's test suite. This ensures that developers have the latest information on known issues, categorized by reason, to facilitate analysis and potential resolution.

Highlights

• XFails Report Update: The reports/xfails_report.txt file has been updated to reflect the current pytest.mark.xfail markers across the test suite. This report provides a detailed breakdown of 10 xfails across 8 unique reasons, including issues related to SM120/121, CUDA graph, attention sink, seq_len=514 failures, nvidia-cutlass-dsl issues, and numerical accuracy problems.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[coderabbitai]

📝 Walkthrough

Walkthrough

A new static report file is added at reports/xfails_report.txt that aggregates xfail test results with headers, totals, detailed tables, and per-reason breakdowns. No code logic or runtime behavior is introduced—this is purely a reporting artifact summarizing existing test failures and conditions.

Changes

Cohort / File(s)	Summary
New Report Artifact reports/xfails_report.txt	Static report file aggregating xfail test results. Includes header, totals (10 xfails, 8 unique reasons), detailed table with reason/count/type, and per-reason breakdowns listing specific test files, test names, type (decorator/runtime/parameter), and conditions.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

• flashinfer-ai/flashinfer#2273: Adds GitHub Actions workflow that runs scripts/xfails_tracker.py to generate/update the reports/xfails_report.txt report file added in this PR.

Suggested reviewers

• jimmyzho
• yzh119

Poem

🐰✨ A report takes shape, neat rows and counts aligned,
Xfails gathered, reasons sorted, nothing left behind,
Ten failures mapped, eight causes crystalline and clear,
A rabbit's delight—organization and cheer! 📋🌟

Pre-merge checks

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore: Update XFails Report' directly and clearly describes the main change: updating the xfails report file. It accurately summarizes the PR's purpose.
Description check	✅ Passed	The PR description includes a summary, changes list, review guidance, and notes about automation. However, it deviates from the template by omitting the 'Related Issues', 'Pre-commit Checks', and 'Tests' sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 30203fb and 812cc1a.

📒 Files selected for processing (1)

• reports/xfails_report.txt

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-12-30T09:34:39.900Z

Learnt from: CR
Repo: flashinfer-ai/flashinfer PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-30T09:34:39.900Z
Learning: Applies to tests/**/*.py : Test implementations should use `flashinfer.utils` functions (`get_compute_capability`, `is_sm90a_supported`, `is_sm100a_supported`, etc.) to skip tests on unsupported GPU architectures

Applied to files:

• reports/xfails_report.txt

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Deploy Docs

🔇 Additional comments (2)

reports/xfails_report.txt (2)

1-97: Report structure and transparency looks good.

The auto-generated report effectively summarizes 10 xfails grouped by reason, with clear file locations, test names, and GPU compute capability conditions where applicable. The structure aligns with the learnings—tests are using get_compute_capability() from flashinfer.utils for architecture-specific skips (lines 35, 40, 95).

As noted in the PR description, reviewers should:

• Confirm the xfail count (currently 10) is expected for the current state of the test suite
• Review the reasons for any new or unexpected xfails
• Consider removing xfails that are no longer needed (especially the temporary ones referencing GitHub PRs like lines 43, 54)

---

76-81: No action needed. The xfail reason is intentional and descriptive.

The pytest.xfail(str(e)) at line 92 is not a coding error. The exception e is a LibraryError containing the descriptive message: "cudnn FP4 GEMM with mxfp4 quantization is not supported on SM120 with cuDNN backend version < 9.14.0." This message becomes the xfail reason at runtime. The report displays "str(e)" because it shows the code as written, not the runtime value. The code includes a TODO comment indicating this check will be removed once cuDNN is updated to version 9.14.0 or later, which is appropriate temporary handling.

Likely an incorrect or invalid review comment.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[gemini-code-assist]

Code Review

This pull request updates the automated xfails report. The report is well-structured, but I've identified one expected failure that uses a non-descriptive reason, str(e). This originates from the test file tests/gemm/test_mm_fp4.py where pytest.xfail is called with a dynamic value. Using a descriptive string literal instead would improve the clarity and utility of this report. My specific comment provides more detail on this.

[gemini-code-assist]

The xfail reason str(e) is not descriptive and makes the report less useful. This is caused by the use of pytest.xfail(str(e)) in tests/gemm/test_mm_fp4.py on line 92. The static analysis script that generates this report cannot resolve the value of str(e) and therefore uses the literal code as the reason.

To improve this, the pytest.xfail call should be updated to use a descriptive string literal. Based on the context in the test file, a more informative reason would be something like:

"cuDNN backend for FP4 GEMM with mxfp4 on SM120+ requires cuDNN 9.14.0+"

This change would make the xfail report more understandable and aid in tracking test failures.