Qualcomm: make --use_fp16 and the fp16 compiler spec agree - #21867
Qualcomm: make --use_fp16 and the fp16 compiler spec agree#21867psiddh wants to merge 2 commits into
Conversation
build_executorch_binary decided precision from quant_dtype alone while deciding whether to quantize from quant_dtype or custom_quantizer, so the two predicates disagreed and each direction was wrong somewhere. The custom op examples set quantizer = None for --use_fp16 but still passed quant_dtype=use_8a8w, so lowering stayed quantized and then built a default quantizer with no annotation for the custom op: a quantized graph holding an unquantized custom op node. They now pass quant_dtype=None, which is the fp16 signal the rest of the file already uses. In the mirror direction, examples that pass only custom_quantizer -- oss_scripts/fastvit.py and oss_scripts/eurobert.py -- quantize the graph and then compile it with kHtpFp16. Deriving use_fp16 from both arguments fixes those without touching them. Authored with Claude Code.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21867
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit d2ef854 with merge base ed65b12 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
There was a problem hiding this comment.
Pull request overview
This PR fixes an inconsistency in the Qualcomm export path where fp16-vs-quantized decisions were derived from quant_dtype in one place but from quant_dtype/custom_quantizer in another, leading to mismatched lowering vs compiler spec (notably for custom-op examples and scripts that only pass a custom_quantizer).
Changes:
- Update custom-op examples to pass
quant_dtype=None(along withquantizer=None) when--use_fp16is set, ensuring fp16 lowering is selected consistently. - Derive HTP
use_fp16from bothquant_dtypeandcustom_quantizerso compiler spec matches whether the graph is actually quantized.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| examples/qualcomm/custom_op/custom_ops_1.py | Ensures fp16 mode is signaled via quant_dtype=None so the example doesn’t accidentally lower/quantize. |
| examples/qualcomm/custom_op/custom_ops_2.py | Same fp16 signaling fix for the second custom-op example. |
| backends/qualcomm/export_utils.py | Aligns HTP compiler use_fp16 decision with the quantization path (quant_dtype and custom_quantizer). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
build_executorch_binary decided precision from quant_dtype alone while deciding whether to quantize from quant_dtype or custom_quantizer, so the two predicates disagreed and each direction was wrong somewhere.
The custom op examples set quantizer = None for --use_fp16 but still passed quant_dtype=use_8a8w, so lowering stayed quantized and then built a default quantizer with no annotation for the custom op: a quantized graph holding an unquantized custom op node. They now pass quant_dtype=None, which is the fp16 signal the rest of the file already uses.
In the mirror direction, examples that pass only custom_quantizer -- oss_scripts/fastvit.py and oss_scripts/eurobert.py -- quantize the graph and then compile it with kHtpFp16. Deriving use_fp16 from both arguments fixes those without touching them.
Authored with Claude Code.
cc @cbilgin