Skip to content

fix rounding of integers with sf - #389

Open
Joshix-1 wants to merge 13 commits into
printfn:mainfrom
Joshix-1:fix-sf-rounding
Open

fix rounding of integers with sf#389
Joshix-1 wants to merge 13 commits into
printfn:mainfrom
Joshix-1:fix-sf-rounding

Conversation

@Joshix-1

Copy link
Copy Markdown

Changes

Fix #308 for big integers


Checklist

  • I used LLMs or similar AI tools when creating this pull request

@Joshix-1 Joshix-1 mentioned this pull request Jul 19, 2026
1 task
@Safari77

Copy link
Copy Markdown
Contributor
> 0xff to 1 sf
approx. 0x550
> 0xff to 2 sf
0xf54

@Joshix-1

Copy link
Copy Markdown
Author

what would be the expected behaviour for the first? 0x100?

@Safari77

Copy link
Copy Markdown
Contributor

yes, I made PR which does that

@Joshix-1

Copy link
Copy Markdown
Author

Fixed it myself.

@Joshix-1
Joshix-1 marked this pull request as draft July 19, 2026 18:12
@Joshix-1
Joshix-1 marked this pull request as ready for review July 19, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorrect significant figure rounding

2 participants