Skip to content

Rten test - #77

Draft
xd009642 wants to merge 6 commits into
mainfrom
rten-test
Draft

xd009642 wants to merge 6 commits into
mainfrom
rten-test

Conversation

@xd009642

Copy link
Copy Markdown
Member

Rten would avoid our libc based issues with ort and the nvidia docker images. Running the benchmarks we get:

  • startup ORT median: 11.83 ms
  • startup RTen median: 1.504 ms
  • 8k chunk ORT median: 305.4 us
  • 8k chunk RTen median: 231 us
  • 16k chunk ORT median: 351.1 us
  • 16k chunk RTen median: 202.3 us

Currently, I've left both side-by-side for comparison but merging this PR will result in ort being deleted and rten taking over the crown.

@xd009642

xd009642 commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Works on my machine - fails on CI :squint:

Rten would avoid our libc based issues with ort and the nvidia docker
images. Running the benchmarks we get:

- startup ORT median: 11.83 ms
- startup RTen median: 1.504 ms
- 8k chunk ORT median: 305.4 us
- 8k chunk RTen median: 231 us
- 16k chunk ORT median: 351.1 us
- 16k chunk RTen median: 202.3 us

Which shows a modest improvement. However, debug builds are always gonna
be very slow (is this worth it?)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant