Skip to content

Latest commit

 

History

History
53 lines (37 loc) · 3.77 KB

CONTRIBUTING.md

File metadata and controls

53 lines (37 loc) · 3.77 KB

How to Contribute

We would love to accept your patches and contributions to this project.

Before you begin

Sign our Contributor License Agreement

Contributions to this project must be accompanied by a Contributor License Agreement (CLA). You (or your employer) retain the copyright to your contribution; this simply gives us permission to use and redistribute your contributions as part of the project.

If you or your current employer have already signed the Google CLA (even if it was for a different project), you probably don't need to do it again.

Visit https://cla.developers.google.com/ to see your current agreements or to sign a new one.

Review our Community Guidelines

This project follows Google's Open Source Community Guidelines.

Contribution process

If you believe a model should be included in our benchmark, please submit a pull request to add it as a new workload.

As of Q1 2026, the benchmark is under active development, and we are refining aspects such as the scoring calculation. We welcome your input on design considerations or other general feedback. Please open an issue to start a discussion.

To propose a new workload, please add it to the resources/experimental directory and submit a pull request. In your pull request, please explain the workload's relevance and provide technical details about its integration.

Adding Experimental Workloads

  • Add any required dependencies in resources/experimental/package.json.
  • Create a subdirectory resources/experimental/src/<workload>/ containing your benchmark entry point(s) (e.g. benchmark.mjs), similar to the existing workloads. If the workload compiles C/C++ or Rust to WebAssembly, include a build-wasm.sh script, check in the prebuilt output under resources/experimental/src/<workload>/build/, and register the workload in resources/experimental/build-wasm-workloads.sh (see resources/experimental/README.md).
  • Ensure that models are self-contained and not loaded remotely from a CDN or the Hugging Face Hub at runtime. Place model files under resources/experimental/models/<workload>/ and track large binary files with Git LFS in .gitattributes.
  • Add licensing information for the library and/or model in resources/experimental/licenses/NOTICE.md and add any required license file in resources/experimental/licenses/.
  • Add an entry and an HtmlWebpackPlugin (using src/console-runner.html as the template) for the new workload in resources/experimental/webpack.common.js.
  • Update .gitignore or .gitattributes if needed.
  • Run npm run build in the repository root directory. Alternatively, you can run npm install and npm run build inside resources/experimental.
  • Add the workload to resources/default-tests.mjs with (optionally) the Experimental- prefix and experimental tag, analogous to the existing workloads.
  • Serve the overall runner via npm run dev in the repository root directory.
  • The experimental workloads are not part of default set of workloads. Browse to http://localhost:8080/?developerMode, select the added workloads alongside any other workloads and click "Start Test" to run the benchmark. You can find more instructions about how to use the Developer Mode in Developer Mode and Custom Parameters.
  • To run individual experimental workloads, navigate to http://localhost:8080/resources/experimental/dist/<workload>.html.

Code Reviews

All submissions, including submissions by project members, require review. We use GitHub pull requests for this purpose. Run npm run fix to format and lint your changes before submitting a pull request.