Softmax update by bugracyln · Pull Request #1494 · fastmachinelearning/hls4ml

bugracyln · 2026-06-25T18:50:11Z

Description

📝 Please include a summary of the change.

The softmax table generation logic was updated. The implementation for writing the softmax tables was revised, and memory attributes were added to enable a more efficient FPGA compilation flow. In addition, the templates were modified to use weights directly from the configuration.

Please also include relevant motivation and context.

The primary motivation for these changes was to bring the oneAPI backend closer to the Vivado backend in terms of implementation.

Memory attributes were added to enable memory banking on the FPGA, allowing for more efficient memory access. The weights are now copied directly into the configuration so that the compiler can recognise the entire table as a set of fixed values. This enables the memory to be implemented more efficiently, resulting in improved resource utilisation during FPGA compilation.

List any dependencies that are required for this change.

N/A

Type of change

For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.

Note: Please delete options that are not relevant.

Bug fix (non-breaking change that fixes an issue)
Documentation update
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality not to work as expected)
A new research paper code implementation
Other (Specify)

Tests

📝 Please describe the tests that you ran to verify your changes.

The changes were primarily verified using black-box tests on an isolated softmax unit. Testing was performed for both quantised and non-quantised implementations. For the quantised version, both configurations, with and without exp and inv table quantisers (QuantiserConfig(...)), were tested.

Additional testing included:

Generating FPGA RTL reports.
Building the emulator.
Performing a hardware compilation using the new Intel oneAPI compiler.

This PR currently supports only the Intel oneAPI compiler. Support for the Altera HLS compiler will be added in a future PR.

The implementation was also evaluated with different table sizes, and the resulting RTL reports were inspected to verify improvements in resource utilisation.

Provide instructions so we can reproduce.

A Python test file and a Keras model containing only a single softmax layer (Softmax or QSoftmax) were used. For the quantised implementation, the input and output quantisers for the exp and inv lookup tables were configured using QuantiserConfig(...). Tests were run with both the quantisers enabled and disabled.

The test configuration included:

Standard Softmax and QSoftmax models.
Explicit exp and inv table input output quantisation.
FPGA RTL generation.
Emulator build.
Hardware compilation with the Altera HLS (newer version of Intel oneAPI)compiler.

Please also list any relevant details for your test configuration.

Test Configuration:

Checklist

I have read the guidelines for contributing.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
My changes generate no new warnings.
I have installed and run pre-commit on the files I edited or added.
I have added tests that prove my fix is effective or that my feature works.

…pi_weights

…api_qmha

…i_qmha

jmitrevs · 2026-07-02T20:00:08Z

        params['type'] = node.get_attr('activation')

+        if (params['type'] == 'softmax') or (params['type'] == 'softmax_multidim'):
+            params.setdefault('n_inner', 1)


These options have a default value, so there's no reason to set it here. See the default value in fpga_backend.py

Removed n_inner and n_outer

jmitrevs · 2026-07-02T20:03:08Z

+
+                if 'exp_table_size' not in params:
+                    params['exp_table_size'] = 2 ** params['inp_norm_t'].precision.width
+                    node.set_attr('exp_table_size', params['exp_table_size'])


I would not make the default different for oneAPI vs Vitis/Vivado. Note that the setting there is:
params.setdefault('exp_table_size', params['table_size']). I am worried that this is too big of a default if, for example, we have a width of 18 bits.

One could argue for a different default, but let's not diverge.

I would suggest just copying https://github.com/fastmachinelearning/hls4ml/blob/main/hls4ml/backends/vivado/passes/core_templates.py#L298-L309 for the default setting

Changed this to use the default size of 1024 bits, but I would suggest something around 4096, perhaps, since at 1e-3 tolerance, I get around 60% of the values wrong. This is with a non-quantised layer too, since we don't have defined exp_table and inv_table sizes for a non-quantised layer, we default to 1024, leading to very coarse table values.

jmitrevs · 2026-07-02T20:12:47Z

+
+            else:
+                # TODO: For latency check the table sizes correctly, match them
+                if 'exp_table_size' not in params:


This is re-implementing set_default.

Got rid of it :)

jmitrevs · 2026-07-02T20:17:14Z

 template <class data_pipe, class res_pipe, typename CONFIG_T> void softmax_legacy_stream() {
-#include "activation_tables/exp_table_legacy.tb"
-#include "activation_tables/invert_table_legacy.tb"
+    //#include "activation_tables/exp_table_legacy.tb"


I would suggest removing the commented out includes that are just historic.

jmitrevs · 2026-07-07T20:01:41Z

-    def __write_exp_table(self, model, path):
-        table_name = 'exp_table'
-        table_size = self.__get_table_size(model, 'softmax')
+    def __get_table_precision(self, model, activation, table_name='table_precision'):


Is this function used?

jmitrevs · 2026-07-07T20:17:32Z

-        fp_bits = 16
-        fp_integer = 6
-        fp_signed = True
+    def __write_exp_table(self, model, path):


Minor comment, feel free to ignore, but would it be useful for this to have a _stable suffix. Also, would the name be better if it was made plural, write_exp_tables_stable (and similar for the other ones) since this now can write out multiple tables.

jmitrevs · 2026-07-07T21:14:40Z

-            real_val = f.exp_float()
-            h_file.write(sep + str(real_val))
-            sep = ', '
+                    # Default fixed point precision


There is no need for default here.

jmitrevs · 2026-07-07T21:16:41Z

-        h_file.close()
+                    # Exp table should use the same precision as exp_table, as seen in Vivado code
+                    # init_exp_table<data_T, CONFIG_T>(exp_table);
+                    for layer in model.get_layers():


Why loop again for layers here within the loop for layers on line 824? This looks like a bug.

jmitrevs · 2026-07-07T21:19:13Z


-        h_file.write('};\n')
-        h_file.close()
+                    # Exp table should use the same precision as exp_table, as seen in Vivado code


I don't understand the comment, that the exp table should use the same precision as exp table. Isn't that true inherently?

jmitrevs · 2026-07-07T21:20:09Z

-            sep = ', '
+                    # Default fixed point precision, in case values from layer attributes cannot be extracted
+                    # 8 bits for integer part, 10 bits for decimal - total, 18
+                    fp_bits = 18


Again I don't think you need defaults here.

jmitrevs · 2026-07-07T21:20:40Z

-        h_file.close()
+                    # Invert table should use the same precision as exp_table, as seen in Vivado code
+                    # init_invert_table<typename CONFIG_T::exp_table_t, CONFIG_T>(invert_table);
+                    for layer in model.get_layers():


And again a layer loop inside of a layer loop.

laurilaatu and others added 30 commits January 26, 2026 20:37

weights for dense

3d463b3

hgq2 homogeneous quant fix

d678573

Merge branch 'hgq2_homo_quant' of github.com:calad0i/hls4ml into onea…

77258bc

…pi_weights

Changes required for oneAPI MHA

59bd96f

Original weight implementation

dbb207b

Merge branch 'main' of github.com:fastmachinelearning/hls4ml into one…

0c59255

…api_qmha

Restore oneAPI weight placement

51efff0

pre-commit

6067bea

Merge branch 'main' into oneapi_qmha

06fda4e

Merge branch 'main' into oneapi_qmha

bf38a6b

Merge branch 'main' into oneapi_qmha

e27fd11

Merge branch 'main' into oneapi_qmha

9f4a448

softmax multidim templates

16ca197

Merge branch 'oneapi_qmha' of github.com:laurilaatu/hls4ml into oneap…

564b692

…i_qmha

pre-commit

974e75a

uncomment

060c398

Merge branch 'main' into oneapi_qmha

f78558c

int_inp_t to config

772b93a

Merge branch 'oneapi_qmha' of github.com:laurilaatu/hls4ml into oneap…

d2b8921

…i_qmha

Merge branch 'main' into oneapi_qmha

a1ad891

Merge branch 'main' into oneapi_qmha

d65544d

Merge branch 'main' into oneapi_qmha

2d6a5cc

softmax fixed

c3a4584

Merge branch 'main' into oneapi_qmha

9b1cf17

table generation cleanup

31b7ad6

Merge pull request fastmachinelearning#4 from bugracyln/smax_fix

70b19d1

fix overflow for legacy softmax, start looking at others

091cd8e

fix stable and streaming legacy softmax

6c9f9a7

minor softmax latency fixes

7a004b2

Merge branch 'main' into sofmax_fix

9d588f2

jmitrevs reviewed Jul 2, 2026

View reviewed changes

suggested fixes and commented sections removed

a90b79a

jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Jul 3, 2026

bugracyln and others added 3 commits July 3, 2026 22:04

formatting quickfix

07ad4a4

minor fix to writer

aa5af6b

Merge branch 'main' into sofmax_fix

49d6ccd

jmitrevs reviewed Jul 7, 2026

View reviewed changes

jmitrevs added please test Trigger testing by creating local PR branch and removed please test Trigger testing by creating local PR branch labels Jul 7, 2026

jmitrevs reviewed Jul 7, 2026

View reviewed changes

jmitrevs and others added 11 commits July 7, 2026 16:38

remove quartus reqirement for signed softmax

3d7979a

add inp_norm_t inferrence

aa678e0

fix test_softmax

5dadf1a

pre-commit fix

d8abde4

change randint to rand

55e7454

make sure softmax widths are transfered properly

947f195

Fix issues with skip, auto

10532da

Merge branch 'main' into sofmax_fix

232b2a3

Merge remote-tracking branch 'jmitrevs/sofmax_fix' into softmax_updated

84e0710

saving before testing

01d70a4

table accuracy improvement and bugfixes

8d8bc8a

Uh oh!

Conversation

bugracyln commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Tests

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmitrevs Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bugracyln commented Jun 25, 2026 •

edited

Loading

jmitrevs Jul 2, 2026 •

edited

Loading