Refactor SSHash indexing to unify parsing modalities and optimize performance

.gitignore

@@ -1,2 +1,2 @@
 .DS_Store
-build
+build*

CMakeLists.txt

@@ -106,6 +106,11 @@ if(SSHASH_BUILD_EXECUTABLES)
     sshash_static
   )
 
+  add_executable(test_minimizer test/test_minimizer.cpp)
+  target_link_libraries(test_minimizer
+    sshash_static
+  )
+
   add_executable(check test/check.cpp)
   target_link_libraries(check
     sshash_static

README.md

@@ -204,30 +204,21 @@ if your queries are to be read from a (multi-line) FASTA file.
 
 ### Example 3
 
-    ./sshash build -i ../data/unitigs_stitched/salmonella_100_k31_ust.fa.gz -k 31 -m 13 --canonical -o salmonella_100.canon.sshash
-
-This example builds a dictionary from the input file `../data/unitigs_stitched/salmonella_100_k31_ust.fa.gz` (same used in Example 2), with k = 31, m = 13, and with the canonical parsing modality (option `--canonical`). The dictionary is serialized on disk to the file `salmonella_100.canon.sshash`.
-
-The "canonical" version of the dictionary offers more speed for only a little space increase, especially under low-hit workloads -- when the majority of k-mers are not found in the dictionary. (For all details, refer to the paper.)
-
-Below a comparison between the dictionary built in Example 2 (not canonical)
-and the one just built (Example 3, canonical).
-
     ./sshash query -i salmonella_100.sshash -q ../data/queries/SRR5833294.10K.fastq.gz
 
-    ./sshash query -i salmonella_100.canon.sshash -q ../data/queries/SRR5833294.10K.fastq.gz
+A k-mer and its reverse complement always share a bucket, so a lookup costs a
+single bucket probe and reports, in `kmer_orientation`, which of the two was
+found. This is the only indexing modality: there is no `--canonical` flag to
+choose, and no space-versus-speed trade-off to make.
 
-Both queries should originate the following report (reported here for reference):
+The query above should originate the following report (reported here for reference):
 
     ==== query report:
     num_kmers = 460000
     num_positive_kmers = 46 (0.01%)
     num_searches = 42/46 (91.3043%)
     num_extensions = 4/46 (8.69565%)
 
-The canonical dictionary can be twice as fast as the regular dictionary
-for low-hit workloads, even on this tiny example, for only +0.3 bits/k-mer.
-
 ### Example 4
 
     ./sshash permute -i ../data/unitigs_stitched/with_weights/ecoli_sakai.ust.k31.fa.gz -k 31 -o ecoli_sakai.permuted.fa

benchmarks/README.md

@@ -14,16 +14,42 @@ To run the benchmarks, from within the `build` directory, run
 
 where `<log_label>` should be replaced by a suitable basename, e.g., the current date.
 
-These are the results obtained on 21/01/26 (see logs [here](results-21-01-26))
+These are the results obtained on 28/08/26 (see logs [here](results-28-08-26))
 on a machine equipped with an AMD Ryzen Threadripper PRO 7985WX processor clocked at 5.40GHz.
 The code was compiled with `gcc` 13.3.0.
-
 The indexes were build with a max RAM usage of 16 GB and 64 threads.
 Queries were run using one thread, instead.
 
-![](results-21-01-26/results.png)
-
-The results can be exported to CSV format with
-
-    python3 ../script/print_csv.py ../benchmarks/results-21-01-26/k31
-    python3 ../script/print_csv.py ../benchmarks/results-21-01-26/k63
+| k  | Collection | m  | Space (bits/kmer) | Space (total GB) | Building time (m:ss) | Positive random lookup (µs/kmer) | Negative random lookup (µs/kmer) | Random Access (µs/kmer) | Streaming Lookup high-hit (ns/kmer) |
+|----|------------|:--:|:-----------------:|:----------------:|:--------------------:|:--------------------------------:|:--------------------------------:|:-----------------------:|:-----------------------------------:|
+||
+| 31 | Cod        | 20 | 7.94              | 0.50             | 0:27                 | 0.43                             | 0.36                             | 0.27                    | 24                                  |
+|    | Kestrel    | 20 | 7.52              | 1.08             | 1:00                 | 0.44                             | 0.39                             | 0.28                    | 39                                  |
+|    | Human      | 21 | 8.77              | 2.75             | 2:35                 | 0.56                             | 0.43                             | 0.35                    | 76                                  |
+|    | NCBI-v     | 19 | 7.43              | 0.35             | 0:14                 | 0.41                             | 0.35                             | 0.26                    | 26                                  |
+|    | SE         | 21 | 10.24             | 1.14             | 0:54                 | 0.62                             | 0.40                             | 0.36                    | 163                                 |
+|    | HPRC       | 21 | 10.64             | 4.94             | 4:36                 | 0.70                             | 0.46                             | 0.53                    | 84                                  |
+||
+| 63 | Cod        | 24 | 4.62              | 0.32             | 0:24                 | 0.57                             | 0.45                             | 0.29                    | 37                                  |
+|    | Kestrel    | 24 | 3.84              | 0.55             | 0:19                 | 0.54                             | 0.48                             | 0.32                    | 44                                  |
+|    | Human      | 25 | 4.92              | 1.70             | 1:13                 | 0.66                             | 0.51                             | 0.36                    | 110                                 |
+|    | NCBI-v     | 23 | 4.06              | 0.21             | 0:07                 | 0.53                             | 0.44                             | 0.28                    | 30                                  |
+|    | SE         | 31 | 7.42              | 1.41             | 1:20                 | 0.97                             | 0.50                             | 0.40                    | 278                                 |
+|    | HPRC       | 31 | 7.63              | 5.65             | 4:46                 | 0.94                             | 0.58                             | 0.62                    | 137                                 |
+
+The results of a run, for both k=31 and k=63 at once, can be exported to CSV
+format (every collected quantity) or to a markdown table (the main quantities)
+with
+
+    python3 ../benchmarks/print_table.py <results_dir>
+    python3 ../benchmarks/print_table.py <results_dir> --md
+
+where `<results_dir>` is the directory written by the scripts above (e.g.,
+`results-21-01-26`), containing the `k31` and `k63` subdirectories.
+
+Note that the scripts now produce a single set of result files per `k`
+(`build.json`, `bench.json`, and `streaming-queries.json`), since the
+regular/canonical distinction is gone: there is one indexing modality only.
+Result directories archived before this change instead hold a `regular-` and a
+`canon-` file for each of those; to re-read them, use the `print_csv.py`
+script from the corresponding commit.