Skip to content

OPENNLP-1894: Add dictionary-based tokenization for Japanese, Korean, and Chinese - #1191

Merged
rzo1 merged 25 commits into
mainfrom
OPENNLP-1894-lattice-cjk
Sep 4, 2026
Merged

OPENNLP-1894: Add dictionary-based tokenization for Japanese, Korean, and Chinese#1191
rzo1 merged 25 commits into
mainfrom
OPENNLP-1894-lattice-cjk

OPENNLP-1894: Address lattice tokenizer review feedback

dba1353
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar