SentencePiece¶
Unsupervised text tokenizer for Neural Network-based text generation.
homepage: https://github.com/google/sentencepiece
Available installations¶
| SentencePiece version | Supported CPU targets | Supported GPU targets | EESSI version | Module |
|---|---|---|---|---|
| 0.2.1 | generic: aarch64, x86_64Arm: a64fx, neoverse_n1, neoverse_v1, nvidia/graceAMD: zen2, zen3, zen4, zen5Intel: haswell, skylake_avx512, sapphirerapids, icelake, cascadelake |
(none) | 2025.06 | SentencePiece/0.2.1-GCC-14.3.0 |
Extensions¶
Overview of extensions included in SentencePiece installations
SentencePiece¶
SentencePiece version |
SentencePiece modules that include it |
|---|---|
| 0.2.1 | SentencePiece/0.2.1-GCC-14.3.0 |
sentencepiece¶
sentencepiece version |
SentencePiece modules that include it |
|---|---|
| 0.2.1 | SentencePiece/0.2.1-GCC-14.3.0 |