Skill instructions (SKILL.md) could not be loaded from local cache or raw GitHub repository.
huggingface-tokenizers
star 0
Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.
By TuYv schedule Updated 6/13/2026 Install via CLI
npx skills add https://github.com/TuYv/ccpm --skill huggingface-tokenizers
Repository Details
star Stars
0
call_split Forks
0
navigation Branch
main
article Path
SKILL.md
More from Creator
TuYv Explore all skills →