[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / raw/paper/iclr2025/kbhiclr2025_tokenizer_free_approaches.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "ICLR2025 Tokenizer-Free Approaches" source: https://www.jemoka.com/posts/kbhiclr2025_tokenizer_free_approaches/ --- Talks ICLR2025 Kilani: MrT5 Tokenizer-Free ICLR2025 Neitemeier: Hierachical Autoregressive Transformers Downsides of Subword Tokenization not learned end to end: vocab is fixed, can’t adapt to difficulty non-smoothness: similar inputs get mapped to very different token sequences [token][ization] typo: [token][zi][ation] <- suddenly bad despite small typo huge vocabs: yes non-adaptive compression ratio: you can’t decide how much to compress (affects FLOPs/document)