Hello (author here), I found the OpenAI tool (
https://platform.openai.com/tokenizer
) for visualising tokens really useful when working on token optimisation strategies (and other token related tasks). But unfortunately that site doesn't support the tokeniser that GPT4 uses (currently).
So I threw together a stackblitz that uses the js-tiktoken to do the same thing.
Hopefully it will be useful for someone else too!
Try it here: https://react-ts-h9yrjh.stackblitz.io/
