Hey @felipelo , my thought was to add /tokenize and /detokenize endpoints to the server, happy to include just an endpoint for counting but let's include it as /tokenize/count or something along those lines.

Hey, can we look into this again? I'm using llama-cpp-python's webserver for an application I'm building and it would be nice to have the tokenizer available as an API. This looks great!

Sorry for the delay, just had to do some cleanup, this will be in the next version.