• In transformers.py, model has to be resized to to length of new tokenizer model.resize_token_embeddings(len(tokenizer)) after line 298 model = AutoModelForCausalLM.from_pretrained(model_name)
• We could reuse transform function and pipeline for different tasks instead of generate to keep the interface consistent between hugging face models and fine tuned models.

• In transformers.py, model has to be resized to to length of new tokenizer model.resize_token_embeddings(len(tokenizer)) after line 298 model = AutoModelForCausalLM.from_pretrained(model_name)

👍

• We could reuse transform function and pipeline for different tasks instead of generate to keep the interface consistent between hugging face models and fine tuned models.

I think there are a few reasons to create a new API:

• Fine tuned models need a new name. You may fine tune gpt2 on several different tasks, so you need a way to specify which gpt2... we could upload these back to huggingface, and use your_username/gpt2-special-tuning, but it seems just as natural to use our existing project_name, since you'll also likely want to test many different base models for fine tuning in the same Project.
• transform takes whatever arbitrary JSON (possibly omitting a model name in favor of a task, which is similar but not the same as a PostgresML deployment), and returns whatever arbitrary JSON that happens to work with the Python call. generate is a more specific API that we know should take a string input and output a string. There are further structured APIs I think we'll want for certain tasks, e.g. predict and predict_logits for text-classification, or functions to get embeddings out of models.

I wouldn't be against extending the PostgresML transform function to accept a fine tuned model name, instead of a hugging face model name, but we'd need a way to disambiguate which model store (huggingface vs fine tuned PostgresML) to reference when a model name is passed.