Edits from finetuning

[granawkins]

Fix the fine-tuning output format, add a way for fine-tune models to exclude the system_prompt, and patch up some issues I found.

Pull Request Checklist

• Documentation has been updated, or this change doesn't require that

[jakethekoenig]

Unfortunately only abante ai org ai keys will be able to use this model so we shouldn't add it to the list.

[PCSwingle]

I'm not sure this makes much sense; like @jakethekoenig mentioned, the finetuned models can only be used by us. How about we just makes the include_system_prompt a config option?

[granawkins]

I get what you mean about the fine-tuned 3.5, makes sense ya I'll remove that one for now.

I do think it's good to build-in support for it though. I was planning to do a lil video about the end-to-end process of making our fine-tuned 3.5, with the thought that any of our users could do the same thing with their data.

Maybe known_models should try to match a base model if it's not immediately recognized? Just like I've done with the encoding function.

[jakethekoenig]

I do think it's good to build-in support for it though. I was planning to do a lil video about the end-to-end process of making our fine-tuned 3.5, with the thought that any of our users could do the same thing with their data.

I like this idea. Back when I was doing this I thought we could eventually integrate it such that you could run a /finetune command and it would run on your own git repo and make fine tuning data, start a fine tuning job and automatically use the model when done. I think that approach to making training data didn't make much sense but the idea of eventually making /finetune a command or more simply distributing .jsonl's and documenting how to use them does make sense given that fine tuning is actually relatively cheap relative to inference.

Maybe known_models should try to match a base model if it's not immediately recognized? Just like I've done with the encoding function.

I like this idea. Mentat should know the costs to use a fine tuned gpt-3.5.

[granawkins]

Added a wrapper class around known_models to handle fine-tuned models.

Removed the new requires_system_prompt stuff - if using a fine-tuned model that doesn't need a system prompt, users can use existing --no-parser-prompt arg:

mentat -a --model ft:gpt-3.5-turbo-1106:abante::8dsQMc4F --no-parser-prompt

[PCSwingle]

Do we have to do asdict? Isn't it technically a dict since it extends Dict?

[granawkins]

thb I spent an hour trying to figure out how to have an overridden keys method return a type that my linter was happy with. Tried typing.KeyView, tried returning self.model.keys() directly - it wouldn't take.

attrs also has an asdict method which we use here and there so it seemed ok.

EDIT: nevermind I figured it out.

[PCSwingle]

LGTM