docs: linting, syntax, formatting & spelling fixes for all files (#761)

* Update README.md * fix: 'ollama run' should be 'ollama pull' * fix: linting, syntax, spelling corrections for all docs * fix: markdown linting rules and missed fixes * fix: readded space to block * fix: changed sh blocks to text * docs: added exception for bare urls in markdown * docs: added exception for in-line html (MD033/no-inline-html) * docs: made python indentation level consistent (4 space tabs) even though I prefer 2. --------- Co-authored-by: Charles Packer <[email protected]>
letta-ai · Jan 2, 2024 · 1acb994 · 1acb994
1 parent 8cf09be
commit 1acb994
Show file tree

Hide file tree

Showing 31 changed files with 343 additions and 181 deletions.
diff --git a/docs/.markdownlint.json b/docs/.markdownlint.json
@@ -0,0 +1,6 @@
+{
+  "MD013": false,
+  "MD028": false,
+  "MD033": false,
+  "MD034": false
+}
diff --git a/docs/adding_wrappers.md b/docs/adding_wrappers.md
@@ -5,7 +5,7 @@ category: 6580da9a40bb410016b8b0c3
 ---
 
 > ⚠️ MemGPT + local LLM failure cases
->
+
 > When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
 
 ### What is a "wrapper"?
@@ -42,10 +42,9 @@ class LLMChatCompletionWrapper(ABC):
 
 You can follow our example wrappers ([located here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm/llm_chat_completion_wrappers)).
 
-
 ### Example with [Airoboros](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1) (llama2 finetune)
 
-To help you get started, we've implemented an example wrapper class for a popular llama2 model **finetuned on function calling** (Airoboros). We want MemGPT to run well on open models as much as you do, so we'll be actively updating this page with more examples. Additionally, we welcome contributions from the community! If you find an open LLM that works well with MemGPT, please open a PR with a model wrapper and we'll merge it ASAP.
+To help you get started, we've implemented an example wrapper class for a popular llama2 model **fine-tuned on function calling** (Airoboros). We want MemGPT to run well on open models as much as you do, so we'll be actively updating this page with more examples. Additionally, we welcome contributions from the community! If you find an open LLM that works well with MemGPT, please open a PR with a model wrapper and we'll merge it ASAP.
 
 ```python
 class Airoboros21Wrapper(LLMChatCompletionWrapper):
@@ -83,9 +82,9 @@ See full file [here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_ll
 
 MemGPT uses function calling to do memory management. With [OpenAI's ChatCompletion API](https://platform.openai.com/docs/api-reference/chat/), you can pass in a function schema in the `functions` keyword arg, and the API response will include a `function_call` field that includes the function name and the function arguments (generated JSON). How this works under the hood is your `functions` keyword is combined with the `messages` and `system` to form one big string input to the transformer, and the output of the transformer is parsed to extract the JSON function call.
 
-In the future, more open LLMs and LLM servers (that can host OpenAI-compatable ChatCompletion endpoints) may start including parsing code to do this automatically as standard practice. However, in the meantime, when you see a model that says it supports “function calling”, like Airoboros, it doesn't mean that you can just load Airoboros into a ChatCompletion-compatable endpoint like WebUI, and then use the same OpenAI API call and it'll just work.
+In the future, more open LLMs and LLM servers (that can host OpenAI-compatible ChatCompletion endpoints) may start including parsing code to do this automatically as standard practice. However, in the meantime, when you see a model that says it supports “function calling”, like Airoboros, it doesn't mean that you can just load Airoboros into a ChatCompletion-compatible endpoint like WebUI, and then use the same OpenAI API call and it'll just work.
 
-1. When a model page says it supports function calling, they probably mean that the model was finetuned on some function call data (not that you can just use ChatCompletion with functions out-of-the-box). Remember, LLMs are just string-in-string-out, so there are many ways to format the function call data. E.g. Airoboros formats the function schema in YAML style (see https://huggingface.co/jondurbin/airoboros-l2-70b-3.1.2#agentfunction-calling) and the output is in JSON style. To get this to work behind a ChatCompletion API, you still have to do the parsing from `functions` keyword arg (containing the schema) to the model's expected schema style in the prompt (YAML for Airoboros), and you have to run some code to extract the function call (JSON for Airoboros) and package it cleanly as a `function_call` field in the response.
+1. When a model page says it supports function calling, they probably mean that the model was fine-tuned on some function call data (not that you can just use ChatCompletion with functions out-of-the-box). Remember, LLMs are just string-in-string-out, so there are many ways to format the function call data. E.g. Airoboros formats the function schema in YAML style (see https://huggingface.co/jondurbin/airoboros-l2-70b-3.1.2#agentfunction-calling) and the output is in JSON style. To get this to work behind a ChatCompletion API, you still have to do the parsing from `functions` keyword arg (containing the schema) to the model's expected schema style in the prompt (YAML for Airoboros), and you have to run some code to extract the function call (JSON for Airoboros) and package it cleanly as a `function_call` field in the response.
 
 2. Partly because of how complex it is to support function calling, most (all?) of the community projects that do OpenAI ChatCompletion endpoints for arbitrary open LLMs do not support function calling, because if they did, they would need to write model-specific parsing code for each one.
 

diff --git a/docs/api.md b/docs/api.md
@@ -7,21 +7,21 @@ category: 658135e7f596b800715c1cee
 ![memgpt llama](https://raw.githubusercontent.com/cpacker/MemGPT/main/docs/assets/memgpt_server.webp)
 
 > ⚠️ API under active development
-> 
+>
 > The MemGPT API is under **active development** and **breaking changes are being made frequently**. Do not expect any endpoints or API schema to persist until an official `v1.0` of the MemGPT API is released.
-> 
+>
 > For support and to track ongoing developments, please visit [the MemGPT Discord server](https://discord.gg/9GEQrxmVyE) where you can chat with the MemGPT team and other developers about the API.
 
 > 📘 Check Discord for the latest development build
-> 
+>
 > Make sure to check [Discord](https://discord.gg/9GEQrxmVyE) for updates on the latest development branch to use. The API reference viewable on this page may only apply to the latest dev branch, so if you plan to experiment with the API we recommend you [install MemGPT from source](https://memgpt.readme.io/docs/contributing#installing-from-source) for the time being.
 
 ## Starting a server process
 
 You can spawn a MemGPT server process using the following command:
 
-```Text shell
+```sh
 memgpt server
 ```
 
-Before attempting to launch a server process, make sure that you have already configured MemGPT (using `memgpt configure`) and are able to successfully create and message an agent using `memgpt run`. For more information, see [our quickstart guide](https://memgpt.readme.io/docs/quickstart).
+Before attempting to launch a server process, make sure that you have already configured MemGPT (using `memgpt configure`) and are able to successfully create and message an agent using `memgpt run`. For more information, see [our quickstart guide](https://memgpt.readme.io/docs/quickstart).
diff --git a/docs/autogen.md b/docs/autogen.md
@@ -1,7 +1,7 @@
 ---
-title: MemGPT + AutoGen 
+title: MemGPT + AutoGen
 excerpt: Creating AutoGen agents powered by MemGPT
-category: 6580dab16cade8003f996d17 
+category: 6580dab16cade8003f996d17
 ---
 
 > 📘 Need help?
@@ -21,6 +21,7 @@ category: 6580dab16cade8003f996d17
 MemGPT includes an AutoGen agent class ([MemGPTAgent](https://github.com/cpacker/MemGPT/blob/main/memgpt/autogen/memgpt_agent.py)) that mimics the interface of AutoGen's [ConversableAgent](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent#conversableagent-objects), allowing you to plug MemGPT into the AutoGen framework.
 
 To create a MemGPT AutoGen agent for use in an AutoGen script, you can use the `create_memgpt_autogen_agent_from_config` constructor:
+
 ```python
 from memgpt.autogen.memgpt_agent import create_memgpt_autogen_agent_from_config
 
@@ -56,6 +57,7 @@ memgpt_autogen_agent = create_memgpt_autogen_agent_from_config(
 ```
 
 Now this `memgpt_autogen_agent` can be used in standard AutoGen scripts:
+
 ```python
 import autogen
 
@@ -97,6 +99,7 @@ Once you've confirmed that you're able to chat with a MemGPT agent using `memgpt
 > If you're using RunPod to run web UI, make sure that you set your endpoint to the RunPod IP address, **not the default localhost address**.
 >
 > For example, during `memgpt configure`:
+>
 > ```text
 > ? Enter default endpoint: https://yourpodaddresshere-5000.proxy.runpod.net
 > ```
@@ -106,6 +109,7 @@ Once you've confirmed that you're able to chat with a MemGPT agent using `memgpt
 Now we're going to integrate MemGPT and AutoGen by creating a special "MemGPT AutoGen agent" that wraps MemGPT in an AutoGen-style agent interface.
 
 First, make sure you have AutoGen installed:
+
 ```sh
 pip install pyautogen
 ```
@@ -117,7 +121,9 @@ In order to run this example on a local LLM, go to lines 46-66 in [examples/agen
 `config_list` is used by non-MemGPT AutoGen agents, which expect an OpenAI-compatible API. `config_list_memgpt` is used by MemGPT AutoGen agents, and requires additional settings specific to MemGPT (such as the `model_wrapper` and `context_window`. Depending on what LLM backend you want to use, you'll have to set up your `config_list` and `config_list_memgpt` differently:
 
 #### web UI example
+
 For example, if you are using web UI, it will look something like this:
+
 ```python
 # Non-MemGPT agents will still use local LLMs, but they will use the ChatCompletions endpoint
 config_list = [
@@ -132,7 +138,7 @@ config_list = [
 config_list_memgpt = [
     {
         "preset": DEFAULT_PRESET,
-        "model": None,  # not required for web UI, only required for Ollama, see: https://memgpt.readme.io/docs/ollama 
+        "model": None,  # not required for web UI, only required for Ollama, see: https://memgpt.readme.io/docs/ollama
         "model_wrapper": "airoboros-l2-70b-2.1",  # airoboros is the default wrapper and should work for most models
         "model_endpoint_type": "webui",
         "model_endpoint": "http://localhost:5000",  # notice port 5000 for web UI
@@ -142,7 +148,9 @@ config_list_memgpt = [
 ```
 
 #### LM Studio example
+
 If you are using LM Studio, then you'll need to change the `api_base` in `config_list`, and `model_endpoint_type` + `model_endpoint` in `config_list_memgpt`:
+
 ```python
 # Non-MemGPT agents will still use local LLMs, but they will use the ChatCompletions endpoint
 config_list = [
@@ -167,7 +175,9 @@ config_list_memgpt = [
 ```
 
 #### OpenAI example
+
 If you are using the OpenAI API (e.g. using `gpt-4-turbo` via your own OpenAI API account), then the `config_list` for the AutoGen agent and `config_list_memgpt` for the MemGPT AutoGen agent will look different (a lot simpler):
+
 ```python
 # This config is for autogen agents that are not powered by MemGPT
 config_list = [
@@ -192,7 +202,9 @@ config_list_memgpt = [
 ```
 
 #### Azure OpenAI example
+
 Azure OpenAI API setup will be similar to OpenAI API, but requires additional config variables. First, make sure that you've set all the related Azure variables referenced in [our MemGPT Azure setup page](https://memgpt.readme.io/docs/endpoints#azure-openai) (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_VERSION`, `AZURE_OPENAI_ENDPOINT`, etc). If you have all the variables set correctly, you should be able to create configs by pulling from the env variables:
+
 ```python
 # This config is for autogen agents that are not powered by MemGPT
 # See Auto
@@ -219,17 +231,17 @@ config_list_memgpt = [
         "azure_endpoint": os.getenv("AZURE_OPENAI_ENDPOINT"),
         "azure_version": os.getenv("AZURE_OPENAI_VERSION"),
         # if you are using Azure for embeddings too, include the following line:
-        "embedding_embedding_endpoint_type": "azure",        
+        "embedding_embedding_endpoint_type": "azure",
     },
 ]
 ```
 
-
 > 📘 Making internal monologue visible to AutoGen
 >
 > By default, MemGPT's inner monologue and function traces are hidden from other AutoGen agents.
 >
 > You can modify `interface_kwargs` to change the visibility of inner monologue and function calling:
+>
 > ```python
 > interface_kwargs = {
 >     "debug": False,  # this is the equivalent of the --debug flag in the MemGPT CLI
@@ -239,11 +251,13 @@ config_list_memgpt = [
 > ```
 
 The only parts of the `agent_groupchat.py` file you need to modify should be the `config_list` and `config_list_memgpt` (make sure to change `USE_OPENAI` to `True` or `False` depending on if you're trying to use a local LLM server like web UI, or OpenAI's API). Assuming you edited things correctly, you should now be able to run `agent_groupchat.py`:
+
 ```sh
 python memgpt/autogen/examples/agent_groupchat.py
 ```
 
 Your output should look something like this:
+
 ```text
 User_proxy (to chat_manager):
 
@@ -287,14 +301,14 @@ Remember, achieving one million dollars in revenue in such a short time frame wo
 --------------------------------------------------------------------------------
 MemGPT_coder (to chat_manager):
 
-Great goal! Generating a million dollars in one month with an app is ambitious, but definitely doable if you approach it the right way. Here are some tips and potential ideas that could help: 
+Great goal! Generating a million dollars in one month with an app is ambitious, but definitely doable if you approach it the right way. Here are some tips and potential ideas that could help:
 
-1. Identify a niche market or trend (for example, AI-powered fitness apps or FinTech solutions). 
-2. Solve a significant problem for many people (such as time management or financial literacy). 
-3. Choose an effective monetization strategy like subscriptions, in-app purchases, or advertising. 
+1. Identify a niche market or trend (for example, AI-powered fitness apps or FinTech solutions).
+2. Solve a significant problem for many people (such as time management or financial literacy).
+3. Choose an effective monetization strategy like subscriptions, in-app purchases, or advertising.
 4. Make sure your app is visually appealing and easy to use to keep users engaged.
 
-Some ideas that might work: 
+Some ideas that might work:
 - AI-powered personal finance management app
 - A virtual assistant app that helps people manage their daily tasks
 - A social networking platform for job seekers or freelancers
@@ -316,15 +330,18 @@ User_proxy (to chat_manager):
 First, follow the instructions in [Example - chat with your data - Creating an external data source](example_data/#creating-an-external-data-source):
 
 To download the MemGPT research paper we'll use `curl` (you can also just download the PDF from your browser):
+
 ```sh
 # we're saving the file as "memgpt_research_paper.pdf"
 curl -L -o memgpt_research_paper.pdf https://arxiv.org/pdf/2310.08560.pdf
 ```
 
 Now that we have the paper downloaded, we can create a MemGPT data source using `memgpt load`:
+
 ```sh
 memgpt load directory --name memgpt_research_paper --input-files=memgpt_research_paper.pdf
 ```
+
 ```text
 loading data
 done loading data
@@ -337,9 +354,11 @@ Generating embeddings: 100%|█████████████████
 Note: you can ignore the "_LLM is explicitly disabled_" message.
 
 Now, you can run `agent_docs.py`, which asks `MemGPT_coder` what a virtual context is:
+
 ```sh
 python memgpt/autogen/examples/agent_docs.py
 ```
+
 ```text
 Ingesting 65 passages into MemGPT_agent
 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.47s/it]

diff --git a/docs/cli_faq.md b/docs/cli_faq.md
@@ -1,11 +1,11 @@
 ---
 title: Frequently asked questions (FAQ)
 excerpt: Check frequently asked questions
-category: 6580d34ee5e4d00068bf2a1d 
+category: 6580d34ee5e4d00068bf2a1d
 ---
 
 > 📘 Open / local LLM FAQ
-> 
+>
 > Questions specific to running your own open / local LLMs with MemGPT can be found [here](local_llm_faq).
 
 ## MemGPT CLI

diff --git a/docs/config.md b/docs/config.md
@@ -1,7 +1,7 @@
 ---
 title: Configuration
 excerpt: Configuring your MemGPT agent
-category: 6580d34ee5e4d00068bf2a1d 
+category: 6580d34ee5e4d00068bf2a1d
 ---
 
 You can set agent defaults by running `memgpt configure`, which will store config information at `~/.memgpt/config` by default.
@@ -18,22 +18,25 @@ The `memgpt run` command supports the following optional flags (if set, will ove
 * `--no-verify`: (bool) Bypass message verification (default=False)
 * `--yes`/`-y`: (bool) Skip confirmation prompt and use defaults (default=False)
 
-You can override the parameters you set with `memgpt configure` with the following additional flags specific to local LLMs: 
+You can override the parameters you set with `memgpt configure` with the following additional flags specific to local LLMs:
 
 * `--model-wrapper`: (str) Model wrapper used by backend (e.g. `airoboros_xxx`)
 * `--model-endpoint-type`: (str) Model endpoint backend type (e.g. lmstudio, ollama)
 * `--model-endpoint`: (str) Model endpoint url (e.g. `localhost:5000`)
 * `--context-window`: (int) Size of model context window (specific to model type)
 
 #### Updating the config location
+
 You can override the location of the config path by setting the environment variable `MEMGPT_CONFIG_PATH`:
-```
+
+```sh
 export MEMGPT_CONFIG_PATH=/my/custom/path/config # make sure this is a file, not a directory
 ```
 
-
 ### Adding Custom Personas/Humans
+
 You can add new human or persona definitions either by providing a file (using the `-f` flag) or text (using the `--text` flag).
+
 ```sh
 # add a human
 memgpt add human [--name <NAME>] [-f <FILENAME>] [--text <TEXT>]
@@ -43,9 +46,11 @@ memgpt add persona [--name <NAME>] [-f <FILENAME>] [--text <TEXT>]
 ```
 
 You can view available persona and human files with the following command:
+
 ```sh
 memgpt list [humans/personas]
 ```
 
 ### Custom Presets
+
 You can customize your MemGPT agent even further with [custom presets](presets) and [custom functions](functions).
diff --git a/docs/contributing.md b/docs/contributing.md
@@ -1,14 +1,14 @@
 ---
 title: How to contribute
 excerpt: Learn how to contribute to the MemGPT project!
-category: 6581eaa89a00e6001012822c 
+category: 6581eaa89a00e6001012822c
 ---
 
 ![memgpt llama](https://raw.githubusercontent.com/cpacker/MemGPT/main/docs/assets/memgpt_library.webp)
 
 MemGPT is an active [open source](https://en.wikipedia.org/wiki/Open_source) project and we welcome community contributions! There are many ways to contribute for both programmers and non-programmers alike.
 
-> 📘 Discord contributor role 
+> 📘 Discord contributor role
 >
 > Contributing to the codebase gets you a **contributor role** on [Discord](https://discord.gg/9GEQrxmVyE). If you're a contributor and we forgot to assign you the role, message the MemGPT team [on Discord](https://discord.gg/9GEQrxmVyE)!
 
@@ -22,4 +22,4 @@ We're always looking to improve our docs (like the page you're reading right now
 
 ## 🦙 Editing the MemGPT source code
 
-If you're interested in editing the MemGPT source code, [check our guide on building and contributing from source](contributing_code).
+If you're interested in editing the MemGPT source code, [check our guide on building and contributing from source](contributing_code).