404 Client Error when using Azure OpenAI endpoints #959

briantani · 2024-02-06T18:37:16Z

Describe the bug
I use memgpt configure to set up the Azure OpenAI backend with these environment variables.

AZURE_OPENAI_EMBEDDING_ENDPOINT=https://{resource}.openai.azure.com
AZURE_OPENAI_ENDPOINT=https://{resource}.openai.azure.com
AZURE_OPENAI_KEY={AZURE API KEY}
AZURE_OPENAI_VERSION=2023-03-15-preview

> memgpt configure
? Select LLM inference provider: azure
? Select default model (recommended: gpt-4): gpt-4
? Select embedding provider: azure
? Select default preset: memgpt_chat
? Select default persona: sam_pov
? Select default human: cs_phd
? Select storage backend for archival data: chroma
? Select chroma backend: persistent
? Select storage backend for recall data: sqlite
📖 Saving config to C:\Users\tanibr\.memgpt\config

Then I run memgpt run and try to converse, this error appear:

An exception occurred when running agent.step():
Traceback (most recent call last):
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\main.py", line 356, in run_agent_loop
    new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\main.py", line 332, in process_agent_step
    new_messages, heartbeat_request, function_failed, token_warning, tokens_accumulated = memgpt_agent.step(
                                                                                          ^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 676, in step
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 596, in step
    response = self._get_ai_reply(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 362, in _get_ai_reply
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 342, in _get_ai_reply
    response = create(
               ^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 346, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 455, in create
    return azure_openai_chat_completions_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 282, in azure_openai_chat_completions_request
    raise http_err
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 271, in azure_openai_chat_completions_request
    response.raise_for_status()  # Raises HTTPError for 4XX/5XX status
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: model_error for url: https://{resource}.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-03-15-preview

I'm able to use this AzureOpenAI with AutoGen for instance, no problem using essencialy the same configuration.
I also tried to use code, and I get the same 404 Client error when the MemGPT agent attempts to call the API. The other agents respond as expected.

# This config is for AutoGen agents that are not powered by MemGPT
config_list = [
    {
        "model": "gpt-4",
        "api_type": "azure",
        "api_key": os.getenv("AZURE_OPENAI_KEY"),
        "api_version": os.getenv("AZURE_OPENAI_VERSION"),
        "base_url": os.getenv("AZURE_OPENAI_ENDPOINT"),
    },
]
llm_config = {"config_list": config_list, "seed": 42}
# This config is for AutoGen agents that powered by MemGPT
config_list_memgpt = [
    {
        "preset": "memgpt_chat",
        "model": "gpt-4",
        "model_wrapper": None,
        "context_window": LLM_MAX_TOKENS["gpt-4"],
        # required setup for Azure
        "model_endpoint_type": "azure",
        "azure_key": os.getenv("AZURE_OPENAI_KEY"),
        "azure_endpoint": os.getenv("AZURE_OPENAI_ENDPOINT"),
        "azure_version": os.getenv("AZURE_OPENAI_VERSION"),
        "embeddings_endpoint": os.getenv("AZURE_OPENAI_EMBEDDING_ENDPOINT"),
    },
]
llm_config_memgpt = {"config_list": config_list_memgpt, "seed": 42}

...

coder_system_message = "You are a 10x engineer, trained in Python."
if not USE_MEMGPT:
    print("Using AutoGen for the coder agent")
    # In the AutoGen example, we create an AssistantAgent to play the role of the coder
    coder = autogen.AssistantAgent(
        name=coder_name,
        system_message=coder_system_message,
        llm_config=llm_config,
    )

else:
    print("Using MemGPT for the coder agent")
    # In our example, we swap this AutoGen agent with a MemGPT agent
    # This MemGPT agent will have all the benefits of MemGPT, ie persistent memory, etc.

    # We can use interface_kwargs to control what MemGPT outputs are seen by the groupchat
    interface_kwargs = {
        "debug": False,
        "show_inner_thoughts": True,
        "show_function_outputs": True,
    }

    coder = create_memgpt_autogen_agent_from_config(
        coder_name,
        llm_config=llm_config_memgpt,
        system_message=coder_system_message,
        interface_kwargs=interface_kwargs,
        default_auto_reply="...", 
    )

Please describe your setup

MemGPT version
- 0.3.0
How did you install memgpt?
- pip install -U pymemgpt
Describe your setup
- What's your OS (Windows/MacOS/Linux)?
  - Windows 10
- How are you running memgpt? (cmd.exe/Powershell/Anaconda Shell/Terminal)
  - Powershell 7 / Conda Environment

The text was updated successfully, but these errors were encountered:

cpacker · 2024-02-12T01:59:29Z

Hi @briantani - I believe this should be fixed by #982, which will be part of the upcoming 0.3.2 release (you can also grab it right now via pymemgpt-nightly).

If this still persists please let me know by reopening the issue.

briantani · 2024-02-12T18:25:27Z

I installed pymemgpt-nightly-0.3.1.dev20240212103909 and ran into the same issue:

This is the error message when I run the code in using python app.py

--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\tanibr\projects\automemgpt\app.py", line 171, in <module>
    user_proxy.initiate_chat(manager, message=prompt_1)
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 793, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 504, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 679, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 1637, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\groupchat.py", line 526, in run_chat
    reply = speaker.generate_reply(sender=self)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 1637, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\autogen\memgpt_agent.py", line 162, in _generate_reply_for_user_message
    ) = self.agent.step(user_message, first_message=False, skip_verify=self.skip_verify)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 676, in step
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 608, in step
    response = self._get_ai_reply(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 359, in _get_ai_reply
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 339, in _get_ai_reply
    response = create(
               ^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 352, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 461, in create
    return azure_openai_chat_completions_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 288, in azure_openai_chat_completions_request
    raise http_err
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 277, in azure_openai_chat_completions_request
    response.raise_for_status()  # Raises HTTPError for 4XX/5XX status
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: model_error for url: https://{resource}.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-03-15-preview

And this is the error I get when I run memgpt run and ask anything in the prompt:

An exception occurred when running agent.step():
Traceback (most recent call last):
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\main.py", line 347, in run_agent_loop
    new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\main.py", line 323, in process_agent_step
    new_messages, heartbeat_request, function_failed, token_warning, tokens_accumulated = memgpt_agent.step(
                                                                                          ^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 676, in step
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 596, in step
    response = self._get_ai_reply(
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 359, in _get_ai_reply
    raise e
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\agent.py", line 339, in _get_ai_reply
    response = create(
               ^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 352, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 461, in create
    return azure_openai_chat_completions_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 288, in azure_openai_chat_completions_request
    raise http_err
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\memgpt\llm_api_tools.py", line 277, in azure_openai_chat_completions_request
    response.raise_for_status()  # Raises HTTPError for 4XX/5XX status
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\tanibr\Anaconda3\envs\memgpt\Lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: model_error for url: https://{resource}.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-03-15-preview
? Retry agent.step()? (Y/n)

Is there anything I can provide to better understand the issue? Or while I did install the nightly version, it didn't have the update yet for some reason?

cpacker · 2024-02-12T20:48:57Z

@briantani thank you for the update!

My guess is that the credentials aren't saved properly (this was patched but you'll still need to re-run memgpt configure to get the credentials set). Can you try checking the contents of your ~/.memgpt/credentials file?

It should have azure fields that look like this (replace the ... and xxx with your own custom info):

[azure]
auth_type = api_key
key = ...
version = 2023-07-01-preview
endpoint = https://xxx.openai.azure.com
deployment = ...
embedding_version = 2023-07-01-preview
embedding_endpoint = https://xxx.openai.azure.com
embedding_deployment = ...

If you're missing any of these fields, try settings the values here, then running memgpt configure again (or manually edit the credentials file).

If you're not missing any of these fields, let me know and I can try to debug your stacktrace.

briantani · 2024-02-13T00:37:21Z

The credentials have the correct value, and running memgpt configure prior to executing the code or running memgpt run does not seem to influence the result. I still get those 404 Client Error: model_error for url: https://{resource].openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-03-15-preview

It seems to be trying to call chat completioins, not even embedding, and it fails. The AutoGen portion of the code, however, seems to be able to contact the chat completions endpoint no problem.

So I placed a breakpoint on the azure_openai_chat_completions_request function, and the true error is this:

{
  "error": {
    "message": "Unrecognized request argument supplied: tools",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

I reproduced it using curl:

curl https://{resource}.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2023-05-15 -H "Content-Type: application/json" -H "api-key: ..." -d '@package.json'
{
  "error": {
    "message": "Unrecognized request argument supplied: tools",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

The package,json was created dumping the request data to a file.

It seems the chat endpoint does not understand the tools and tool_choice properties. If I remove them prior to sending the request, the API responds as expected.

cpacker · 2024-02-13T01:11:13Z

Ah OK this makes more sense now - it seems like you're using Azure OpenAI version 2023-05-15

According to the Azure OpenAI REST API docs, tools had not been added yet then. From browsing the swagger specs, it seems like it may have been added in 2023-06-01-preview?

Is it possible for you to upgrade your Azure OpenAI version to any of these?

2023-06-01-preview (retiring 2024-04-02) Swagger spec
2023-07-01-preview (retiring 2024-04-02) Swagger spec
2023-08-01-preview (retiring 2024-04-02) Swagger spec
2023-09-01-preview (retiring 2024-04-02) Swagger spec
2023-12-01-preview (required for Vision scenarios) Swagger spec

If not, no worries, I can also add some sort of fallback inside the MemGPT code that checks if you're using an Azure spec older than 2023-06-01-preview, and toggles this line of code accordingly. You can also try modifying this by hand yourself (setting use_tool_naming to False) to check that it works.

briantani · 2024-02-13T03:38:11Z

Using a more recent API preview did the work. This solves my problem. I was using the same API preview for consistency but didn't know about the tools since I wasn't testing tools yet.

Thanks for taking the time to help me.

cpacker · 2024-02-13T18:44:03Z

Awesome, really glad to hear it @briantani !

Thank you so much for your detailed bug reporting.

sarahwooders added the bug Something isn't working label Feb 6, 2024

cpacker added bug Something isn't working and removed bug Something isn't working labels Feb 6, 2024

github-project-automation bot added this to 🐛 MemGPT issue tracker Feb 6, 2024

github-project-automation bot moved this to To triage in 🐛 MemGPT issue tracker Feb 6, 2024

cpacker moved this from To triage to Ready in 🐛 MemGPT issue tracker Feb 8, 2024

cpacker self-assigned this Feb 8, 2024

cpacker moved this from Ready to In progress in 🐛 MemGPT issue tracker Feb 11, 2024

cpacker closed this as completed Feb 12, 2024

github-project-automation bot moved this from In progress to Done in 🐛 MemGPT issue tracker Feb 12, 2024

Nilbug mentioned this issue Apr 3, 2024

Client error from memgpt_autogen : HTTPError: 404 Client Error: Not Found for url: https://api.memgpt.ai/chat/completions #1211

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

404 Client Error when using Azure OpenAI endpoints #959

404 Client Error when using Azure OpenAI endpoints #959

briantani commented Feb 6, 2024

cpacker commented Feb 12, 2024 •

edited

Loading

briantani commented Feb 12, 2024

cpacker commented Feb 12, 2024

briantani commented Feb 13, 2024

cpacker commented Feb 13, 2024

briantani commented Feb 13, 2024

cpacker commented Feb 13, 2024

404 Client Error when using Azure OpenAI endpoints #959

404 Client Error when using Azure OpenAI endpoints #959

Comments

briantani commented Feb 6, 2024

cpacker commented Feb 12, 2024 • edited Loading

briantani commented Feb 12, 2024

cpacker commented Feb 12, 2024

briantani commented Feb 13, 2024

cpacker commented Feb 13, 2024

briantani commented Feb 13, 2024

cpacker commented Feb 13, 2024

cpacker commented Feb 12, 2024 •

edited

Loading