Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BFCL] ModuleNotFoundError after Restructuring in Commit 70d6722: Incorrect Import Paths #614

Closed
kobe0938 opened this issue Aug 29, 2024 · 1 comment · Fixed by #615
Closed
Labels
BFCL-General General BFCL Issue

Comments

@kobe0938
Copy link
Contributor

Describe the issue
After updating to commit 70d6722, the LLM inference command for the evaluation dataset fails on gorilla model. Specifically, running the command:
python openfunctions_evaluation.py --model gorilla-openfunctions-v2 --test-category multiple --num-threads 1
results in the following error:
ModuleNotFoundError: No module named 'model_handler'

The root cause appears to be the relocation of the model_handler and eval_checker folders into a new folder named bfcl during the specified commit, without updating the relevant import paths in the code.

ID datapoint

  1. Datapoint / Model Handler permalink: https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/gorilla_handler.py
    https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/oss_handler.py
  2. Issue: Failure to locate the model_handler module due to incorrect import paths.
  3. Gorilla repo commit #: 70d6722

What is the issue
The error disrupts the use of the LLM inference functionality for evaluating datasets, preventing execution with the current module paths.
Proposed Changes
Update the import statements in the following files to reflect the new directory structure:

oss_handler.py
Current: from model_handler.constant import DEFAULT_SYSTEM_PROMPTProposed:from bfcl.model_handler.constant import DEFAULT_SYSTEM_PROMPT gorilla_handler.pyCurrent:from model_handler.constant import DEFAULT_SYSTEM_PROMPTProposed:from bfcl.model_handler.constant import DEFAULT_SYSTEM_PROMPT`

Additional context
The changes above restore functionality to the LLM inference process by correcting the module paths affected by the restructuring in commit 70d6722. A pull request with these fixes will be submitted shortly to address the problem.

@HuanzhiMao
Copy link
Collaborator

Hi @kobe0938,
Thanks for the issue! Could you try #612? It should have been fixed there.

@HuanzhiMao HuanzhiMao added the BFCL-General General BFCL Issue label Aug 29, 2024
kobe0938 pushed a commit to kobe0938/gorilla that referenced this issue Aug 29, 2024
kobe0938 added a commit to kobe0938/gorilla that referenced this issue Aug 29, 2024
HuanzhiMao pushed a commit that referenced this issue Aug 29, 2024
Bug fix of #614 [](url)

This pull request resolves the `ModuleNotFoundError` reported in issue
#614 by updating the import paths affected by the directory
restructuring in commit 70d6722. The adjustments ensure that the LLM
inference functionality works as expected.

**Changes:**
- `oss_handler.py` and `gorilla_handler.py` import paths updated to
include the new `bfcl` directory prefix.

**Testing:**
Run the following command to verify that the issue has been resolved:
`python openfunctions_evaluation.py --model gorilla-openfunctions-v2
--test-category multiple --num-threads 1
`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BFCL-General General BFCL Issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants