You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the issue
After updating to commit 70d6722, the LLM inference command for the evaluation dataset fails on gorilla model. Specifically, running the command: python openfunctions_evaluation.py --model gorilla-openfunctions-v2 --test-category multiple --num-threads 1
results in the following error: ModuleNotFoundError: No module named 'model_handler'
The root cause appears to be the relocation of the model_handler and eval_checker folders into a new folder named bfcl during the specified commit, without updating the relevant import paths in the code.
What is the issue
The error disrupts the use of the LLM inference functionality for evaluating datasets, preventing execution with the current module paths. Proposed Changes
Update the import statements in the following files to reflect the new directory structure:
Additional context
The changes above restore functionality to the LLM inference process by correcting the module paths affected by the restructuring in commit 70d6722. A pull request with these fixes will be submitted shortly to address the problem.
The text was updated successfully, but these errors were encountered:
Bug fix of #614 [](url)
This pull request resolves the `ModuleNotFoundError` reported in issue
#614 by updating the import paths affected by the directory
restructuring in commit 70d6722. The adjustments ensure that the LLM
inference functionality works as expected.
**Changes:**
- `oss_handler.py` and `gorilla_handler.py` import paths updated to
include the new `bfcl` directory prefix.
**Testing:**
Run the following command to verify that the issue has been resolved:
`python openfunctions_evaluation.py --model gorilla-openfunctions-v2
--test-category multiple --num-threads 1
`
Describe the issue
After updating to commit 70d6722, the LLM inference command for the evaluation dataset fails on gorilla model. Specifically, running the command:
python openfunctions_evaluation.py --model gorilla-openfunctions-v2 --test-category multiple --num-threads 1
results in the following error:
ModuleNotFoundError: No module named 'model_handler'
The root cause appears to be the relocation of the model_handler and eval_checker folders into a new folder named bfcl during the specified commit, without updating the relevant import paths in the code.
ID datapoint
https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/oss_handler.py
What is the issue
The error disrupts the use of the LLM inference functionality for evaluating datasets, preventing execution with the current module paths.
Proposed Changes
Update the import statements in the following files to reflect the new directory structure:
oss_handler.py
Current:
from model_handler.constant import DEFAULT_SYSTEM_PROMPTProposed:
from bfcl.model_handler.constant import DEFAULT_SYSTEM_PROMPTCurrent:
from model_handler.constant import DEFAULT_SYSTEM_PROMPTProposed:
from bfcl.model_handler.constant import DEFAULT_SYSTEM_PROMPT`Additional context
The changes above restore functionality to the LLM inference process by correcting the module paths affected by the restructuring in commit 70d6722. A pull request with these fixes will be submitted shortly to address the problem.
The text was updated successfully, but these errors were encountered: