Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] crawl4ai tool not able to scrape data #2032

Open
GauravKasat opened this issue Feb 6, 2025 · 1 comment
Open

[Bug] crawl4ai tool not able to scrape data #2032

GauravKasat opened this issue Feb 6, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@GauravKasat
Copy link

from agno.agent import Agent
from agno.tools.crawl4ai import Crawl4aiTools
import warnings
warnings.simplefilter("ignore") # Ignore all warnings
import os
os.environ["OPENAI_API_KEY"] = "api-key"

agent = Agent(tools=[Crawl4aiTools(max_length=None)], show_tool_calls=True)
agent.print_response("Tell me the price and rating mentioned on https://www.amazon.in/Apple-iPhone-Pro-Max-256/dp/B0CHWV2WYK?th=1")

Its not able to bring down the price and rating specified for my use case

@GauravKasat GauravKasat added the bug Something isn't working label Feb 6, 2025
@isikepalaku
Copy link

add cannot run in docker as agent:

2025-02-10 16:26:08 from api.routes.v1_router import v1_router
2025-02-10 16:26:08 File "/app/api/routes/v1_router.py", line 3, in
2025-02-10 16:26:08 from api.routes.playground import playground_router
2025-02-10 16:26:08 File "/app/api/routes/playground.py", line 7, in
2025-02-10 16:26:08 from agents.trend_kejahatan import get_crime_trend_agent
2025-02-10 16:26:08 File "/app/agents/trend_kejahatan.py", line 10, in
2025-02-10 16:26:08 from agno.tools.crawl4ai import Crawl4aiTools
2025-02-10 16:26:08 File "/usr/local/lib/python3.12/site-packages/agno/tools/crawl4ai.py", line 7, in
2025-02-10 16:26:08 from crawl4ai import AsyncWebCrawler, CacheMode
2025-02-10 16:26:08 File "/usr/local/lib/python3.12/site-packages/crawl4ai/init.py", line 3, in
2025-02-10 16:26:08 from .async_webcrawler import AsyncWebCrawler, CacheMode
2025-02-10 16:26:08 File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_webcrawler.py", line 14, in
2025-02-10 16:26:08 from .async_database import async_db_manager
2025-02-10 16:26:08 File "/usr/local/lib/python3.12/site-packages/crawl4ai/async_database.py", line 24, in
2025-02-10 16:26:08 os.makedirs(DB_PATH, exist_ok=True)
2025-02-10 16:26:08 File "", line 225, in makedirs
2025-02-10 16:26:08 PermissionError: [Errno 13] Permission denied: '/app/.crawl4ai'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants