Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Serious bugs about “CRAWLAB_SPIDER_PATH” #150

Closed
Zerorigin opened this issue Aug 20, 2019 · 1 comment
Closed

Serious bugs about “CRAWLAB_SPIDER_PATH” #150

Zerorigin opened this issue Aug 20, 2019 · 1 comment
Labels
bug Something isn't working

Comments

@Zerorigin
Copy link

　　上传巨型爬虫项目压缩文件(如包含未经清理的 data/logs 等其它项目无关大文件)至 Crawlab 服务端或将 CRAWLAB_SPIDER_PATH 目录映射到宿主本地大文件目录时，会导致 Crawlab 容器异常占用内存，并持续读写硬盘直至宿主机内存被撑爆(如果未在使用 docker 部署时限制内存使用上限的话 )。负面影响，严重影响宿主机性能，造成宿主长时间无法正常提供其它服务(如SSH远程会话等)。
　
P.S：项目组说提个 issue 方便追踪，所以就随便写写了。
@tikazyq tikazyq added the bug Something isn't working label Aug 20, 2019
@tikazyq tikazyq mentioned this issue Aug 20, 2019
Merged
@tikazyq
Copy link
Collaborator

tikazyq commented Aug 20, 2019 •
edited
Loading

fixed in #151

@tikazyq tikazyq closed this as completed Aug 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tikazyq @Zerorigin