Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watchtower 服务 bug:反复重试访问 *.docker.io 可能导致服务崩溃 #308

Closed
6 tasks done
Xheldon opened this issue Dec 7, 2024 · 7 comments · Fixed by #309
Closed
6 tasks done

Watchtower 服务 bug:反复重试访问 *.docker.io 可能导致服务崩溃 #308

Xheldon opened this issue Dec 7, 2024 · 7 comments · Fixed by #309
Labels
bug Something isn't working

Comments

@Xheldon
Copy link

Xheldon commented Dec 7, 2024

  • 我已确认目前没有类似 issue
  • 我已确认我已升级到最新版本
  • 我已完整浏览项目 README 和项目文档并未找到解决方案
  • 我理解并愿意跟进此 issue,协助测试和提供反馈
  • 我将以礼貌和尊重的态度提问,不得使用不文明用语 (包括在此发布评论的所有人同样适用, 不遵守的人将被 block)
  • 我理解并认可上述内容,并理解项目维护者精力有限,不遵循规则的 issue 可能会被无视或直接关闭

使用的是 docker-compose -f docker-compose.watch.yaml up -d 命令,带 watch 的,部署完挺好的,隔夜发现服务挂了,重启正常,然后一直到现在都正常就没管了(可能因为我中途重启过几次,无意间掩盖了请求过多导致的问题)。

但是今天排查其他问题,无意间发现 watchtower 失败会一直重试,回想起这可能是之前导致这个服务挂掉的原因。

证据如图:

Surge 网络请求(注意这个是活动 tab,是是一直动态变化,一个请求 index.docker.io 失败后就从这个列表移除,然后又出现一个 index.docker.io 的请求,其他 *.docker.io 请求类似)

CleanShot_2024_12_08_at_01_22_02@2x

docker 内的 chatnio 日志:(注意看右下角,很长的日志,这才启动不到1分钟!)

CleanShot_2024_12_08_at_01_23_46@2x

@Xheldon Xheldon added the bug Something isn't working label Dec 7, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Title: Watchtower service bug: Repeatedly retrying to access *.docker.io may cause the service to crash

  • I have confirmed that there is currently no similar issue
  • I have confirmed that I have upgraded to the latest version
  • I have fully browsed the project README and project documentation and did not find a solution
  • I understand and am willing to follow up on this issue, assist in testing and provide feedback
  • I will ask questions in a polite and respectful manner and shall not use uncivilized language (the same applies to everyone who posts comments here, those who do not comply will be blocked)
  • I understand and agree with the above content, and understand that project maintainers have limited energy. Issues that do not follow the rules may be ignored or closed directly

I used the docker-compose -f docker-compose.watch.yaml up -d command, with watch, and the deployment was fine. I found that the service was down overnight, restarted it normally, and then I left it alone until now. (Maybe because I restarted several times midway and unintentionally covered up the problem caused by too many requests).

But today when I was troubleshooting other issues, I accidentally discovered that the watchtower would keep retrying if it failed. I recalled that this might be the reason why the service hung up before.

The evidence is shown in the picture:

Surge network request (note that this is an active tab, which is always changing dynamically. After a request to index.docker.io fails, it will be removed from this list, and then another request to index.docker.io appears, and other *.docker.io request similar)

CleanShot_2024_12_08_at_01_22_02@2x

Chatnio log in docker: (Pay attention to the lower right corner, the very long log, it took less than 1 minute to start!)

CleanShot_2024_12_08_at_01_23_46@2x

@Xheldon
Copy link
Author

Xheldon commented Dec 7, 2024

另外还有一个小问题我就不单独提 issue 了,就是后台管理里面的服务日志,点击删除了 chatnio.log 日志后,就再也获取不到新日志了(提示找不到文件),刷新也没用,除非重启 docker 服务才行。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is also a small issue that I won’t mention separately, which is the service log in the background management. After clicking to delete the chatnio.log log, I can no longer get new logs (it prompts that the file cannot be found), and the refresh fails. It's useless unless you restart the docker service.

@zmh-program
Copy link
Member

另外还有一个小问题我就不单独提 issue 了,就是后台管理里面的服务日志,点击删除了 chatnio.log 日志后,就再也获取不到新日志了(提示找不到文件),刷新也没用,除非重启 docker 服务才行。

这个我记得好久之前修复过,禁止删除latest log chunk,不知道开源版下没下放。后续 #285 会修复。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


There is another small problem that I won’t mention separately. It is the service log in the background management. After clicking to delete the chatnio.log log, I can no longer get new logs (it prompts that the file cannot be found). Refresh It's useless unless you restart the docker service.

I remember that this was fixed a long time ago, prohibiting the deletion of the latest log chunk. I don’t know if it has been decentralized in the open source version. #285 will be fixed in the future.

@zmh-program
Copy link
Member

已从 30s 设置到 3600s。 git pull 一下重新启动。 未复现,大概率的原因是未携带 docker credentials 因此导致速率限制。
如仍有问题,请 reopen 此 issue。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Set from 30s to 3600s. git pull and restart. It has not been reproduced. The most likely reason is that docker credentials are not carried, which results in rate limitation.
If you still have problems, please reopen this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants