logs() hangs after container has quit

[ghost]

Hi,

I'm using version 1.6 of docker-py on Python 3.4.3+ on Ubuntu 15.10

bash# pip3 freeze | grep docker-py && python3 --version && docker version
docker-py==1.6.0
Python 3.4.3+
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.3
 Git commit:   a34a1d5
 Built:        Fri Nov 20 17:56:04 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:20:08 UTC 2015
 OS/Arch:      linux/amd64

This hangs after my container finishes what it's doing:

 machine.start(container=backup)
 for line in machine.logs(container=backup, stream=True):
   log(line.decode("utf-8").strip(), name)

I don't want to have to return to wrapping shell commands, uggg :-(

I have seen various closed issues related to this. Am I missing some subtlety in the docs or is this a problem again?

Thanks!

[TomasTomecek]

minimal reproducer would be great

[ghost]

Hi Tomas:

config = machine.create_host_config(binds=['%s:/backup' % source], privileged=True)
restore = machine.create_container(
    image='muse_backup',
    command=['/restore.sh', directory, location],
    entrypoint=entrypoint,
    host_config=config,
)
log('Restore container ID: ' + restore.get('Id'), name)
machine.start(container=restore)
for line in machine.logs(container=restore, stream=True):
    log(line.decode("utf-8").strip(), name)

[TomasTomecek]

Looks like that engine did some changes. This is working for me:

#!/usr/bin/python3
import docker
d = docker.AutoVersionClient()
restore = d.create_container(
    image='fedora:23',
    command=["python3", "-m", "http.server", "8000"],
    tty=True  # this does the trick
)
d.start(container=restore)
for line in d.logs(container=restore, stream=True):
    print(line, end="")  # engine likely turns off line buffering and sends every character

[ghost]

Hi Tomas,

I'll give that a try.

What is AutoVersionClient? Does it take the same parameters as Client?

Thanks!

[TomasTomecek]

AutoVersionClient uses latest API from engine. Yes, it has exactly the same interface as Client, it's just shortcut for Client(version="auto").

[ghost]

Thanks Tomas,

I now have this working on Docker Engine 1.9.1 (I'll test 1.10.0 later). As your code comment mentioned, stream is spitting out every character without buffering.

 machine.start(container=backup)
 line = ''
 for char in machine.logs(container=backup, stream=True):
     if char == '\n':
         log(line, name)
         line = ''
     else:
         line = line + char

[ghost]

Seems to be working fine on 1.10.0 too.

[messense]

Same issue when streaming logs.

Python 3.5

docker-py==1.10.3
requests==2.10.0

Docker version 1.12.1, build 23cf638

[tlusk]

Same issue with 1.10.3, even with the tty=True trick from #919 (comment)

[tedmiston]

I am still seeing this issue with the latest version of the Docker client (2.5.1). Using tty=True plus the streaming fix by @DarkerMatter in #919 (comment) fixed it for us.

[txomon]

This is still happening to us, although I'm not sure on why. I have tried out the solution (tty=True + per line composing) but it seems like it still fails to log.

In my case this is happening with:
docker==3.7.3
requests==2.22.0
urllib3==1.25.6

This is happening to me with airflow just in case.

I have ended up doing a polling with wait(timeout=10) + logs(since=ts) in an infinite loop waiting for wait() to yield a result. This will give you terrible performance in many cases, but should allow most of the usecases.

I have no idea why is this really failing though