HTTP Proxy broken in Version 17.12.0-ce-rc2-mac41 (21193)

[alexellis]

Expected behavior

docker pull should work when no direct internet access is available but the proxy is configured properly

Actual behavior

$ docker pull functions/alpine:latest
Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 192.168.65.1:53: no such host

I.e. fails.. this worked prior to upgrading

Information

• Full output of the diagnostics from "Diagnose & Feedback" in the menu
• A reproducible case if this is a bug, Dockerfiles FTW
• Page URL if this is a docs issue or the name of a man page

Client:
 Version:	17.12.0-ce-rc2
 API version:	1.35
 Go version:	go1.9.2
 Git commit:	f9cde63
 Built:	Tue Dec 12 06:39:10 2017
 OS/Arch:	darwin/amd64

Server:
 Engine:
  Version:	17.12.0-ce-rc2
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.2
  Git commit:	f9cde63
  Built:	Tue Dec 12 06:45:30 2017
  OS/Arch:	linux/amd64
  Experimental:	true

Steps to reproduce the behavior

Use corporate network
Configure proxy
Upgrade to Version 17.12.0-ce-rc2-mac41 (21193) / 0f8a7d28d0

Try a Docker pull

[guillaumerose]

Let me try to reproduce this one.
We heavily changed the code in this area. We now use vpnkit transparent http proxy by default. With this, we don't have to pass proxy variables to the VM. I will double-check that.

[alexellis]

What about settings in either ~/.docker/config.json or ~/.docker/daemon.json cc @dave-tucker

[djs55]

When I use "little snitch" to block outgoing web traffic and try the docker pull I get:

$ docker pull functions/alpine:latest
Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp 34.200.28.105:443: getsockopt: connection refused

-- note this is a connection refused from the IP of the registry. By contrast your failure looks DNS related: lookup registry-1.docker.io on 192.168.65.1:53: no such host (where 192.168.65.1:53 is where the local DNS server lives)

If I set Whale menu -> Preferences -> Proxies -> Use system proxy and then set System Preferences -> Network -> Advanced -> Proxies -> (Web proxy & Secure Web proxy) and then retry the docker pull:

$ docker pull functions/alpine:latest
latest: Pulling from functions/alpine
Digest: sha256:cdb0747993851b2eea897ff1d1e7ae0f7b086f2fbefee3f0b4e448deb075f9b6
Status: Image is up to date for functions/alpine:latest

Do you have an already-pulled container lying around (e.g. alpine)? Could you try some DNS resolutions with something like ping www.google.com?

[djs55]

Ah, you have no direct internet access at all -- not even DNS. I guess that is the problem. I'll have to think about it a bit more!

[alexellis]

If I set DNS to our internal server i.e. not 8.8.8.8 or what gets assigned by the local router then I can now pull images. This is odd.. I'm sure the settings are the same as prior to the upgrade. DNS should go through the proxy right?

[djs55]

I'm glad you can pull images now :) but I'm still confused :/

There's a DNS proxy running in vpnkit on the magic internal IP 192.168.65.1 which uses an Apple API (DNSServiceRef*) for resolving names. The idea is to use the Mac's resolver so that it should work with complicated VPN setups without having to replicate all the "zone"/"domain" stuff ourselves.

I assume you changed your DNS on the Mac with System Preferences -> Network -> Advanced -> DNS -- is that right? Does it fail again if you switch it back? If so, could you try something like dig registry-1.docker.io @<IP of server which doesn't work> on the host?

Thanks!

[alexellis]

I'm on the next version of the beta now and still having issues. When I connect remotely via the VPN I can't pull images from the Hub. The normal network DNS server entry doesn't work, Google's doesn't work and the ISP doesn't work. Until this update I'm sure DNS resolution happened through the http/s
proxy itself.

[djs55]

Hi, Thanks for the update. Is it expected that the environment has no DNS servers, with absolutely everything sent via HTTP proxies? If so to make this work again we'll have to add a non-transparent HTTP proxy to vpnkit and point the environment variables at it. The transparent proxy (that we have at the minute) requires the client to have working DNS (which is usually -- but perhaps not always -- fine) Thanks, Dave

…

On 20 Dec 2017 12:24, "Alex Ellis" ***@***.***> wrote: I'm on the next version of the beta now and still having issues. When I connect remotely via the VPN I can't pull images from the Hub. The normal network DNS server entry doesn't work, Google's doesn't work and the ISP doesn't work. Until this update I'm sure DNS resolution happened through the http/s proxy itself. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2320 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMHuh1AfI4tGHug95r5YrfTXLihsLflks5tCPxvgaJpZM4RCCvl> .

[guillaumerose]

Hello,
So you're VPN gives you a http proxy with an IP and not a name ? Every http resolution go through the proxy then ?
Guillaume

[gtardif]

Hi @alexellis, to follow-up on this, could you also let us know what version of D4Mac you were using before upgrading to 17.12.0-ce-rc2-mac41, where you did not have this issue ? was it the previous edge : 17.11.0, or stable 17.09.x (or older) ?

[alexellis]

Not over the Christmas break.. no, but in the New Year I can provide more information when back at work. Basically everything has to go through the HTTP proxy including DNS.

[djs55]

@alexellis There's an experimental build if you'd like to give it a go: https://download-stage.docker.com/mac/pr/21584/Docker.dmg . It should allow docker pull to work through the proxy. For docker build and docker run you'll need a ~/.docker/config.json which looks something like this: (maybe you already have one of these)

{
  "proxies":
  {
    "default":
    {
      "httpProxy": "http://user:[email protected]:3128",
      "httpsProxy": "http://user:[email protected]:3128",
      "noProxy": "http://user:[email protected]:3128",
      "ftpProxy": "http://user:[email protected]:3128"
    }
  }
}

Let me know how it goes.
Happy New Year!

[gcstang]

I know I'm not the original poster but I had this same issue and after using build 21584 and adding the proxies section to my config.json it worked. Does this mean we no longer put our proxy into the UI?

[alexellis]

Thanks I'll see if I can get some time at work to try this tomorrow 👍

[djs55]

@gcstang the settings are not completely unified yet. The ~/.docker/config.json is only for the CLI docker run and docker build. The setting in the UI affects the docker engine itself and activates a transparent proxy so stray HTTP on port 80/443 from containers gets transparently redirected for completeness.

I'm glad it worked for you though -- thanks for letting me know!

[gcstang]

@djs55 does that mean that the command line export settings will no longer work and will need to be set in config.json for docker run, build, pull, etc... ?

I also noticed that my install is telling me there is a newer build 21612, does that build have what you did in this version or should I hold off from upgrading?

I pulled the latest photon image and attempted to perform makecache with tdnf and it couldn't reach the internet through proxy no matter what I did internal to the container. I reverted to Docker_17.11.0-ce-mac40_20561 and everything worked properly via export HTTPS_PROXY, HTTP_PROXY in the container so it seems there are more issues with these 17.12.x builds.

[gtardif]

@alexellis if you get a chance to try this, you might want to try https://download-stage.docker.com/mac/pr/21609/Docker.dmg instead of the previously mentioned build, that will be closer to the next release.

[gcstang]

@gtardif Does that version have any other changes in it? Are there differences that 21612 doesn't have? The prior version that worked for me was edge : 17.11.0

[gtardif]

@gcstang The downloads we mentionned above are test versions and you can't rely on the auto-updates (you will get random things under development, incomplete, untested, etc.).
Build 21609 has a few improvements compared to 21584, especially it doesn't lock a port for the proxy that might be used by other apps.
After testing, you should switch back to an edge version to get auto-updates as before. Either from https://download.docker.com/mac/edge/Docker.dmg, or https://download.docker.com/mac/edge/20561/Docker.dmg for the previous edge 17.11.0

[alexellis]

Related to #2386 - CC/ @alvarow

[docker-robott]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked