Fix problems with copying files and Docker-Compose on Windows #514

kiview · 2017-12-10T15:38:06Z

Copying files into the container did not work on Docker for Windows.
Also the docker socket needs to be mounted differently into the docker-compose container on Windows.

UNIX path separator needs to be used when building docker-compose env file path for containerized docker-compose.

For docker socket mounting on Windows, the socket path must be prefixed with an additional "/"

kiview · 2017-12-10T16:55:53Z

Core will now build on Windows.

bsideup · 2017-12-11T09:23:44Z

core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java

@@ -10,6 +10,7 @@
 import com.google.common.util.concurrent.Uninterruptibles;
 import org.apache.commons.lang.StringUtils;
 import org.apache.commons.lang.SystemUtils;
+import org.jetbrains.annotations.NotNull;


I would rather not use JB's annotations in our code base. Adds another dependency

Oh that was not intended, wonder how it got in, thanks for spotting it.

We are already using in in multiple places btw.

➜ testcontainers-java git:(docker-for-windows-compat) ✗ grep -r "import org.jetbrains.annotations.NotNull;" * core/src/main/java/org/testcontainers/dockerclient/WindowsClientProviderStrategy.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/dockerclient/AuditLoggingDockerClient.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/dockerclient/UnixSocketClientProviderStrategy.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/utility/MountableFile.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/utility/AuditLogger.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/utility/CommandLine.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/utility/ComparableVersion.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/containers/GenericContainer.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/containers/FixedHostPortGenericContainer.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/containers/VncRecordingSidekickContainer.java:import org.jetbrains.annotations.NotNull; core/src/main/java/org/testcontainers/containers/startupcheck/MinimumDurationRunningStartupCheckStrategy.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/junit/wait/HttpWaitStrategyTest.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/junit/wait/AbstractWaitStrategyTest.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/junit/wait/LogMessageWaitStrategyTest.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/junit/wait/HostPortWaitStrategyTest.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/junit/BaseDockerComposeTest.java:import org.jetbrains.annotations.NotNull; core/src/test/java/org/testcontainers/utility/MountableFileTest.java:import org.jetbrains.annotations.NotNull; modules/selenium/src/main/java/org/testcontainers/containers/BrowserWebDriverContainer.java:import org.jetbrains.annotations.NotNull; modules/selenium/src/test/java/org/testcontainers/junit/BaseWebDriverContainerTest.java:import org.jetbrains.annotations.NotNull; modules/postgresql/src/main/java/org/testcontainers/containers/PostgreSQLContainer.java:import org.jetbrains.annotations.NotNull; modules/mysql/src/main/java/org/testcontainers/containers/MySQLContainer.java:import org.jetbrains.annotations.NotNull; modules/jdbc/src/main/java/org/testcontainers/containers/JdbcDatabaseContainer.java:import org.jetbrains.annotations.NotNull; modules/nginx/src/main/java/org/testcontainers/containers/NginxContainer.java:import org.jetbrains.annotations.NotNull;

I don't really care about using it in this place, shall I remove it?

I lose track of what the best No[nt]Null annotation is 😞
I don't think there's any harm in leaving this here, and if we want to get rid of the JetBrain annotation altogether we could do it everywhere at once.

I removed it, since I don't think it added anything useful.

bsideup · 2017-12-11T09:24:33Z

core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java

@@ -395,6 +396,10 @@ default void validateFileList(List<File> composeFiles) {
 * Use Docker Compose container.
 */
 class ContainerisedDockerCompose extends GenericContainer<ContainerisedDockerCompose> implements DockerCompose {
+
+    private static final String DOCKER_SOCKET_PATH = "//var/run/docker.sock";


should not start with double / (you add it in getDockerSocketHostPath)

You are right, still works in Linux interestingly ^^

well, on *nix systems, you can use as many / as you want :D

$ ls -la /////usr total 0 drwxr-xr-x@ 10 root wheel 320 Sep 27 08:04 . drwxr-xr-x 31 root wheel 992 Nov 30 13:02 .. drwxr-xr-x 976 root wheel 31232 Nov 30 13:02 bin drwxr-xr-x 263 root wheel 8416 Nov 16 11:38 include drwxr-xr-x 311 root wheel 9952 Nov 30 13:02 lib drwxr-xr-x 235 root wheel 7520 Nov 30 13:02 libexec drwxr-xr-x 15 root wheel 480 Sep 26 22:58 local drwxr-xr-x 246 root wheel 7872 Nov 30 12:51 sbin drwxr-xr-x 47 root wheel 1504 Sep 27 08:04 share drwxr-xr-x 5 root wheel 160 Sep 21 06:32 standalone

rnorth · 2017-12-11T19:52:52Z

core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java

@@ -10,6 +10,7 @@
 import com.google.common.util.concurrent.Uninterruptibles;
 import org.apache.commons.lang.StringUtils;
 import org.apache.commons.lang.SystemUtils;
+import org.jetbrains.annotations.NotNull;


I lose track of what the best No[nt]Null annotation is 😞
I don't think there's any harm in leaving this here, and if we want to get rid of the JetBrain annotation altogether we could do it everywhere at once.

For docker socket mounting on Windows, the socket path must be prefixed with an additional "/". Also used Windows style paths when interacting with files for Docker for Windows compatibility.

Bumps [influxdb-java](https://github.com/influxdata/influxdb-java) from 2.10 to 2.14. <details> <summary>Changelog</summary> *Sourced from [influxdb-java's changelog](https://github.com/influxdata/influxdb-java/blob/master/CHANGELOG.md).* > ## 2.14 [2018-10-12] > > ### Fixes > > - Fixed chunked query exception handling [Issue #523](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/523) > - Memory leak in StringBuilder cache for Point.lineprotocol() [Issue #526](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/521) > > ## 2.13 [2018-09-12] > > ### Fixes > - MessagePack queries: Exception during parsing InfluxDB version [macOS] [PR #487](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/487) > - The InfluxDBResultMapper is able to handle results with a different time precision [PR #501](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/501) > - UDP target host address is cached [PR #502](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/502) > - Error messages from server not parsed correctly when using msgpack [PR #506](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/506) > - Response body must be closed properly in case of JSON response [PR #514](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/514) > - Time is serialized not consistently in MsgPack and Json, missing millis and nanos in MsgPack[PR #517](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/517) > > ### Features > > - Support for Basic Authentication [PR #492](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/492) > - Added possibility to reuse client as a core part of [influxdb-java-reactive](https://github.com/bonitoo-io/influxdb-java-reactive) client [PR #493](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/493) > - Retry capability for writing of BatchPoints [PR #503](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/503) > - Added `BiConsumer` with capability to discontinue a streaming query [Issue #515](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/515) > - Added `onComplete` action that is invoked after successfully end of streaming query [Issue #515](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/515) > > ## 2.12 [2018-07-31] > > ### Fixes > > - Remove code which checks for unsupported influxdb versions [PR #474](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/474) > - Unpredictable errors when OkHttpClient.Builder instance is reused [PR #478](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/478) > > ### Features > > - Support for MessagePack [PR #471](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/471) > - Cache version per influxdb instance and reduce ping() calls for every query call [PR #472](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/472) > - FAQ list for influxdb-java [PR #475](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/475) > > ### Improvements > > - Test: Unit test to ensure tags should be sorted by key in line protocol (to reduce db server overheads) [PR #476](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/476) > > ## 2.11 [2018-07-02] > > ### Features > > - Allow write precision of TimeUnit other than Nanoseconds [PR #321](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/321) > - Support dynamic measurement name in InfluxDBResultMapper [PR #423](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/423) > - Debug mode which allows HTTP requests being sent to the database to be logged [PR #450](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/450) > - Fix problem of connecting to the influx api with URL which does not points to the url root (e.g. localhots:80/influx-api/) [PR #400] (https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/400) ></table> ... (truncated) </details> <details> <summary>Commits</summary> - [`91d0f09`](influxdata/influxdb-java@91d0f09) [maven-release-plugin] prepare release influxdb-java-2.14 - [`8ffaeb9`](influxdata/influxdb-java@8ffaeb9) Revert "[maven-release-plugin] prepare release influxdb-java-2.14" - [`2781da2`](influxdata/influxdb-java@2781da2) [maven-release-plugin] prepare release influxdb-java-2.14 - [`19c69ed`](influxdata/influxdb-java@19c69ed) [maven-release-plugin] prepare for next development iteration - [`c6d7f25`](influxdata/influxdb-java@c6d7f25) [maven-release-plugin] prepare release influxdb-java-2.14 - [`2f4c594`](influxdata/influxdb-java@2f4c594) Merge pull request [#531](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/531) from heshengbang/master - [`f653e62`](influxdata/influxdb-java@f653e62) Easy to use try-with-resources, add README.md - [`c7be9b0`](influxdata/influxdb-java@c7be9b0) Easy to use try-with-resources - [`4590d18`](influxdata/influxdb-java@4590d18) - added automated SNAPSHOT publishing to Maven Central repository - [`ce65a41`](influxdata/influxdb-java@ce65a41) - added automated SNAPSHOT publishing to Maven Central repository - Additional commits viewable in [compare view](influxdata/influxdb-java@influxdb-java-2.10...influxdb-java-2.14) </details> <br /> [![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=org.influxdb:influxdb-java&package-manager=gradle&previous-version=2.10&new-version=2.14)](https://dependabot.com/compatibility-score.html?dependency-name=org.influxdb:influxdb-java&package-manager=gradle&previous-version=2.10&new-version=2.14) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- **Note:** This repo was added to Dependabot recently, so you'll receive a maximum of 5 PRs for your first few update runs. Once an update run creates fewer than 5 PRs we'll remove that limit. You can always request more updates by clicking `Bump now` in your [Dependabot dashboard](https://app.dependabot.com). <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme Additionally, you can set the following in the `.dependabot/config.yml` file in this repo: - Update frequency (including time of day and day of week) - Automerge options (never/patch/minor, and dev/runtime dependencies) - Pull request limits (per update run and/or open at any time) - Out-of-range updates (receive only lockfile updates, if desired) - Security updates (receive only security updates, if desired) Finally, you can contact us by mentioning @dependabot. </details>

trajano · 2019-01-04T02:55:06Z

core/src/main/java/org/testcontainers/containers/DockerComposeContainer.java

        addEnv("DOCKER_HOST", "unix:///docker.sock");
        setStartupCheckStrategy(new IndefiniteWaitOneShotStartupCheckStrategy());
        setWorkingDirectory(containerPwd);
    }

+    private String getDockerSocketHostPath() {
+        return SystemUtils.IS_OS_WINDOWS


I think this is making an assumption that we are only using docker via their sockets interface. However in Docker Machine setups this may not be available It should be able to use the DOCKER_HOSTand DOCKER_CERT_PATH and DOCKER_TLS values.

I just checked master it appears to make an assumption that the socket file does exist.

kiview added 4 commits December 10, 2017 16:13

Fix problems with copying files and Docker-Compose on Windows

aa9357a

For docker socket mounting on Windows, the socket path must be prefixed with an additional "/"

Force git to checkout *.sh files with LF line endings

43faf02

Use UNIX path separator for env files in ContainerisedDockerCompose

029c65d

Update CHANGELOG

f4986ae

Use qualified imports to avoid codacy warning

38a35d1

kiview requested review from rnorth and bsideup December 10, 2017 16:59

bsideup reviewed Dec 11, 2017

View reviewed changes

bsideup added the client/docker-for-windows label Dec 11, 2017

Remove duplicate leading '/' in docker socket path for compose

ae906cb

rnorth approved these changes Dec 11, 2017

View reviewed changes

rnorth added this to the 1.5.0 milestone Dec 11, 2017

Remove @NotNull annotation

4b5f861

kiview merged commit 87b8501 into master Dec 11, 2017

kiview deleted the docker-for-windows-compat branch December 16, 2017 20:07

kiview mentioned this pull request Feb 26, 2018

Resolve issues with Docker Compose when running under Docker for Windows #306

Closed

trajano reviewed Jan 4, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix problems with copying files and Docker-Compose on Windows #514

Fix problems with copying files and Docker-Compose on Windows #514

kiview commented Dec 10, 2017 •

edited

Loading

kiview commented Dec 10, 2017

bsideup Dec 11, 2017

kiview Dec 11, 2017 •

edited

Loading

kiview Dec 11, 2017

rnorth Dec 11, 2017

kiview Dec 11, 2017

bsideup Dec 11, 2017

kiview Dec 11, 2017

bsideup Dec 11, 2017

rnorth Dec 11, 2017

trajano Jan 4, 2019 •

edited

Loading

Fix problems with copying files and Docker-Compose on Windows #514

Fix problems with copying files and Docker-Compose on Windows #514

Conversation

kiview commented Dec 10, 2017 • edited Loading

kiview commented Dec 10, 2017

Choose a reason for hiding this comment

kiview Dec 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trajano Jan 4, 2019 • edited Loading

Choose a reason for hiding this comment

kiview commented Dec 10, 2017 •

edited

Loading

kiview Dec 11, 2017 •

edited

Loading

trajano Jan 4, 2019 •

edited

Loading