-
Notifications
You must be signed in to change notification settings - Fork 597
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NIO support for TMP_DIR #4469
NIO support for TMP_DIR #4469
Conversation
@@ -358,7 +360,7 @@ private void generateDuplicateIndexes() { | |||
final int maxInMemory = (int) Math.min((Runtime.getRuntime().maxMemory() * 0.25) / SortingLongCollection.SIZEOF, | |||
(double) (Integer.MAX_VALUE - 5)); | |||
logger.info("Will retain up to " + maxInMemory + " duplicate indices before spilling to disk."); | |||
this.duplicateIndexes = new SortingLongCollection(maxInMemory, TMP_DIR.toArray(new File[TMP_DIR.size()])); | |||
this.duplicateIndexes = new SortingLongCollection(maxInMemory, (Path[]) TMP_DIR.stream().map(IOUtils::getPath).toArray()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of a cast, you can provide an argument to toArray()
, e.g. TMP_DIR.stream().map(IOUtils::getPath).toArray(Path[]::new)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
@Argument(common=true, optional=true) | ||
public List<File> TMP_DIR = new ArrayList<>(); | ||
@Argument(common=true, optional=true, doc = "List of temp directories to use.") | ||
public List<String> TMP_DIR = new ArrayList<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you kebabify this argument as part of this PR (--temp-dir
)? Looks like we missed this one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, we just did a usage check on this argument, and it's ONLY used in MarkDuplicates
. Given this, @lbergelson and I think that we should just remove this argument completely. Would you be ok with removing it @magicDGS, or are you currently depending on it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I depend on it, but I can move it to the tools that require it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@droazen Do we think a lot of users use this instead -Djava.io.tmpdir
? Maybe it's worth keeping a shortcut too? Sorry to reconsider.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was reconsidering it too (that's why I didn't add a commit), because maybe there are other codepaths using teh java property, which is set here, instead of directly using the parameter. Next commit will not have this fix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But the name was changed on StandardArgumentDefinitions.TMP_DIR_NAME
after rebase.
logger.warn("Temp directory {}: unable to set permissions due to {}", p, e.getMessage()); | ||
} | ||
// TODO - this should be p.toAbsolutePath().toUri().toString() to allow other FileSystems to be used (see above) | ||
System.setProperty("java.io.tmpdir", p.toAbsolutePath().toString()); // in loop so that last one takes effect |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lbergelson suggests a special case here for file URIs to strip the file://
prefix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in the next commit.
91cf73c
to
74bf120
Compare
Rebased to resolve conflicts and addressing comments. |
// only set the permissions if they change | ||
if (permissions.addAll(Arrays.asList( | ||
PosixFilePermission.OWNER_READ, PosixFilePermission.GROUP_READ, PosixFilePermission.OTHERS_READ, | ||
PosixFilePermission.OWNER_WRITE, PosixFilePermission.GROUP_WRITE, PosixFilePermission.OTHERS_WRITE))) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the way, this is related with #4513 - should probably resolve here.
Codecov Report
@@ Coverage Diff @@
## master #4469 +/- ##
===============================================
- Coverage 86.657% 86.552% -0.104%
+ Complexity 29049 29026 -23
===============================================
Files 1808 1809 +1
Lines 134688 134751 +63
Branches 14938 14935 -3
===============================================
- Hits 116716 116630 -86
- Misses 12559 12714 +155
+ Partials 5413 5407 -6
|
@Argument(fullName = StandardArgumentDefinitions.TMP_DIR_NAME, common=true, optional=true) | ||
public List<File> TMP_DIR = new ArrayList<>(); | ||
@Argument(fullName = StandardArgumentDefinitions.TMP_DIR_NAME, common=true, optional=true, doc = "List of temp directories to use.") | ||
public List<String> TMP_DIR = new ArrayList<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're going to keep this argument in GATK, can you make the type into a String
instead of List<String>
, since it's not doing anything intelligent right now with the List?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I modify that, it changes the behavior of MarkDuplicatesGATK
and EstimateLibraryComplexityGATK
(I guess that they are setting the temporary directory properly)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can add a separate argument to those tools if they really need to take a List of temporary directories. But having the GATK-wide argument take a List and then ignore all but the last element is confusing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True. Done.
if (permissions.addAll(Arrays.asList( | ||
PosixFilePermission.OWNER_READ, PosixFilePermission.GROUP_READ, PosixFilePermission.OTHERS_READ, | ||
PosixFilePermission.OWNER_WRITE, PosixFilePermission.GROUP_WRITE, PosixFilePermission.OTHERS_WRITE))) { | ||
Files.setPosixFilePermissions(p, permissions); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you get rid of this code to change the permissions on the temp dir, and throw a UserException
instead if the temp dir is not writable/readable?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, although in that case is a change of behavior - if the temp dir is not used at all, this can fail for every tool...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we always use the temp dir in GATK -- if for nothing else, then to extract and load the GKL library for compression/decompression.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the GKL library always extracted, even if not used at all? I don't think that it is (or should be) like that...
* @param path path to get the absolute name. | ||
* @return a String with the absolute name. | ||
*/ | ||
public static String getAbsolutePathName(final Path path) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The name of this method doesn't really tell you what it's doing -- can you rename to getAbsolutePathWithoutFileProtocol()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* the file:// prefix. | ||
* | ||
* @param path path to get the absolute name. | ||
* @return a String with the absolute name. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"a String with the absolute name, and the file:// protocol removed, if it was present"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
try { | ||
Files.createDirectories(p); | ||
} catch (IOException e) { | ||
// intentionally ignoring |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't create the temp dir if it doesn't exist -- just throw a UserException
in this case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I personally like specifying a non-existent directory for temporary files that is created by the program instead of explicitly using mkdirs
- anyway, I can change in the next commit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed in the next commit.
public void testGetAbsolutePathName(final String uriString, final String expected) { | ||
Assert.assertEquals(IOUtils.getAbsolutePathName(IOUtils.getPath(uriString)), expected); | ||
} | ||
|
||
@Test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a unit test to prove that the temp dir works over NIO?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which kind of test do you need there? Retrieving the property and writing to it? Or running tools using it with NIO (e.g., MarkDuplicatesGATK
)? Let me know for next round of comments...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about a test that:
-
Declares a dummy
CommandLineProgram
that executescreateTempFile()
and then asserts that it exists in itsdoWork()
method -
Runs the above command line program with
--tmp-dir
set to a jimfs URI. You can create a jimfs like this:
try (FileSystem jimfs = Jimfs.newFileSystem(Configuration.unix())) {
final Path tmpDirPath = jimfs.getPath("tmp");
- You should also include a test case that runs the dummy
CommandLineProgram
with--tmp-dir
set to afile://
URI, since that is a different code path.
You could put the test in a new CommandLineProgramIntegrationTest
class
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which createTempFile
should be use in the test? There are several implementations (htsjdk, gatk, Files
, File
), and all of them have their own peculiarities. The ones that work with a File
object won't work with this change. Thus, I am not sure how to proceed here - for the tools using the tmpDir this should work because the codebase uses the htsjdk Path
implementation, but otherwise I am not sure how to get this done well...
On the other hand, there is a test class already for CLP (CommandLineProgramUnitTest
) - should I still create the *IntegrationTest
class or just use that one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Second review complete, back to @magicDGS for changes
Back to you, still without tests for tmp dir working with NIO due to lack of information: should it be at the tool level (e.g., running |
Friendly ping here @droazen |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some more comments, back to you @magicDGS !
return new Object[][] { | ||
{"/local/example.txt", "/local/example.txt"}, | ||
{"/local/file://example.txt", "/local/file:/example.txt"}, | ||
{"file:///local/example.txt", "/local/example.txt"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a test case with a non-file URI (such as gcs://
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess that you mean adding the gs://
example, that is the prefix in GCS. Otherwise it fails because IOUtils.getPath
does not recognize it as a non-file URI.
Done and test passing.
if (permissions.addAll(Arrays.asList( | ||
PosixFilePermission.OWNER_READ, PosixFilePermission.GROUP_READ, PosixFilePermission.OTHERS_READ, | ||
PosixFilePermission.OWNER_WRITE, PosixFilePermission.GROUP_WRITE, PosixFilePermission.OTHERS_WRITE))) { | ||
Files.setPosixFilePermissions(p, permissions); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we always use the temp dir in GATK -- if for nothing else, then to extract and load the GKL library for compression/decompression.
public void testGetAbsolutePathName(final String uriString, final String expected) { | ||
Assert.assertEquals(IOUtils.getAbsolutePathName(IOUtils.getPath(uriString)), expected); | ||
} | ||
|
||
@Test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about a test that:
-
Declares a dummy
CommandLineProgram
that executescreateTempFile()
and then asserts that it exists in itsdoWork()
method -
Runs the above command line program with
--tmp-dir
set to a jimfs URI. You can create a jimfs like this:
try (FileSystem jimfs = Jimfs.newFileSystem(Configuration.unix())) {
final Path tmpDirPath = jimfs.getPath("tmp");
- You should also include a test case that runs the dummy
CommandLineProgram
with--tmp-dir
set to afile://
URI, since that is a different code path.
You could put the test in a new CommandLineProgramIntegrationTest
class
if (this.TMP_DIR == null) this.TMP_DIR = new ArrayList<>(); | ||
if (this.TMP_DIR.isEmpty()) TMP_DIR.add(IOUtil.getDefaultTmpDir()); | ||
// TODO - this should use the HTSJDK IOUtil.getDefaultTmpDirPath, which is somehow broken in the current HTSJDK version | ||
if (TMP_DIR == null || TMP_DIR.isEmpty()) TMP_DIR = IOUtils.getAbsolutePathWithoutFileProtocol(IOUtils.getPath(System.getProperty("java.io.tmpdir"))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curly braces around if statement body
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leftover from previous implementation. Changed as it is clearer.
@Argument(fullName = StandardArgumentDefinitions.TMP_DIR_NAME, common=true, optional=true) | ||
public List<File> TMP_DIR = new ArrayList<>(); | ||
@Argument(fullName = StandardArgumentDefinitions.TMP_DIR_NAME, common=true, optional=true, doc = "Temp directory to use.") | ||
public String TMP_DIR; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you rename this variable to tmpDir
(but keep the argument name itself as-is)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
// TODO: it may be that the program does not need a tmp dir | ||
// TODO: if it fails, the problem can be discovered downstream | ||
// TODO: should log a warning instead? | ||
throw new UserException.BadInput(String.format("--%s (%s) should exists and have read/write access", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use UserException.BadTmpDir
instead (here and below)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, "should exists" should be "should exist"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@magicDGS You probably want to rebase before making this change since I just made some changes to BadTmpDir
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using BadTmpDir
as currently implemented is not really pointing out where the error is: if the property was not set, and fails on checking the access it will point to a different tmp directory. I added a new constructor for the exception with a Path
to provide the one in use.
Done.
73a98f8
to
407c209
Compare
@droazen - Still some questions about integration tests (on the comment with your suggestion, but here too):
Waiting for your feedback before doing something that does not make sense... |
@magicDGS Could you add overloads of |
407c209
to
280da96
Compare
Sorry for the delay here, @droazen - I had some technical problems with my hardware and did not have time to work on this. Back to you with implemented tests! |
@magicDGS Could you rebase this branch one last time? Once it's been rebased, I'll give it a final look and (hopefully) hit merge. Thanks! |
280da96
to
be00971
Compare
Rebased @droazen - back to you! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@magicDGS This PR looks good now. I just had a few remaining comments on the test code, then we should be able to merge.
@@ -0,0 +1,98 @@ | |||
/* | |||
* The MIT License (MIT) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove the license from this file. GATK4 has a repository-wide BSD 3-clause license.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed (sorry, my IDE attach sometimes old LICENSE config...)
/** | ||
* @author Daniel Gomez-Sanchez (magicDGS) | ||
*/ | ||
public class CommandLineProgramIntegrationTest extends GATKBaseTest { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Integration test classes should extend CommandLineProgramTest
rather than GATKBaseTest
directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
*/ | ||
public class CommandLineProgramIntegrationTest extends GATKBaseTest { | ||
|
||
private static FileSystem jimfs; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Create the Jimfs
filesystem within your test case rather than at the class level. Static variables in test code tend to cause problems down the line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found a solution by creating a new method in GATKBaseTest
to close Jimfs
for a Path
. I refactored to remove this static field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For whatever reason, that method fails - it looks like GATK's IOUtil.getPath
does not recognize Jimfs
if the instance is created in the test method. Again, no action here.
public Object[][] tmpDirs() throws Exception { | ||
return new Object[][]{ | ||
{createTempDir("local").toPath()}, | ||
{Files.createDirectory(jimfs.getPath("tmp"))} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be 3 test cases here rather than 2 in order to cover all the branches in your code: a local directory without a file://
scheme specified, a local directory with file://
specified, and a Jimfs URI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, the one with the file://
prefix is already tested - in the test, the argument is converted with Path.toUri().toString()
, which adds the file://
. I refactored the method to has the String
argument directly, and added a test case for a local file without prefix.
} | ||
|
||
@Test(dataProvider = "tmpDirs", singleThreaded = true) | ||
public void testCreateTempPath(final Path tmpDir) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test should be named something like testTmpDirArgument
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
public Object[][] absoluteNames() { | ||
return new Object[][] { | ||
{"/local/example.txt", "/local/example.txt"}, | ||
{"/local/file://example.txt", "/local/file:/example.txt"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a strange one... can you explain why "/local/file://example.txt"
becomes "/local/file:/example.txt"
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Due to normalization of the path (removal of redundant /
) - I added a comment here to clarify for the future.
|
||
final Path p = (Path) new TestCreateTempPathClp().instanceMain(new String[]{ | ||
"--" + StandardArgumentDefinitions.TMP_DIR_NAME, tmpDir.toUri().toString()}); | ||
Assert.assertTrue(Files.exists(p)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also assert that the java.io.tmpdir
property has been changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
* | ||
* @return A file in the temporary directory starting with name, ending with extension, which will be deleted after the program exits. | ||
*/ | ||
public static Path createTempPath(String name, String extension) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to have a standalone unit test for this method in IOUtilsUnitTest
(since the test in CommandLineProgramIntegrationTest
is testing the --tmp-dir
argument rather than this method directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
@@ -166,6 +166,21 @@ private void innerTestGetPath(String s) throws IOException { | |||
Assert.assertTrue(size>0); | |||
} | |||
|
|||
@DataProvider | |||
public Object[][] absoluteNames() { | |||
return new Object[][] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also include a test case with a non-absolute path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Assert.assertTrue(Files.exists(p)); | ||
|
||
// get back to the previous tmp dir | ||
System.setProperty("java.io.tmpdir", previousTmpDir); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also put this in a finally
clause to be sure that after the test it always resets the previous tmp directory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A small number of remaining comments on the tests. Back to @magicDGS.
try { | ||
final Path p = (Path) new TestCreateTempPathClp().instanceMain(new String[] { | ||
"--" + StandardArgumentDefinitions.TMP_DIR_NAME, tmpDirArg}); | ||
Assert.assertTrue(Files.exists(p)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should also assert here that the parent directory of p is equal to the tmpDirArg
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
final Path p = (Path) new TestCreateTempPathClp().instanceMain(new String[] { | ||
"--" + StandardArgumentDefinitions.TMP_DIR_NAME, tmpDirArg}); | ||
Assert.assertTrue(Files.exists(p)); | ||
Assert.assertNotEquals(System.getProperty("java.io.tmpdir"), previousTmpDir); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally you would check here that java.io.tmpdir
has been set to tmpDirArg
, not just that it's different from previousTmpDir
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
/** | ||
* Closes the {@link java.nio.file.FileSystem} for a path if it is an instance from {@link Jimfs} | ||
*/ | ||
public static void closeFileSystemIfJimnfs(final Path path) throws IOException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure this method should be up in GATKBaseTest
, since you reported that you weren't able to use it in CommandLineProgramIntegrationTest
. You should use the same method to create the Jimfs instances across your different test classes. At this point I'd be satisfied if you just deleted this method, and used the @BeforeClass
/@AfterClass
approach to set up the Jimfs in IOUtilsUnitTest
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
*/ | ||
public class CommandLineProgramIntegrationTest extends CommandLineProgramTest { | ||
|
||
private static FileSystem jimfs; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you make this non-static?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
@droazen - I also rebased locally to check the conflicting files: the differences are a new constant ( |
@droazen - friendly ping here! |
@magicDGS It looks like all of my comments here have been addressed, but the branch is in conflict. If you do a last rebase I can (finally) hit merge! |
28d90b5
to
0936dfb
Compare
Rebased - waiting for test passing and hopefully you can hit merge @droazen! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Looks good! Thanks for your patience across the many review iterations of this branch.
Thanks for the review @droazen! |
Extracted from #3998