Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] #9045

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open

[DO NOT MERGE] #9045

Changes from 3 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
9ca31cb
first attempt at a changed environment
jamesemery Nov 12, 2024
1cf15f9
updating pytorch to newer version that carries some risk
jamesemery Nov 15, 2024
10acfbd
gore
jamesemery Nov 15, 2024
2cf671b
back to basics
jamesemery Nov 15, 2024
0eeb8b6
uncommenting some
jamesemery Nov 15, 2024
6be3e4d
trying two more dependencies
jamesemery Nov 15, 2024
a0e5dbb
enabling two more
jamesemery Nov 15, 2024
966333d
was it cyvcf?
jamesemery Nov 18, 2024
090dfef
mmninga?
jamesemery Nov 18, 2024
c4ed905
another minor alteration of python environment
jamesemery Nov 19, 2024
788f442
bioconda cyvcf2 is probably a conflict
jamesemery Nov 19, 2024
e99aeee
bioconda cyvcf2 is probably a conflict
jamesemery Nov 19, 2024
edba11b
Reverting to 2.1.0
jamesemery Dec 4, 2024
701fc34
trying to bundle the cuda installation via conda
jamesemery Dec 9, 2024
9a81630
pairing up the correct cuda compatable pytroch version
jamesemery Dec 9, 2024
265eafc
change the channel
jamesemery Dec 9, 2024
49221aa
testing with only cuda backend?
jamesemery Dec 9, 2024
bf01960
change the channel
jamesemery Dec 9, 2024
b953413
testing if the other mkl invocation was the problem
jamesemery Dec 10, 2024
bfefb7b
testing if the other mkl invocation was the problem
jamesemery Dec 10, 2024
ca5809f
finally! python has lead us from gods light
jamesemery Dec 10, 2024
4235198
finally! python has lead us from gods light
jamesemery Dec 10, 2024
684d0ea
this should probably work
jamesemery Dec 10, 2024
8d66d26
put the code in the right place, now need to build a pythonscriptexec…
jamesemery Dec 19, 2024
5914af0
hopefully what is needed for docker building proper
jamesemery Jan 10, 2025
1b66e04
did this really stop working?
jamesemery Jan 10, 2025
102a225
python installation is dizzying
jamesemery Jan 10, 2025
1d685a2
moving the cuda drivers into the right stage of the build
jamesemery Jan 10, 2025
705b5e0
names have power
jamesemery Jan 13, 2025
094b42e
fixing a silly mistake
jamesemery Jan 13, 2025
578c7b1
made a somewhat robust mechanism for converting to the python argumen…
jamesemery Jan 15, 2025
7e85678
some more script editing
jamesemery Jan 16, 2025
d9f9c40
some more script editing
jamesemery Jan 16, 2025
aaee681
added the next few tasks from the main wdl
jamesemery Jan 17, 2025
15644f2
added the next few tasks from the main wdl
jamesemery Jan 17, 2025
36ae7f2
added docstore bindings to the permutect wdls
jamesemery Jan 21, 2025
f78d4a1
maybe the branches are the problem
jamesemery Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions scripts/gatkcondaenv.yml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,21 @@ dependencies:
- conda-forge::pytensor=2.18.3
- conda-forge::scipy=1.11.4
- conda-forge::h5py=3.10.0
- conda-forge::pytorch=2.1.0=*mkl*100
- conda-forge::pytorch=2.3.0=*mkl*100
- conda-forge::pytorch-lightning=2.4.0 # supports Pytorch >= 2.1 and <= 2.4, used by NVScoreVariants
- conda-forge::tensorboard=2.8.0
- conda-forge::scikit-learn=1.3.2
- conda-forge::matplotlib=3.8.2
- conda-forge::pandas=2.1.3
- conda-forge::tqdm=4.66.1
- conda-forge::dill=0.3.7 # used for pickling lambdas in TrainVariantAnnotationsModel
- conda-forge::biopython=1.84 # used by NVScoreVariants
- conda-forge::setuptools>=57.0.0
- conda-forge::psutil>=5.9.2
- conda-forge::protobuf<3.20,>=3.9.2 # Protobuf constraint for TensorFlow compatibility
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we get rid of tensorflow?

Copy link
Collaborator Author

@jamesemery jamesemery Nov 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea. Its in the dependencies for the other project:

# Protobuf 4.0 is incompatible with TF. Force < 3.20 until they unblock upgrade.
# See: http://b/182876485
# See: https://github.com/protocolbuffers/protobuf/issues/9954#issuecomment-1128283911
# See: https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/tools/pip_package/setup.py?q=protobuf
protobuf >= 3.9.2, < 3.20

- conda-forge::intervaltree~=3.1.0
- conda-forge::cyvcf2~=0.30.15
- conda-forge::mmap-ninja>=0.2.4

# core R dependencies; these should only be used for plotting and do not take precedence over core python dependencies!
- r-base=4.3.1
Expand All @@ -55,4 +62,4 @@ dependencies:

# pip installs should be avoided, as pip may not respect the dependencies found by the conda solver
- pip:
- gatkPythonPackageArchive.zip
- gatkPythonPackageArchive.zip
Loading