Join the team (post here to be added to the bioconda team)

[johanneskoester]

Everybody is welcome to contribute a package. Simply reply to this issue and you will be added to the bioconda team.

Edit: After you post here, we will email you an invitation through github to join the bioconda team. Click the link in that invitation in order to be added.

Edit: To have the Bioconda logo display in your profile, navigate to https://github.com/orgs/bioconda/people and find yourself. Click on 'Private' and select 'Public'.

[hyeshik]

Johannes,
This is a great idea! I often felt the need of an OS-independent package catalogue for computational biology that works within the user space. After few minutes of trial with Conda, I could agree that it's a wonderful platform for such system. I would like to contribute packages that I use as one in the team. I have an experience as a package maintainer (so-called ports committer) in the FreeBSD operating system for ~5 years in the early 2000s.

[johanneskoester]

Great! I have send you an invitation.

[daler]

Hi Johannes -
I'd like like help with this, too. I like the idea of having one obvious place to find and contribute bio-related conda packages. I already have a handful of conda packages that could be useful here (https://conda.binstar.org/daler).

[johanneskoester]

Hi Ryan!
Great to hear that! I have sent you an invitation. Feel free to add your packages!

[johanneskoester]

Btw Hyeshik, please also create an anaconda.org account so I can add you to the bioconda team over there, as well.

[daler]

FYI, https://github.com/chapmanb/bcbio-conda has a lot of conda packages already. I'm not sure how much of that is tied specifically to versions needed in the bcbio-* packages though.

[johanneskoester]

Good to know! Maybe we can invite Brad to migrate their stuff once we have reached a critical mass with Bioconda.

[hyeshik]

@johanneskoester Thank you! I just created an account on anaconda.org.

[johanneskoester]

Great. Added you!

[hyeshik]

@johanneskoester I seems that I don't have a permission to write the recipes github repo. Can I get one? Thank you!

[johanneskoester]

Of course. Sorry for that, does it work now?

[hyeshik]

It works. Thank you!

[dkoppstein]

@johanneskoester, I am enthusiastic about this idea as well, especially since I've been using wonky Makefiles to deal with software environments for my company =) I'd love to contribute in any way possible, although I don't have much experience with conda apart from being an end-user. My anaconda.org account is dkoppstein.

[johanneskoester]

Hi,
glad to hear that you want to join us, thanks! Conda packaging is really easy. Basically, it is just some metadata plus a shell script with the commands you would use if you install a tool manually.

[chapmanb]

Johannes;
This is a great initiative, thank you for putting it together. I'd love to contribute as well. We have a lot of packages prepared for bcbio dependencies and could happily move over to more community driven packaging:

https://anaconda.org/bcbio

It may also be worth getting in touch with the CGAT folks, who have a wide variety of conda tools as well:

https://anaconda.org/cgat

What Linux platform do you target builds for? In bcbio we need to support people running on older platforms so build everything in a CentOS5 docker container. We have a ready to run script that does this, only re-building new recipes that are missing from anaconda.org:

https://github.com/chapmanb/bcbio-conda#readme

Thanks again for organizing this.

[johanneskoester]

Hi Brad,
I'm very glad you want to join. We already thought about contacting you. Great work with the bcbio. So, feel free to move stuff over!
Regarding the builds, that is a very good question. So far, my impression was that if we target e.g. linux-64 and include all dependencies for build and run as conda packages, the resulting builds should be independent of the underlying linux platform. E.g. conda is shipping its own libstdc++ etc, right?
Am I missing something here?

Best,
Johannes

[chapmanb]

Johannes;
Thanks so much for including me. Regarding builds, unfortunately it is not that isolated. You'll compile against system packages which will cause failures on different systems. glibc is the most common cause of these. For instance, the current bedtools build fails to run on CentOS6:

~/test/anaconda/bin/bedtools 
/cm/shared/apps/bcbio/20141204-devel/data/anaconda/bin/bedtools: /lib64/libc.so.6: version `GLIBC_2.15' not found (required by /cm/shared/apps/bcbio/20141204-devel/data/anaconda/bin/bedtools)
/cm/shared/apps/bcbio/20141204-devel/data/anaconda/bin/bedtools: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /cm/shared/apps/bcbio/20141204-devel/data/anaconda/bin/bedtools)

This is the same issue you're seeing in #2.

The anaconda folks build their packages on CentOS5 which avoids most of these issues (although they still do get system-specific things -- the curl package doesn't work on non-RedHat systems due to certificate differences). With a CentOS5 based automated build you can avoid the most common compatibility issues and I haven't had any problems with portability of packages so far. I could replicate the setup we have in bcbio-conda here if that would help.

[johanneskoester]

Hi Brad,
thanks for the insight! But what about packages like glibc. If you manage to build against that, wouldn't bedtools work on all platforms, regardless of the system glibc?

[johanneskoester]

Oh, I have just seen that this package just seems to copy the system libs. Strange...
Ok, in this case, it would be great if you could replicate your CentOS setup for us, thank you!

[johanneskoester]

Alternatively, we could try to get the binstar builds up and running. I experimented a bit with them, but I never managed to trigger a build. I can only submit them, but they don't seem to be assigned to a build queue.

[dkoppstein]

@johanneskoester, do you envision this being Linux-64 only or also Mac/Windows? If the latter, perhaps it makes sense to try to appropriate funds for organization status so we can use those types of build nodes (assuming we can get binstar to work).

FWIW I'm totally fine with it being Linux-64 only.

[johanneskoester]

I think linux is most important. I have uploaded some MacOS X builds because I need them for a Snakemake tutorial I am giving. But in general, I would say we should start with Linux and see where the journey takes us.

I would propose the following plan:

1. Build for linux-64/CentOS 5 with Brad's setup.
2. Once the automated binstar builds work, build for linux-64 via the binstar queue.
3. Depending on user requests, decide at some point if there is a market for extending builds to MacOS X.

I think I won't go for Windows, because many tools don't support it at all. I also don't want to encourage people to use Win for bioinformatics ;-).

[chapmanb]

Johannes and David;
Thanks for the thoughts on the build ideas. I pushed scripts which we can use for building and uploading packages on Linux and MacOSX. For Linux, it uses a CentOS 5 docker container which will provide hopefully widely compatible binaries. For MacOSX, we should just build directly on a Mac machine but this will require marking tools that don't build on Mac, or where the recipe uses Linux binaries.

I would love to eventually use the anaconda.org builds, but explored this a year ago and didn't have any luck getting it set up, even with offering to pay for build boxes. I got the feeling it's still under development on their side but the situation could have changed in the interim.

Let me know if you have any problems running the scripts, I'm happy to improve the docs or scripts as needed. Thanks again.

[tomkinsc]

I would like to join!

[sauloal]

Hey,

I think reproducibility is a hot topic now. We are also working on a similar project: https://github.com/BioDocker/biodocker

Apparently there's a huge double work here (and we are not the only ones).

It would be interesting to partner and for sure we could install your conda packages inside our docker images instead of downloading from source, reducing the download size.

Also, could you please tell me what are the advantages of conda over docker? I've personally never used it but the multiplatform nature and no "mount/port forward/etc" crap seem very nice.

regards

[daler]

The biggest disadvantage for me personally is that Docker is not allowed on our HPC cluster! In contrast, conda installs everything in my home directory without the need for any elevated privileges.

Conda is mostly for installing executables and libraries. You can't do things like run an isolated mysql server using conda. But that's exactly the sort of thing docker is good at.

While conceptually there's a lot of overlap between the projects, it looks like there's not much domain overlap yet: biodocker has lots of proteomic packages while bioconda has lots of sequencing packages. One way to minimize duplicated work while taking advantage of both projects would be to 1) port existing dockerfiles into conda packages under bioconda and then 2) pull conda packages into docker containers built under biodocker.

[sauloal]

@daler I see. Thank you very much. I was also struggling on how to run the programs on our HPC too. this is great.

BTW. indeed the initial creators are from the proteomics field but I'm from the genomics and I've started porting several programs:
https://github.com/BioDocker/sandbox

I really like the idea to create packages here and installing them inside docker. makes the docker images lighter and the same package can be used inside docker, outside docker and in HPC. 3 in one.

I like this idea very much. We'll keep talking.

[percyfal]

Hi Johannes,

great initiative, I'd be pleased to chip in. I recently started using conda and I haven't gotten round to learning how to build packages, but looking at your recipes should help.

Cheers,

Per

[baraaorabi]

Hello! I am a grad student at @vpc-ccg. Can I join the team? Thanks!

[epruesse]

Welcome @baraaorabi! :)

[epruesse]

And welcome @Rakshith-Manandi - sorry about the mixup! :)

[sjanssen2]

Hallo team. I am trying to contribute my first PR #14233 , but tests are failing. I could need some help please. Local building works.
SEPP places short sequences into a larger reference phylogeny. We use that to compute trees for 16S amplicon data via Qiime2.

[roshni-b]

I would like to join the team and contribute too. Thank you!

[epruesse]

Welcome @roshni-b!

[bsipos]

Hi all! I would like to join the team.
Thanks!

[dpryan79]

Welcome @bsipos!

[sjanssen2]

I would like to join the team and contribute too. Thank you!

[dpryan79]

@sjanssen2 We just changed our policy (I merged the change just after the request from Botond) such that we prefer people ping @bioconda/core when they make a PR. We can add you then (the main reason being so we can interact during your first PR to the project). So please just ping us when you make your next PR and we'll get you added then :)

[sjanssen2]

Hi @dpryan79 I am confused ;-) I wanted to become a member to draw your attention to my open PR #14233 Shall I just tag @bioconda/core in there? Do I have to make a new PR?

[dpryan79]

@sjanssen2 Perfect! Just tag us there.

[iprada]

Hi,

I am planning to contribute with a recipe. I would like to join the team.

cheers,

Iñigo

[epruesse]

Welcome @iprada!

Just get started and open a PR for your new recipe. We'll help you from there!

[stolpeo]

I want to join the team :-)

[dpryan79]

@stolpeo Just request this when you open your next PR and we'll help you from there!

[Indianhedgehog]

Hi, please add me..i want to contribute with a recipe.

[dpryan79]

@HyperOdin Please ping us when you make a PR and we can add you then.

[tlkaufmann]

Just in case someone stumbles upon this issue looking for a way to be added:

Have a look at the Gitter channel and ask there to be added as a member