Welcome to esm Discussions!

[ebetica]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[yufengwhy]

Is there any Quickstart to get ESM3 embedding via (residue) sequence?

[santiag0m]

Check #2

[eelx001]

Check #3

[waitou-miu]

Great work, thank you for your contributions to the community.
I'm trying to extract embeddings from protein structures using esm3, but I'm encountering OOM errors on an A100-40G with structures longer than 2000 residues. Can the model inference be parallelized across multiple GPUs?

[ebetica]

Unfortunately not. Can you try the following?

with torch.autocast(enabled=enabled, device_type='cuda', dtype=torch.bfloat16):
  esm3.forward_and_sample(...)

And if that doesn't work, try removing the mask parameter here. As long as you're using the inference API, it's not needed, and may select a more efficient kernel.

esm/esm/layers/attention.py

Line 67 in e9f0602

query_BHLD, key_BHLD, value_BHLD, mask_BHLL

Let me know if either of these work and we can merge it into the open source codebase.

[waitou-miu]

Thank you for your response.
Both methods work, but on an A100-40G, it can only process protein structures up to around 2300 residues.

[RohanAlibutud]

What would be the recommended method for extracting amino acid probabilities for a given sequence? i.e. finding the "favorability" of a particular residue at a particular site. I've previously done this with ESM2's batch converter, but am unsure about the proper ESM3 implementation. Right now, I have this script:

ESM3_Inference.py

But I'm not certain about the order of amino acids used.

[damian238]

I have an Intel Core i7-10700F CPU @ 2.90GHz, 16 GB RAM (15.8 GB usable), running Windows 10 Home version 22H2 with WSL2 and Ubuntu. My GPU is an NVIDIA GeForce RTX 3060 with 12 GB of VRAM and CUDA version 12.4.

Is there any possibility this system is strong enough to work with strings that are a few hundred peptides long? If so I would definitely put in the effort to get this program working on my system. I'm an old codger, but still quick of wit (perhaps even enough to pull off a few miracles). Looking forward to learning!