Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HNSW CPU Hierarchy #465

Draft
wants to merge 6 commits into
base: branch-24.12
Choose a base branch
from

Conversation

divyegala
Copy link
Member

This PR adds an option to build the full HNSW hierarchy on the CPU when converting a CAGRA index to an hnswlib index. It also updates hnswlib to v0.7.0

@divyegala divyegala added feature request New feature or request non-breaking Introduces a non-breaking change labels Nov 13, 2024
@divyegala divyegala self-assigned this Nov 13, 2024
@divyegala divyegala changed the title update hnswlib to 0.7.0 HNSW CPU Hierarchy Nov 13, 2024
@jakirkham
Copy link
Member

Does this need to be added to the Conda recipes as well?

@divyegala
Copy link
Member Author

@jakirkham no actually we don't want to use the hnswlib conda package because their python distribution messes up with our python version requirements. I will be removing it from our conda dev environments as well.

We'll rely on CMake instead to get us hnswlib. There's no double compilation issue anyway since it's a header-only library.

@jakirkham
Copy link
Member

jakirkham commented Nov 14, 2024

Oh ok. Had not looked into the nuances of this package or our usage of it

Indeed if we don't use the Python package, we should drop it

There doesn't appear to be a header-only C++ package currently. So just fetching it from source seems like the thing to do

Are the headers just used in the build to produce a library? Or are we shipping their headers too? If the latter, we may want to look at siloing it somehow to avoid clobbering or affecting other installs a user may have

@divyegala
Copy link
Member Author

@jakirkham we don't need to ship the headers, they are self-contained within the libcuvs shared object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake cpp feature request New feature or request non-breaking Introduces a non-breaking change Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants