🏗️GNES is under construction and rapid release cycles

[hanxiao]

🙇Hello there, I'd like to thank you for your interests in GNES, greatly appreciated!

After reading through the README, you may realize that GNES has an ambitious goal and the workload to enable any-to-any semantic search is quite something. Right now we are rapidly iterating GNES in terms of its usability, stability. This includes:

• adding new APIs;
• refactoring old APIs;
• restructuring the project structure;
• writing tutorial;
• building a healthy GNES eco-system;
• making demos

⏰A new release is scheduled every Friday's evening (GMT+8), the release procedure can be found here.

Please be patient if you encounter some issues, and please understand that we may not respond in time regarding your issues. Thanks for your understanding. 🙏

Meanwhile, if you'd like to understand the design principles behind GNES, feel free to read this blog post: Generic Neural Elastic Search: From bert-as-service and Go Way Beyond

For other questions, please contact our team via [email protected]

[hanxiao]

Thanks all ❤️ for trying GNES and giving feedback at the early stage. I'd like to introduce you the new GNES Flow API (available since v0.0.46), enables a pythonic and intuitive way for building workflow in GNES. As an example, an indexing workflow can be simply defined as:

flow = (Flow(check_version=False, ctrl_with_ipc=True)
        .add_preprocessor(name='prep', yaml_path='yaml/prep.yml', replicas=3)
        .add_encoder(yaml_path='yaml/incep.yml', replicas=6)
        .add_indexer(name='vec_idx', yaml_path='yaml/vec.yml')
        .add_indexer(name='doc_idx', yaml_path='yaml/doc.yml', recv_from='prep')
        .add_router(name='sync', yaml_path='BaseReduceRouter', num_part=2, recv_from=['vec_idx', 'doc_idx']))

# then use it for indexing
with flow(backend='process') as fl:
    fl.index(bytes_gen=read_flowers(), batch_size=64)

🔰 You can find some resources here to help you getting started quickly:

• Image search demo using GNES Flow, Jupyter Notebook
• GNES Flow module documentation
• Getting started in GNES readme

🙇 Give it a try and we welcome your feedback and contribution.