Exemples of use cases

[bratao]

Hi,
This is a VERY GREAT looking project. Thank you for your effort.

But from the documentation is not clear what kind of uses are possible with this project.
For example, in a text search, I would imagine that gnes would enable me to find similar documents. Is that right?

And how it compared with Vespa.ai (https://github.com/vespa-engine) ?

Some examples of what is possible with Gnes would be nice!

[Ravikiran2611]

yeah some more examples with explanation of what is happening (how data is loaded and how model is loaded , how the path for model is specified )could be even more useful for beginners like me . Hope you take this asa a request
Thanks

[hanxiao]

@bratao @Ravikiran2611 thanks again ❤️ for trying GNES and giving feedback at the early stage. I'd like to introduce you the new GNES Flow API (available since v0.0.46), enables a pythonic and intuitive way for building workflow in GNES. As an example, an indexing workflow can be simply defined as:

flow = (Flow(check_version=False, ctrl_with_ipc=True)
        .add_preprocessor(name='prep', yaml_path='yaml/prep.yml', replicas=3)
        .add_encoder(yaml_path='yaml/incep.yml', replicas=6)
        .add_indexer(name='vec_idx', yaml_path='yaml/vec.yml')
        .add_indexer(name='doc_idx', yaml_path='yaml/doc.yml', recv_from='prep')
        .add_router(name='sync', yaml_path='BaseReduceRouter', num_part=2, recv_from=['vec_idx', 'doc_idx']))

# then use it for indexing
with flow(backend='process') as fl:
    fl.index(bytes_gen=read_flowers(), batch_size=64)

🔰 You can find some resources here to help you getting started quickly:

• Image search demo using GNES Flow, Jupyter Notebook
• GNES Flow module documentation
• Getting started in GNES readme

🙇 Give it a try and we welcome your feedback and contribution.

[Ravikiran2611]

@hanxiao can i add a new model to gnes so that i can use it in a yaml file that can be specified in gnes.flow