Batch mode for multiple lines or files

[antoniozh]

Hi,
is it possible to run the prediction container on multiple lines or files in a row? Would this be useful for analyzling large Go repositories at once? The prediction model would be loaded only once then and we would save a lot of time

[Cortys]

While the model does in principle support batch predictions, the container CLI currently does not.

Batch predictions could definitely improve performance if the number of queries is sufficiently large. However, the overall prediction time for a single query is dominated by the CFG creation preprocessing step, which does not really benefit from batching (model loading and inference are comparatively fast). Simply running the prediction container multiple times in parallel therefore probably is not significantly slower than a single batched container run.