Update greedy_causal_lm.cpp to read EOS Token

[anzr299]

Details: Made*changes to accommodate the dynamic EOS Token
Tickets: #277 132861

[pavel-esir]

could you please also update beam search sample? EOS token is specified in parameters for beam search
https://github.com/openvinotoolkit/openvino.genai/blob/master/text_generation/causal_lm/cpp/beam_search_causal_lm.cpp#L53

[pavel-esir]

@anzr299 thx for the PR. Left some comments. Please also check why compilation fails

[pavel-esir]

Please pay attention to failure issues: add type declaration for SPECIAL_EOS_TOKEN and so on

Please look to the comments i left in the code. Please try to build locally and check if it works. If you have any questions i am ready to answer

[anzr299]

I am having some trouble building it locally and hence I continued pushing to check the build. I would require some assistance with building it.

[pavel-esir]

I am having some trouble building it locally and hence I continued pushing to check the build. I would require some assistance with building it.

Yes, sure. Could you please share some details, what kind of errors do you face? When you follow this steps https://github.com/ilya-lavrenov/openvino.genai/tree/ct-beam-search/text_generation/causal_lm/cpp#linuxmacos when the problem appears? During configure or compile stage?
You can run them separately one after another

cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
cmake --build ./build/ -j

[anzr299]

@pavel-esir I have made the changes and tested it locally, can you please review it now?

[anzr299]

@ilya-lavrenov @pavel-esir I have pushed the latest changes, I have fixed the error which appeared earlier regarding reading the model which was due to it being called before the path was set.
Although I have a question, does the SPECIAL_EOS_TOKEN have a fixed type of an integer?

Also, I have updated beam_causal_lm to include reading the EOS token from the model runtime info.

[anzr299]

@ilya-lavrenov @pavel-esir I have pushed the latest changes and fixed the problem, could you please review it now?

[anzr299]

@ilya-lavrenov @apaniukov @pavel-esir The specific check that fails is because the model Qwen-7B-Chat's runtime information does not contain an EOS token. Hence the program responds with an error as expected saying that the EOS token is not found in the runtime information.

[ilya-lavrenov]

Please, update OpenVINO tokenizers submodule https://github.com/openvinotoolkit/openvino.genai/tree/master/thirdparty to catch up latest commit openvinotoolkit/openvino_tokenizers#109 which fixes Qwen-7B-Chat

[anzr299]

I am still updating the tokenizers submodule.

[anzr299]

@ilya-lavrenov I have updated the tokenizer submodule to the latest commit.

[anzr299]

@ilya-lavrenov I have just merged changes from the main repo. I think the error regarding building openvino tokenizer only for speculative decoding and stable diffusions could be fixed with the latest commit.

[anzr299]

@ilya-lavrenov I am looking deeper into this issue. The tokenizer submodule wasn't updated and hence the error regarding the specific header file added in this commit: "openvinotoolkit/openvino_tokenizers@35d7dcb" appears which was added after the old submodule update

[anzr299]

@ilya-lavrenov Is there anything I can assist with in regards to the tokenizer issue?

[ilya-lavrenov]

Hi @anzr299
I've ported fix for Qwen model to releases/2024/0 branch, because latest master is not compatible with 2024.0 which is used in stable diffusion tests.
So, please, update to use releases/2024/0

[anzr299]

Hi @anzr299 I've ported fix for Qwen model to releases/2024/0 branch, because latest master is not compatible with 2024.0 which is used in stable diffusion tests. So, please, update to use releases/2024/0

Done

fixed

[ilya-lavrenov]

thank you for the contribution!