Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/fluent-ui-migration' into dkalin…
Browse files Browse the repository at this point in the history
…in/text-models
  • Loading branch information
ActiveChooN committed Nov 29, 2024
2 parents 93a8a15 + 168d0b5 commit 4f71c4d
Show file tree
Hide file tree
Showing 89 changed files with 5,796 additions and 789 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/linux-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
- uses: subosito/flutter-action@v2
with:
channel: 'stable'
flutter-version: '3.24.0'
flutter-version: '3.24.5'
- name: Install dependencies
run: sudo apt-get install -y clang cmake ninja-build pkg-config libgtk-3-dev liblzma-dev libmpv-dev
- name: Install project dependencies
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/macos-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
- uses: subosito/flutter-action@v2
with:
channel: 'stable'
flutter-version: '3.24.0'
flutter-version: '3.24.5'
- name: Install project dependencies
run: flutter pub get
- name: Generate intermediates
Expand Down
9 changes: 5 additions & 4 deletions .github/workflows/windows-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,12 @@ jobs:

# Step 9: Install vcpkg and ffmpeg
- name: Install vcpkg and ffmpeg
shell: powershell
run: |
git clone https://github.com/microsoft/vcpkg.git C:\vcpkg
if (!(Test-Path "C:\vcpkg")) { git clone https://github.com/microsoft/vcpkg.git C:\vcpkg }
C:\vcpkg\bootstrap-vcpkg.bat
C:\vcpkg\vcpkg install ffmpeg
shell: cmd
cd openvino_bindings/third_party
C:\vcpkg\vcpkg install
# Step 10: Download and Install OpenVINO Runtime
- name: Download and Install OpenVINO Runtime 24.5.0
Expand Down Expand Up @@ -126,7 +127,7 @@ jobs:
- uses: subosito/flutter-action@v2
with:
channel: 'stable'
flutter-version: '3.24.0'
flutter-version: '3.24.5'
- name: Install project dependencies
run: flutter pub get
- name: Generate intermediates
Expand Down
118 changes: 99 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,122 @@
<div align="center">

# OpenVINO™ Test Drive

[![codecov](https://codecov.io/gh/openvinotoolkit/openvino_testdrive/graph/badge.svg?token=DH98FAPH65)](https://codecov.io/gh/openvinotoolkit/openvino_testdrive)
[![openvino](https://img.shields.io/badge/openvino-2024.4-blue)]()

</div>

Get started with OpenVINO™ Test Drive, an application that allows you to run generative AI and vision models trained by [Intel® Geti™](https://docs.geti.intel.com/) directly on your computer or edge device using [OpenVINO™ Runtime](https://github.com/openvinotoolkit/openvino).


<p align="center">
<img src="./docs/readme.gif" width="600" alt="sample">
</p>

With use of OpenVINO™ Test Drive you can:
+ **Chat with LLMs** and evaluating model performance on your computer or edge device
+ **Experiment with different text prompts** to generate images using Stable Diffusion and Stable DiffusionXL models (coming soon)
+ **Transcribe speech from video** using Whisper models, including generation of timestamps (coming soon)
+ **Run and visualize results of models** trained by Intel® Geti™ using single image inference or batch inference mode

## Installation

Download the latest release from the [Releases repository](https://storage.openvinotoolkit.org/repositories/openvino_testdrive/packages/).

> [!NOTE]
> To verify downloaded file integrity, you can generate a SHA-256 of the downloaded file and compare it to the SHA-256 from corresponding `.sha256` file published in Releases repository.
<details>
<summary>Installation on Windows</summary>

1. Downloading the zip archive [Releases repository](https://storage.openvinotoolkit.org/repositories/openvino_testdrive/packages/) `Windows` folder .

<p align="left">
<img src="./docs/win_inst.gif" width="500">
</p>

2. Extract zip archive double-click the MSIX installation package, click `Install` button and it will display the installation process

Get started with OpenVINO Test Drive, an application that allows you to run LLMs and models trained by [Intel Geti](https://geti.intel.com/) directly on your computer or edge device using OpenVINO.
<p align="left">
<img src="./docs/win_inst2.gif" width="350">
</p>

# Features
### LLM models
+ **Text Generation**: Generate text and engage in chat experiences.
+ **Performance metrics**: Evaluate model performance on your computer or edge device.
### Computer vision models
+ **Single Image Inference**: Perform inference on individual images.
+ **Batch Inference**: Conduct inference on batches of images.
3. Click on the application name on Windows app list to launch OpenVINO™ Test Drive.

# High level architecture
![Design Graph](./design_graph.png)
</details>

# Using the Test Drive
## Quick start

Upon starting the application, you can import a model using either Huggingface for LLMs or “from local disk” for Geti models.
Upon starting the application, you can import a model using either Hugging Face for LLMs or upload Intel® Geti models from local disk.

![Preview](./preview.png)
### Text generation and LLM performance evaluation

# Getting Started
1. Find a model on Hugging Face and import it
<p align="left">
<img src="./docs/llm_import.gif" width="500">
</p>

## Release
2. Chat with LLMs via `Playground` tab.
<p align="left">
<img src="./docs/llm_model_chat.gif" width="500">
</p>

Download the latest release from the [Releases page](https://github.com/openvinotoolkit/openvino_testdrive/releases).
3. Use `Performance metrics` tab to get model performance metrics on your computer or edge device
<p align="left">
<img src="./docs/metrics.gif" width="500">
</p>

### Images inference with models trained by Intel® Geti™

1. Download deployment code for the model in OpenVINO format trained by Intel® Geti™.

<p align="left">
<img src="./docs/geti_download.gif" width="500">
</p>

> [!NOTE]
> Please check [Intel® Geti™ documentation](https://docs.geti.intel.com) for more details.
2. Import deployment code into OpenVINO™ Test Drive using `Import model` -> `Local disk` button.

<p align="left">
<img src="./docs/geti_import.gif" width="500">
</p>

3. Run and visualize results of inference on individual images using `Live inference` tab.

<p align="left">
<img src="./docs/geti_cv.gif" width="500">
</p>

4. For batch inference, use `Batch inference` tab and provide paths to folder with input images in a `Source folder` and specify `Destination folder` for output batch inference results. Click on `Start` to start batch inference.

<p align="left">
<img src="./docs/geti_batch.gif" width="500">
</p>

## Build

The application requires the flutter SDK and the dependencies for your specific platform to be installed.
Secondly, the bindings and its dependencies for your platform to be added to `./bindings`.

1. [Install flutter sdk](https://docs.flutter.dev/get-started/install). Make sure to follow the guide for flutter dependencies.
2. [Download the bindings](https://github.com/intel-sandbox/applications.ai.geti.flutter.inference/releases) and extract them to ./bindings folder
2. Build the bindings and put them to `./bindings` folder. OpenVINO™ Test Drive uses bindings to OpenVINO™ GenAI and OpenVINO™ Vision ModelAPI located in `./openvino_bindings` folder. See [readme](./openvino_bindings/README.md) for more details.
3. Once done you can start the application: `flutter run`

## Build bindings
## Ecosystem

- [OpenVINO™](https://github.com/openvinotoolkit/openvino) - software toolkit for optimizing and deploying deep learning models.
- [GenAI Repository](https://github.com/openvinotoolkit/openvino.genai) and [OpenVINO Tokenizers](https://github.com/openvinotoolkit/openvino_tokenizers) - resources and tools for developing and optimizing Generative AI applications.
- [Intel® Geti™](https://docs.geti.intel.com/) - software for building computer vision models.
- [OpenVINO™ Vision ModelAPI](https://github.com/openvinotoolkit/model_api) - a set of wrapper classes for particular tasks and model architectures, simplifying data preprocess and postprocess as well as routine procedures.

## Contributing

For those who would like to contribute to the OpenVINO™ Test Drive, please check out [Contribution Guidelines](CONTRIBUTING.md) for more details.

The Test Drive uses c bindings to OpenVINO. These are located in `./openvino_bindings` folder. See [readme.md](./openvino_bindings/README.md).
## License

OpenVINO™ Test Drive repository is licensed under [Apache License Version 2.0](LICENSE).
By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
Loading

0 comments on commit 4f71c4d

Please sign in to comment.