Add example of using generate_events_async with streaming.

examples/notebooks/generate_events_and_streaming.ipynb

@@ -0,0 +1,195 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "source": [
+    "# Using `generate_events_async` and Streaming\n",
+    "\n",
+    "This guide will teach you how to use the `LLMRails.generate_events_async` method and stream the bot response. This is useful when your rails need to process other events, in addition to user and bot messages. \n",
+    "\n",
+    "**Important**: the streaming option does not work with the synchronous method `LLMRails.generate_events`.\n",
+    "\n",
+    "**Note**: this guide assumes you have successfully installed NeMo Guardrails and the OpenAI package. If not, please refer to the [Hello World](../../docs/getting_started/1_hello_world) guide."
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "53e0d6f2f984979d"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "# Setting the TOKENIZERS_PARALLELISM to get rid of the forking warning\n",
+    "os.environ[\"TOKENIZERS_PARALLELISM\"] = \"false\""
+   ],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-11-23T08:47:33.941631Z",
+     "start_time": "2023-11-23T08:47:33.939231Z"
+    }
+   },
+   "id": "4b18190855adfe3a"
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "## Step 1: create a config \n",
+    "\n",
+    "Let's create a simple config:"
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "35fb674a4026ec51"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "outputs": [],
+   "source": [
+    "from nemoguardrails import RailsConfig, LLMRails\n",
+    "\n",
+    "YAML_CONFIG = \"\"\"\n",
+    "models:\n",
+    "  - type: main\n",
+    "    engine: openai\n",
+    "    model: gpt-4\n",
+    "\n",
+    "streaming: True\n",
+    "\"\"\"\n",
+    "\n",
+    "config = RailsConfig.from_content(yaml_content=YAML_CONFIG)\n",
+    "app = LLMRails(config)"
+   ],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-11-23T08:47:39.137061Z",
+     "start_time": "2023-11-23T08:47:33.942980Z"
+    }
+   },
+   "id": "d9bac50b3383915e"
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "Next, we need to create a streaming handler and register it in the current async context by setting the value of the AsyncIO context variable `streaming_handler_var`, create a demo task that prints the tokens and make the `generate_events_async` call:"
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "9036d5ce62e352f0"
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "CHUNK: Hello\n",
+      "CHUNK: !\n",
+      "CHUNK:  As\n",
+      "CHUNK:  an\n",
+      "CHUNK:  AI\n",
+      "CHUNK: ,\n",
+      "CHUNK:  I\n",
+      "CHUNK:  don\n",
+      "CHUNK: 't\n",
+      "CHUNK:  have\n",
+      "CHUNK:  feelings\n",
+      "CHUNK: ,\n",
+      "CHUNK:  but\n",
+      "CHUNK:  I\n",
+      "CHUNK: 'm\n",
+      "CHUNK:  here\n",
+      "CHUNK:  and\n",
+      "CHUNK:  ready\n",
+      "CHUNK:  to\n",
+      "CHUNK:  assist\n",
+      "CHUNK:  you\n",
+      "CHUNK: .\n",
+      "CHUNK:  How\n",
+      "CHUNK:  can\n",
+      "CHUNK:  I\n",
+      "CHUNK:  help\n",
+      "CHUNK:  you\n",
+      "CHUNK:  today\n",
+      "CHUNK: ?\n",
+      "There were 12 new events.\n"
+     ]
+    }
+   ],
+   "source": [
+    "import asyncio \n",
+    "from nemoguardrails.streaming import StreamingHandler\n",
+    "from nemoguardrails.context import streaming_handler_var\n",
+    "\n",
+    "# Create the streaming handler and register it.\n",
+    "streaming_handler = StreamingHandler()\n",
+    "streaming_handler_var.set(streaming_handler)\n",
+    "\n",
+    "# For demo purposes, create a task that prints the tokens.\n",
+    "async def process_tokens():\n",
+    "    async for chunk in streaming_handler:\n",
+    "        print(f\"CHUNK: {chunk}\")\n",
+    "\n",
+    "asyncio.create_task(process_tokens())\n",
+    "\n",
+    "# Call the events-based API.\n",
+    "events = [{\n",
+    "  \"type\": \"UtteranceUserActionFinished\",\n",
+    "  \"final_transcript\": \"Hello! How are you?\"\n",
+    "}]\n",
+    "\n",
+    "new_events = await app.generate_events_async(events)\n",
+    "print(f\"There were {len(new_events)} new events.\")"
+   ],
+   "metadata": {
+    "collapsed": false,
+    "ExecuteTime": {
+     "end_time": "2023-11-23T08:47:42.846315Z",
+     "start_time": "2023-11-23T08:47:39.143972Z"
+    }
+   },
+   "id": "60fa80f584cce58c"
+  },
+  {
+   "cell_type": "markdown",
+   "source": [
+    "As expected, the tokens were printed as they were generated, and at the end we get the complete list of events that were generated. For more details on the structure of the events, check out the [Event-based API Guide](../../docs/user_guides/advanced/event-based-api.md)."
+   ],
+   "metadata": {
+    "collapsed": false
+   },
+   "id": "29f1381b93da53b4"
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 2
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython2",
+   "version": "2.7.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}