From distributed TensorFlow experiment to scoring.ipynb

{
    "nbformat_minor": 1, 
    "cells": [
        {
            "source": "<table style=\"border: none\" align=\"left\">\n   <tr style=\"border: none\">\n      <th style=\"border: none\"><font face=\"verdana\" size=\"4\" color=\"black\"><b>From tensorflow experiment to scoring with</b></font><font face=\"monospace\" size=\"4\" color=\"darkblue\"><b> watson-machine-learning-client</b></font></th>\n      <th style=\"border: none\"><img src=\"https://github.com/pmservice/customer-satisfaction-prediction/blob/master/app/static/images/ml_icon_gray.png?raw=true\" alt=\"Watson Machine Learning icon\" height=\"40\" width=\"40\"></th>\n   </tr> \n   <tr style=\"border: none\">\n       <td style=\"border: none\"><img src=\"https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/hand-written-digit-recognition/images/experiment_banner.png\" width=\"800\" alt=\"Icon\"></td>\n   </tr>\n</table>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "This notebook contains steps and code to demonstrate support of Deep Learning model training and scoring in Watson Machine Learning Service. This notebook introduces commands for getting data, training_definition persistance to Watson Machine Learning repository, model training, model persistance, model deployment and scoring.\n\nSome familiarity with Python is helpful. This notebook uses Python 3.\n\n\n## Learning goals\n\nThe learning goals of this notebook are:\n\n-  Working with Watson Machine Learning experiments to train Deep Learning models (TensorFlow).\n-  Saving trained models in Watson Machine Learning repository.\n-  Online deployment and scoring of trained model.\n\n\n## Contents\n\nThis notebook contains the following parts:\n\n1.\t[Setup](#setup)\n2.\t[Training definition](#definition)\n3.\t[Experiment definition](#experiment)\n4.\t[Experiment Run](#run)\n5.\t[Deploy and Score](#scoring)\n6.\t[Summary and next steps](#summary)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"setup\"></a>\n## 1. Setup\n\nBefore you use the sample code in this notebook, you must perform the following setup tasks:\n\n-  Create a [Watson Machine Learning Service](https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/) instance (a lite plan is offered). \n-  Create [Cloud Object Storage (COS)](https://console.bluemix.net/catalog/infrastructure/cloud-object-storage) instance (a lite plan is offered).\n    - After you create COS instance, go to your COS dashboard.\n    - In \"Service credentials\" tab, click on \"New Credential\",\n    - Add inline configuration parameter: {\"HMAC\":true}, click \"Add\".\n\n    This configuration parameter will add section below to instance credentials which will be used later on,\n    ```\n      \"cos_hmac_keys\": {\n            \"access_key_id\": \"722432c254bc4eaa96e05897bf2779e2\",\n            \"secret_access_key\": \"286965ac10ecd4de8b44306288c7f5a3e3cf81976a03075c\"\n       }\n    ```", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.1 Working with Cloud Object Storage", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Boto library allows Python developers to manage Cloud Object Storage.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import ibm_boto3\nfrom ibm_botocore.client import Config\nimport os, json, warnings"
        }, 
        {
            "source": "We define the endpoint we will use. You can find this information in \"Endpoint\" section of your Cloud Object Storage intance's dashbord.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "cos_credentials = {\n  \"apikey\": \"***\",\n  \"cos_hmac_keys\": {\n    \"access_key_id\": \"***\",\n    \"secret_access_key\": \"***\"\n  },\n  \"endpoints\": \"https://cos-service.bluemix.net/endpoints\",\n  \"iam_apikey_description\": \"***\",\n  \"iam_apikey_name\": \"***\",\n  \"iam_role_crn\": \"crn:v1:bluemix:public:iam::::serviceRole:Writer\",\n  \"iam_serviceid_crn\": \"***\",\n  \"resource_instance_id\": \"***\"\n}\n\napi_key = cos_credentials['apikey']\nservice_instance_id = cos_credentials['resource_instance_id']\nauth_endpoint = 'https://iam.bluemix.net/oidc/token'\nservice_endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'"
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by DSX for sharing."
        }, 
        {
            "source": "We create Boto resource by providing type, endpoint_url and credentials.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "cos = ibm_boto3.resource('s3',\n                         ibm_api_key_id=api_key,\n                         ibm_service_instance_id=service_instance_id,\n                         ibm_auth_endpoint=auth_endpoint,\n                         config=Config(signature_version='oauth'),\n                         endpoint_url=service_endpoint)"
        }, 
        {
            "source": "Let's create the buckets we will use to store training data and training results.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Creating bucket \"train-data-experiment-2018-03-28-09-34\"...\nCreating bucket \"train-result-experiment-2018-03-28-09-34\"...\n"
                }
            ], 
            "source": "import datetime\n\ndate_time = datetime.datetime.now()\ndate_time = date_time.strftime(\"%Y-%m-%d-%H-%M\")\n\nbuckets = {'data': 'train-data-experiment-' + date_time, 'results': 'train-result-experiment-' + date_time}\n\nfor bucket in buckets.values():\n    if not cos.Bucket(bucket) in cos.buckets.all():\n        print('Creating bucket \"{}\"...'.format(bucket))\n        try:\n            cos.create_bucket(Bucket=bucket)\n        except ibm_boto3.exceptions.ibm_botocore.client.ClientError as e:\n            print('Error: {}.'.format(e.response['Error']['Message']))\n    else:\n        print('Bucket \"{}\" already exists.'.format(bucket))"
        }, 
        {
            "source": "Now we should have 2 new buckets created.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.2 Downloading training data and upload it to COS buckets.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Let's download our training data and upload them to 'training-data' bucket.\nWe will create a list of links for training dataset", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "data_links = ['http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz']"
        }, 
        {
            "source": "Below snippet will create 'MNIST_DATA' folder and download the files from links.\n\n**Tip:** First, please install wget library by the command below\n`!pip install wget`", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 21, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['MNIST_DATA/train-labels-idx1-ubyte.gz',\n 'MNIST_DATA/t10k-labels-idx1-ubyte.gz',\n 'MNIST_DATA/train-images-idx3-ubyte.gz',\n 'MNIST_DATA/t10k-images-idx3-ubyte.gz']"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "import wget, glob\n\ndata_dir = 'MNIST_DATA'\nif not os.path.isdir(data_dir):\n    os.mkdir(data_dir)\n\nfor link in data_links:\n    if not os.path.isfile(os.path.join(data_dir, os.path.join(link.split('/')[-1]))):\n        wget.download(link, out=data_dir)\n\nglob.glob('MNIST_DATA/*')"
        }, 
        {
            "source": "Upload the data files to created buckets.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "train-labels-idx1-ubyte.gz is uploaded.\nt10k-labels-idx1-ubyte.gz is uploaded.\ntrain-images-idx3-ubyte.gz is uploaded.\nt10k-images-idx3-ubyte.gz is uploaded.\n"
                }
            ], 
            "source": "bucket_name = buckets['data']\nbucket_obj = cos.Bucket(bucket_name)\n\nfor filename in os.listdir(data_dir):\n    with open(os.path.join(data_dir, filename), 'rb') as data: \n        bucket_obj.upload_file(os.path.join(data_dir, filename), filename)\n        print('{} is uploaded.'.format(filename))"
        }, 
        {
            "source": "Let's see the list of created buckets and their contents.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Object key: t10k-images-idx3-ubyte.gz, size: 1610.2kB\nObject key: t10k-labels-idx1-ubyte.gz, size:   4.4kB\nObject key: train-images-idx3-ubyte.gz, size: 9680.1kB\nObject key: train-labels-idx1-ubyte.gz, size:  28.2kB\n"
                }
            ], 
            "source": "for obj in bucket_obj.objects.all():\n    print('Object key: {}, size: {:5.1f}kB'.format(obj.key, obj.size/1024))"
        }, 
        {
            "source": "We are done with Cloud Object Storage, we are ready to train our model!", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.3 Work with the Watson Machine Learning instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Authenticate to Watson Machine Learning service on Bluemix.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Action**: Put authentication information from your instance of Watson Machine Learning service here.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "wml_credentials = {\n  \"url\": \"https://ibm-watson-ml.mybluemix.net\",\n  \"access_key\": \"***\",\n  \"username\": \"***\",\n  \"password\": \"***\",\n  \"instance_id\": \"***\"\n}"
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by DSX for sharing."
        }, 
        {
            "source": "#### Install `watson-machine-learning-client` from pypi", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client"
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Requirement already up-to-date: watson-machine-learning-client in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\nRequirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client)\nRequirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\n"
                }
            ], 
            "source": "!pip install --upgrade watson-machine-learning-client --upgrade"
        }, 
        {
            "source": "#### Import `watson-machine-learning-client` and authenticate to service instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient"
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client = WatsonMachineLearningAPIClient(wml_credentials)"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "1.0.53\n"
                }
            ], 
            "source": "print(client.version)"
        }, 
        {
            "source": "**Note:** `watson-machine-learning-client` documentation can be found here [http://wml-api-pyclient.mybluemix.net/](http://wml-api-pyclient.mybluemix.net/).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"definition\"></a>\n## 2. Training definitions", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 2.2 Save training definition ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Prepare training definition metadata", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_tf_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - distributed\",\n            client.repository.DefinitionMetaNames.AUTHOR_EMAIL: \"lc@pl.ibm.com\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"PS_HOSTS_COUNT=1 ./launcher.py python3 ${MODEL_DIR}/mnist_dist.py --data_dir ${DATA_DIR}\"\n            }"
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_horovod_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - horovod\",\n            client.repository.DefinitionMetaNames.AUTHOR_EMAIL: \"lc@pl.ibm.com\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow-horovod\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"python tensorflow_mnist.py\"\n            }"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_ddl_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - ddl\",\n            client.repository.DefinitionMetaNames.AUTHOR_EMAIL: \"lc@pl.ibm.com\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow-ddl\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"python mnist-train-ddl.py; python transform.py\"\n            }"
        }, 
        {
            "source": "#### Get sample model definition content files from git", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 22, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['tf_horovod.zip', 'tf_ddl.zip', 'tf_distributed.zip']"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "filename_tf = 'tf_distributed.zip'\nfilename_horovod = 'tf_horovod.zip'\nfilename_ddl = 'tf_ddl.zip'\n\nif os.path.isfile(filename_tf):\n    os.remove(filename_tf)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/native/' + filename_tf)\n\nif os.path.isfile(filename_horovod):\n    os.remove(filename_horovod)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/horovod/' + filename_horovod)\n\nif os.path.isfile(filename_ddl):\n    os.remove(filename_ddl)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/ddl/' + filename_ddl)\n\nglob.glob('tf_*.zip')"
        }, 
        {
            "source": "#### Publish training definition in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "##### Store definition: native distributed tensorflow", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/06fa6000-9b1b-4cf0-8932-f35cf449361e\n"
                }
            ], 
            "source": "definition_tf_details = client.repository.store_definition(filename_tf, model_definition_tf_metadata)\n\ndefinition_tf_url = client.repository.get_definition_url(definition_tf_details)\ndefinition_tf_uid = client.repository.get_definition_uid(definition_tf_details)\nprint(definition_tf_url)"
        }, 
        {
            "source": "##### Store definition: horovod", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/a473639d-4280-4157-b360-7df30f965788\n"
                }
            ], 
            "source": "definition_horovod_details = client.repository.store_definition(filename_horovod, model_definition_horovod_metadata)\n\ndefinition_horovod_url = client.repository.get_definition_url(definition_horovod_details)\ndefinition_horovod_uid = client.repository.get_definition_uid(definition_horovod_details)\nprint(definition_horovod_url)"
        }, 
        {
            "source": "##### Store definition: ibm DDL", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 25, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/c5988360-4241-4565-8e6c-05fde3c8b27e\n"
                }
            ], 
            "source": "definition_ddl_details = client.repository.store_definition(filename_ddl, model_definition_ddl_metadata)\n\ndefinition_ddl_url = client.repository.get_definition_url(definition_ddl_details)\ndefinition_ddl_uid = client.repository.get_definition_uid(definition_ddl_details)\nprint(definition_ddl_url)"
        }, 
        {
            "source": "#### LIST stored definitions", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------------  ------------------------  ------------------\nGUID                                  NAME                                      CREATED                   FRAMEWORK\nea8c0dd5-8a6b-4e5e-a9cd-74a0c573c1de  MNIST-MLP                                 2018-03-15T10:20:58.646Z  tensorflow\n8095a87c-5c94-4f43-b5fd-468b2d72a3c8  MNIST-CNN                                 2018-03-15T10:21:00.033Z  tensorflow\nbebc485e-bcc9-45b7-9bce-1ba20acc3b53  MNIST-MLP                                 2018-03-15T10:35:04.997Z  tensorflow\nf26539ff-43b5-4100-8915-7b0be22b2211  MNIST-CNN                                 2018-03-15T10:35:05.815Z  tensorflow\nbed45da1-6372-4cb1-b889-e0d402cfe62a  MNIST-MLP                                 2018-03-15T15:41:40.335Z  tensorflow\nc80aaeef-1c03-4474-8891-535aa6dae81e  MNIST-CNN                                 2018-03-15T15:41:41.174Z  tensorflow\nf819f724-8e1c-4903-a04c-eee5dfc048a4  style transfer van gogh                   2018-03-15T20:33:03.354Z  tensorflow\nf5c4e713-27d9-4529-a473-10dc88383a52  style transfer kandinsky                  2018-03-15T20:33:05.032Z  tensorflow\naf72a944-cc83-4aaf-9152-9f25c1912bf4  style transfer van gogh                   2018-03-15T20:57:13.232Z  tensorflow\n4fb3987a-f979-4e1e-a26f-1acbe11a08e7  style transfer kandinsky                  2018-03-15T20:57:13.892Z  tensorflow\n1421337a-c2a8-4153-8554-accdc55bbd27  style transfer van gogh                   2018-03-15T21:45:46.889Z  tensorflow\n0f8fa74a-f821-475c-85c9-a2b507ed6c63  style transfer kandinsky                  2018-03-15T21:45:48.344Z  tensorflow\ncd31e290-6636-4586-8097-e3c3ac7fef8c  style transfer van gogh                   2018-03-16T08:37:18.880Z  tensorflow\n76b144fc-100c-45ee-80ae-ef0d44a87cd6  style transfer kandinsky                  2018-03-16T08:37:19.383Z  tensorflow\n7dc7e7cc-ae5b-4db2-8705-32174324010a  MNIST-MLP                                 2018-03-16T09:59:57.138Z  tensorflow\n6a024dde-d6bc-45d3-8463-895452134c29  MNIST-CNN                                 2018-03-16T09:59:57.888Z  tensorflow\n32c64603-13d2-4ec6-b3ca-4ee4650fbd8e  style transfer van gogh                   2018-03-16T10:04:24.779Z  tensorflow\n18498298-3e01-43a0-a1f7-955ecc118a4c  style transfer kandinsky                  2018-03-16T10:04:25.661Z  tensorflow\n9e446ccc-8b1d-4e46-a549-655f9b16a577  style transfer van gogh                   2018-03-16T10:33:54.588Z  tensorflow\n1f54975c-0104-4fcb-910c-ae57e3272d7c  style transfer kandinsky                  2018-03-16T10:33:55.170Z  tensorflow\n3501d90f-8599-4154-a8a5-d6ddafac15a8  MNIST-MLP                                 2018-03-16T11:53:19.116Z  tensorflow\n7c1b279f-7556-4beb-bee3-5516a0daa433  MNIST-CNN                                 2018-03-16T11:53:19.907Z  tensorflow\nd56f3240-b49e-4b30-98e8-fb1ca8ab6dd6  MNIST-MLP                                 2018-03-16T12:35:29.322Z  tensorflow\n970df64d-d363-4524-881d-75320928c695  MNIST-CNN                                 2018-03-16T12:35:30.591Z  tensorflow\n11ceb998-31d9-424e-94d7-51eb8476f143  training def for product-line-prediction  2018-03-16T18:08:30.465Z  mllib\n960d8f27-e636-438d-a8b9-65cc9033e5f6  training def for drug-selection           2018-03-16T18:10:24.191Z  mllib\n60994f57-61e6-4bcc-b025-18622d22c98f  My definition name                        2018-03-28T07:55:40.796Z  tensorflow\n06fa6000-9b1b-4cf0-8932-f35cf449361e  Tensorflow - distributed                  2018-03-28T09:45:06.201Z  tensorflow\na473639d-4280-4157-b360-7df30f965788  Tensorflow - horovod                      2018-03-28T09:45:07.779Z  tensorflow-horovod\nc5988360-4241-4565-8e6c-05fde3c8b27e  Tensorflow - ddl                          2018-03-28T09:45:08.735Z  tensorflow-ddl\n------------------------------------  ----------------------------------------  ------------------------  ------------------\n"
                }
            ], 
            "source": "client.repository.list_definitions()"
        }, 
        {
            "source": "<a id=\"experiment\"></a>\n## 3. Experiment definition", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.1 Save experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Get the list of supported configuration params", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--------------------------  ----  --------\nMETA_PROP NAME              TYPE  REQUIRED\nNAME                        str   Y\nTAGS                        list  N\nDESCRIPTION                 str   N\nAUTHOR_NAME                 str   N\nAUTHOR_EMAIL                str   Y\nEVALUATION_METHOD           str   N\nEVALUATION_METRICS          list  N\nTRAINING_REFERENCES         list  Y\nTRAINING_DATA_REFERENCE     dict  Y\nTRAINING_RESULTS_REFERENCE  dict  Y\n--------------------------  ----  --------\n"
                }
            ], 
            "source": "client.repository.ExperimentMetaNames.show()"
        }, 
        {
            "source": "#### Experiment configuration dictionary\nCreate experiment that will train two models based on previously stored definitions.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "TRAINING_DATA_REFERENCE = {\n                            \"connection\": {\n                                \"endpoint_url\": service_endpoint,\n                                \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n                                \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n                            },\n                            \"source\": {\n                                \"bucket\": buckets[0],\n                            },\n                            \"type\": \"s3\"\n                           }\nTRAINING_RESULTS_REFERENCE = {\n                                \"connection\": {\n                                    \"endpoint_url\": service_endpoint,\n                                    \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n                                    \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n                                },\n                                \"target\": {\n                                    \"bucket\": buckets[1],\n                                },\n                                \"type\": \"s3\"\n                             }"
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "experiment_metadata = {\n            client.repository.ExperimentMetaNames.NAME: \"Distributed MNIST experiment\",\n            client.repository.ExperimentMetaNames.AUTHOR_EMAIL: \"lc@pl.ibm.com\",\n            client.repository.ExperimentMetaNames.TRAINING_DATA_REFERENCE: TRAINING_DATA_REFERENCE,\n            client.repository.ExperimentMetaNames.TRAINING_RESULTS_REFERENCE: TRAINING_RESULTS_REFERENCE,\n            client.repository.ExperimentMetaNames.TRAINING_REFERENCES: [\n                        {\n                            \"name\": \"tf-distributed\",\n                            \"training_definition_url\": definition_tf_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        },                        \n                        {\n                            \"name\": \"tf-horovod\",\n                            \"training_definition_url\": definition_horovod_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        },                        \n                        {\n                            \"name\": \"tf-ddl\",\n                            \"training_definition_url\": definition_ddl_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        }\n                    ],\n                }"
        }, 
        {
            "source": "#### Store experiment in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "d763b2eb-1b9d-440e-8c25-09a0f06ee017\n"
                }
            ], 
            "source": "experiment_details = client.repository.store_experiment(meta_props=experiment_metadata)\n\nexperiment_uid = client.repository.get_experiment_uid(experiment_details)\nprint(experiment_uid)"
        }, 
        {
            "source": "#### LIST stored experiments", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------  ------------------------\nGUID                                  NAME                          CREATED\n0c8803bf-122f-4066-be77-39d6aef4de3d  MNIST experiment              2018-03-15T10:35:54.116Z\n207928e4-2634-48b2-80d1-acbc3eed3db2  MNIST experiment              2018-03-15T10:21:23.857Z\n298cc42b-3138-43a2-a690-bf18292e7f65  STYLE experiment              2018-03-16T10:34:03.736Z\n3b274502-a203-430f-b2b2-903f1e713d40  MNIST experiment              2018-03-16T10:00:46.078Z\n496a28ea-7887-4b3e-b4c0-28d21b831333  MNIST experiment              2018-03-15T15:41:47.219Z\n4bae6fcb-f490-45b7-bdaf-559030b6a0f9  STYLE experiment              2018-03-15T21:46:05.795Z\nceb102b9-a05a-4e42-8212-4ac1e0fe115e  STYLE experiment              2018-03-16T08:37:30.628Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  Distributed MNIST experiment  2018-03-28T09:46:25.089Z\nd8cf0def-3a2c-4cee-83b9-79b5eedff2eb  STYLE experiment              2018-03-15T20:57:23.876Z\ndc222702-508e-4c27-aae5-a0af07c3a810  MNIST experiment              2018-03-16T12:35:40.804Z\nf35cea6e-4461-45d2-b1b7-d2250fc1a59e  STYLE experiment              2018-03-15T20:35:12.928Z\nff7bada4-7530-4c1d-8de1-59684727e021  STYLE experiment              2018-03-16T10:04:35.729Z\n------------------------------------  ----------------------------  ------------------------\n"
                }
            ], 
            "source": "client.repository.list_experiments()"
        }, 
        {
            "source": "#### UPDATE experiment\nYou can update experiment definition by calling below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "experiment_details = client.repository.update_experiment(experiment_uid, experiment_metadata)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "#### GET experiment definition details", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "details = client.repository.get_experiment_details(experiment_uid)"
        }, 
        {
            "source": "#### Delete experiment definition from repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "client.repository.delete(experiment_uid)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"run\"></a>\n## 4. Run experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 4.1 Run experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Tip:** To run experiment in background set optional parameter `asynchrounous=True` (or remove it)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "\n\n#############################################################\n\nRunning '5abc65a2-0ab4-480b-a356-23bea3442d3c' experiment run\n\n#############################################################\n\n\n\n\n--------------------------------------------------------------------\nRun of '5abc65a2-0ab4-480b-a356-23bea3442d3c' finished successfully.\n--------------------------------------------------------------------\n\n\n"
                }
            ], 
            "source": "experiment_run_details = client.experiments.run(experiment_uid, asynchronous=False)"
        }, 
        {
            "source": "As we can see experiment run has fininshed.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### GET experiment run UID", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "98590d53-c70a-470c-8fde-dcb017a8dfaf\n"
                }
            ], 
            "source": "experiment_run_id = client.experiments.get_run_uid(experiment_run_details)\nprint(experiment_run_id)"
        }, 
        {
            "source": "#### GET experiment details\nUse below method to get all information about particular experiment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "experiment_details = client.experiments.get_details(experiment_uid)"
        }, 
        {
            "source": "You can use below print method to display experiment details (change cell type to code).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "print(json.dumps(experiment_details, indent=2))", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "#### GET run details\nUse below method to get particular experiment run details.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "run_details = client.experiments.get_run_details(experiment_run_id)"
        }, 
        {
            "source": "#### GET experiment run status\nYou can check experiment run status by calling `client.experiments.get_status(run_uid)` method. This is quite useful when running experiment in background.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'current_iteration': 1, 'submitted_at': '2018-03-28T10:08:02Z', 'state': 'completed', 'current_at': '2018-03-28T10:08:02Z'}\n"
                }
            ], 
            "source": "status = client.experiments.get_status(experiment_run_id)\nprint(status)"
        }, 
        {
            "source": "#### LIST training runs trigered by experiment run", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------  --------------  ---------  --------------------  --------------------  -----------\nGUID (training)     NAME            STATE      SUBMITTED             FINISHED              PERFORMANCE\ntraining-3F-0Q8kmg  tf-distributed  completed  2018-03-28T10:08:04Z  2018-03-28T10:10:01Z  -\ntraining-GcLAQ8kiR  tf-horovod      completed  2018-03-28T10:08:08Z  2018-03-28T10:10:36Z  -\ntraining-PuUAQUkig  tf-ddl          completed  2018-03-28T10:08:09Z  2018-03-28T10:11:13Z  -\n------------------  --------------  ---------  --------------------  --------------------  -----------\n"
                }
            ], 
            "source": "client.experiments.list_training_runs(experiment_run_id)"
        }, 
        {
            "source": "#### MONITOR experiment run\nYou can monitor experiment run by calling `client.experiments.monitor_logs(run_uid)`. This method will stream training logs content to console.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": null, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.experiments.monitor_logs(experiment_run_id)"
        }, 
        {
            "source": "#### LIST experiment runs\nYou can list all experiment run using `client.experiments.list()` method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ------------------------------------  ----------------------------  ---------  --------------------\nGUID (experiment)                     GUID (run)                            NAME (experiment)             STATE      CREATED\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  5abc65a2-0ab4-480b-a356-23bea3442d3c  Distributed MNIST experiment  completed  2018-03-28T09:53:03Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  98590d53-c70a-470c-8fde-dcb017a8dfaf  Distributed MNIST experiment  completed  2018-03-28T10:08:02Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  b280a72e-9c9f-4e1b-8fe8-f5037271407c  Distributed MNIST experiment  completed  2018-03-28T09:47:07Z\n------------------------------------  ------------------------------------  ----------------------------  ---------  --------------------\n"
                }
            ], 
            "source": "client.experiments.list_runs()"
        }, 
        {
            "source": "#### GET training runs uids", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'tf-horovod': 'training-GcLAQ8kiR', 'tf-distributed': 'training-3F-0Q8kmg', 'tf-ddl': 'training-PuUAQUkig'}\n"
                }
            ], 
            "source": "training_runs = {rd['training_reference_name']: rd['training_guid'] for rd in run_details['entity']['training_statuses']}\n\nprint(training_runs)"
        }, 
        {
            "source": "Training run uid can be used to monitor selected training within experiment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "training_uids = client.experiments.get_training_uids(run_details)"
        }, 
        {
            "execution_count": null, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.training.monitor_logs(training_uids[0])"
        }, 
        {
            "source": "<a id=\"model\"></a>\n### 4.2 Store trained model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Call this method to save model in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Saving model: tf-horovod...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/84070d90-6db7-4a12-8da6-26aaf18e9378/versions/7debe8e8-7d3a-49fd-ba49-9b5409537b47', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/84070d90-6db7-4a12-8da6-26aaf18e9378', 'ml_pipeline_url': '', 'ml_asset_guid': '84070d90-6db7-4a12-8da6-26aaf18e9378'}, 'metadata': {'guid': 'training-GcLAQ8kiR', 'created_at': '2018-03-28T10:08:05Z', 'modified_at': '2018-03-28T10:08:05Z', 'url': '/v3/models/training-GcLAQ8kiR'}}\nSaving model: tf-distributed...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca/versions/1675377e-c7ae-4290-b2b4-37a538330fe3', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca', 'ml_pipeline_url': '', 'ml_asset_guid': 'b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca'}, 'metadata': {'guid': 'training-3F-0Q8kmg', 'created_at': '2018-03-28T10:08:02Z', 'modified_at': '2018-03-28T10:08:02Z', 'url': '/v3/models/training-3F-0Q8kmg'}}\nSaving model: tf-ddl...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/91149430-5588-4ee6-ac6a-695cd2392aa8/versions/c05acb65-e9dd-48b0-88a4-3935f6fd87d2', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/91149430-5588-4ee6-ac6a-695cd2392aa8', 'ml_pipeline_url': '', 'ml_asset_guid': '91149430-5588-4ee6-ac6a-695cd2392aa8'}, 'metadata': {'guid': 'training-PuUAQUkig', 'created_at': '2018-03-28T10:08:08Z', 'modified_at': '2018-03-28T10:08:08Z', 'url': '/v3/models/training-PuUAQUkig'}}\n"
                }
            ], 
            "source": "saved_model_details = {}\n\nfor training_run in training_runs:\n    print(\"Saving model: {}...\".format(training_run))\n    saved_model_details[training_run] = client.repository.store_model(training_runs[training_run], {\"name\": \"My cool mnist model {}\".format(training_run)})\n    print(saved_model_details[training_run])"
        }, 
        {
            "source": "**Tip:** Run below code to extract stored model guid from dictionary.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Saved model guid: {'tf-horovod': '84070d90-6db7-4a12-8da6-26aaf18e9378', 'tf-distributed': 'b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca', 'tf-ddl': '91149430-5588-4ee6-ac6a-695cd2392aa8'}\n"
                }
            ], 
            "source": "saved_model_uids = {training_run: client.repository.get_model_uid(saved_model_details[training_run]) for training_run in saved_model_details}\nprint(\"Saved model guid: {}\".format(saved_model_uids))"
        }, 
        {
            "source": "#### Call this method to list stored models", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------  ------------------------  --------------\nGUID                                  NAME                                CREATED                   FRAMEWORK\n43986a55-0f0e-493b-9b98-0712f918b0ba  MNIST CNN model                     2018-03-15T16:12:06.503Z  tensorflow-1.5\na5a952e1-fa6f-4c64-88f7-293b367b7e1f  MNIST best model                    2018-03-16T10:12:16.773Z  tensorflow-1.5\nc755c740-dab6-4d5d-b873-1060885d55d5  MNIST best model                    2018-03-16T10:12:31.990Z  tensorflow-1.5\n861b6b73-6861-49c0-867c-d171d6d9d545  MNIST best model                    2018-03-16T13:00:40.461Z  tensorflow-1.5\nf7219c8f-fccb-4d26-a1bd-f51544090eae  training-S3ttnlgig_2 best model     2018-03-16T13:32:33.747Z  tensorflow-1.5\n327242a2-d2c0-400f-b673-4f33ad7ec17d  product-line-prediction             2018-03-16T18:08:31.260Z  mllib-2.0\na0cb79ca-e937-4cbf-b7e5-23dad2d2ecec  drug-selection                      2018-03-16T18:10:25.046Z  mllib-2.0\n97822727-4bd1-4df0-95df-8ff9536ea01d  My cool mnist model                 2018-03-28T09:02:31.509Z  tensorflow-1.5\n84070d90-6db7-4a12-8da6-26aaf18e9378  My cool mnist model tf-horovod      2018-03-28T11:15:53.446Z  tensorflow-1.5\nb6e59ce5-7d24-4ce1-90f8-f6a918ad71ca  My cool mnist model tf-distributed  2018-03-28T11:15:56.061Z  tensorflow-1.5\n91149430-5588-4ee6-ac6a-695cd2392aa8  My cool mnist model tf-ddl          2018-03-28T11:15:59.813Z  tensorflow-1.5\n------------------------------------  ----------------------------------  ------------------------  --------------\n"
                }
            ], 
            "source": "client.repository.list_models()"
        }, 
        {
            "source": "### 4.3 Delete experiment-run", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can delete experiment runs and results by running below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "client.experiments.delete(experiment_run_id)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "## 5. Create online deployment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can deployed stored model as webservice (online) by running below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '76dd8d4a-0cab-435a-9430-d488c7144bf2' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='76dd8d4a-0cab-435a-9430-d488c7144bf2'\n------------------------------------------------------------------------------------------------\n\n\n"
                }
            ], 
            "source": "deployment_details = client.deployments.create(name=\"Mnist model deployment - tf-ddl\", model_uid=saved_model_uids['tf-ddl'])"
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/91149430-5588-4ee6-ac6a-695cd2392aa8/deployments/76dd8d4a-0cab-435a-9430-d488c7144bf2/online\n"
                }
            ], 
            "source": "scoring_url = client.deployments.get_scoring_url(deployment_details)\nprint(scoring_url)"
        }, 
        {
            "source": "## 6. Scoring", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "Prepare sample scoring data to score deployed model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "image1 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.639,0.757,0.596,0.361,0.200,0.200,0.200,0.200,0.122,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.992,0.988,0.992,0.988,0.992,0.988,0.992,0.753,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.082,0.400,0.243,0.400,0.400,0.239,0.718,0.157,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.239,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.361,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.678,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.992,0.988,0.875,0.796,0.796,0.796,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.482,0.996,0.992,0.878,0.796,0.796,0.875,0.996,0.278,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.953,0.992,0.510,0.078,0.000,0.000,0.078,0.992,0.910,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.596,0.992,0.718,0.000,0.000,0.000,0.000,0.000,0.518,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.592,0.078,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.443,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.322,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.757,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.796,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.996,0.992,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.400,0.000,0.000,0.000,0.000,0.000,0.082,0.796,0.992,0.510,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.800,0.482,0.000,0.000,0.000,0.000,0.161,0.678,0.992,0.796,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.635,0.875,0.400,0.082,0.400,0.639,0.953,0.992,0.671,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.918,0.992,1.000,0.992,1.000,0.675,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.118,0.514,0.753,0.435,0.196,0.039,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]\nimage2 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.196,0.878,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.275,0.114,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.475,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.580,0.659,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.016,0.765,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.376,0.824,0.043,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.271,0.988,0.525,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.447,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.176,0.925,0.851,0.047,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.753,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.659,0.969,0.208,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.071,1.000,0.992,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.329,0.949,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.553,0.992,0.741,0.020,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.125,0.910,0.980,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.059,0.882,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.525,0.988,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.988,0.643,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.655,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.035,0.800,0.820,0.071,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.992,0.992,0.420,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.780,0.333,0.333,0.333,0.333,0.506,0.643,0.765,0.988,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.667,0.961,0.988,0.988,0.988,0.988,0.910,0.906,0.984,0.988,0.988,0.035,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.192,0.329,0.329,0.329,0.329,0.000,0.000,0.631,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.176,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.502,0.992,0.992,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.529,0.988,0.957,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.910,0.925,0.435,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.702,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]"
        }, 
        {
            "execution_count": 34, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np"
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT4AAACFCAYAAADCQpQyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAB2lJREFUeJzt3U+IjV8cx/Fz+ZV/STFkQVMTNf4khSzGRLOwIAtD+VeTJAtlJClCyb9hYWEW/mStJH9SQjYSCxYipqz8m5KFQUqSML/Fbzq/8z3uvWau53nuc+/n/Vp9T2eac9Sdj+c59zzPKfT39zsAUDKs2hMAgKwRfADkEHwA5BB8AOQQfADkEHwA5BB8AOQQfADkEHwA5PyT8Xg8JpIfhWpPoI7wuc6PQX2uueIDIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyAn6zcw14SOjg7TfvPmja9nzpxp+lpaWkx7xowZFY3Z0NDg68bGxop+B/D48WPTbm9v9/WrV69SH//27dumHf49TJ06NfXxB4srPgByCD4Acgr9/Zmek5LbQ1n6+vp8vWDBAtPX29vr62HD7P8Vv379Mu2wfyh9ra2tvu7s7DR94e1KgjhsKDm5+Vx3dXWZdnd3t6/fvXuX+vjbtm0z7ffv3/v6woULqY/vOGwIAIoj+ADIIfgAyGE7y4Bwy0q4puecc6dPn/b1li1bTF+4Nuicc1euXPF1vLXl+fPnJccPt8kcOXKkZF9zc3PJ3wFNP3788PWNGzeqOBPn5s+fb9onTpzw9devX03f6NGjM5lTMVzxAZBD8AGQw61uEfGWlXLCJy6c+/1WOBRuWSln7969ps3tLcq5c+eOrx88eGD6du3alelcPn78aNrh8g63ugBQRQQfADkEHwA5rPEVET9OlrVFixZVdXzkW09Pj2mvX7/e101NTaYvXi9O27Vr1zIdr1Jc8QGQQ/ABkMOt7oDwa/ehbGcBsnb48GHT/vLli6/jJzfGjBmT+nzCLSx37941fXn9W8rnrAAgRQQfADkEHwA5rPENuHr1qq+rvZ0FiF2+fNnXN2/eNH3Tp0/3dfz28CyEa47xmt6SJUt8PW7cuKym9Edc8QGQQ/ABkMNhQwOGDx/u6/hyfcqUKb6eOHHioH/n7t27TTulQ4MqxWFDyUn9c71mzRpfhy+7dc65kydP+nrr1q1pT8W8tNc55xYuXOjrT58+mb7wtrytrS3dif2Hw4YAoBiCD4Acgg+AHNntLEePHjXtcK3zb7azhAcor1692vTdunXL10uXLq14DNS/z58/m/bDhw9L/mwW63qhs2fPmvaHDx98HR+wldG63pBxxQdADsEHQI7UrW74Bpauri7TVyj8/y14/PLG7du3+3rChAllxwgv+ydNmmT6Ojo6fB0eEOPc77cI0Pb9+3fTfvv2ra/Xrl2b9XSMFy9elOybNWtWhjOpHFd8AOQQfADkEHwA5Eit8XV3d/s6Ptw43F5y8ODBiscI1wD37Nlj+sJ1xXv37pk+1vgQGjt2rGnPmTPH10+fPjV94RuQx48fn8p8wm1aly5dKvlzLS0tqYyfNK74AMgh+ADIIfgAyJFa49u8ebOvw317zjl36NChxMdbuXKlaR8/fjzxMVCfRo4cadrTpk3zdfxaquXLl/t6586dFY337Nkz03758qVpv3792tflTk7L66lqsdqYJQAkiOADIEfqVrexsdHXp06dSn28eMsKhxihUgcOHPB1/Nb069ev+3rdunUV/f74Ucx4KSh8FLOcTZs2VTR+1rjiAyCH4AMgh+ADIKeu1/jir/3PnTvn6/hQ5jSEh5Q7Vztf9SN/wkcaL168aPqePHni63KvjCpn1apVZfs3btzo6/Pnz5f8uXgbTl7xlwhADsEHQE5d3+oeO3bMtP/09uQk9PX1+TrezhJuEWhtbU19LtAwd+7conWSmpqaBvVzPT09pj179uw0pvPXuOIDIIfgAyCH4AMgp+7W+MI1tvCtsc7Zt7OkMZ5zzi1btszX8WM/4eltvHEZtSR83LLco5d5XdOLccUHQA7BB0BO3d3qNjQ0+Do+eCW+La1EeCi5c87t27fPtB89euTrefPmmb7wYHKgloRPHdXDE0i1/y8AgCEi+ADIIfgAyKm7Nb5Q/NV6+HaWcC3QOefa29t9Hb/V5f79+76O37gSH0we/p4zZ86YviwemQPS8O3bt5J9I0aMyHAmyeCKD4Acgg+AnEJ8cEnKMh0s3nqyePFiX8eHp4S70eOv68O+8FbWOec2bNhg2nF/jhX+/CMYpEw/19UwefJkX//8+dP07d+/39ednZ2ZzamEQX2uueIDIIfgAyCH4AMgp67X+GK9vb2+Dre2OGfflhyv04VvS25ubjZ9o0aNSnKKWWKNLzl1v8a3YsUKX+/YscP0tbW1ZT2dcljjA4BiCD4AcqRudWFwq5scPtf5wa0uABRD8AGQQ/ABkEPwAZBD8AGQQ/ABkEPwAZBD8AGQQ/ABkEPwAZCT9WFDPCaFesTnusZwxQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQM6/vfWVmGCcY80AAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x2ac3c72fc320>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "for i, image in enumerate([image1, image2]):\n    plt.subplot(2, 2, i + 1)\n    plt.axis('off')\n    plt.imshow((np.reshape(image, (28, 28)) * 255).astype(np.uint8), cmap=plt.cm.gray_r, interpolation='nearest')"
        }, 
        {
            "source": "**Tip**: If plot fails please upgrade dask library.\n\n!pip install dask --upgrade", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Build scoring dictionary consisting of two digits and send it to deployed model to get predictions.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 36, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "scoring_data = {\"values\": [image1, image2]}"
        }, 
        {
            "execution_count": 37, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Scoring result: {'values': [8, 9]}\n"
                }
            ], 
            "source": "predictions = client.deployments.score(scoring_url, scoring_data)\nprint(\"Scoring result: \" + str(predictions))"
        }, 
        {
            "source": "<HR>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"summary\"></a>\n## 7. Summary and next steps     ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": " You successfully completed this notebook! You learned how to use `watson-machine-learning-client` to run experiments. Check out our _[Online Documentation](https://dataplatform.ibm.com/docs/content/analyze-data/wml-ai.html?context=analytics)_ for more samples, tutorials, documentation, how-tos, and blog posts. ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "### Authors\n\n**Wojciech Sobala**, Data Scientist in IBM working on Watson Machine Learning service (public Cloud).\n\n**Lukasz Cmielowski**, PhD, is a Automation Architect and Data Scientist in IBM with a track record of developing enterprise-level applications that substantially increases clients' ability to turn data into actionable knowledge.\n\n", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Copyright \u00a9 2017 IBM. This notebook and its source code are released under the terms of the MIT License.", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}