Use distributed TensorFlow to predict digits.ipynb

{
    "nbformat_minor": 1, 
    "cells": [
        {
            "source": "<div style=\"background:#F5F7FA; height:100px; padding: 2em; font-size:14px;\">\n<span style=\"font-size:18px;color:#152935;\">Want to do more?</span><span style=\"border: 1px solid #3d70b2;padding: 15px;float:right;margin-right:40px; color:#3d70b2; \"><a href=\"https://ibm.co/wsnotebooks\" target=\"_blank\" style=\"color: #3d70b2;text-decoration: none;\">Sign Up</a></span><br>\n<span style=\"color:#5A6872;\"> Try out this notebook with your free trial of IBM Watson Studio.</span>\n</div>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<table style=\"border: none\" align=\"left\">\n   <tr style=\"border: none\">\n      <th style=\"border: none\"><font face=\"verdana\" size=\"4\" color=\"black\"><b>Use distributed TensorFlow to predict digits</b></font></th>\n      <th style=\"border: none\"><img src=\"https://github.com/pmservice/customer-satisfaction-prediction/blob/master/app/static/images/ml_icon_gray.png?raw=true\" alt=\"Watson Machine Learning icon\" height=\"40\" width=\"40\"></th>\n   </tr> \n   <tr style=\"border: none\">\n       <td style=\"border: none\"><img src=\"https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/hand-written-digit-recognition/images/experiment_banner.png\" width=\"800\" alt=\"Icon\"></td>\n   </tr>\n</table>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "This notebook contains steps and code to demonstrate support of Deep Learning model training and scoring in Watson Machine Learning Service. This notebook introduces commands for getting data, training_definition persistance to Watson Machine Learning repository, model training, model persistance, model deployment and scoring.\n\nSome familiarity with Python is helpful. This notebook uses Python 3.\n\n\n## Learning goals\n\nThe learning goals of this notebook are:\n\n-  Working with Watson Machine Learning experiments to train Deep Learning models (TensorFlow).\n-  Saving trained models in Watson Machine Learning repository.\n-  Online deployment and scoring of trained model.\n\n\n## Contents\n\nThis notebook contains the following parts:\n\n1.\t[Set up the environment](#setup)\n2.\t[Training definition](#definition)\n3.\t[Experiment definition](#experiment)\n4.\t[Experiment Run](#run)\n5.\t[Deploy and Score](#scoring)\n6.\t[Summary and next steps](#summary)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"setup\"></a>\n## 1. Set up the environment\n\nBefore you use the sample code in this notebook, you must perform the following setup tasks:\n\n-  Create a [Watson Machine Learning (WML) Service](https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/) instance (a free plan is offered and information about how to create the instance is [here](https://dataplatform.ibm.com/docs/content/analyze-data/wml-setup.html))\n-  Create a [Cloud Object Storage (COS)](https://console.bluemix.net/catalog/infrastructure/cloud-object-storage) instance (a lite plan is offered and information about how to order storage is [here](https://console.bluemix.net/docs/services/cloud-object-storage/basics/order-storage.html#order-storage)).\n    - After you create COS instance, go to your COS dashboard.\n    - In **Service credentials** tab, click **New Credential**.\n    - Add the inline configuration parameter: {[\"HMAC\"](https://console.bluemix.net/docs/services/cloud-object-storage/hmac/credentials.html#using-hmac-credentials):true}, click **Add**.\n\n    This configuration parameter adds the following section to the instance credentials, (for use later in this notebook):\n    ```\n      \"cos_hmac_keys\": {\n            \"access_key_id\": \"722432c254bc4eaa96e05897bf2779e2\",\n            \"secret_access_key\": \"286965ac10ecd4de8b44306288c7f5a3e3cf81976a03075c\"\n       }\n    ```", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.1 Work with Cloud Object Storage (COS)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Import the Boto library, which allows Python developers to manage COS.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import ibm_boto3\nfrom ibm_botocore.client import Config\nimport os, json, warnings"
        }, 
        {
            "source": "Define the endpoint you will use. \n\n1. To do this, first enter your COS credentials in the following cell. \nYou can find these credentials in your COS instance dashboard under the **Service credentials** tab.\n**Note** the HMAC key, described in [set up the environment](#setup) is included in these credentials.\n\n2. Go to the **Endpoint** tab in the COS instance's dashboard to get the endpoint information.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "cos_credentials = {\n  \"apikey\": \"***\",\n  \"cos_hmac_keys\": {\n    \"access_key_id\": \"***\",\n    \"secret_access_key\": \"***\"\n  },\n  \"endpoints\": \"https://cos-service.bluemix.net/endpoints\",\n  \"iam_apikey_description\": \"***\",\n  \"iam_apikey_name\": \"***\",\n  \"iam_role_crn\": \"crn:v1:bluemix:public:iam::::serviceRole:Writer\",\n  \"iam_serviceid_crn\": \"***\",\n  \"resource_instance_id\": \"***\"\n}\n\napi_key = cos_credentials['apikey']\nservice_instance_id = cos_credentials['resource_instance_id']\nauth_endpoint = 'https://iam.bluemix.net/oidc/token'\nservice_endpoint = 'https://s3-api.us-geo.objectstorage.softlayer.net'"
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by DSX for sharing."
        }, 
        {
            "source": "We create Boto resource by providing type, endpoint_url and credentials.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "cos = ibm_boto3.resource('s3',\n                         ibm_api_key_id=api_key,\n                         ibm_service_instance_id=service_instance_id,\n                         ibm_auth_endpoint=auth_endpoint,\n                         config=Config(signature_version='oauth'),\n                         endpoint_url=service_endpoint)"
        }, 
        {
            "source": "Let's create the buckets we will use to store training data and training results.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Creating bucket \"train-data-experiment-2018-03-28-09-34\"...\nCreating bucket \"train-result-experiment-2018-03-28-09-34\"...\n"
                }
            ], 
            "source": "import datetime\n\ndate_time = datetime.datetime.now()\ndate_time = date_time.strftime(\"%Y-%m-%d-%H-%M\")\n\nbuckets = {'data': 'train-data-experiment-' + date_time, 'results': 'train-result-experiment-' + date_time}\n\nfor bucket in buckets.values():\n    if not cos.Bucket(bucket) in cos.buckets.all():\n        print('Creating bucket \"{}\"...'.format(bucket))\n        try:\n            cos.create_bucket(Bucket=bucket)\n        except ibm_boto3.exceptions.ibm_botocore.client.ClientError as e:\n            print('Error: {}.'.format(e.response['Error']['Message']))\n    else:\n        print('Bucket \"{}\" already exists.'.format(bucket))"
        }, 
        {
            "source": "Now we should have 2 new buckets created.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.2 Download training data and upload it to COS buckets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Let's download our training data and upload them to 'training-data' bucket.\nWe will create a list of links for training dataset", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "data_links = ['http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz',\n              'http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz']"
        }, 
        {
            "source": "Below snippet will create 'MNIST_DATA' folder and download the files from links.\n\n**Tip:** First, please install wget library by the command below\n`!pip install wget`", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 21, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['MNIST_DATA/train-labels-idx1-ubyte.gz',\n 'MNIST_DATA/t10k-labels-idx1-ubyte.gz',\n 'MNIST_DATA/train-images-idx3-ubyte.gz',\n 'MNIST_DATA/t10k-images-idx3-ubyte.gz']"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "import wget, glob\n\ndata_dir = 'MNIST_DATA'\nif not os.path.isdir(data_dir):\n    os.mkdir(data_dir)\n\nfor link in data_links:\n    if not os.path.isfile(os.path.join(data_dir, os.path.join(link.split('/')[-1]))):\n        wget.download(link, out=data_dir)\n\nglob.glob('MNIST_DATA/*')"
        }, 
        {
            "source": "Upload the data files to created buckets.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "train-labels-idx1-ubyte.gz is uploaded.\nt10k-labels-idx1-ubyte.gz is uploaded.\ntrain-images-idx3-ubyte.gz is uploaded.\nt10k-images-idx3-ubyte.gz is uploaded.\n"
                }
            ], 
            "source": "bucket_name = buckets['data']\nbucket_obj = cos.Bucket(bucket_name)\n\nfor filename in os.listdir(data_dir):\n    with open(os.path.join(data_dir, filename), 'rb') as data: \n        bucket_obj.upload_file(os.path.join(data_dir, filename), filename)\n        print('{} is uploaded.'.format(filename))"
        }, 
        {
            "source": "Let's see the list of created buckets and their contents.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Object key: t10k-images-idx3-ubyte.gz, size: 1610.2kB\nObject key: t10k-labels-idx1-ubyte.gz, size:   4.4kB\nObject key: train-images-idx3-ubyte.gz, size: 9680.1kB\nObject key: train-labels-idx1-ubyte.gz, size:  28.2kB\n"
                }
            ], 
            "source": "for obj in bucket_obj.objects.all():\n    print('Object key: {}, size: {:5.1f}kB'.format(obj.key, obj.size/1024))"
        }, 
        {
            "source": "We are done with Cloud Object Storage, we are ready to train our model!", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 1.3 Work with the Watson Machine Learning instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Authenticate to Watson Machine Learning service on IBM Cloud.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Authenticate to the Watson Machine Learning service on IBM Cloud.\n\n**Tip**: Authentication information (your credentials) can be found in the [Service Credentials](https://console.bluemix.net/docs/services/service_credentials.html#service_credentials) tab of the service instance that you created on IBM Cloud. <BR>If you cannot see the **instance_id** field in **Service Credentials**, click **New credential (+)** to generate new authentication information. \n\n**Action**: Enter your Watson Machine Learning service instance credentials here.\n", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "wml_credentials = {\n  \"url\": \"https://ibm-watson-ml.mybluemix.net\",\n  \"access_key\": \"***\",\n  \"username\": \"***\",\n  \"password\": \"***\",\n  \"instance_id\": \"***\"\n}"
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by DSX for sharing."
        }, 
        {
            "source": "#### Install `watson-machine-learning-client` from pypi", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client"
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Requirement already up-to-date: watson-machine-learning-client in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages\nRequirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client)\nRequirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\n"
                }
            ], 
            "source": "!pip install --upgrade watson-machine-learning-client --upgrade"
        }, 
        {
            "source": "#### Import `watson-machine-learning-client` and authenticate to service instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient"
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client = WatsonMachineLearningAPIClient(wml_credentials)"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "1.0.53\n"
                }
            ], 
            "source": "print(client.version)"
        }, 
        {
            "source": "**Note:** `watson-machine-learning-client` documentation can be found here [http://wml-api-pyclient.mybluemix.net/](http://wml-api-pyclient.mybluemix.net/).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"definition\"></a>\n## 2. Training definitions", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 2.2 Save training definition ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Prepare training definition metadata", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_tf_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - distributed\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"PS_HOSTS_COUNT=1 ./launcher.py python3 ${MODEL_DIR}/mnist_dist.py --data_dir ${DATA_DIR}\"\n            }"
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_horovod_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - horovod\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow-horovod\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"python tensorflow_mnist.py\"\n            }"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model_definition_ddl_metadata = {\n            client.repository.DefinitionMetaNames.NAME: \"Tensorflow - ddl\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_NAME: \"tensorflow-ddl\",\n            client.repository.DefinitionMetaNames.FRAMEWORK_VERSION: \"1.5\",\n            client.repository.DefinitionMetaNames.RUNTIME_NAME: \"python\",\n            client.repository.DefinitionMetaNames.RUNTIME_VERSION: \"3.5\",\n            client.repository.DefinitionMetaNames.EXECUTION_COMMAND: \"python mnist-train-ddl.py; python transform.py\"\n            }"
        }, 
        {
            "source": "#### Get sample model definition content files from git", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 22, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['tf_horovod.zip', 'tf_ddl.zip', 'tf_distributed.zip']"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "filename_tf = 'tf_distributed.zip'\nfilename_horovod = 'tf_horovod.zip'\nfilename_ddl = 'tf_ddl.zip'\n\nif os.path.isfile(filename_tf):\n    os.remove(filename_tf)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/native/' + filename_tf)\n\nif os.path.isfile(filename_horovod):\n    os.remove(filename_horovod)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/horovod/' + filename_horovod)\n\nif os.path.isfile(filename_ddl):\n    os.remove(filename_ddl)\nwget.download('https://github.com/pmservice/wml-sample-models/raw/master/tensorflow/distributed-hand-written-digits/ddl/' + filename_ddl)\n\nglob.glob('tf_*.zip')"
        }, 
        {
            "source": "#### Publish training definition in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "##### Store definition: native distributed tensorflow", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/06fa6000-9b1b-4cf0-8932-f35cf449361e\n"
                }
            ], 
            "source": "definition_tf_details = client.repository.store_definition(filename_tf, model_definition_tf_metadata)\n\ndefinition_tf_url = client.repository.get_definition_url(definition_tf_details)\ndefinition_tf_uid = client.repository.get_definition_uid(definition_tf_details)\nprint(definition_tf_url)"
        }, 
        {
            "source": "##### Store definition: horovod", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/a473639d-4280-4157-b360-7df30f965788\n"
                }
            ], 
            "source": "definition_horovod_details = client.repository.store_definition(filename_horovod, model_definition_horovod_metadata)\n\ndefinition_horovod_url = client.repository.get_definition_url(definition_horovod_details)\ndefinition_horovod_uid = client.repository.get_definition_uid(definition_horovod_details)\nprint(definition_horovod_url)"
        }, 
        {
            "source": "##### Store definition: ibm DDL", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 25, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/ml_assets/training_definitions/c5988360-4241-4565-8e6c-05fde3c8b27e\n"
                }
            ], 
            "source": "definition_ddl_details = client.repository.store_definition(filename_ddl, model_definition_ddl_metadata)\n\ndefinition_ddl_url = client.repository.get_definition_url(definition_ddl_details)\ndefinition_ddl_uid = client.repository.get_definition_uid(definition_ddl_details)\nprint(definition_ddl_url)"
        }, 
        {
            "source": "#### LIST stored definitions", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------------  ------------------------  ------------------\nGUID                                  NAME                                      CREATED                   FRAMEWORK\nea8c0dd5-8a6b-4e5e-a9cd-74a0c573c1de  MNIST-MLP                                 2018-03-15T10:20:58.646Z  tensorflow\n8095a87c-5c94-4f43-b5fd-468b2d72a3c8  MNIST-CNN                                 2018-03-15T10:21:00.033Z  tensorflow\nbebc485e-bcc9-45b7-9bce-1ba20acc3b53  MNIST-MLP                                 2018-03-15T10:35:04.997Z  tensorflow\nf26539ff-43b5-4100-8915-7b0be22b2211  MNIST-CNN                                 2018-03-15T10:35:05.815Z  tensorflow\nbed45da1-6372-4cb1-b889-e0d402cfe62a  MNIST-MLP                                 2018-03-15T15:41:40.335Z  tensorflow\nc80aaeef-1c03-4474-8891-535aa6dae81e  MNIST-CNN                                 2018-03-15T15:41:41.174Z  tensorflow\nf819f724-8e1c-4903-a04c-eee5dfc048a4  style transfer van gogh                   2018-03-15T20:33:03.354Z  tensorflow\nf5c4e713-27d9-4529-a473-10dc88383a52  style transfer kandinsky                  2018-03-15T20:33:05.032Z  tensorflow\naf72a944-cc83-4aaf-9152-9f25c1912bf4  style transfer van gogh                   2018-03-15T20:57:13.232Z  tensorflow\n4fb3987a-f979-4e1e-a26f-1acbe11a08e7  style transfer kandinsky                  2018-03-15T20:57:13.892Z  tensorflow\n1421337a-c2a8-4153-8554-accdc55bbd27  style transfer van gogh                   2018-03-15T21:45:46.889Z  tensorflow\n0f8fa74a-f821-475c-85c9-a2b507ed6c63  style transfer kandinsky                  2018-03-15T21:45:48.344Z  tensorflow\ncd31e290-6636-4586-8097-e3c3ac7fef8c  style transfer van gogh                   2018-03-16T08:37:18.880Z  tensorflow\n76b144fc-100c-45ee-80ae-ef0d44a87cd6  style transfer kandinsky                  2018-03-16T08:37:19.383Z  tensorflow\n7dc7e7cc-ae5b-4db2-8705-32174324010a  MNIST-MLP                                 2018-03-16T09:59:57.138Z  tensorflow\n6a024dde-d6bc-45d3-8463-895452134c29  MNIST-CNN                                 2018-03-16T09:59:57.888Z  tensorflow\n32c64603-13d2-4ec6-b3ca-4ee4650fbd8e  style transfer van gogh                   2018-03-16T10:04:24.779Z  tensorflow\n18498298-3e01-43a0-a1f7-955ecc118a4c  style transfer kandinsky                  2018-03-16T10:04:25.661Z  tensorflow\n9e446ccc-8b1d-4e46-a549-655f9b16a577  style transfer van gogh                   2018-03-16T10:33:54.588Z  tensorflow\n1f54975c-0104-4fcb-910c-ae57e3272d7c  style transfer kandinsky                  2018-03-16T10:33:55.170Z  tensorflow\n3501d90f-8599-4154-a8a5-d6ddafac15a8  MNIST-MLP                                 2018-03-16T11:53:19.116Z  tensorflow\n7c1b279f-7556-4beb-bee3-5516a0daa433  MNIST-CNN                                 2018-03-16T11:53:19.907Z  tensorflow\nd56f3240-b49e-4b30-98e8-fb1ca8ab6dd6  MNIST-MLP                                 2018-03-16T12:35:29.322Z  tensorflow\n970df64d-d363-4524-881d-75320928c695  MNIST-CNN                                 2018-03-16T12:35:30.591Z  tensorflow\n11ceb998-31d9-424e-94d7-51eb8476f143  training def for product-line-prediction  2018-03-16T18:08:30.465Z  mllib\n960d8f27-e636-438d-a8b9-65cc9033e5f6  training def for drug-selection           2018-03-16T18:10:24.191Z  mllib\n60994f57-61e6-4bcc-b025-18622d22c98f  My definition name                        2018-03-28T07:55:40.796Z  tensorflow\n06fa6000-9b1b-4cf0-8932-f35cf449361e  Tensorflow - distributed                  2018-03-28T09:45:06.201Z  tensorflow\na473639d-4280-4157-b360-7df30f965788  Tensorflow - horovod                      2018-03-28T09:45:07.779Z  tensorflow-horovod\nc5988360-4241-4565-8e6c-05fde3c8b27e  Tensorflow - ddl                          2018-03-28T09:45:08.735Z  tensorflow-ddl\n------------------------------------  ----------------------------------------  ------------------------  ------------------\n"
                }
            ], 
            "source": "client.repository.list_definitions()"
        }, 
        {
            "source": "<a id=\"experiment\"></a>\n## 3. Experiment definition", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.1 Save experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Get the list of supported configuration params", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--------------------------  ----  --------\nMETA_PROP NAME              TYPE  REQUIRED\nNAME                        str   Y\nTAGS                        list  N\nDESCRIPTION                 str   N\nAUTHOR_NAME                 str   N\nAUTHOR_EMAIL                str   Y\nEVALUATION_METHOD           str   N\nEVALUATION_METRICS          list  N\nTRAINING_REFERENCES         list  Y\nTRAINING_DATA_REFERENCE     dict  Y\nTRAINING_RESULTS_REFERENCE  dict  Y\n--------------------------  ----  --------\n"
                }
            ], 
            "source": "client.repository.ExperimentMetaNames.show()"
        }, 
        {
            "source": "#### Experiment configuration dictionary\nCreate experiment that will train two models based on previously stored definitions.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "TRAINING_DATA_REFERENCE = {\n                            \"connection\": {\n                                \"endpoint_url\": service_endpoint,\n                                \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n                                \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n                            },\n                            \"source\": {\n                                \"bucket\": buckets[0],\n                            },\n                            \"type\": \"s3\"\n                           }\nTRAINING_RESULTS_REFERENCE = {\n                                \"connection\": {\n                                    \"endpoint_url\": service_endpoint,\n                                    \"access_key_id\": cos_credentials['cos_hmac_keys']['access_key_id'],\n                                    \"secret_access_key\": cos_credentials['cos_hmac_keys']['secret_access_key']\n                                },\n                                \"target\": {\n                                    \"bucket\": buckets[1],\n                                },\n                                \"type\": \"s3\"\n                             }"
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "experiment_metadata = {\n            client.repository.ExperimentMetaNames.NAME: \"Distributed MNIST experiment\",\n            client.repository.ExperimentMetaNames.TRAINING_DATA_REFERENCE: TRAINING_DATA_REFERENCE,\n            client.repository.ExperimentMetaNames.TRAINING_RESULTS_REFERENCE: TRAINING_RESULTS_REFERENCE,\n            client.repository.ExperimentMetaNames.TRAINING_REFERENCES: [\n                        {\n                            \"name\": \"tf-distributed\",\n                            \"training_definition_url\": definition_tf_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        },                        \n                        {\n                            \"name\": \"tf-horovod\",\n                            \"training_definition_url\": definition_horovod_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        },                        \n                        {\n                            \"name\": \"tf-ddl\",\n                            \"training_definition_url\": definition_ddl_url,\n                            \"compute_configuration\": {\"name\": \"p100\", \"nodes\": 2}\n                        }\n                    ],\n                }"
        }, 
        {
            "source": "#### Store experiment in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "d763b2eb-1b9d-440e-8c25-09a0f06ee017\n"
                }
            ], 
            "source": "experiment_details = client.repository.store_experiment(meta_props=experiment_metadata)\n\nexperiment_uid = client.repository.get_experiment_uid(experiment_details)\nprint(experiment_uid)"
        }, 
        {
            "source": "#### LIST stored experiments", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------  ------------------------\nGUID                                  NAME                          CREATED\n0c8803bf-122f-4066-be77-39d6aef4de3d  MNIST experiment              2018-03-15T10:35:54.116Z\n207928e4-2634-48b2-80d1-acbc3eed3db2  MNIST experiment              2018-03-15T10:21:23.857Z\n298cc42b-3138-43a2-a690-bf18292e7f65  STYLE experiment              2018-03-16T10:34:03.736Z\n3b274502-a203-430f-b2b2-903f1e713d40  MNIST experiment              2018-03-16T10:00:46.078Z\n496a28ea-7887-4b3e-b4c0-28d21b831333  MNIST experiment              2018-03-15T15:41:47.219Z\n4bae6fcb-f490-45b7-bdaf-559030b6a0f9  STYLE experiment              2018-03-15T21:46:05.795Z\nceb102b9-a05a-4e42-8212-4ac1e0fe115e  STYLE experiment              2018-03-16T08:37:30.628Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  Distributed MNIST experiment  2018-03-28T09:46:25.089Z\nd8cf0def-3a2c-4cee-83b9-79b5eedff2eb  STYLE experiment              2018-03-15T20:57:23.876Z\ndc222702-508e-4c27-aae5-a0af07c3a810  MNIST experiment              2018-03-16T12:35:40.804Z\nf35cea6e-4461-45d2-b1b7-d2250fc1a59e  STYLE experiment              2018-03-15T20:35:12.928Z\nff7bada4-7530-4c1d-8de1-59684727e021  STYLE experiment              2018-03-16T10:04:35.729Z\n------------------------------------  ----------------------------  ------------------------\n"
                }
            ], 
            "source": "client.repository.list_experiments()"
        }, 
        {
            "source": "#### UPDATE experiment\nYou can update experiment definition by calling below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "experiment_details = client.repository.update_experiment(experiment_uid, experiment_metadata)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "#### GET experiment definition details", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "details = client.repository.get_experiment_details(experiment_uid)"
        }, 
        {
            "source": "#### Delete experiment definition from repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "client.repository.delete(experiment_uid)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"run\"></a>\n## 4. Run experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 4.1 Run experiment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Tip:** To run experiment in background set optional parameter `asynchrounous=True` (or remove it)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "\n\n#############################################################\n\nRunning '5abc65a2-0ab4-480b-a356-23bea3442d3c' experiment run\n\n#############################################################\n\n\n\n\n--------------------------------------------------------------------\nRun of '5abc65a2-0ab4-480b-a356-23bea3442d3c' finished successfully.\n--------------------------------------------------------------------\n\n\n"
                }
            ], 
            "source": "experiment_run_details = client.experiments.run(experiment_uid, asynchronous=False)"
        }, 
        {
            "source": "As we can see experiment run has fininshed.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### GET experiment run UID", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "98590d53-c70a-470c-8fde-dcb017a8dfaf\n"
                }
            ], 
            "source": "experiment_run_id = client.experiments.get_run_uid(experiment_run_details)\nprint(experiment_run_id)"
        }, 
        {
            "source": "#### GET experiment details\nUse below method to get all information about particular experiment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "experiment_details = client.experiments.get_details(experiment_uid)"
        }, 
        {
            "source": "You can use below print method to display experiment details (change cell type to code).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "print(json.dumps(experiment_details, indent=2))", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "#### GET run details\nUse below method to get particular experiment run details.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "run_details = client.experiments.get_run_details(experiment_run_id)"
        }, 
        {
            "source": "#### GET experiment run status\nYou can check experiment run status by calling `client.experiments.get_status(run_uid)` method. This is quite useful when running experiment in background.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'current_iteration': 1, 'submitted_at': '2018-03-28T10:08:02Z', 'state': 'completed', 'current_at': '2018-03-28T10:08:02Z'}\n"
                }
            ], 
            "source": "status = client.experiments.get_status(experiment_run_id)\nprint(status)"
        }, 
        {
            "source": "#### LIST training runs trigered by experiment run", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------  --------------  ---------  --------------------  --------------------  -----------\nGUID (training)     NAME            STATE      SUBMITTED             FINISHED              PERFORMANCE\ntraining-3F-0Q8kmg  tf-distributed  completed  2018-03-28T10:08:04Z  2018-03-28T10:10:01Z  -\ntraining-GcLAQ8kiR  tf-horovod      completed  2018-03-28T10:08:08Z  2018-03-28T10:10:36Z  -\ntraining-PuUAQUkig  tf-ddl          completed  2018-03-28T10:08:09Z  2018-03-28T10:11:13Z  -\n------------------  --------------  ---------  --------------------  --------------------  -----------\n"
                }
            ], 
            "source": "client.experiments.list_training_runs(experiment_run_id)"
        }, 
        {
            "source": "#### MONITOR experiment run\nYou can monitor experiment run by calling `client.experiments.monitor_logs(run_uid)`. This method will stream training logs content to console.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": null, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.experiments.monitor_logs(experiment_run_id)"
        }, 
        {
            "source": "#### LIST experiment runs\nYou can list all experiment run using `client.experiments.list()` method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ------------------------------------  ----------------------------  ---------  --------------------\nGUID (experiment)                     GUID (run)                            NAME (experiment)             STATE      CREATED\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  5abc65a2-0ab4-480b-a356-23bea3442d3c  Distributed MNIST experiment  completed  2018-03-28T09:53:03Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  98590d53-c70a-470c-8fde-dcb017a8dfaf  Distributed MNIST experiment  completed  2018-03-28T10:08:02Z\nd763b2eb-1b9d-440e-8c25-09a0f06ee017  b280a72e-9c9f-4e1b-8fe8-f5037271407c  Distributed MNIST experiment  completed  2018-03-28T09:47:07Z\n------------------------------------  ------------------------------------  ----------------------------  ---------  --------------------\n"
                }
            ], 
            "source": "client.experiments.list_runs()"
        }, 
        {
            "source": "#### GET training runs uids", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'tf-horovod': 'training-GcLAQ8kiR', 'tf-distributed': 'training-3F-0Q8kmg', 'tf-ddl': 'training-PuUAQUkig'}\n"
                }
            ], 
            "source": "training_runs = {rd['training_reference_name']: rd['training_guid'] for rd in run_details['entity']['training_statuses']}\n\nprint(training_runs)"
        }, 
        {
            "source": "Training run uid can be used to monitor selected training within experiment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "training_uids = client.experiments.get_training_uids(run_details)"
        }, 
        {
            "execution_count": null, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.training.monitor_logs(training_uids[0])"
        }, 
        {
            "source": "<a id=\"model\"></a>\n### 4.2 Store trained model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Call this method to save model in Watson Machine Learning repository", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Saving model: tf-horovod...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/84070d90-6db7-4a12-8da6-26aaf18e9378/versions/7debe8e8-7d3a-49fd-ba49-9b5409537b47', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/84070d90-6db7-4a12-8da6-26aaf18e9378', 'ml_pipeline_url': '', 'ml_asset_guid': '84070d90-6db7-4a12-8da6-26aaf18e9378'}, 'metadata': {'guid': 'training-GcLAQ8kiR', 'created_at': '2018-03-28T10:08:05Z', 'modified_at': '2018-03-28T10:08:05Z', 'url': '/v3/models/training-GcLAQ8kiR'}}\nSaving model: tf-distributed...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca/versions/1675377e-c7ae-4290-b2b4-37a538330fe3', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca', 'ml_pipeline_url': '', 'ml_asset_guid': 'b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca'}, 'metadata': {'guid': 'training-3F-0Q8kmg', 'created_at': '2018-03-28T10:08:02Z', 'modified_at': '2018-03-28T10:08:02Z', 'url': '/v3/models/training-3F-0Q8kmg'}}\nSaving model: tf-ddl...\n{'entity': {'ml_asset_version_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/91149430-5588-4ee6-ac6a-695cd2392aa8/versions/c05acb65-e9dd-48b0-88a4-3935f6fd87d2', 'ml_asset_url': 'https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/91149430-5588-4ee6-ac6a-695cd2392aa8', 'ml_pipeline_url': '', 'ml_asset_guid': '91149430-5588-4ee6-ac6a-695cd2392aa8'}, 'metadata': {'guid': 'training-PuUAQUkig', 'created_at': '2018-03-28T10:08:08Z', 'modified_at': '2018-03-28T10:08:08Z', 'url': '/v3/models/training-PuUAQUkig'}}\n"
                }
            ], 
            "source": "saved_model_details = {}\n\nfor training_run in training_runs:\n    print(\"Saving model: {}...\".format(training_run))\n    saved_model_details[training_run] = client.repository.store_model(training_runs[training_run], {\"name\": \"My cool mnist model {}\".format(training_run)})\n    print(saved_model_details[training_run])"
        }, 
        {
            "source": "**Tip:** Run below code to extract stored model guid from dictionary.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Saved model guid: {'tf-horovod': '84070d90-6db7-4a12-8da6-26aaf18e9378', 'tf-distributed': 'b6e59ce5-7d24-4ce1-90f8-f6a918ad71ca', 'tf-ddl': '91149430-5588-4ee6-ac6a-695cd2392aa8'}\n"
                }
            ], 
            "source": "saved_model_uids = {training_run: client.repository.get_model_uid(saved_model_details[training_run]) for training_run in saved_model_details}\nprint(\"Saved model guid: {}\".format(saved_model_uids))"
        }, 
        {
            "source": "#### Call this method to list stored models", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------  ------------------------  --------------\nGUID                                  NAME                                CREATED                   FRAMEWORK\n43986a55-0f0e-493b-9b98-0712f918b0ba  MNIST CNN model                     2018-03-15T16:12:06.503Z  tensorflow-1.5\na5a952e1-fa6f-4c64-88f7-293b367b7e1f  MNIST best model                    2018-03-16T10:12:16.773Z  tensorflow-1.5\nc755c740-dab6-4d5d-b873-1060885d55d5  MNIST best model                    2018-03-16T10:12:31.990Z  tensorflow-1.5\n861b6b73-6861-49c0-867c-d171d6d9d545  MNIST best model                    2018-03-16T13:00:40.461Z  tensorflow-1.5\nf7219c8f-fccb-4d26-a1bd-f51544090eae  training-S3ttnlgig_2 best model     2018-03-16T13:32:33.747Z  tensorflow-1.5\n327242a2-d2c0-400f-b673-4f33ad7ec17d  product-line-prediction             2018-03-16T18:08:31.260Z  mllib-2.0\na0cb79ca-e937-4cbf-b7e5-23dad2d2ecec  drug-selection                      2018-03-16T18:10:25.046Z  mllib-2.0\n97822727-4bd1-4df0-95df-8ff9536ea01d  My cool mnist model                 2018-03-28T09:02:31.509Z  tensorflow-1.5\n84070d90-6db7-4a12-8da6-26aaf18e9378  My cool mnist model tf-horovod      2018-03-28T11:15:53.446Z  tensorflow-1.5\nb6e59ce5-7d24-4ce1-90f8-f6a918ad71ca  My cool mnist model tf-distributed  2018-03-28T11:15:56.061Z  tensorflow-1.5\n91149430-5588-4ee6-ac6a-695cd2392aa8  My cool mnist model tf-ddl          2018-03-28T11:15:59.813Z  tensorflow-1.5\n------------------------------------  ----------------------------------  ------------------------  --------------\n"
                }
            ], 
            "source": "client.repository.list_models()"
        }, 
        {
            "source": "### 4.3 Delete experiment-run", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can delete experiment runs and results by running below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "client.experiments.delete(experiment_run_id)", 
            "cell_type": "raw", 
            "metadata": {}
        }, 
        {
            "source": "## 5. Create online deployment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can deployed stored model as webservice (online) by running below method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '76dd8d4a-0cab-435a-9430-d488c7144bf2' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='76dd8d4a-0cab-435a-9430-d488c7144bf2'\n------------------------------------------------------------------------------------------------\n\n\n"
                }
            ], 
            "source": "deployment_details = client.deployments.create(name=\"Mnist model deployment - tf-ddl\", model_uid=saved_model_uids['tf-ddl'])"
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/91149430-5588-4ee6-ac6a-695cd2392aa8/deployments/76dd8d4a-0cab-435a-9430-d488c7144bf2/online\n"
                }
            ], 
            "source": "scoring_url = client.deployments.get_scoring_url(deployment_details)\nprint(scoring_url)"
        }, 
        {
            "source": "## 6. Scoring", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "Prepare sample scoring data to score deployed model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "image1 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.639,0.757,0.596,0.361,0.200,0.200,0.200,0.200,0.122,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.992,0.988,0.992,0.988,0.992,0.988,0.992,0.753,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.082,0.400,0.243,0.400,0.400,0.239,0.718,0.157,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.239,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.361,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.678,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.992,0.988,0.875,0.796,0.796,0.796,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.482,0.996,0.992,0.878,0.796,0.796,0.875,0.996,0.278,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.953,0.992,0.510,0.078,0.000,0.000,0.078,0.992,0.910,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.596,0.992,0.718,0.000,0.000,0.000,0.000,0.000,0.518,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.200,0.592,0.078,0.000,0.000,0.000,0.000,0.000,0.200,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.443,0.992,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.322,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.757,0.988,0.400,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.796,0.718,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.996,0.992,0.161,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.082,0.875,0.400,0.000,0.000,0.000,0.000,0.000,0.082,0.796,0.992,0.510,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.800,0.482,0.000,0.000,0.000,0.000,0.161,0.678,0.992,0.796,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.635,0.875,0.400,0.082,0.400,0.639,0.953,0.992,0.671,0.078,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.918,0.992,1.000,0.992,1.000,0.675,0.322,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.118,0.514,0.753,0.435,0.196,0.039,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]\nimage2 = [0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.196,0.878,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.275,0.114,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.475,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.580,0.659,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.016,0.765,0.906,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.376,0.824,0.043,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.271,0.988,0.525,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.447,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.176,0.925,0.851,0.047,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.753,0.988,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.659,0.969,0.208,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.071,1.000,0.992,0.082,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.329,0.949,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.553,0.992,0.741,0.020,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.125,0.910,0.980,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.059,0.882,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.525,0.988,0.827,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.988,0.643,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.655,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.035,0.800,0.820,0.071,0.000,0.000,0.000,0.000,0.000,0.000,0.086,0.992,0.992,0.420,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.663,0.988,0.780,0.333,0.333,0.333,0.333,0.506,0.643,0.765,0.988,0.988,0.416,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.161,0.667,0.961,0.988,0.988,0.988,0.988,0.910,0.906,0.984,0.988,0.988,0.035,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.192,0.329,0.329,0.329,0.329,0.000,0.000,0.631,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.176,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.502,0.992,0.992,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.498,0.988,0.988,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.529,0.988,0.957,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.910,0.925,0.435,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.702,0.259,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000]"
        }, 
        {
            "execution_count": 34, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np"
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT4AAACFCAYAAADCQpQyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAB2lJREFUeJzt3U+IjV8cx/Fz+ZV/STFkQVMTNf4khSzGRLOwIAtD+VeTJAtlJClCyb9hYWEW/mStJH9SQjYSCxYipqz8m5KFQUqSML/Fbzq/8z3uvWau53nuc+/n/Vp9T2eac9Sdj+c59zzPKfT39zsAUDKs2hMAgKwRfADkEHwA5BB8AOQQfADkEHwA5BB8AOQQfADkEHwA5PyT8Xg8JpIfhWpPoI7wuc6PQX2uueIDIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyCH4AMgh+ADIIfgAyAn6zcw14SOjg7TfvPmja9nzpxp+lpaWkx7xowZFY3Z0NDg68bGxop+B/D48WPTbm9v9/WrV69SH//27dumHf49TJ06NfXxB4srPgByCD4Acgr9/Zmek5LbQ1n6+vp8vWDBAtPX29vr62HD7P8Vv379Mu2wfyh9ra2tvu7s7DR94e1KgjhsKDm5+Vx3dXWZdnd3t6/fvXuX+vjbtm0z7ffv3/v6woULqY/vOGwIAIoj+ADIIfgAyGE7y4Bwy0q4puecc6dPn/b1li1bTF+4Nuicc1euXPF1vLXl+fPnJccPt8kcOXKkZF9zc3PJ3wFNP3788PWNGzeqOBPn5s+fb9onTpzw9devX03f6NGjM5lTMVzxAZBD8AGQw61uEfGWlXLCJy6c+/1WOBRuWSln7969ps3tLcq5c+eOrx88eGD6du3alelcPn78aNrh8g63ugBQRQQfADkEHwA5rPEVET9OlrVFixZVdXzkW09Pj2mvX7/e101NTaYvXi9O27Vr1zIdr1Jc8QGQQ/ABkMOt7oDwa/ehbGcBsnb48GHT/vLli6/jJzfGjBmT+nzCLSx37941fXn9W8rnrAAgRQQfADkEHwA5rPENuHr1qq+rvZ0FiF2+fNnXN2/eNH3Tp0/3dfz28CyEa47xmt6SJUt8PW7cuKym9Edc8QGQQ/ABkMNhQwOGDx/u6/hyfcqUKb6eOHHioH/n7t27TTulQ4MqxWFDyUn9c71mzRpfhy+7dc65kydP+nrr1q1pT8W8tNc55xYuXOjrT58+mb7wtrytrS3dif2Hw4YAoBiCD4Acgg+AHNntLEePHjXtcK3zb7azhAcor1692vTdunXL10uXLq14DNS/z58/m/bDhw9L/mwW63qhs2fPmvaHDx98HR+wldG63pBxxQdADsEHQI7UrW74Bpauri7TVyj8/y14/PLG7du3+3rChAllxwgv+ydNmmT6Ojo6fB0eEOPc77cI0Pb9+3fTfvv2ra/Xrl2b9XSMFy9elOybNWtWhjOpHFd8AOQQfADkEHwA5Eit8XV3d/s6Ptw43F5y8ODBiscI1wD37Nlj+sJ1xXv37pk+1vgQGjt2rGnPmTPH10+fPjV94RuQx48fn8p8wm1aly5dKvlzLS0tqYyfNK74AMgh+ADIIfgAyJFa49u8ebOvw317zjl36NChxMdbuXKlaR8/fjzxMVCfRo4cadrTpk3zdfxaquXLl/t6586dFY337Nkz03758qVpv3792tflTk7L66lqsdqYJQAkiOADIEfqVrexsdHXp06dSn28eMsKhxihUgcOHPB1/Nb069ev+3rdunUV/f74Ucx4KSh8FLOcTZs2VTR+1rjiAyCH4AMgh+ADIKeu1/jir/3PnTvn6/hQ5jSEh5Q7Vztf9SN/wkcaL168aPqePHni63KvjCpn1apVZfs3btzo6/Pnz5f8uXgbTl7xlwhADsEHQE5d3+oeO3bMtP/09uQk9PX1+TrezhJuEWhtbU19LtAwd+7conWSmpqaBvVzPT09pj179uw0pvPXuOIDIIfgAyCH4AMgp+7W+MI1tvCtsc7Zt7OkMZ5zzi1btszX8WM/4eltvHEZtSR83LLco5d5XdOLccUHQA7BB0BO3d3qNjQ0+Do+eCW+La1EeCi5c87t27fPtB89euTrefPmmb7wYHKgloRPHdXDE0i1/y8AgCEi+ADIIfgAyKm7Nb5Q/NV6+HaWcC3QOefa29t9Hb/V5f79+76O37gSH0we/p4zZ86YviwemQPS8O3bt5J9I0aMyHAmyeCKD4Acgg+AnEJ8cEnKMh0s3nqyePFiX8eHp4S70eOv68O+8FbWOec2bNhg2nF/jhX+/CMYpEw/19UwefJkX//8+dP07d+/39ednZ2ZzamEQX2uueIDIIfgAyCH4AMgp67X+GK9vb2+Dre2OGfflhyv04VvS25ubjZ9o0aNSnKKWWKNLzl1v8a3YsUKX+/YscP0tbW1ZT2dcljjA4BiCD4AcqRudWFwq5scPtf5wa0uABRD8AGQQ/ABkEPwAZBD8AGQQ/ABkEPwAZBD8AGQQ/ABkEPwAZCT9WFDPCaFesTnusZwxQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQA7BB0AOwQdADsEHQM6/vfWVmGCcY80AAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x2ac3c72fc320>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "for i, image in enumerate([image1, image2]):\n    plt.subplot(2, 2, i + 1)\n    plt.axis('off')\n    plt.imshow((np.reshape(image, (28, 28)) * 255).astype(np.uint8), cmap=plt.cm.gray_r, interpolation='nearest')"
        }, 
        {
            "source": "**Tip**: If plot fails please upgrade dask library.\n\n!pip install dask --upgrade", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Build scoring dictionary consisting of two digits and send it to deployed model to get predictions.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 36, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "scoring_data = {\"values\": [image1, image2]}"
        }, 
        {
            "execution_count": 37, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Scoring result: {'values': [8, 9]}\n"
                }
            ], 
            "source": "predictions = client.deployments.score(scoring_url, scoring_data)\nprint(\"Scoring result: \" + str(predictions))"
        }, 
        {
            "source": "<HR>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"summary\"></a>\n## 7. Summary and next steps     ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": " You successfully completed this notebook! You learned how to use `watson-machine-learning-client` to run experiments. Check out our [Online Documentation](https://dataplatform.ibm.com/docs/content/analyze-data/wml-setup.html) for more samples, tutorials, documentation, how-tos, and blog posts. ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "### Authors\n\n**Wojciech Sobala**, is a Data Scientist at IBM working on Watson Machine Learning service (public Cloud).\n\n**Lukasz Cmielowski**, PhD, is a Automation Architect and Data Scientist at IBM with a track record of developing enterprise-level applications that substantially increases clients' ability to turn data into actionable knowledge.\n\n", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Copyright \u00a9 2017, 2018 IBM. This notebook and its source code are released under the terms of the MIT License.", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}