From scikit-learn model to scoring.ipynb

{
    "nbformat_minor": 1, 
    "cells": [
        {
            "source": "<table style=\"border: none\" align=\"left\">\n   <tr style=\"border: none\">\n      <th style=\"border: none\"><font face=\"verdana\" size=\"5\" color=\"black\"><b>From scikit-learn model to scoring with watson machine learning client</b></th>\n      <th style=\"border: none\"><img src=\"https://github.com/pmservice/customer-satisfaction-prediction/blob/master/app/static/images/ml_icon_gray.png?raw=true\" alt=\"Watson Machine Learning icon\" height=\"40\" width=\"40\"></th>\n   </tr>\n   <tr style=\"border: none\">\n       <th style=\"border: none\"><img src=\"https://github.com/pmservice/wml-sample-models/raw/master/scikit-learn/hand-written-digits-recognition/images/numbers_banner-04.png\" width=\"600\" alt=\"Icon\"> </th>\n   </tr>\n</table>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "This notebook contains steps and code to work with [watson-machine-learning-client](https://pypi.python.org/pypi/watson-machine-learning-client) library available in PyPI repository. This notebook introduces commands for getting data and for basic data exploration, pipeline creation, model training and evaluation, model persistance to Watson Machine Learning repository, model deployment, and scoring.\n\nSome familiarity with Python is helpful. This notebook uses Python 3.5, scikit-learn, watson-machine-learning-client package.\n\nYou will use a toy dataset available in scikit-learn, **sklearn.datasets.load_digits**, which contains hand-written digits images. Use the toy dataset to recognize hand-written digits.\n\n## Learning goals\n\nThe learning goals of this notebook are:\n\n-  Load a sample dataset from scikit-learn.\n-  Explore data.\n-  Prepare data for training and evaluation.\n-  Create Scikit-learn machine learning pipeline.\n-  Train and evaluate a model.\n-  Persist a model in Watson Machine Learning repository.\n-  Deploy model for online scoring using client library.\n-  Score sample records using client library.\n\n\n## Contents\n\nThis notebook contains the following parts:\n\n1.\t[Setup](#setup)\n2.\t[Load and explore data](#load)\n3.\t[Create scikit-learn model](#model)\n4.\t[Persist model](#persistence)\n5.\t[Deploy and score in a Cloud](#scoring)\n6.\t[Summary and next steps](#summary)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"setup\"></a>\n## 1. Setup\n\nBefore you use the sample code in this notebook, you must perform the following setup task:\n\n-  Create a [Watson Machine Learning Service](https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/) instance (a free plan is offered). \n- Configure local python environment\n  + python 3.5\n  + scikit-learn 0.17.1\n  + watson-machine-learning-client", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Tip**: You can install libraries from [PyPI](https://pypi.python.org/pypi) running cell below.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client"
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Collecting watson-machine-learning-client\n  Downloading watson_machine_learning_client-1.0.53-py3-none-any.whl (561kB)\n\u001b[K    100% |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 563kB 1.6MB/s eta 0:00:01\n\u001b[?25hRequirement not upgraded as not directly required: certifi in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: urllib3 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: requests in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: pandas in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: lomond in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tabulate in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: tqdm in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from watson-machine-learning-client)\nRequirement not upgraded as not directly required: chardet<3.1.0,>=3.0.2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: idna<2.7,>=2.5 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from requests->watson-machine-learning-client)\nRequirement not upgraded as not directly required: python-dateutil>=2 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: pytz>=2011k in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: numpy>=1.9.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from pandas->watson-machine-learning-client)\nRequirement not upgraded as not directly required: six>=1.10.0 in /opt/conda/envs/DSX-Python35/lib/python3.5/site-packages (from lomond->watson-machine-learning-client)\nInstalling collected packages: watson-machine-learning-client\n  Found existing installation: watson-machine-learning-client 1.0.12\n    Uninstalling watson-machine-learning-client-1.0.12:\n      Successfully uninstalled watson-machine-learning-client-1.0.12\nSuccessfully installed watson-machine-learning-client-1.0.53\n"
                }
            ], 
            "source": "!pip install watson-machine-learning-client --upgrade"
        }, 
        {
            "source": "<a id=\"load\"></a>\n## 2. Load and explore data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will load the data from scikit-learn sample datasets and perform a basic exploration.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 3, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import sklearn\nfrom sklearn import datasets\n\ndigits = datasets.load_digits()"
        }, 
        {
            "source": "Loaded toy dataset consists of 8x8 pixels images of hand-written digits.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Let's display first digit data and label using **data** and **target**.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 4, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "[[  0.   0.   5.  13.   9.   1.   0.   0.]\n [  0.   0.  13.  15.  10.  15.   5.   0.]\n [  0.   3.  15.   2.   0.  11.   8.   0.]\n [  0.   4.  12.   0.   0.   8.   8.   0.]\n [  0.   5.   8.   0.   0.   9.   8.   0.]\n [  0.   4.  11.   0.   1.  12.   7.   0.]\n [  0.   2.  14.   5.  10.  12.   0.   0.]\n [  0.   0.   6.  13.  10.   0.   0.   0.]]\n"
                }
            ], 
            "source": "print(digits.data[0].reshape((8, 8)))"
        }, 
        {
            "execution_count": 5, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 5, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "0"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "digits.target[0]"
        }, 
        {
            "source": "In next step you will count data examples.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Number of samples: 1797\n"
                }
            ], 
            "source": "samples_count = len(digits.images)\n\nprint(\"Number of samples: \" + str(samples_count))"
        }, 
        {
            "source": "<a id=\"model\"></a>\n## 3. Create a Scikit-learn model\n\nIn this section you will learn how to prepare data, create a Scikit-learn machine learning pipeline, and train a model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.1: Prepare data\n\nIn this subsection you will split your data into: train, test, and score datasets.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Number of training records: 1257\nNumber of testing records : 360\nNumber of scoring records : 180\n"
                }
            ], 
            "source": "train_data = digits.data[: int(0.7*samples_count)]\ntrain_labels = digits.target[: int(0.7*samples_count)]\n\ntest_data = digits.data[int(0.7*samples_count): int(0.9*samples_count)]\ntest_labels = digits.target[int(0.7*samples_count): int(0.9*samples_count)]\n\nscore_data = digits.data[int(0.9*samples_count): ]\n\nprint(\"Number of training records: \" + str(len(train_data)))\nprint(\"Number of testing records : \" + str(len(test_data)))\nprint(\"Number of scoring records : \" + str(len(score_data)))"
        }, 
        {
            "source": "As you can see our data has been successfully split into three datasets: \n\n-  The train data set, which is the largest group, is used for training.\n-  The test data set will be used for model evaluation and is used to test the assumptions of the model.\n-  The score data set will be used for scoring in Cloud.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.2: Create pipeline and train a model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will create scikit-learn machine learning pipeline and then train the model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In the first step you need to import the scikit-learn machine learning packages that will be needed in the subsequent steps.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.pipeline import Pipeline\nfrom sklearn import preprocessing\nfrom sklearn import svm, metrics"
        }, 
        {
            "source": "Standardize features by removing the mean and scaling to unit variance.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "scaler = preprocessing.StandardScaler()"
        }, 
        {
            "source": "Next, define estimators you want to use for classification. Support Vector Machines (SVM) with radial basis function as kernel is used in the following example.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "clf = svm.SVC(kernel='rbf')"
        }, 
        {
            "source": "Let's build the pipeline now. A pipeline consists of transformer and an estimator.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "pipeline = Pipeline([('scaler', scaler), ('svc', clf)])"
        }, 
        {
            "source": "Now, you can train your SVM model by using the previously defined **pipeline** and **train data**.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "model = pipeline.fit(train_data, train_labels)"
        }, 
        {
            "source": "You can check your **model quality** now. To evaluate the model, use **test data**.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "Evaluation report: \n\n             precision    recall  f1-score   support\n\n          0       1.00      0.97      0.99        37\n          1       0.97      0.97      0.97        34\n          2       1.00      0.97      0.99        36\n          3       1.00      0.94      0.97        35\n          4       0.78      0.97      0.87        37\n          5       0.97      0.97      0.97        38\n          6       0.97      0.86      0.91        36\n          7       0.92      0.97      0.94        35\n          8       0.91      0.89      0.90        35\n          9       0.97      0.92      0.94        37\n\navg / total       0.95      0.94      0.95       360\n\n"
                }
            ], 
            "source": "predicted = model.predict(test_data)\n\nprint(\"Evaluation report: \\n\\n%s\" % metrics.classification_report(test_labels, predicted))"
        }, 
        {
            "source": "You can tune your model now to achieve better accuracy. For simplicity of this example tuning section is omitted.", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "<a id=\"persistence\"></a>\n## 4. Persist model", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "In this section you will learn how to store your model in Watson Machine Learning repository by using commont python client.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Tip**: You can check documentation for watson-machine-learning-client by visiting [documentation webpage](https://wml-api-pyclient.mybluemix.net).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 4.1: Work with your instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "First, you must import client libraries.", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient"
        }, 
        {
            "source": "Authenticate to Watson Machine Learning service on IBM Cloud.", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "**Action**: Put authentication information from your instance of Watson Machine Learning service here.</div>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "wml_credentials={\n  \"url\": \"https://ibm-watson-ml.mybluemix.net\",\n  \"access_key\": \"****\",\n  \"username\": \"****\",\n  \"password\": \"****\",\n  \"instance_id\": \"****\"\n}"
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by DSX for sharing."
        }, 
        {
            "source": "**Tip**: Authentication information can be found on **Service Credentials** tab of service instance created on IBM  Cloud. <BR>If you cannot see **instance_id** field in **Serice Credentials** generate new credentials by pressing **New credential (+)** button. ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "#### Create API client by running below code.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client = WatsonMachineLearningAPIClient(wml_credentials)"
        }, 
        {
            "source": "#### Get instance details.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import json\n\ninstance_details = client.service_instance.get_details()"
        }, 
        {
            "source": "### 4.2: Publish model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Publish model in Watson Machine Learning repository on Cloud.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Define model name, autor name and email.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "published_model = client.repository.store_model(model=model, meta_props={'name':'Digits prediction model'}, \\\n                                                training_data=train_data, training_target=train_labels)"
        }, 
        {
            "source": "### 4.3: Get model details", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{\n  \"metadata\": {\n    \"modified_at\": \"2018-03-28T13:00:55.963Z\",\n    \"guid\": \"ae7ea0bf-3a48-453d-984a-07595ad0ef8a\",\n    \"url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a\",\n    \"created_at\": \"2018-03-28T13:00:55.885Z\"\n  },\n  \"entity\": {\n    \"deployments\": {\n      \"count\": 0,\n      \"url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/deployments\"\n    },\n    \"input_data_schema\": {\n      \"labels\": {\n        \"fields\": [\n          {\n            \"type\": \"int\",\n            \"name\": \"l1\"\n          }\n        ],\n        \"type\": \"ndarray\"\n      },\n      \"features\": {\n        \"fields\": [\n          {\n            \"type\": \"float\",\n            \"name\": \"f0\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f1\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f2\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f3\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f4\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f5\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f6\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f7\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f8\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f9\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f10\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f11\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f12\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f13\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f14\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f15\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f16\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f17\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f18\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f19\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f20\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f21\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f22\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f23\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f24\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f25\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f26\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f27\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f28\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f29\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f30\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f31\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f32\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f33\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f34\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f35\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f36\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f37\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f38\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f39\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f40\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f41\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f42\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f43\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f44\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f45\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f46\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f47\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f48\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f49\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f50\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f51\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f52\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f53\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f54\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f55\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f56\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f57\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f58\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f59\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f60\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f61\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f62\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f63\"\n          }\n        ],\n        \"type\": \"ndarray\"\n      }\n    },\n    \"learning_configuration_url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/learning_configuration\",\n    \"runtime_environment\": \"python-3.5\",\n    \"feedback_url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/feedback\",\n    \"author\": {},\n    \"name\": \"Digits prediction model\",\n    \"model_type\": \"scikit-learn-0.19\",\n    \"training_data_schema\": {\n      \"labels\": {\n        \"fields\": [\n          {\n            \"type\": \"int\",\n            \"name\": \"l1\"\n          }\n        ],\n        \"type\": \"ndarray\"\n      },\n      \"features\": {\n        \"fields\": [\n          {\n            \"type\": \"float\",\n            \"name\": \"f0\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f1\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f2\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f3\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f4\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f5\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f6\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f7\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f8\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f9\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f10\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f11\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f12\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f13\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f14\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f15\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f16\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f17\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f18\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f19\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f20\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f21\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f22\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f23\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f24\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f25\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f26\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f27\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f28\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f29\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f30\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f31\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f32\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f33\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f34\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f35\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f36\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f37\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f38\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f39\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f40\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f41\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f42\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f43\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f44\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f45\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f46\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f47\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f48\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f49\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f50\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f51\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f52\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f53\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f54\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f55\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f56\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f57\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f58\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f59\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f60\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f61\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f62\"\n          },\n          {\n            \"type\": \"float\",\n            \"name\": \"f63\"\n          }\n        ],\n        \"type\": \"ndarray\"\n      }\n    },\n    \"label_col\": \"l1\",\n    \"latest_version\": {\n      \"guid\": \"8600c69e-493f-4513-bca9-389ca119aede\",\n      \"url\": \"https://ibm-watson-ml.mybluemix.net/v3/ml_assets/models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/versions/8600c69e-493f-4513-bca9-389ca119aede\",\n      \"created_at\": \"2018-03-28T13:00:55.963Z\"\n    },\n    \"learning_iterations_url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/learning_iterations\",\n    \"evaluation_metrics_url\": \"https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/evaluation_metrics\"\n  }\n}\n"
                }
            ], 
            "source": "published_model_uid = client.repository.get_model_uid(published_model)\nmodel_details = client.repository.get_details(published_model_uid)\n\nprint(json.dumps(model_details, indent=2))"
        }, 
        {
            "source": "#### Get all models", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------  ------------------------  -----------------\nGUID                                  NAME                                CREATED                   FRAMEWORK\n43986a55-0f0e-493b-9b98-0712f918b0ba  MNIST CNN model                     2018-03-15T16:12:06.503Z  tensorflow-1.5\na5a952e1-fa6f-4c64-88f7-293b367b7e1f  MNIST best model                    2018-03-16T10:12:16.773Z  tensorflow-1.5\nc755c740-dab6-4d5d-b873-1060885d55d5  MNIST best model                    2018-03-16T10:12:31.990Z  tensorflow-1.5\n861b6b73-6861-49c0-867c-d171d6d9d545  MNIST best model                    2018-03-16T13:00:40.461Z  tensorflow-1.5\nf7219c8f-fccb-4d26-a1bd-f51544090eae  training-S3ttnlgig_2 best model     2018-03-16T13:32:33.747Z  tensorflow-1.5\n327242a2-d2c0-400f-b673-4f33ad7ec17d  product-line-prediction             2018-03-16T18:08:31.260Z  mllib-2.0\na0cb79ca-e937-4cbf-b7e5-23dad2d2ecec  drug-selection                      2018-03-16T18:10:25.046Z  mllib-2.0\n97822727-4bd1-4df0-95df-8ff9536ea01d  My cool mnist model                 2018-03-28T09:02:31.509Z  tensorflow-1.5\n84070d90-6db7-4a12-8da6-26aaf18e9378  My cool mnist model tf-horovod      2018-03-28T11:15:53.446Z  tensorflow-1.5\nb6e59ce5-7d24-4ce1-90f8-f6a918ad71ca  My cool mnist model tf-distributed  2018-03-28T11:15:56.061Z  tensorflow-1.5\n91149430-5588-4ee6-ac6a-695cd2392aa8  My cool mnist model tf-ddl          2018-03-28T11:15:59.813Z  tensorflow-1.5\nae7ea0bf-3a48-453d-984a-07595ad0ef8a  Digits prediction model             2018-03-28T13:00:55.885Z  scikit-learn-0.19\n------------------------------------  ----------------------------------  ------------------------  -----------------\n"
                }
            ], 
            "source": "models_details = client.repository.list_models()"
        }, 
        {
            "source": "### 4.3: Load model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this subsection you will learn how to load back saved model from specified instance of Watson Machine Learning.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "loaded_model = client.repository.load(published_model_uid)"
        }, 
        {
            "source": "You can make predictions to check that model has been loaded correctly.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_predictions = loaded_model.predict(test_data[:10])"
        }, 
        {
            "execution_count": 25, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "[4 0 5 3 6 9 6 4 7 5]\n"
                }
            ], 
            "source": "print(test_predictions)"
        }, 
        {
            "source": "As you can see you are able to make predictions, so model is loaded back correctly. You have already learned how save to and load the model from Watson Machine Learning repository.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 4.4: Delete model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can delete published model from Watson Machine Learning repository using below code. The code is commented out at this stage since the model will be needed later for deployment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# client.repository.delete(published_model_uid)"
        }, 
        {
            "source": "<a id=\"scoring\"></a>\n## 5. Deploy and score in a Cloud", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will learn how to create online scoring and to score a new data record by using the Watson Machine Learning Client.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 5.1: Create model deployment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "#### Create online deployment for published model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "\n\n#######################################################################################\n\nSynchronous deployment creation for uid: '278bff75-ee3d-42c6-a3d1-32dfa51698c1' started\n\n#######################################################################################\n\n\nINITIALIZING\nDEPLOY_SUCCESS\n\n\n------------------------------------------------------------------------------------------------\nSuccessfully finished deployment creation, deployment_uid='278bff75-ee3d-42c6-a3d1-32dfa51698c1'\n------------------------------------------------------------------------------------------------\n\n\n"
                }
            ], 
            "source": "created_deployment = client.deployments.create(published_model_uid, \"Deployment of scikit model\")"
        }, 
        {
            "source": "**Note**: Here we use deployment url saved in published_model object. In next section we show how to retrive deployment url from Watson Mchine Learning instance.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Now you can print an online scoring endpoint. ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/deployments/278bff75-ee3d-42c6-a3d1-32dfa51698c1/online\n"
                }
            ], 
            "source": "scoring_endpoint = client.deployments.get_scoring_url(created_deployment)\n\nprint(scoring_endpoint)"
        }, 
        {
            "source": "### 5.2: Get deployments", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "deployments = client.deployments.get_details()"
        }, 
        {
            "source": "You can get deployment_url by parsing deployment details for last deployed model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/3f6e5c2b-4880-46aa-9d79-62e90ccc9d56/published_models/ae7ea0bf-3a48-453d-984a-07595ad0ef8a/deployments/278bff75-ee3d-42c6-a3d1-32dfa51698c1\n"
                }
            ], 
            "source": "deployment_url = client.deployments.get_url(created_deployment)\n\nprint(deployment_url)"
        }, 
        {
            "source": "### 5.3: Score", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can use below method to do test scoring request against deployed model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Action**: Prepare scoring payload with records to score.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "scoring_payload = {\"values\": [list(score_data[0]), list(score_data[1])]}"
        }, 
        {
            "source": "Use ``client.deployments.score()`` method to run scoring.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "predictions = client.deployments.score(scoring_endpoint, scoring_payload)"
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{\n  \"fields\": [\n    \"prediction\"\n  ],\n  \"values\": [\n    [\n      5\n    ],\n    [\n      2\n    ]\n  ]\n}\n"
                }
            ], 
            "source": "print(json.dumps(predictions, indent=2))"
        }, 
        {
            "source": "### 5.4: Delete deployment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Use the following method to delete deployment.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.deployments.delete(client.deployments.get_uid(created_deployment))"
        }, 
        {
            "source": "You can check depoyments by running ``list`` method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 34, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  -------------------------------  ------  --------------  ------------------------  --------------\nGUID                                  NAME                             TYPE    STATE           CREATED                   FRAMEWORK\n0efa6832-80ea-4637-baed-771827562112  MNIST keras deployment           online  DEPLOY_FAILURE  2018-03-15T16:12:48.062Z  tensorflow-1.5\n1bf606e8-e8c4-4143-8727-5b2d1ac97b7e  MNIST keras deployment           online  DEPLOY_SUCCESS  2018-03-16T13:00:52.773Z  tensorflow-1.5\n2436e614-5396-4b78-ac02-3e6a69479b1c  MNIST keras deployment           online  DEPLOY_FAILURE  2018-03-15T16:13:29.910Z  tensorflow-1.5\n4bcae2dc-b309-4e0d-8858-2c04fe4c6383  Mnist model deployment           online  DEPLOY_SUCCESS  2018-03-28T09:02:49.220Z  tensorflow-1.5\n4fb26af0-e4b5-4fcc-9156-2fe6dcad41a2  drug deployment                  online  DEPLOY_SUCCESS  2018-03-16T18:11:33.498Z  mllib-2.0\n687c8e42-86e8-4b2c-a1bb-c9f2ebebfa23  MNIST keras deployment           online  DEPLOY_FAILURE  2018-03-15T16:21:39.495Z  tensorflow-1.5\n76dd8d4a-0cab-435a-9430-d488c7144bf2  Mnist model deployment - tf-ddl  online  DEPLOY_SUCCESS  2018-03-28T11:16:20.200Z  tensorflow-1.5\n8c05418f-547a-43e4-b43d-95b0da98a064  test                             online  DEPLOY_SUCCESS  2018-03-16T13:33:17.610Z  tensorflow-1.5\n8e29bf15-96b3-42be-8fc4-4e0554d395c7  MNIST keras deployment           online  DEPLOY_FAILURE  2018-03-15T16:14:00.816Z  tensorflow-1.5\nf1cc5088-c2cb-49aa-af3c-4825407b194b  product line                     online  DEPLOY_SUCCESS  2018-03-16T18:12:29.167Z  mllib-2.0\n------------------------------------  -------------------------------  ------  --------------  ------------------------  --------------\n"
                }
            ], 
            "source": "client.deployments.list()"
        }, 
        {
            "source": "### 5.5: Delete model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "client.repository.delete(published_model_uid)"
        }, 
        {
            "source": "You can check your stored models by running below ``list`` method.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 36, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ----------------------------------  ------------------------  --------------\nGUID                                  NAME                                CREATED                   FRAMEWORK\n43986a55-0f0e-493b-9b98-0712f918b0ba  MNIST CNN model                     2018-03-15T16:12:06.503Z  tensorflow-1.5\na5a952e1-fa6f-4c64-88f7-293b367b7e1f  MNIST best model                    2018-03-16T10:12:16.773Z  tensorflow-1.5\nc755c740-dab6-4d5d-b873-1060885d55d5  MNIST best model                    2018-03-16T10:12:31.990Z  tensorflow-1.5\n861b6b73-6861-49c0-867c-d171d6d9d545  MNIST best model                    2018-03-16T13:00:40.461Z  tensorflow-1.5\nf7219c8f-fccb-4d26-a1bd-f51544090eae  training-S3ttnlgig_2 best model     2018-03-16T13:32:33.747Z  tensorflow-1.5\n327242a2-d2c0-400f-b673-4f33ad7ec17d  product-line-prediction             2018-03-16T18:08:31.260Z  mllib-2.0\na0cb79ca-e937-4cbf-b7e5-23dad2d2ecec  drug-selection                      2018-03-16T18:10:25.046Z  mllib-2.0\n97822727-4bd1-4df0-95df-8ff9536ea01d  My cool mnist model                 2018-03-28T09:02:31.509Z  tensorflow-1.5\n84070d90-6db7-4a12-8da6-26aaf18e9378  My cool mnist model tf-horovod      2018-03-28T11:15:53.446Z  tensorflow-1.5\nb6e59ce5-7d24-4ce1-90f8-f6a918ad71ca  My cool mnist model tf-distributed  2018-03-28T11:15:56.061Z  tensorflow-1.5\n91149430-5588-4ee6-ac6a-695cd2392aa8  My cool mnist model tf-ddl          2018-03-28T11:15:59.813Z  tensorflow-1.5\n------------------------------------  ----------------------------------  ------------------------  --------------\n"
                }
            ], 
            "source": "client.repository.list_models()"
        }, 
        {
            "source": "<a id=\"summary\"></a>\n## 6. Summary and next steps     ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": " You successfully completed this notebook! You learned how to use scikit-learn machine learning as well as Watson Machine Learning for model creation and deployment. Check out our _[Online Documentation](https://console.ng.bluemix.net/docs/services/PredictiveModeling/index.html?pos=2)_ for more samples, tutorials, documentation, how-tos, and blog posts. ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "### Authors\n\n**Wojciech Sobala**, Data Scientist at IBM developing enterprise-level applications that substantially increases clients' ability to turn data into actionable knowledge.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Copyright \u00a9 2017 IBM. This notebook and its source code are released under the terms of the MIT License.", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}