From PMML model to scoring.ipynb

{
    "nbformat_minor": 1, 
    "cells": [
        {
            "source": "<div style=\"background:#F5F7FA; height:100px; padding: 2em; font-size:14px;\">\n<span style=\"font-size:18px;color:#152935;\">Want to do more?</span><span style=\"border: 1px solid #3d70b2;padding: 15px;float:right;margin-right:40px; color:#3d70b2; \"><a href=\"https://ibm.co/wsnotebooks\" target=\"_blank\" style=\"color: #3d70b2;text-decoration: none;\">Sign Up</a></span><br>\n<span style=\"color:#5A6872;\"> Try out this notebook with your free trial of IBM Watson Studio.</span>\n</div>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<table style=\"border: none\" align=\"left\">\n   <tr style=\"border: none\">\n      <th style=\"border: none\"><font face=\"verdana\" size=\"5\" color=\"black\"><b>Predicting Iris species with IBM Watson Machine Learning</b></th>\n      <th style=\"border: none\"><img src=\"https://github.com/pmservice/customer-satisfaction-prediction/blob/master/app/static/images/ml_icon_gray.png?raw=true\" alt=\"Watson Machine Learning icon\" height=\"40\" width=\"40\"></th>\n  <tr style=\"border: none\">\n       <th style=\"border: none\"><img src=\"https://github.com/pmservice/wml-sample-models/blob/master/spark/customer-satisfaction-prediction/images/users_banner_2-03.png?raw=true\" width=\"600\" alt=\"Icon\"> </th>\n   </tr>\n</table>", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "This notebook contains steps from storing sample PMML model to starting scoring new data. \n\nSome familiarity with python is helpful. This notebook uses Python 3.\n\nYou will use a **Iris** data set, which details measurements of iris perianth. Use the details of this data set to predict iris species.\n\n## Learning goals\n\nThe learning goals of this notebook are:\n\n-  Working with the WML instance\n-  Online deployment of PMML model\n-  Scoring of deployed model\n\n\n## Contents\n\nThis notebook contains the following parts:\n\n1.\t[Setup](#setup)\n2.\t[Work with service instance](#instance) \n3.\t[Persist, deploy and score in an IBM Cloud](#scoring)\n4.\t[Summary and next steps](#summary)", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"setup\"></a>\n## 1. Setup\n\nBefore you use the sample code in this notebook, you must perform the following setup tasks:\n\n-  Create a [Watson Machine Learning Service](https://console.ng.bluemix.net/catalog/services/ibm-watson-machine-learning/) instance required for model storing and deployment (a free plan is offered).\n-   Install [watson-machine-learning-client](https://pypi.python.org/pypi?%3Aaction=search&term=watson-machine-learning-client&submit=search) (not required on DSx).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"instance\"></a>\n## 2. Working with the instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will learn how to get details of your instance.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Install python API client from pypi.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "!rm -rf $PIP_BUILD/watson-machine-learning-client"
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "!pip install watson-machine-learning-client --upgrade"
        }, 
        {
            "source": "**Action**: Put credentials from your instance of Watson Machine Learning service in IBM Cloud here.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "wml_credentials = {\n  \"url\": \"https://ibm-watson-ml.mybluemix.net\",\n  \"access_key\": \"***\",\n  \"username\": \"***\",\n  \"password\": \"***\",\n  \"instance_id\": \"***\"\n}"
        }, 
        {
            "execution_count": 3, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from watson_machine_learning_client import WatsonMachineLearningAPIClient"
        }, 
        {
            "execution_count": 4, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "wml_client = WatsonMachineLearningAPIClient(wml_credentials)"
        }, 
        {
            "source": "Details about your Watson Machine Learning service are available in `instance_details`.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 5, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'metadata': {'created_at': '2018-01-30T10:39:51.938Z', 'url': 'https://ibm-watson-ml.mybluemix.net/v3/wml_instances/fd1db635-c9f3-4d83-afc7-32efec367e5e', 'modified_at': '2018-01-30T14:16:00.601Z', 'guid': 'fd1db635-c9f3-4d83-afc7-32efec367e5e'}, 'entity': {'region': 'us-south', 'account': {'id': '8dba8395f5b8dc6fcab49d43161ab9f1', 'type': 'TRIAL', 'name': \"Wojciech Sobala's Account\"}, 'deployments': {'url': 'https://ibm-watson-ml.mybluemix.net/v3/wml_instances/fd1db635-c9f3-4d83-afc7-32efec367e5e/deployments'}, 'source': 'Bluemix', 'status': 'Active', 'plan_id': '0f2a3c2c-456b-40f3-9b19-726d2740b11c', 'published_models': {'url': 'https://ibm-watson-ml.mybluemix.net/v3/wml_instances/fd1db635-c9f3-4d83-afc7-32efec367e5e/published_models'}, 'organization_guid': 'a7b3ef43-5dc2-41fc-8fa7-29b0846e5807', 'plan': 'standard', 'space_guid': 'a6b628f9-3f64-45c0-bf38-739876b63fec', 'usage': {'deployment_count': {'current': 1, 'limit': 1000}, 'expiration_date': '2018-02-01T00:00:00.000Z', 'computation_time': {'current': 0}, 'prediction_count': {'current': 3}, 'model_count': {'current': 0, 'limit': 1000}}, 'owner': {'country_code': 'POL', 'ibm_id': '3100020VJR', 'beta_user': True, 'user_id': '1c54455e-c23b-4506-b184-27494b507536', 'email': 'wojciech.sobala@pl.ibm.com'}}}\n"
                }
            ], 
            "source": "print(wml_client.instance_details)"
        }, 
        {
            "source": "Getting stored models on WML instance", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ------------------------------  ------------------------  -----------------  -----\nGUID                                  NAME                            CREATED                   FRAMEWORK          TYPE\n51cdbdea-28ea-488d-9edd-dab2a8c44208  Sentiment Prediction            2018-01-30T14:15:06.339Z  mllib-2.0          model\nb7e9fc69-f0db-444f-89b4-fe7aa5e8a4f1  Handwritten Digits Recognition  2018-01-30T14:15:24.106Z  scikit-learn-0.17  model\n------------------------------------  ------------------------------  ------------------------  -----------------  -----\n"
                }
            ], 
            "source": "wml_client.repository.list_models()"
        }, 
        {
            "source": "Geting list of deployed models.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ------------------  ------  ------------------------  -----------------\nGUID                                  NAME                TYPE    CREATED                   FRAMEWORK\n06faa2cd-e12b-4595-b96d-555d9fae6363  Digits recognition  online  2018-01-30T14:15:59.342Z  scikit-learn-0.17\n------------------------------------  ------------------  ------  ------------------------  -----------------\n"
                }
            ], 
            "source": "wml_client.deployments.list()"
        }, 
        {
            "source": "<a id=\"scoring\"></a>\n## 3. Persist, deploy and score model in a Cloud", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "In this section you will learn how to create online deployment and to score a new data record by using the python API client for Watson Machine Learning service.\nFor more information, see the [python API client documentation](http://wml-api-pyclient.mybluemix.net/).", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.1: Persist model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "**Action**: Download sample PMML model from git project using wget.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "import wget, os\n\nsample_dir = 'pmml_sample_model'\nif not os.path.isdir(sample_dir):\n    os.mkdir(sample_dir)\n    \nfilename=os.path.join(sample_dir, 'iris_chaid.xml')\nif not os.path.isfile(filename):\n    filename = wget('https://github.com/pmservice/wml-sample-models/raw/master/pmml/iris-species/model/iris_chaid.xml', out=sample_dir)"
        }, 
        {
            "source": "Store downloaded file in Watson Machine Learning repository.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  ------------------------------  ------------------------  -----------------  -----\nGUID                                  NAME                            CREATED                   FRAMEWORK          TYPE\n51cdbdea-28ea-488d-9edd-dab2a8c44208  Sentiment Prediction            2018-01-30T14:15:06.339Z  mllib-2.0          model\nb7e9fc69-f0db-444f-89b4-fe7aa5e8a4f1  Handwritten Digits Recognition  2018-01-30T14:15:24.106Z  scikit-learn-0.17  model\nfae8e498-136e-4c35-81bd-a9a69a50b1c7  CHAID PMML model for Iris data  2018-01-30T14:17:19.921Z  pmml-4.2           model\n------------------------------------  ------------------------------  ------------------------  -----------------  -----\n"
                }
            ], 
            "source": "props_pmml = {wml_client.repository.ModelMetaNames.NAME: 'CHAID PMML model for Iris data',\n              wml_client.repository.ModelMetaNames.AUTHOR_NAME: \"IBM\", \n              wml_client.repository.ModelMetaNames.AUTHOR_EMAIL: \"ibm@ibm.com\", \n              wml_client.repository.ModelMetaNames.FRAMEWORK_NAME: \"pmml\", \n              wml_client.repository.ModelMetaNames.FRAMEWORK_VERSION: \"4.2\"}\n\nmodel_details = wml_client.repository.store_model(filename, props_pmml)\nwml_client.repository.list_models()"
        }, 
        {
            "source": "**Note:** You can see that model is successfully stored in Watson Machine Learning Service.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "### 3.2: Create online deployment", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can use commands bellow to create online deployment for stored model.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "------------------------------------  -----------------------  ------  ------------------------  -----------------\nGUID                                  NAME                     TYPE    CREATED                   FRAMEWORK\n06faa2cd-e12b-4595-b96d-555d9fae6363  Digits recognition       online  2018-01-30T14:15:59.342Z  scikit-learn-0.17\n56f3501f-844c-4cb7-9bbc-02131da8f6bf  Iris species prediction  online  2018-01-30T14:17:22.742Z  pmml-4.2\n------------------------------------  -----------------------  ------  ------------------------  -----------------\n"
                }
            ], 
            "source": "model_uid = wml_client.repository.get_model_uid(model_details)\nmodel_deployment = wml_client.deployments.create(model_uid, name = 'Iris species prediction')\n\nwml_client.deployments.list()"
        }, 
        {
            "source": "### 3.3: Scoring", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "You can check online scoring endpoint by sending sample record using python API client.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "https://ibm-watson-ml.mybluemix.net/v3/wml_instances/fd1db635-c9f3-4d83-afc7-32efec367e5e/published_models/fae8e498-136e-4c35-81bd-a9a69a50b1c7/deployments/56f3501f-844c-4cb7-9bbc-02131da8f6bf/online\n"
                }
            ], 
            "source": "model_scoring_url = wml_client.deployments.get_scoring_url(model_deployment)\n\nprint(model_scoring_url)"
        }, 
        {
            "source": "You can see that scoring url is available for sending scoring requests.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "{'fields': ['PredictedValue', 'Confidence'], 'values': [['setosa', 1.0]]}\n"
                }
            ], 
            "source": "scoring_data = {'fields': ['Sepal.Length', 'Sepal.Width', 'Petal.Length', 'Petal.Width'], 'values': [[5.1, 3.5, 1.4, 0.2]]}\npredictions = wml_client.deployments.score(model_scoring_url, scoring_data) \n\nprint(predictions)"
        }, 
        {
            "source": "As we can see this is Iris Setosa flower.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "<a id=\"summary\"></a>\n## 4. Summary and next steps     ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": " You successfully completed this notebook! You learned how to use Watson Machine Learning for PMML model deployment and scoring. Check out our _[Online Documentation](https://console.ng.bluemix.net/docs/services/PredictiveModeling/index.html?pos=2)_ for more samples, tutorials, documentation, how-tos, and blog posts. ", 
            "cell_type": "markdown", 
            "metadata": {
                "collapsed": true
            }
        }, 
        {
            "source": "### Authors\n\nWojciech Sobala, Data Scientist in IBM.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Copyright \u00a9 2018 IBM. This notebook and its source code are released under the terms of the MIT License.", 
            "cell_type": "markdown", 
            "metadata": {}
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}