docs: models usage examples (#1669)

docs/features/models/NER.ipynb

@@ -80,14 +80,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!python -m deeppavlov install ner_ontonotes_bert_torch"
+    "!python -m deeppavlov install ner_ontonotes_bert"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "`ner_ontonotes_bert_torch` is the name of the model's *config_file*. [What is a Config File?](http://docs.deeppavlov.ai/en/master/intro/configuration.html) \n",
+    "`ner_ontonotes_bert` is the name of the model's *config_file*. [What is a Config File?](http://docs.deeppavlov.ai/en/master/intro/configuration.html) \n",
     "\n",
     "Configuration file defines the model and describes its hyperparameters. To use another model, change the name of the *config_file* here and further.\n",
     "The full list of NER models with their config names can be found in the [table](#3.-Models-list).\n",
@@ -126,7 +126,7 @@
    "source": [
     "from deeppavlov import build_model\n",
     "\n",
-    "ner_model = build_model('ner_ontonotes_bert_torch', download=True, install=True)"
+    "ner_model = build_model('ner_ontonotes_bert', download=True, install=True)"
    ]
   },
   {
@@ -180,7 +180,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! python -m deeppavlov interact ner_ontonotes_bert_torch -d"
+    "! python -m deeppavlov interact ner_ontonotes_bert -d"
    ]
   },
   {
@@ -198,7 +198,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! python -m deeppavlov predict ner_ontonotes_bert_torch -f <file-name>"
+    "! python -m deeppavlov predict ner_ontonotes_bert -f <file-name>"
    ]
   },
   {
@@ -224,7 +224,7 @@
    "source": [
     "from deeppavlov import evaluate_model\n",
     "\n",
-    "model = evaluate_model('ner_ontonotes_bert_torch', download=True)"
+    "model = evaluate_model('ner_ontonotes_bert', download=True)"
    ]
   },
   {
@@ -240,7 +240,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! python -m deeppavlov evaluate ner_ontonotes_bert_torch"
+    "! python -m deeppavlov evaluate ner_ontonotes_bert"
    ]
   },
   {
@@ -275,7 +275,7 @@
     "from deeppavlov import train_model\n",
     "from deeppavlov.core.commands.utils import parse_config\n",
     "\n",
-    "model_config = parse_config('ner_ontonotes_bert_torch')\n",
+    "model_config = parse_config('ner_ontonotes_bert')\n",
     "\n",
     "# dataset that the model was trained on\n",
     "print(model_config['dataset_reader']['data_path'])"
@@ -401,7 +401,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! python -m deeppavlov train ner_ontonotes_bert_torch"
+    "! python -m deeppavlov train ner_ontonotes_bert"
    ]
   },
   {

docs/integrations/rest_api.rst

@@ -117,3 +117,26 @@ Here are POST request payload examples for some of the library models:
 |                                         | |  "question_raw":["What strained the relationship between Great Britain and its colonies?"]}                                                       |
 +-----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+
 
+REST API Usage Example
+======================
+
+To start server with ``squad_bert`` model run:
+
+.. code:: bash
+
+    python -m deeppavlov riseapi squad_bert -id
+
+To get response from this model on another terminal run:
+
+.. code:: bash
+
+    curl -X POST http://0.0.0.0:5000/model -H 'Content-Type: application/json' -d '{
+        "context_raw": [
+            "All work and no play makes Jack a dull boy.",
+            "I used to be an adventurer like you, then I took an arrow in the knee."
+        ],
+        "question_raw": [
+            "What makes Jack a dull boy?",
+            "Who I used to be?"
+        ]
+    }'

docs/intro/installation.rst

@@ -1,43 +1,68 @@
 Installation
 ============
 
+DeepPavlov supports **Linux** platform, **Python 3.6-3.10**.
 
-We support ``Linux`` platform, ``Python 3.6``, ``3.7``, ``3.8``, ``3.9`` and ``3.10``.
-
-.. note::
-
-    * ``Python 3.5`` is not supported!
+Install with pip
+~~~~~~~~~~~~~~~~
 
+You should install DeepPavlov in a `virtual environment <https://docs.python.org/3/library/venv.html>`_. If you’re
+unfamiliar with Python virtual environments, take a look at this
+`guide <https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/>`_. A virtual
+environment makes it easier to manage different projects, and avoid compatibility issues between dependencies.
 
 #. Create a virtual environment:
 
     .. code:: bash
 
         python -m venv env
 
-#. Activate the environment:
+#. Activate the virtual environment on Linux (`source` could be replaced with `.`):
 
-    * Linux
-
-        .. code:: bash
+    .. code:: bash
 
-            source ./env/bin/activate
+        source env/bin/activate
 
-#. Install the package inside this virtual environment:
+#. Install DeepPavlov inside this virtual environment:
 
     .. code:: bash
 
         pip install deeppavlov
 
+Install from source
+~~~~~~~~~~~~~~~~~~~
+
+Install DeepPavlov **dev** branch from source with the following command:
+
+    .. code:: bash
+
+        pip install git+http://github.com/deeppavlov/DeepPavlov@dev
+
+This command installs the bleeding edge dev version rather than the latest release version. The dev version is useful
+for staying up-to-date with the latest developments. For instance, if a bug has been fixed since the last release but
+a new release hasn’t been rolled out yet. However, this means the dev version may not always be stable.
+
+Editable install
+~~~~~~~~~~~~~~~~
+
+You will need an editable install if you want to make changes in the DeepPavlov source code that immediately take place
+without requiring a new installation.
+
+Clone the repository and install DeepPavlov with the following commands:
+
+    .. code:: bash
+
+        git clone http://github.com/deeppavlov/DeepPavlov.git
+        pip install -e DeepPavlov
 
 Docker Images
--------------
+~~~~~~~~~~~~~
 
 We have built several DeepPavlov based Docker images, which include:
 
     * DeepPavlov based Jupyter notebook Docker image;
     * Docker images which serve some of our models and allow to access them
-      via REST API (``riseapi`` mode).
+      via REST API (:doc:`riseapi </integrations/rest_api>` mode).
 
 Here is our `DockerHub repository <https://hub.docker.com/u/deeppavlov/>`_ with
 images and deployment instructions.

docs/intro/quick_start.rst

@@ -198,17 +198,97 @@ Text Question Answering
 Text Question Answering component answers a question based on a given context (e.g,
 a paragraph of text), where the answer to the question is a segment of the context.
 
-.. table::
-    :widths: auto
+.. code:: python
 
-    +----------+------------------------------------------------------------------------------------+-------------------------------------------+
-    | Language | DeepPavlov config                                                                  | Demo                                      |
-    +==========+====================================================================================+===========================================+
-    | En       | :config:`squad_bert <squad/squad_bert.json>`                                       | https://demo.deeppavlov.ai/#/en/textqa    |
-    +----------+------------------------------------------------------------------------------------+-------------------------------------------+
-    | Ru       | :config:`squad_ru_bert <squad/squad_ru_bert.json>`                                 | https://demo.deeppavlov.ai/#/ru/textqa    |
-    +----------+------------------------------------------------------------------------------------+-------------------------------------------+
+    from deeppavlov import build_model
 
+    model = build_model('squad_bert', download=True, install=True)
+    contexts = ['DeepPavlov is a library for NLP and dialog systems.', 'All work and no play makes Jack a dull boy']
+    questions = ['What is DeepPavlov?', 'What makes Jack a dull boy?']
+    answer, answers_start_idx, score = model(contexts, questions)
+    print(answer)
+
+.. code:: bash
+
+    ['a library for NLP and dialog systems', 'All work and no play']
+
+To get list of available models for Text Question Answering see :doc:`documentation </features/models/SQuAD>`.
+
+Open-Domain Question Answering
+==============================
+
+Open Domain Question Answering (ODQA) answers any question based on the document collection covering a wide range of
+topics. The ODQA task combines two challenges of document retrieval (finding the relevant articles) with that of machine
+comprehension of text (identifying the answer span from those articles). This component can be used to answer questions
+based on the company knowledge base.
+
+.. code:: python
+
+    from deeppavlov import build_model
+
+    model = build_model('en_odqa_infer_wiki', download=True, install=True)
+    questions = ["What is the name of Darth Vader's son?", 'Who was the first president of France?']
+    answer, answer_score, answer_place = model(questions)
+    print(answer)
+
+.. code:: bash
+
+    ['Luke Skywalker', 'Louis-Napoleon Bonaparte']
+
+To get list of available models for Open-Domain Question Answering see :doc:`documentation </features/models/ODQA>`.
+
+Knowledge Base Question Answering
+=================================
+
+Knowledge Base Question Answering (KBQA) answers any question based on Knowledge Base (Knowledge Graph) -
+a comprehensive repository of information about a given domain or a number of domains that reflects the ways we model
+knowledge about a given subject or subjects, in terms of concepts, entities, properties, and relationships. KBQA models
+validate questions against a preconfigured list of question templates, disambiguate entities using Entity Linking,
+and answer questions asked in natural language.
+
+.. code:: python
+
+    from deeppavlov import build_model
+
+    model = build_model('kbqa_cq_en', download=True, install=True)
+    questions = ['What is the currency of Sweden?', 'When did the Korean War end?']
+    answers, answer_ids, query = model(questions)
+    print(answers)
+
+.. code:: bash
+
+    ['Swedish krona', '27 July 1953']
+
+To get list of available models for Knowledge Base Question Answering see :doc:`documentation </features/models/KBQA>`.
+
+Classification (insult and paraphrase detection, sentiment analysis, topic classification)
+==========================================================================================
+
+Insult detection predicts whether a text (e.g, post or speech in some public discussion) is considered insulting to one
+of the persons it is related to.
+
+Sentiment analysis is a task of classifying the polarity of the the given sequence.
+
+The models trained for the paraphrase detection task identify whether two sentences expressed with different words
+convey the same meaning.
+
+Topic classification refers to the task of classifying an utterance by the topic which belongs to the conversational
+domain.
+
+.. code:: python
+
+    from deeppavlov import build_model
+
+    model = build_model('insults_kaggle_bert', download=True, install=True)
+    phrases = ['You are kind of stupid', 'You are a wonderful person!']
+    labels = model(phrases)
+    print(labels)
+
+.. code:: bash
+
+    ['Insult', 'Not Insult']
+
+To get list of available models for Classification see :doc:`documentation </features/models/classification>`.
 
 Name Entity Recognition
 =======================
@@ -217,47 +297,63 @@ Named Entity Recognition (NER) classifies tokens in text into predefined categor
 (tags), such as person names, quantity expressions, percentage expressions, names
 of locations, organizations, as well as expression of time, currency and others.
 
-.. table::
-    :widths: auto
+.. code:: python
+
+    from deeppavlov import build_model
+
+    model = build_model('ner_ontonotes_bert', download=True, install=True)
+    phrases = ['Bob Ross lived in Florida', 'Elon Musk founded Tesla']
+    tokens, tags = model(phrases)
+    print(tokens, tags, sep='\n')
+
+.. code:: bash
+
+    [['Bob', 'Ross', 'lived', 'in', 'Florida'], ['Elon', 'Musk', 'founded', 'Tesla']]
+    [['B-PERSON', 'I-PERSON', 'O', 'O', 'B-GPE'], ['B-PERSON', 'I-PERSON', 'O', 'B-ORG']]
+
+To get list of available models for Name Entity Recognition see :doc:`documentation </features/models/NER>`.
+
+Entity Extraction
+=================
+
+Entity Detection is the task of identifying entity mentions in text with corresponding entity types.
+Entity Linking is the task of finding knowledge base entity ids for entity mentions in text.
+Entity Extraction configs perform subsequent Entity Detection and Entity Linking of extracted entity mentions.
+
+.. code:: python
+
+    from deeppavlov import build_model
+
+    model = build_model('entity_extraction_en', download=True, install=True)
+    phrases = ['Forrest Gump is a comedy-drama film directed by Robert Zemeckis and written by Eric Roth.']
+    entity_substr, tags, entity_offsets, entity_ids, entity_conf, entity_pages, entity_labels = model(phrases)
+    print(entity_substr, tags, entity_ids, entity_labels, sep='\n')
 
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
-    | Language | DeepPavlov config                                                                              | Demo                                      |
-    +==========+================================================================================================+===========================================+
-    | Multi    | :config:`ner_ontonotes_bert_mult <ner/ner_ontonotes_bert_mult.json>`                           | https://demo.deeppavlov.ai/#/mu/ner       |
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
-    | En       | :config:`ner_ontonotes_bert_mult <ner/ner_ontonotes_bert_mult.json>`                           | https://demo.deeppavlov.ai/#/en/ner       |
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
-    | Ru       | :config:`ner_rus_bert <ner/ner_rus_bert.json>`                                                 | https://demo.deeppavlov.ai/#/ru/ner       |
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
+.. code:: bash
 
+    [['forrest gump', 'robert zemeckis', 'eric roth']]
+    [['WORK_OF_ART', 'PERSON', 'PERSON']]
+    [[['Q134773', 'Q552213', 'Q12016774'], ['Q187364', 'Q36951156'], ['Q942932', 'Q89320386', 'Q89909683']]]
+    [[['Forrest Gump', 'Forrest Gump', 'Forrest Gump'], ['Robert Zemeckis', 'Welcome to Marwen'], ['Eric Roth', 'Eric Roth', 'Eric W Roth']]]
 
-Insult Detection
-================
+To get list of available models for Entity Extraction see :doc:`documentation </features/models/entity_extraction>`.
 
-Insult detection predicts whether a text (e.g, post or speech in some
-public discussion) is considered insulting to one of the persons it is
-related to.
+Spelling Correction
+===================
 
-.. table::
-    :widths: auto
+Spelling Correction models detect and correct spelling errors in texts.
 
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
-    | Language | DeepPavlov config                                                                              | Demo                                      |
-    +==========+================================================================================================+===========================================+
-    | En       | :config:`insults_kaggle_bert <classifiers/insults_kaggle_bert.json>`                           | https://demo.deeppavlov.ai/#/en/insult    |
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
+.. code:: python
 
+    from deeppavlov import build_model
 
-Paraphrase Detection
-====================
+    model = build_model('brillmoore_wikitypos_en', download=True, install=True)
+    phrases_w_typos = ['I think this is the begining of a beautifull frendship.', "I'll be bak"]
+    correct_phrases = model(phrases_w_typos)
+    print(correct_phrases)
 
-Detect if two given texts have the same meaning.
+.. code:: bash
 
-.. table::
-    :widths: auto
+    ['i think this is the beginning of a beautiful friendship.', "i'll be back"]
 
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
-    | Language | DeepPavlov config                                                                              | Demo                                      |
-    +==========+================================================================================================+===========================================+
-    | Ru       | :config:`paraphraser_rubert <classifiers/paraphraser_rubert.json>`                             | None                                      |
-    +----------+------------------------------------------------------------------------------------------------+-------------------------------------------+
+To get list of available models for Spelling Correction see :doc:`documentation </features/models/spelling_correction>`.