Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add instructions for using clip server hosted by jina #848

Merged
merged 5 commits into from
Oct 31, 2022
Merged

docs: add instructions for using clip server hosted by jina #848

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d64f62e
docs: add instructions for using clip server hosted by jina
ZiniuYu Oct 28, 2022
89b0ffd
fix: update landing page
ZiniuYu Oct 28, 2022
45d071b
fix: update readme
ZiniuYu Oct 28, 2022
4dba979
fix: toctree
ZiniuYu Oct 28, 2022
db31471
fix: address comment
ZiniuYu Oct 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,15 @@ CLIP-as-service is a low-latency high-scalability service for embedding images a

## Try it!

An always-online server `api.clip.jina.ai` loaded with `ViT-L/14-336px` is there for you to play & test.
Before you start, make sure you have created access token from our [console website](https://console.clip.jina.ai/get_started),
or CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat).
An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play & test.
Before you start, make sure you have obtained an access token from our [console website](https://console.clip.jina.ai/get_started),
or via CLI as described in [this guide](https://docs.jina.ai/jina-ai-cloud/login/#create-a-new-pat).

```bash
jina auth token create <name of PAT> -e <expiration days>
```

Then, you need to set the created token in HTTP request header `Authorization` as `<your access token>`,
or configure it in the parameter `credential` of the client in python.
Then, you need to configure the access token in the parameter `credential` of the client in python or set it in the HTTP request header `Authorization` as `<your access token>`.

⚠️ Our demo server `demo-cas.jina.ai` is sunset and no longer available after **15th of Sept 2022**.

Expand Down
95 changes: 95 additions & 0 deletions docs/hosting/by-jina.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Hosted by Jina AI
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference betwee hostted by Jina AI and hosted by jCloud? I think this title is misleading. Maybe we can just use # Cloud Hosting?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this page would explain the SaaS product.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One is hosted by Jina, right now the user can only access it but not control it.
The other is hosted on Jcloud, the user has full control of it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this title does not tell the difference


Just like any other machine learning models, CLIP models have better performance when running on GPU. However, it is not always possible to have a GPU machine at hand, and it could be costly to configure a GPU machine. To make CLIP models more accessible, we provide a hosted service for CLIP models. You can send requests to our hosted service and get the embedding results back.

An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play or develop your CLIP applications. The server is available for **encoding** and **ranking** tasks.

Before you start, make sure you have obtained an access token from our [console website](https://console.clip.jina.ai/get_started), or via CLI as described in [this guide](https://docs.jina.ai/jina-ai-cloud/login/#create-a-new-pat)

```bash
jina auth token create <name of PAT> -e <expiration days>
```

(by-jina-python)=
## Connect in Python

We provide two ways to send requests to our hosted service: via gRPCs and via HTTPs.

| Protocol | Address |
| -------- | ------------------------------- |
| gRPCs | `grpcs://api.clip.jina.ai:2096` |
| HTTPs | `https://api.clip.jina.ai:8443` |


To use the service, you need select the protocol by specifying corresponding address in the client. For example, if you want to use gRPCs, you need to specify the address as `grpcs://api.clip.jina.ai:2096`.

Then, you need to configure the access token in the parameter `credential` of the client:


````{tab} via gRPCs

```{code-block} python
---
emphasize-lines: 4
---
from clip_client import Client

c = Client(
'grpcs://api.clip.jina.ai:2096', credential={'Authorization': '<your access token>'}
)

r = c.encode(
[
'First do it',
'then do it right',
'then do it better',
'https://picsum.photos/200',
]
)
```

````
````{tab} via HTTPs

```{code-block} python
---
emphasize-lines: 4
---
from clip_client import Client

c = Client(
'https://api.clip.jina.ai:8443', credential={'Authorization': '<your access token>'}
)

r = c.encode(
[
'First do it',
'then do it right',
'then do it better',
'https://picsum.photos/200',
]
)
```

````

(by-jina-curl)=
## Connect using plain HTTP request via `curl`

You can also send requests to our hosted service using plain HTTP request via `curl` by configuring the access token in the HTTP request header `Authorization` as `<your access token>`.


```{code-block} bash
---
emphasize-lines: 4
---
curl \
-X POST https://api.clip.jina.ai:8443/post \
-H 'Content-Type: application/json' \
-H 'Authorization: <your access token>' \
-d '{"data":[{"text": "First do it"},
{"text": "then do it right"},
{"text": "then do it better"},
{"uri": "https://picsum.photos/200"}],
"execEndpoint":"/"}'
```
99 changes: 53 additions & 46 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,46 +7,31 @@

## Try it!

An always-online server `api.clip.jina.ai` loaded with `ViT-L/14-336px` is there for you to play & test.

Before you start, make sure you have created access token from our [console website](https://console.clip.jina.ai/get_started),
or CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat).
An always-online server `api.clip.jina.ai` loaded with `ViT-L-14-336::openai` is there for you to play & test.
Before you start, make sure you have created an access token from our [console website](https://console.clip.jina.ai/get_started),
or via CLI as described in [this guide](https://github.com/jina-ai/jina-hubble-sdk#create-a-new-pat).

```bash
jina auth token create <name of PAT> -e <expiration days>
```

Then, you need to set the created token in HTTP request header `Authorization` as `<your access token>`,
or configure it in the parameter `credential` of the client in python.


````{tab} via HTTPS 🔐

```bash
curl \
-X POST https://api.clip.jina.ai:8443/post \
-H 'Content-Type: application/json' \
-H 'Authorization: <your access token>' \
-d '{"data":[{"text": "First do it"},
{"text": "then do it right"},
{"text": "then do it better"},
{"uri": "https://picsum.photos/200"}],
"execEndpoint":"/"}'
```

````
Then, you need to configure the access token in the parameter `credential` of the client in python or set it in the HTTP request header `Authorization` as `<your access token>`.

````{tab} via gRPC ⚡⚡

```bash
pip install clip-client
```

```python
```{code-block} python
---
emphasize-lines: 5
---
from clip_client import Client

c = Client(
'grpcs://api.clip.jina.ai:2096', credential={'Authorization': '<your access token>'}
'grpcs://api.clip.jina.ai:2096',
credential={'Authorization': '<your access token>'}
)

r = c.encode(
Expand All @@ -62,11 +47,30 @@ print(r)

````

````{tab} via HTTPS 🔐

```{code-block} bash
---
emphasize-lines: 4
---
curl \
-X POST https://api.clip.jina.ai:8443/post \
-H 'Content-Type: application/json' \
-H 'Authorization: <your access token>' \
-d '{"data":[{"text": "First do it"},
{"text": "then do it right"},
{"text": "then do it better"},
{"uri": "https://picsum.photos/200"}],
"execEndpoint":"/"}'
```

````

## Install

![PyPI](https://img.shields.io/pypi/v/clip_client?color=%23ffffff&label=%20) is the latest version.

Make sure you have Python 3.7+. You can install client and server independently. You **don't** have to install both: e.g. installing `clip_server` on a GPU machine and `clip_client` on a local laptop.
Make sure you are using Python 3.7+. You can install the client and server independently. It is **not required** to install both: e.g. you can install `clip_server` on a GPU machine and `clip_client` on a local laptop.

````{tab} Client

Expand Down Expand Up @@ -112,49 +116,50 @@ pip install "clip_server[tensorrt]"
````



## Quick check

After install, you can run the following commands for a quick connectivity check.
After installing, you can run the following commands for a quick connectivity check.

### Start the server

````{tab} Run PyTorch Server
````{tab} Start PyTorch Server
```bash
python -m clip_server
```
````

````{tab} Run ONNX Server
````{tab} Start ONNX Server
```bash
python -m clip_server onnx-flow.yml
```
````

````{tab} Run TensorRT Server
````{tab} Start TensorRT Server
```bash
python -m clip_server tensorrt-flow.yml
```
````

At the first time, it will download the default pretrained model, which may take a minute. Then you will get the following address information:
At the first time starting the server, it will download the default pretrained model, which may take a while depending on your network speed. Then you will get the address information similar to the following:

```text
🔗 Protocol GRPC
🏠 Local access 0.0.0.0:51000
🔒 Private network 192.168.3.62:51000
🌐 Public address 87.105.159.191:51000
╭────────────── 🔗 Endpoint ───────────────╮
│ 🔗 Protocol GRPC │
│ 🏠 Local 0.0.0.0:51000 │
│ 🔒 Private 192.168.31.62:51000 │
| 🌍 Public 87.105.159.191:51000 |
╰──────────────────────────────────────────╯
```

It means the server is ready to serve. Note down the three addresses showed above, you will need them later.
This means the server is ready to serve. Note down the three addresses shown above, you will need them later.

### Connect from client

```{tip}
Depending on the location of the client and server. You may use different IP addresses:
- Client and server are on the same machine: use local address e.g. `0.0.0.0`
- Client and server are behind the same router: use private network address e.g. `192.168.3.62`
- Server is in public network: use public network address e.g. `87.105.159.191`
- Client and server are on the same machine: use local address, e.g. `0.0.0.0`
- Client and server are connected to the same router: use private network address, e.g. `192.168.3.62`
- Server is in public network: use public network address, e.g. `87.105.159.191`
```

Run the following Python script:
Expand All @@ -169,11 +174,12 @@ c.profile()
will give you:

```text
Roundtrip 16ms 100%
├── Client-server network 12ms 75%
└── Server 4ms 25%
├── Gateway-CLIP network 0ms 0%
└── CLIP model 4ms 100%
Roundtrip 16ms 100%
├── Client-server network 8ms 49%
└── Server 8ms 51%
├── Gateway-CLIP network 2ms 25%
└── CLIP model 6ms 75%
{'Roundtrip': 15.684750003856607, 'Client-server network': 7.684750003856607, 'Server': 8, 'Gateway-CLIP network': 2, 'CLIP model': 6}
```

It means the client and the server are now connected. Well done!
Expand All @@ -200,8 +206,9 @@ user-guides/faq
:caption: Hosting
:hidden:

hosting/colab
hosting/by-jina
hosting/on-jcloud
hosting/colab
```

```{toctree}
Expand Down