Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Commit

Permalink
update docs and notebook port config to resolve comments
Browse files Browse the repository at this point in the history
  • Loading branch information
mydmdm committed Jun 28, 2019
1 parent 949634f commit 2ae6eb8
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 9 deletions.
13 changes: 11 additions & 2 deletions contrib/python-sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Besides above benefits, this project also provides powerful runtime support, whi

- [Get started](#Get-started)
- [Installation](#Installation)
- [Dependencies](#Dependencies)
- [Define your clusters](#Define-your-clusters)
- [How-to guide for the CLI tool](#How-to-guide-for-the-CLI-tool)
- [Cluster and storage management](#Cluster-and-storage-management)
Expand All @@ -32,6 +33,7 @@ Besides above benefits, this project also provides powerful runtime support, whi
- [How to preview the generated job config but not submit it](#How-to-preview-the-generated-job-config-but-not-submit-it)
- [`Jupyter` notebook](#Jupyter-notebook)
- [How to run a local notebook with remote resources](#How-to-run-a-local-notebook-with-remote-resources)
- [How to launch a remote jupyter server and connect it](#How-to-launch-a-remote-jupyter-server-and-connect-it)
- [Other FAQ of CLI](#Other-FAQ-of-CLI)
- [How to select a cluster to use until I change it](#How-to-select-a-cluster-to-use-until-I-change-it)
- [How to simplify the command](#How-to-simplify-the-command)
Expand Down Expand Up @@ -65,9 +67,12 @@ opai -h
python -c "from openpaisdk import __version__; print(__version__)"
```

And you may also change it to another branch (only take effect in the job container) by `opai set sdk-branch=<your/branch>`
And you may also change it to another branch (only take effect in the job container) by `opai set sdk-branch=<your/branch>`.

_The package require python3, and we tested it on `py3.5+` environment_
### Dependencies

- The package requires python3 (mainly because of `type hinting`), and we only tested it on `py3.5+` environment. _Only commands `job sub` and `job notebook` require installing this project inside container, others don't make any constraints of `python` version in the docker container._
- [`Pylon`](https://github.com/microsoft/pai/tree/master/docs/pylon) is required to parse the REST api path like `/reset-server/`.

## Define your clusters

Expand Down Expand Up @@ -264,6 +269,10 @@ This command requires options as the `opai job sub` does. This command would
- _Local_ - wait and query the job state until its status to be `SUCCEEDED`
- _Local_ - download `<html-result>` to local and open it with web browser

### How to launch a remote jupyter server and connect it

Sometimes user may want to launch a remote jupyter server and do some work on it interactively. To do this, just add `--interactive` in `job notebook` command. After submitting the job, a link like `http://x.x.x.x:port/notebooks/<notebook>` will be opened in your browser. Since it takes a while to start the container, please wait and refresh the page until the notebook opens. Use the default token `abcd` (unless it is overriden by `--token <token>`) to login the notebook.

## Other FAQ of CLI

### How to select a cluster to use until I change it
Expand Down
12 changes: 5 additions & 7 deletions contrib/python-sdk/openpaisdk/job.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,9 @@ def from_notebook(self, nb_file: str, image: str, cluster: dict, resources: dict
interactive_mode, nb_file = True, ""
html_file = os.path.splitext(nb_file)[0] + ".html" if not interactive_mode else ""
if interactive_mode:
resources.setdefault("ports", {})["jupyter"] = 1
cmds = [
"jupyter notebook --no-browser --ip 0.0.0.0 --port 8888 --NotebookApp.token={} --allow-root {}".format(token, nb_file),
"jupyter notebook --no-browser --ip 0.0.0.0 --port $PAI_CONTAINER_HOST_jupyter_PORT_LIST --NotebookApp.token={} --allow-root --NotebookApp.file_to_run={}".format(token, nb_file),
]
else:
cmds = [
Expand All @@ -232,14 +233,11 @@ def from_notebook(self, nb_file: str, image: str, cluster: dict, resources: dict
assert state == "RUNNING", "why not running {}".format(state)
while True:
try:
status = client.jobs(self.name)
ip = status["taskRoles"]["main"]["taskStatuses"][0]["containerIp"]
status = client.jobs(self.name)["taskRoles"]["main"]["taskStatuses"][0]
browser_open("http://%s:%s/notebooks/%s" % (status["containerIp"], status["containerPorts"]["jupyter"], nb_file))
break
except:
ip = None
time.sleep(10)
if ip:
break
browser_open("http://%s:8888/?token=%s" % (ip, token))
else:
state = client.wait([self.name])[0]
if state != "SUCCEEDED":
Expand Down

0 comments on commit 2ae6eb8

Please sign in to comment.