Skip to content

Commit a4318e7

Browse files
ckunkiahsimbtkilias
authored
Doc/#203 updated documentation (#208)
* Updated user guide * Updated changes file * Based on feedback from PM * Added screenshot * Use docker_client.login instead parameter auth_config which doesn't exist for docker_client.build Co-authored-by: Mikhail Beck <mikhail.beck@exasol.com> Co-authored-by: Torsten Kilias <torsten.kilias@exasol.com>
1 parent cb659d3 commit a4318e7

21 files changed

+440
-356
lines changed

README.md

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,20 @@
11
# Exasol AI-Lab
22

3-
## Overview
3+
The Exasol AI-Lab is a pre-configured container designed to empower data scientists. It streamlines common data science and AI tasks, including data loading, preparation, exploration, model training, and deployment. Whether you’re a seasoned practitioner or just getting started, the AI-Lab provides a hassle-free experience.
44

5-
This project provides an automated mechanism to build and export virtual machines images (AWS AMI, VMDK,...)
6-
enabling users to try out data science algorithms in Jupyter notebooks connected to the Exasol database.
5+
![Transformers Extension](doc/user_guide/ai-lab-screenshot.png)
76

8-
## Where to find the VM images
7+
Key Features:
8+
* Jupyter Notebook Environment: The heart of the AI-Lab is a robust Jupyter Notebook environment. It’s where you’ll work on your data science projects.
9+
* Exasol Integration: Leverage Exasol’s power for your AI and machine learning use cases. The AI-Lab includes essential Exasol packages, extensions, and configuration tasks.
10+
* Example Notebooks: Jumpstart your work with ready-to-use example notebooks. Explore classic machine learning scenarios (think scikit-learn), seamlessly integrate Exasol with AWS Sagemaker, and tap into Huggingface models directly within Exasol.
911

10-
The release process will automatically store the links to the images in the [release notes](https://github.com/exasol/ai-lab/releases/latest), as there will be a specific AMI per release.
11-
Please check the user guide about details of the image.
12+
Feel free to explore the Exasol AI-Lab and unleash your data science potential!
1213

13-
## Links
14+
The AI-Lab is available in multiple [Editions](doc/user_guide/editions.md) involving different technology stacks, see also common [System Requirements](doc/user_guide/system-requirements.md).
15+
16+
After downloading the required files and having started the AI-Lab you can connect to AI-Lab's [Jupyter Service](doc/user_guide/jupyter.md).
17+
18+
## Additional Links
1419

15-
* [User Guide](doc/user_guide/user_guide.md)
1620
* [Developer Guide](doc/developer_guide/developer_guide.md)

doc/changes/changelog.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# Changes
22

3-
* [0.2.1](changes_0.2.1.md)
3+
* [1.0.0](changes_1.0.0.md)
44
* [0.2.0](changes_0.2.0.md)
55
* [0.1.0](changes_0.1.0.md)

doc/changes/changes_0.2.1.md renamed to doc/changes/changes_1.0.0.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# AI-Lab 0.2.1, released 2024-??-??
1+
# AI-Lab 1.0.0 released 2024-02-20
22

3-
Code name: Fix Cloud Storage Notebook
3+
Code name: First Official Release
44

55
## Summary
66

@@ -11,6 +11,8 @@ Impact and delimitation
1111
* Also AWS codebuild image need to be upgraded from `aws/codebuild/standard:5.0` to `6.0`.
1212
* The Jupyterlab notebooks and their libraries remain on Python `3.8` for now.
1313

14+
Additionally the release updates the User Guide.
15+
1416
## AI-Lab-Release
1517

1618
Version: 0.2.1
@@ -41,7 +43,7 @@ n/a
4143

4244
## Documentation
4345

44-
n/a
46+
* #203: Updated User Guide
4547

4648
## Refactoring
4749

doc/developer_guide/testing.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ docker image in the tests in `integration/test_create_dss_docker_image.py`
1111
simply add CLI option `--dss-docker-image` when calling `pytest`:
1212

1313
```shell
14-
poetry run pytest --dss-docker-image exasol/ai-lab:0.2.0
14+
poetry run pytest --dss-docker-image exasol/ai-lab:1.0.0
1515
```
1616

1717
#### Executing tests involving AWS resources

doc/user_guide/ai-lab-screenshot.png

111 KB
Loading

doc/user_guide/ami_usage.md renamed to doc/user_guide/ami-usage.md

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
1-
# Using the Amazon Machine Image (AMI Edition)
1+
# AI-Lab AMI Edition (Amazon Machine Image)
22

3-
__Important__: The AMI is currently only available in the AWS region `eu-central-1`. If you want to use the image in another region, you need to copy it before. Check the [AWS documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/CopyingAMIs.html) for details about how to copy the image.
3+
The ID of the AMI (Amazon Machine Image) is mentioned in the [release notes](https://github.com/exasol/ai-lab/releases/latest) and can be used to start an EC2-instance in your AWS account.
4+
5+
The naming scheme is: "_Exasol-AI-Lab-${VERSION}_", e.g. "_Exasol-AI-Lab-5.0.0_"
6+
7+
The AMI is currently only available in the AWS region `eu-central-1`.
8+
9+
Check the [AWS documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/CopyingAMIs.html) for details about how to copy the image.
410

511
## Overview
612

@@ -19,15 +25,19 @@ __Important__: The AMI is currently only available in the AWS region `eu-central
1925
- Create inbound rules:
2026
- One of type `ssh`
2127
- If you plan to connect to the Jupyter lab, add another rule of type "Custom TCP" with port 49494.
22-
- **Important**: With this rule you expose the Jupyter lab to the internet; anybody who has access to the password will be able to execute commands. For a minimum of security you should change the default Jupyter password. Details about how to do that will be shown in the login screen when you login via ssh. However, you should consider to use only `ssh` with port forwarding.
28+
- **Important**: With this rule you expose the Jupyter lab to the internet; anybody who has access to the password will be able to execute commands. For a minimum of security you should change the default Jupyter password. Details about how to do that will be shown in the login screen when you log in via ssh. However, you should consider to use only `ssh` with port forwarding.
2329
4. Go back to the EC2 console
2430
5. Launch the EC2 instance:
2531
- In the navigation bar on the left select "Instances"
2632
- Click button "Launch instances"
27-
- At field "Application and OS Images" select the AMI id of the sandbox (found in the [release notes](https://github.com/exasol/ai-lab/releases/latest))
33+
- At field "Application and OS Images" select the AMI ID of the sandbox (found in the [release notes](https://github.com/exasol/ai-lab/releases/latest))
2834
- Select an appropriate instance type (at least "t2.small" or similar)
2935
- Choose your key pair
30-
- Choose the security group which your created in step 3.
36+
- Choose the security group which you created in step 3.
3137
- For the storage we recommend to keep the pre-selected 100GB volume
3238
- Click button "Launch instance"
3339
6. As soon as the machine becomes available you can connect per ssh with user `ubuntu`: `ssh -i your_key.pem ubuntu@the_new_ec_instance`
40+
41+
## Login
42+
43+
See [Login to AMI and VM Editions](login-vm-and-ami.md) for logging in to the system.

doc/user_guide/docker/docker-usage.md

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# AI-Lab Docker Edition
2+
3+
Using Exasol AI-Lab Docker Edition requires some specific [prerequisites](prerequisites.md) but also offers additional benefits.
4+
5+
The [Operating System and Setup Guide](prerequisites.md#operating-systems-and-setups) helps you with the initial system setup.
6+
7+
AI-Lab also offers a [short introduction](intro.md) to Docker Images and Containers if you are new to this technology.
8+
9+
## Defining Environment Variables
10+
11+
The Unix shell commands in the following sections will use some environment variables. By this you can adapt the commands to your specific preferences while still being able to execute them literally:
12+
* Variable `VERSION` refers to the version of Exasol AI-Lab Docker Edition you want to use, alternatively you can use `latest`.
13+
* Variable `VOLUME` is expected to contain the name of your Docker volume, see [Managing User Data](managing-user-data.md).
14+
* Variable `LISTEN_IP` defines the range of IP-addresses allowed to connect to the forwarded Jupyter port.
15+
* `0.0.0.0` means all IP-addresses are allowed.
16+
* For local setups, we recommend `127.0.0.1`.
17+
* Please contact your IT department if there are security restrictions.
18+
* Variable `CONTAINER_NAME` defines the name for the AI-Lab Docker container. Giving the running container a name makes it easier to refer to it when [stopping](#stopping-the-ai-lab-docker-container) or [restarting](#restarting-a-stopped-container) it.
19+
20+
Here is an example:
21+
22+
```shell
23+
VERSION=1.0.0
24+
LISTEN_IP=0.0.0.0
25+
VOLUME=my-vol
26+
CONTAINER_NAME=ai-lab
27+
```
28+
29+
## Creating a Docker Container for the AI- Lab from the AI-Lab Docker Image
30+
31+
You can use an Exasol database with AI-Lab in two ways
32+
* [External Exasol database](#ai-lab-with-external-exasol-database-only)
33+
* Integrated [Exasol Docker-DB](#ai-lab-with-integrated-exasol-docker-db) managed by AI-Lab
34+
35+
### AI-Lab with External Exasol Database Only
36+
37+
In this scenario the AI-Lab Docker container does not need access to the Docker daemon.
38+
39+
The following command will
40+
* Download the Docker image for the specified version `$VERSION` of the AI-Lab if the image of the specified version is not yet available in your Docker service
41+
* Run a Docker container using this image
42+
* Mount the volume `$VOLUME` to the directory `/root/notebooks` inside the container
43+
* If the volume does not exist yet, then it will be created automatically.
44+
* Forward port `49494` on the [daemon machine](prerequisites.md) to allow connections from all IP addresses matched by `$LISTEN_IP`
45+
46+
```shell
47+
docker run \
48+
--name ${CONTAINER_NAME} \
49+
--volume ${VOLUME}:/root/notebooks \
50+
--publish ${LISTEN_IP}:49494:49494 \
51+
exasol/ai-lab:${VERSION}
52+
```
53+
54+
If you want to use a newer version of the AI-Lab then please [delete the Docker volumes](managing-user-data.md#replacing-the-docker-volume) created with older versions.
55+
56+
Additional options
57+
* Add option `--detach` to run the container in the background but please note that the initial welcome message with instructions will be hidden then, see also Command [`docker logs`](https://docs.docker.com/engine/reference/commandline/container_logs/) and section [Stopping the AI-Lab Docker Container](#stopping-the-ai-lab-docker-container).
58+
* If port `49494` is not available on your daemon machine you can forward port `49494` of the Jupyter server in the Docker container to another port, e.g. `55555`, on the daemon machine with `--publish ${LISTEN_IP}:55555:49494`
59+
60+
### AI-Lab with Integrated Exasol Docker-DB
61+
62+
In this scenario you must enable the AI-Lab Docker container to access the Docker daemon.
63+
64+
**Please note**
65+
* Additional [Limitations and security risks](prerequisites.md#enabling-exasol-ai-lab-to-use-docker-features) apply.
66+
* Only file system objects on the daemon machine can be mounted. This applies to ordinary directories as well as the `docker.sock`.
67+
* On Windows mounting `docker.sock` only works with Docker Desktop with WSL 2.
68+
```shell
69+
docker run \
70+
--name ${CONTAINER_NAME} \
71+
--volume ${VOLUME}:/root/notebooks \
72+
--volume /var/run/docker.sock:/var/run/docker.sock \
73+
--publish ${LISTEN_IP}:49494:49494 \
74+
exasol/ai-lab:${VERSION}
75+
```
76+
77+
## Stopping the AI-Lab Docker Container
78+
79+
If you used one of the commands given in preceding sections without option `--detach` then you can stop the AI-Lab Docker container by simply pressing Ctrl-C.
80+
81+
If you used option `--detach` then you need to use the following command:
82+
83+
```shell
84+
docker stop ${CONTAINER_NAME}
85+
```
86+
87+
See also https://docs.docker.com/engine/reference/commandline/container_stop/.
88+
89+
## Restarting a Stopped Container
90+
91+
In general we recommend to restart a stopped container instead of creating a new one. This has the benefit of keeping [additional dependencies](#installing-additional-dependencies) that you did install:
92+
93+
```shell
94+
docker start ${CONTAINER_NAME}
95+
```
96+
97+
**Please note**
98+
* If you accidently created a new AI-Lab Docker container and the stopped container still exists please remove the new container and restart the existing one.
99+
* If there is no stopped AI-Lab container and you are using Exasol Docker-DB you need to link the newly created container to the Exasol Docker-DB network `db_network_DemoDb`.
100+
101+
```shell
102+
docker network connect db_network_DemoDb <CONTAINER>
103+
```
104+
105+
See also https://docs.docker.com/engine/reference/commandline/network_connect/.
106+
107+
## Connecting to the Jupyter Service
108+
109+
When starting AI-Lab as a Docker container the command line will display a welcome message showing connection instructions and a reminder to change the default password:
110+
111+
```
112+
$ docker run --publish 0.0.0.0:$PORT:49494 exasol/ai-lab:$VERSION
113+
Server for Jupyter has been started successfully.
114+
115+
You can connect with http://<host>:<port>
116+
117+
If using a Docker daemon on your local machine and did forward the
118+
port to the same port then you can connect with http://localhost:49494.
119+
120+
┬ ┬┌─┐┌┬┐┌─┐┌┬┐┌─┐ ┬ ┬┌─┐┬ ┬┬─┐ ┬┬ ┬┌─┐┬ ┬┌┬┐┌─┐┬─┐ ┌─┐┌─┐┌─┐┌─┐┬ ┬┌─┐┬─┐┌┬┐ ┬
121+
│ │├─┘ ││├─┤ │ ├┤ └┬┘│ ││ │├┬┘ ││ │├─┘└┬┘ │ ├┤ ├┬┘ ├─┘├─┤└─┐└─┐││││ │├┬┘ ││ │
122+
└─┘┴ ─┴┘┴ ┴ ┴ └─┘ ┴ └─┘└─┘┴└─ └┘└─┘┴ ┴ ┴ └─┘┴└─ ┴ ┴ ┴└─┘└─┘└┴┘└─┘┴└──┴┘ o
123+
124+
The default password is "ai-lab".
125+
To update the password, log in to the Docker container as the user root and run
126+
/root/jupyterenv/bin/jupyter-lab server password
127+
```
128+
129+
Using an internet browser you then can connect to the Jupyter server running in the Docker container in order to follow the tutorials presented by a set of Jupyter notebooks, see [Connecting to Jupyter Service](../jupyter.md#open-jupyter-in-your-browser).
130+
131+
For the parameter `<host>`: If your daemon machine is identical to the machine your browser is running on then you can replace `<host>` by `localhost` otherwise please use the IP address of the daemon machine.
132+
133+
The following section explains how to log in to the Docker container to change settings, such as the default password.
134+
135+
## Logging in to the Docker container
136+
137+
To update the password you must log in to the Docker container.
138+
139+
First, you need to find out the container's ID. The following command shows the list of currently running Docker containers.
140+
141+
```shell
142+
docker ps
143+
```
144+
145+
Here is a sample output
146+
147+
```
148+
CONTAINER ID IMAGE COMMAND NAMES
149+
1199447716d4 image:2 "entrypoint" funny_rabbit
150+
```
151+
152+
The following command enables you to log in as the user `root` to the specified container:
153+
154+
```shell
155+
docker exec --user root -it ${CONTAINER_NAME} bash
156+
```
157+
158+
## Installing Additional Dependencies
159+
160+
See [User Guide](../jupyter.md#installing-additional-dependencies).
161+
162+
Please note: Removing the docker container `docker rm <container>` will discard all dependencies that have been installed additionally.

0 commit comments

Comments
 (0)