Skip to content

Releases: exasol/ai-lab

3.1.0: Additional fixes on top of release 3.0.0

10 Sep 13:04
@ckunki ckunki
3.1.0
69ae6ea
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
3.1.0: Additional fixes on top of release 3.0.0 Latest
Latest

Summary

This release updates jupyterlab to version 4.2.5 and applies some fixes to the Jupyter notebooks.

Refactoring

  • #324 Used pytest-plugins in notebook tests.

Bug Fixes

  • #326
    • Scikit-learn notebook: call model's fit and predict with numpy arrays;
    • Ibis notebook: added a link to this notebook on the front page;
    • Configuration: added internal bucket-fs host name and port;
    • Upgraded jupyterlab to 4.2.5.

Dependency Updates

pyproject.toml

  • Updated dependency boto3:1.35.2 to 1.35.11
  • Updated dependency rich:13.7.1 to 13.8.0
  • Updated dependency pygithub:2.3.0 to 2.4.0
  • Updated dependency cfn-lint:1.10.3 to 1.12.1
  • Updated dependency localstack:3.6.0 to 3.7.1

jupyter_requirements.txt

  • Updated dependency jupyterlab:4.1.1 to 4.2.5

notebook_requirements.txt

  • Updated dependency exasol-notebook-connector:0.2.9 to 0.3.0

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:3.1.0

AMI Images (Filter=3.1.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-091554ce41898f1c0 Exasol-AI-Lab-3.1.0 Image Description yes 561494727831/Exasol-AI-Lab-3.1.0 2024-09-10T10:34:04.000Z available 3.1.0

VM Images, Prefix=ai_lab/3.1.0)

Key Size URL
0 ai_lab/3.1.0/exasol-ai-lab-3.1.0.vhd 14.98 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/3.1.0/exasol-ai-lab-3.1.0.vhd
1 ai_lab/3.1.0/exasol-ai-lab-3.1.0.vmdk 10.44 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/3.1.0/exasol-ai-lab-3.1.0.vmdk
Assets 3
Loading

3.0.0: More Notebooks, TE-Update, Exasol SaaS, Python 3.10

26 Aug 09:59
@tkilias tkilias
3.0.0
c18369d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
3.0.0: More Notebooks, TE-Update, Exasol SaaS, Python 3.10

Summary

This release adds Jupyter tutorials for using the IBIS dataframe library and for building Exasol Script Language Containers (SLC).

Transitive dependency transformers-extension (TE) via notebook-connector has been updated to TE version 2.0.0. This fixes an error in saving and loading of the model but required a breaking change regarding the storage format of the TE model.

Users of the AI-Lab must redeploy TE into their database and upload the models once again.

The release adds support for parameters for SaaS instances of Exasol database to the configuration page and fixes vulnerability CVE-2024-23342 by updating dependencies and also updates the operating system from ubuntu 20.04 to 22.04 and Python version to 3.10 in the published images for Docker, AMI, and virtual machines.

Additionally, this release fixes the following vulnerabilities by updating dependencies:

  • Vulnerability CVE-2024-23342 in transitive dependency via localstack to ecdsa vulnerably to Minerva timing attack on P-256 in python-ecdsa.
  • Vulnerability CVE-2024-5206 in dependency scikit-learn versions below 1.5.0 caused by sensitive data leakage.
  • Vulnerability CVE-2024-35195 in dependency requests in versions below 2.32.0 caused by requests Session object not verifying requests after making first request with verify=False.
  • Vulnerability CVE-2024-37891 in transitive dependency via boto3 to urllib3 in versions below 2.2.2 caused by proxy-authorization request header not to be stripped during cross-origin redirects.

The release ignores vulnerability CVE-2024-33663 in transitive dependency via localstack to python-jose 3.3.0 caused by algorithm confusion with OpenSSH ECDSA keys as there is no newer version of python-jose available and the dependency only affects tests.

Features

  • #277: Added the SaaS database parameters to the configuration page.
  • #279: Made the notebooks tests running in SaaS as well as in the Docker-DB.
  • #19: Added SLC notebook
  • #301: Added CloudFront distribution for example data S3 bucket
  • #273: Added jupyterenv/bin to environment variable PATH for running Jupyter Server

Security

  • #207: Fixed vulnerability CVE-2024-23342 by updating dependency ecdsa
  • #298: Fixed vulnerabilities by updating dependencies

Bug Fixes

  • #303: Fixed AWS Codebuild
  • #313: Fixed differing versions of dependency scikit-learn

Documentation

Refactoring

  • #267: Switched CodeBuildWaiter to use tenacity
  • #276: Started using the new ITDE Manager interface in the notebook-connector 0.2.9
  • #282: Updated python version to Python 3.10
  • #295: Made notebook-tests mandatory for merge
  • #193: Ignored warnings in notebook tests
  • #297: Reduced log level for transitive libraries in notebook tests
  • #307: Made the notebook tests running in parallel; moved common steps from test jobs to a composite action
  • #308: Removed redundant dependencies from file notebook_requirements.txt.
  • #318: Re-enabled disabled notebook tests

Dependency Updates

Jupyter Environment Dependencies

In File notebook_requirements.txt:

  • Removed dependency to uncertainties
  • Updated scikit-learn:1.0.2 to 1.5.1
  • Updated matplotlib:3.7.4 to 3.9.2
  • Updated jupysql:0.10.10 to 0.10.12
  • Relaxed declaration of dependency stopwatch.py:2.0.1 to 2.* to avoid inconsistencies with ITDE
  • Updated exasol-notebook-connector:0.2.8 to 0.2.9
  • Updated ipywidgets:8.1.1 to 8.1.3

Dependencies in ai-lab/pyproject.toml

  • Updated boto3:1.34.144 to 1.35.2
  • Updated ansible:9.8.0 to 10.3.0
  • Updated rich:12.6.0 to 13.7.1
  • Updated pandas:1.5.3 to 2.2.2
  • Updated tenacity:8.5.0 to 9.0.0
  • Updated importlib-metadata:7.2.1 to 8.4.0
  • Updated pytest-check-links:0.9.3 to 0.10.1
  • Updated pytest:7.4.4 to 8.3.2
  • Updated cfn-lint:0.65.1 to 1.10.3
  • Updated localstack:3.5.0 to 3.6.0
  • Updated docker:6.1.3 to 7.1.0
  • Updated fabric:2.7.1 to 3.2.2
  • Updated requests:2.31.0 to 2.32.3
  • Removed explicit declaration of dependency numpy

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:3.0.0

AMI Images (Filter=3.0.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-08cf9ff725cd94595 Exasol-AI-Lab-3.0.0 Image Description yes 561494727831/Exasol-AI-Lab-3.0.0 2024-08-26T07:04:43.000Z available 3.0.0

VM Images, Prefix=ai_lab/3.0.0)

Key Size URL
0 ai_lab/3.0.0/exasol-ai-lab-3.0.0.vhd 15.65 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/3.0.0/exasol-ai-lab-3.0.0.vhd
1 ai_lab/3.0.0/exasol-ai-lab-3.0.0.vmdk 11.08 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/3.0.0/exasol-ai-lab-3.0.0.vmdk
Assets 3
Loading

2.0.0: Use non-privileged user for running JupyterLab

28 Mar 15:28
@ckunki ckunki
2.0.0
d6fffbc
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
2.0.0: Use non-privileged user for running JupyterLab

Summary

The following changes are especially important if you are using the AI-Lab's Docker Edition and are mounting a volume containing your private notebook files and the Secure Configuration Storage (SCS) into the AI-Lab's Docker container.

Major changes

  1. The mount-point for Jupyter notebook files and the SCS has moved from /root/notebooks to /home/jupyter/notebooks.
  2. Some of the notebooks have been updated, especially the Cloud Storage Extension notebook.

In case you are using the AI-Lab's Docker Edition with mounted volume, then please

  1. Change your commands to use the new mount point as described in the User Guide and
  2. Find the updated notebooks in folder /home/jupyter/notebook-defaults as the AI-Lab does not overwrite existing files, to avoid losing manual changes.

AI-Lab-Release

Version: 2.0.0

Features

  • #223: Added support to add docker image tag "latest"
  • #204: Updated developer guide
  • #177: Disabled core dumps
  • #255: Changed owner of notebooks to jupyter in entrypoint.py

Security

n/a

Bug Fixes

  • #241: Fixed non-root-user access

Documentation

  • #204: Updated developer guide
  • #219: Described Virtual Box setup in user guide

Refactoring

  • #217: Changed notebook-connector dependency, now installing it from PyPi.
  • #220: Changed default ports in the external database configuration.
  • #221: Changed wording in the main configuration notebook, as suggested by PM.
  • #66: Used a non-root user to run Jupyter in the Docker Image ai-lab
  • #149: Split AWS tests
  • #252: Added tests for access to Docker socket

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:2.0.0

AMI Images (Filter=2.0.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-040f01ce7b0ba402a Exasol-AI-Lab-2.0.0 Image Description yes 561494727831/Exasol-AI-Lab-2.0.0 2024-03-28T13:12:48.000Z available 2.0.0

VM Images, Prefix=ai_lab/2.0.0)

Key Size URL
0 ai_lab/2.0.0/exasol-ai-lab-2.0.0.vhd 14.6 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/2.0.0/exasol-ai-lab-2.0.0.vhd
1 ai_lab/2.0.0/exasol-ai-lab-2.0.0.vmdk 10.03 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/2.0.0/exasol-ai-lab-2.0.0.vmdk
Assets 3
Loading

1.0.0: First Official Release

20 Feb 20:54
@ckunki ckunki
1.0.0
a4318e7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
1.0.0: First Official Release

Summary

This release fixes the Cloud Storage notebook and also fixes vulnerabilities by updating dependencies in file poetry.lock and GitHub workflows.

Impact and delimitation

  • Updating the dependencies required to upgrade the build environment from Python 3.8 to 3.10.
  • Also AWS codebuild image need to be upgraded from aws/codebuild/standard:5.0 to 6.0.
  • The Jupyterlab notebooks and their libraries remain on Python 3.8 for now.

Additionally the release updates the User Guide.

AI-Lab-Release

Version: 1.0.0

Features

n/a

Security

Bug Fixes

  • #205: Error on cloud storage notebook init

Documentation

  • #203: Updated User Guide

Refactoring

n/a

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:1.0.0

AMI Images (Filter=1.0.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-0186ee45a0b054ad8 Exasol-AI-Lab-1.0.0 Image Description yes 561494727831/Exasol-AI-Lab-1.0.0 2024-02-20T19:23:14.000Z available 1.0.0

VM Images, Prefix=ai_lab/1.0.0)

Key Size URL
0 ai_lab/1.0.0-rc-20240220_153140/exasol-ai-lab-1.0.0-rc-20240220_153140.vhd 5.34 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/1.0.0-rc-20240220_153140/exasol-ai-lab-1.0.0-rc-20240220_153140.vhd
1 ai_lab/1.0.0-rc-20240220_153140/exasol-ai-lab-1.0.0-rc-20240220_153140.vmdk 2.26 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/1.0.0-rc-20240220_153140/exasol-ai-lab-1.0.0-rc-20240220_153140.vmdk
2 ai_lab/1.0.0/exasol-ai-lab-1.0.0.vhd 5.52 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/1.0.0/exasol-ai-lab-1.0.0.vhd
3 ai_lab/1.0.0/exasol-ai-lab-1.0.0.vmdk 2.43 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/1.0.0/exasol-ai-lab-1.0.0.vmdk
Assets 3
Loading

0.2.0: Post release fixes

15 Feb 20:02
@ckunki ckunki
0.2.0
ee1e750
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
0.2.0: Post release fixes

Summary

This release comes with a number of updates and improvements and enhanced and fixed documentation.

AI-Lab-Release

Version: 0.2.0

Bug Fixes

  • #163: Fixed version number of VM images etc.
  • #161: Fixed the bug in the Transformers' Translation notebook.

Documentation

  • #125: Explained login to docker container
  • #174: Added to FAQ: How to install additional python packages into the Docker container

Refactoring

  • #160: Implemented the PM's recommendations of 2024-01-24.
  • #120: Passing the secret store object (sb_config) as a parameter to all functions that need it.
  • #165: Reduced log output in Codebuild ai-lab
  • #184: Changed notebook tests to only run if the commit message contains a special string
  • #167: Replacing the term "Docker-DB" with "Exasol Docker-DB" in all notebooks and documentation.
  • #168: Renaming the section name “Access Configuration” to "Open Secure Configuration Storage".
  • #170: Renaming the section name "Set up" to "Setup".
  • #182: Renaming the secret store global variable from "sb_config" to "ai_lab_config".
  • #169: Renaming the default database schema from "IDA" to "AI_LAB".
  • #128: Removed unused dependencies
  • #188: Start using the new namespace of the notebook-connector 0.2.7.

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:0.2.0

AMI Images (Filter=0.2.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-0e5f5f9120944ac60 Exasol-AI-Lab-0.2.0 Image Description yes 561494727831/Exasol-AI-Lab-0.2.0 2024-02-15T17:54:52.000Z available 0.2.0

VM Images, Prefix=ai_lab/0.2.0)

Key Size URL
0 ai_lab/0.2.0/exasol-ai-lab-0.2.0.vhd 5.81 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/0.2.0/exasol-ai-lab-0.2.0.vhd
1 ai_lab/0.2.0/exasol-ai-lab-0.2.0.vmdk 2.46 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/0.2.0/exasol-ai-lab-0.2.0.vmdk
Assets 3
Loading

0.1.0: Initial release

31 Jan 23:58
@tkilias tkilias
0.1.0
1cc2138
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
0.1.0: Initial release

Summary

Initial release of the Exasol AI Lab. The project enables users to try out data science algorithms
in Jupyter notebooks connected to the Exasol database. This release provides:

  • A JupyterLab environment with reference implementation notebooks of data science task using Exasol
  • A Secure Configuration Store to easily and securely manage your credentials to a Exasol database and other external service
  • The creation of an Amazon Machine Image (AMI), Virtual Machine Images, and a Docker Image for a specific version
    of the Exasol AI Lab project.

AI-Lab-Release

Version: 0.1.0

Features

  • #11: Created a notebook to show training with scikit-learn in the notebook
  • #15: Installed exasol-notebook-connector via ansible
  • #30: Added script to build the Data Science Sandbox as Docker Image
  • #33: Added a notebook to securely manage sandbox configuration
  • #30: Used the Secret Store in the Learning-in-the-notebook tutorial
  • #41: Refactored the Transformer Extension notebook - made it use the Secret Store
  • #53: Moved Jupyter notebooks to folder visible to ansible
  • #16: Installed Jupyter notebooks via ansible
  • #67: Removed apt cache to reduce image size
  • #23: Fixed AWS Code build
  • #69: Added entry point to start Jupyter server
  • #84: Fixed retrieval and display of jupyter password
  • #36: Supported pushing Docker image to a Docker registry
  • #115: Added VOLUME entry to Docker Image
  • #78: Described persistent storage of notebook files and credential store
  • #76: Added display of usage instructions for AI-Lab Docker edition
  • #137: Set Jupyter lab default URL to AI-Lab start page
  • #75: Changed default port of Jupyter server to 49494
  • #79: Renamed data science sandbox to exasol-ai-lab
  • #150: Used multipart upload for VM images
  • #145: Added Docker Test Library to prepare Notebook tests
  • #151: Setup SageMaker Credentials for notebook testing in the CI
  • #155: Added a Notebook Link Checker to Github Actions
  • #157: Added link checker for the documentation
  • #136: Implemented notebook testing

Bug Fixes

  • #1: Fixed CI build
  • #61: Change initial password of Jupyter notebooks to "dss"

Refactoring

  • #5: Renamed all occurrences of "script language developer" by "data science"
  • #56: Moved jupyter notebook files again
  • #63: Improved logging of Ansible tasks
  • #46: Enabled to suppress ansible output
  • #52: Created CI build workflow to build and push Docker Image

Documentation

  • #58: Added location of Juypter notebooks to developer guide
  • #145: Split Development Guide

AMI Region availability

The AMI is currently only available in the AWS region eu-central-1. If you want to use the image in another region, you need to copy it before. Check the AWS documentation for details about how to copy the image.

Docker Images

docker pull exasol/ai-lab:0.1.0

AMI Images (Filter=0.1.0)

Image ID Name Description Public Image Location Creation Date State Asset-Tag-Value
0 ami-04042d10bc710115d Exasol-AI-Lab-0.1.0 Image Description yes 561494727831/Exasol-AI-Lab-0.1.0 2024-01-31T22:15:24.000Z available 0.1.0

VM Images, Prefix=ai_lab/0.1.0)

Key Size URL
0 ai_lab/0.1.0/exasol-ai-lab-0.1.0.vhd 5.68 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/0.1.0/exasol-ai-lab-0.1.0.vhd
1 ai_lab/0.1.0/exasol-ai-lab-0.1.0.vmdk 2.41 GB https://d3adquuoo89zuc.cloudfront.net/ai_lab/0.1.0/exasol-ai-lab-0.1.0.vmdk
Assets 3
Loading