added mechanisms for mapping secondary data folder and individual secrets files

Author: mfhepp

.dockerignore

@@ -1,5 +1,6 @@
 .git
 .github
+.secrets
 docs
 test
 LICENSE
@@ -11,4 +12,4 @@ default-stack.md
 versions.md
 output/
 src/output/
-src/data/
+src/data/

.gitignore

@@ -1,3 +1,8 @@
+# Project
+.secrets
+archive
+todo
+
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.py[cod]

README.md

@@ -310,6 +310,9 @@ It is **strongly recommended to use an absolute path in the alias** (otherwise,
 
 You can build isolated containers with Juypter Notebook and JupyterLab.
 
+**Note:** This functionality is likely to become a separate project, see [Issue 15](https://github.com/mfhepp/py4docker/issues/15)
+
+
 ### Building a Notebook Image
 
 #### Using the default environment file `notebook.yaml`
@@ -390,6 +393,45 @@ nbh openai
 nbh foo
 ```
 
+#### Mapping a secondary data directory to `/mnt/data`
+
+You can map any other directory from your system as read-only bind volume to `/mnt/data` inside the Docker container like so:
+
+```bash
+# /home/foo/bar will be accessible as /mnt/data inside the container:
+./run_notebook.sh --data-dir /home/foo/bar
+```
+
+#### Mapping API tokens and other secrets from local files to `/mnt/secrets/`
+
+You can map one or more local files containing access tokens as a read-only bind mounts to `/mnt/secrets/` inside the Docker container like so:
+
+```bash
+./run_notebook.sh --add-secret ~/Documents/.access_tokens/TESTTOKEN1 FOO \
+                  --add-secret ~/Documents/.access_tokens/TESTTOKEN2 BAR
+```
+You will then be able to access them inside the notebook like so:
+
+```bash
+# Inside a notebook cell, run Bash commands with a ! directive;
+!cat /mnt/secrets/FOO
+!cat /mnt/secrets/BAR
+# Contents of the two files TESTTOKEN1 and TESTTOKEN2
+SUPERSECRET_TOKEN1
+API_TOKEN_FOR_ACME
+```
+
+A Python example is in [examples/secrets_test.ipynb](examples/secrets_test.ipynb).
+
+**Warnings:** 
+
+1. This is a ***simplistic*** substitute for Docker's mechanisms for managing secrets, but IMO more secure than using environment variables that may be leaked in logfiles etc. **Keep in mind that in the current version, ALL files inside that directory will be available from inside the container!**
+2. Make sure **that you DO NOT LEAK YOUR SECRETS TO YOUR Git repository**.
+3. Make sure **THAT YOUR SECRETS folder is NOT below your current working directory.** Otherwise, it will be accessible for read- and write-access from within  `/usr/app/data` (e.g. as  `/usr/app/data/.access_tokens/`)!!!
+4. On OSX, do not use `~/.access_tokens`, but rather `~/Documents/.access_tokens`, `~/Documents/.access_tokens`, or any place in the predefined subfolders below the user directory, because
+    1. OSX grants ANY user on your machine read-access to any user's home folder.
+    2. The OSX permissions model for applications will ask you only if an application tries to access one of the specific folders ***below the user directory***. I.e., any application COULD READ from `/Users/yourusername/.access_tokens`!!!
+
 ## Advanced Topics
 
 ### Access to the Local File System

examples/secrets_test.ipynb

@@ -0,0 +1,146 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "3c697b0f-4fa4-4dd0-a86b-90b5ed0fbaaa",
+   "metadata": {},
+   "source": [
+    "# Example: Accessing API tokens and credentials\n",
+    "\n",
+    "## Starting Jupyter\n",
+    "\n",
+    "Start Jupyter with\n",
+    "\n",
+    "```bash\n",
+    "./run_notebook.sh --add-secret ~/.my_access_tokens/TESTTOKEN1 FOO --add-secret ~/.my_access_tokens/TESTTOKEN2 BAR\n",
+    "```\n",
+    "\n",
+    "The two **files** \n",
+    "\n",
+    "`~/.my_access_tokens/TESTTOKEN1`\n",
+    "\n",
+    "and\n",
+    "\n",
+    "`~/.my_access_tokens/TESTTOKEN2`\n",
+    "\n",
+    "must exist on the host systems!\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "91d1053e-31d8-43a0-9c53-819f9f15f7cc",
+   "metadata": {},
+   "source": [
+    "## Accessing the contents of the secrets from a notebook"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "11af385e-37bf-4d72-adce-2e0e9caa8489",
+   "metadata": {},
+   "source": [
+    "### With shell commands"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "92b59d9d-6bc2-40ca-afb7-abb7903f7dc8",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "SUPERSECRET_TOKEN1\n",
+      "API_TOKEN_FOR_ACME\n"
+     ]
+    }
+   ],
+   "source": [
+    "!cat /mnt/secrets/FOO\n",
+    "!cat /mnt/secrets/BAR"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "67300fec-69df-42af-b962-5eab6799cd0b",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "total 16\n",
+      "drwxr-xr-x 2 root root 4096 May 10 11:43 .\n",
+      "drwxr-xr-x 1 root root 4096 May 10 11:43 ..\n",
+      "-rw-r--r-- 1 root root   19 May 10 11:32 BAR\n",
+      "-rw-r--r-- 1 root root   19 May 10 11:32 FOO\n"
+     ]
+    }
+   ],
+   "source": [
+    "!ls -la /mnt/secrets/"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "29b93823-53ea-4dc8-98bc-077440b8b54a",
+   "metadata": {},
+   "source": [
+    "### Python"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "cde13d39-2d0a-4cba-992d-2e9e06511d4d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "SUPERSECRET_TOKEN1\n",
+      "API_TOKEN_FOR_ACME\n"
+     ]
+    }
+   ],
+   "source": [
+    "SECRET_1_PATH = \"/mnt/secrets/FOO\"\n",
+    "SECRET_2_PATH = \"/mnt/secrets/BAR\"\n",
+    "\n",
+    "with open(SECRET_1_PATH, \"r\") as file:\n",
+    "    SECRET_TOKEN = file.read().strip()\n",
+    "\n",
+    "with open(SECRET_2_PATH, \"r\") as file:\n",
+    "    API_KEY_ACME = file.read().strip()\n",
+    "\n",
+    "print(SECRET_TOKEN)\n",
+    "print(API_KEY_ACME)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

notebook.yaml.lock

@@ -45,12 +45,12 @@ dependencies:
 - importlib_metadata=7.1.0=hd8ed1ab_0
 - importlib_resources=6.4.0=pyhd8ed1ab_0
 - ipykernel=6.29.3=pyhd33586a_0
-- ipython=8.22.2=pyh707e725_0
-- ipywidgets=8.1.2=pyhd8ed1ab_0
+- ipython=8.24.0=pyh707e725_0
+- ipywidgets=8.1.2=pyhd8ed1ab_1
 - isodate=0.6.1=pyhd8ed1ab_0
 - isoduration=20.11.0=pyhd8ed1ab_0
 - jedi=0.19.1=pyhd8ed1ab_0
-- jinja2=3.1.3=pyhd8ed1ab_0
+- jinja2=3.1.4=pyhd8ed1ab_0
 - json5=0.9.25=pyhd8ed1ab_0
 - jsonpointer=2.4=py312h996f985_3
 - jsonschema=4.22.0=pyhd8ed1ab_0
@@ -68,7 +68,7 @@ dependencies:
 - jupyterlab_pygments=0.3.0=pyhd8ed1ab_1
 - jupyterlab_server=2.27.1=pyhd8ed1ab_0
 - jupyterlab_widgets=3.0.10=pyhd8ed1ab_0
-- jupytext=1.16.1=pyhd8ed1ab_0
+- jupytext=1.16.2=pyhd8ed1ab_1
 - keyutils=1.6.1=h4e544f5_0
 - krb5=1.21.2=hc419048_0
 - ld_impl_linux-aarch64=2.40=hba4e955_0
@@ -77,17 +77,17 @@ dependencies:
 - libedit=3.1.20191231=he28a2e2_2
 - libexpat=2.6.2=h2f0025b_0
 - libffi=3.4.2=h3557bc0_5
-- libgcc-ng=13.2.0=he277a41_6
-- libgfortran-ng=13.2.0=he9431aa_6
-- libgfortran5=13.2.0=h87d9d71_6
-- libgomp=13.2.0=he277a41_6
+- libgcc-ng=13.2.0=he277a41_7
+- libgfortran-ng=13.2.0=he9431aa_7
+- libgfortran5=13.2.0=h87d9d71_7
+- libgomp=13.2.0=he277a41_7
 - libiconv=1.17=h31becfc_2
 - liblapack=3.9.0=22_linuxaarch64_openblas
 - libnsl=2.0.1=h31becfc_0
 - libopenblas=0.3.27=pthreads_h5a5ec62_0
 - libsodium=1.0.18=hb9de7d4_1
 - libsqlite=3.45.3=h194ca79_0
-- libstdcxx-ng=13.2.0=h3f4de04_6
+- libstdcxx-ng=13.2.0=h3f4de04_7
 - libuuid=2.38.1=hb4cce97_0
 - libxcrypt=4.4.36=h31becfc_1
 - libxml2=2.12.6=h3091e33_2
@@ -106,7 +106,7 @@ dependencies:
 - nbconvert-core=7.16.4=pyhd8ed1ab_0
 - nbconvert-pandoc=7.16.4=hd8ed1ab_0
 - nbformat=5.10.4=pyhd8ed1ab_0
-- ncurses=6.4.20240210=h0425590_0
+- ncurses=6.5=h0425590_0
 - nest-asyncio=1.6.0=pyhd8ed1ab_0
 - notebook=7.1.3=pyhd8ed1ab_0
 - notebook-shim=0.2.4=pyhd8ed1ab_0
@@ -134,7 +134,7 @@ dependencies:
 - ptyprocess=0.7.0=pyhd3deb0d_0
 - pure_eval=0.2.2=pyhd8ed1ab_0
 - pycparser=2.22=pyhd8ed1ab_0
-- pygments=2.17.2=pyhd8ed1ab_0
+- pygments=2.18.0=pyhd8ed1ab_0
 - pyparsing=3.1.2=pyhd8ed1ab_0
 - pyshacl=0.25.0=pyhd8ed1ab_0
 - pysocks=1.7.1=pyha2e5f31_6
@@ -146,8 +146,8 @@ dependencies:
 - python_abi=3.12=4_cp312
 - pytz=2024.1=pyhd8ed1ab_0
 - pyyaml=6.0.1=py312hdd3e373_1
-- pyzmq=26.0.2=py312h7059f03_0
-- qtconsole-base=5.5.1=pyha770c72_0
+- pyzmq=26.0.3=py312h7059f03_0
+- qtconsole-base=5.5.2=pyha770c72_0
 - qtpy=2.4.1=pyhd8ed1ab_0
 - rdflib=7.0.0=pyhd8ed1ab_0
 - readline=8.2=h8fc344f_1
@@ -156,7 +156,7 @@ dependencies:
 - rfc3339-validator=0.1.4=pyhd8ed1ab_0
 - rfc3986-validator=0.1.1=pyh9f0ad1d_0
 - rich=13.7.1=pyhd8ed1ab_0
-- rpds-py=0.18.0=py312h2a5c9d6_0
+- rpds-py=0.18.1=py312heb99873_0
 - send2trash=1.8.3=pyh0d859eb_0
 - setuptools=69.5.1=pyhd8ed1ab_0
 - shellingham=1.5.4=pyhd8ed1ab_0
@@ -168,7 +168,6 @@ dependencies:
 - terminado=0.18.1=pyh0d859eb_0
 - tinycss2=1.3.0=pyhd8ed1ab_0
 - tk=8.6.13=h194ca79_0
-- toml=0.10.2=pyhd8ed1ab_0
 - tomli=2.0.1=pyhd8ed1ab_0
 - tornado=6.4=py312h9ef2f89_0
 - traitlets=5.14.3=pyhd8ed1ab_0