Author: George Papadimitriou
This github repository can be used to instantiate a local development environment for the Fabric Testbed. It creates a Jupyterhub environment similar to the one offered by Fabric, that runs in docker containers locally in your laptop or desktop. The benefit of this environment is that your data live locally and you can use github to version your code and track your changes.
Docker. This folder contains all the scripts and recipes needed to prepare the Docker image for the deployment.
Docker/build.sh. This is the script to be invoked to generate the Docker image. It passes the needed build arguments automatically, such as the fablib version to be used.
fabric_config. This folder contains all the needed configuration for successfully interacting with Fabric Testbed.
fabric_config/keys. This folder is used for your bastion and sliver ssh keys. The folder's content has been excluded from tracking via the .gitignore file.
fabric_config/tokens. This folder is used for your fabric token. Each project requires its own token. The folder's content has been excluded from tracking via the .gitignore file.
fabric_config/fabric_rc. This file contains Fabric variables that are used by fablib to prepopulate fields.
fabric_config/ssh_config. This file contains ssh configuration that is used from within the container to interact with fabric vm instances.
helpers. This folder contains some helper scripts to help keep your token alive for as long as possible.
projects. This folder is allocated for you jupyter notebooks and project specific code.
During the build process the fabric user will be created in the image with uid=your_current_uid and gid=your_current_primary_gid. This will make attaching volumes and accessing files from within and outside the container easy.
cd Docker
./build.sh <FABLIB_VERSION> (e.g., 1.5.0)
Place your Bastion and Sliver keys in the fabric_config/keys folder. The file names should follow the naming convention as seen bellow. To generate keys please use Fabric's portal. https://portal.fabric-testbed.net/experiments#sshKeys
fabric_bastion_key
fabric_bastion_key.pub
fabric_sliver_key
fabric_sliver_key.pub
Create a new file fabric_config/tokens/fabric.json and paste your Fabric token.
To generate a token follow the instructions on Fabric's documentation.
https://learn.fabric-testbed.net/knowledge-base/obtaining-and-using-fabric-api-tokens/
Note: Fabric tokens have an expiration date. The docker deployment will try to keep your token alive for as long as possible by polling the Fabric resources every 8 hours. However, this can fail and you will have to fetch a new token and replace the contents of the fabric_config/tokens/fabric.json file.
Replace the placeholders (lines 4,5) with your project id and bastion username.
line 4: export FABRIC_PROJECT_ID=<YOUR_PROJECT_ID>
line 5: export FABRIC_BASTION_USERNAME=<YOUR_BASTION_USERNAME>
Replace the placeholders (lines 6,14) with your bastion username.
line 6: User <YOUR_BASTION_USERNAME>
line 14: ProxyJump <YOUR_BASTION_USERNAME>@bastion-1.fabric-testbed.net:22
Step 6: Setup Access to Chameleon Resource (You can skip this step if you don't need Chameleon resources)
Connect to the Chameleon site you would like to acuire resource for. And follow the instructions on Chameleon's documentation to generate and download applications credentials. https://chameleoncloud.readthedocs.io/en/latest/technical/cli.html?highlight=credentials#creating-an-application-credential After you have acquired the RC file save it in ./chameleon_config/chameleon_rc (the name of the file is important).
To start the service you need to export the FABLIB_VERSION (e.g., 1.5.0) that you have used to build the docker container image, and then invoke the docker compose command. This can happen in a single line command as follows.
export FABLIB_VERSION=1.5.0 && docker compose up -d
Use any browser on you local machine and hit the following URL.
https://localhost:9098
The jupyterhub page will request for an access token. To retrieve this you can check the docker logs of the container running the serivce.
docker logs fabric-dev-env-${USER}
Or you can run the jupyter server list command inside the container.
docker exec fabric-dev-env-${USER} jupyter server list
You don't have to take down the service, but if you want you can use the following command.
export FABLIB_VERSION=1.5.0 && docker compose down