Skip to content
This repository was archived by the owner on Apr 26, 2023. It is now read-only.

Scipion Cloud for training

Jump to bottom
Laura edited this page Nov 17, 2017 · 30 revisions

AWS for Scipion training (only for the i2pc group)

Creating AWS instances for training

Only AWS admin user can do that, but these are the steps.

  • Login to AWS console
  • Go to EC2 services
  • Select AMIs on the left side menu
  • Select i2pc-training-2 and click on Launch

AWS console - AMIs

  • Choose the instance type appropriate for the training. For the usual Workflow (ExtendedBetagal) we normally use p2.xlarge (GPU).
  • Click ‘Next: Configure instance details’ and select the number of instances to be created.
  • Click ‘Next: Add Storage’ and specify the disk size (30 GB by default).
  • Click ‘Next: Add tags’ and specify the following tags:

AWS console - Add tags

(UserName must be i2pc-training but Name can be chosen, although it must contain the word training).

  • Click ‘Next: Configure Security group’ and select the one named ‘https+ssh’.

AWS console - Security Groups

It is advised to modify this setting to set up an IP range when possible.

  • Click on ‘Review and launch’ and press ‘Launch’. A pop-up window will appear to select a security key pair.

AWS console - Key pairs

  • Click on ‘Launch Instances’ and go to ‘View Instances’ and wait until instances has been initialized (Status Checks)

Instances are by default created with a dynamic IP address. This means that when the instance is restarted the IP changes. AWS allows to create ‘Elastic IPs’ that are fix IPs that can be assigned to an instance. This way even if the instance is restarted the IP is kept, which which might interesting for a multi-day course. The steps to do this are:

  • Go to the Left side menu ‘Network and Security’/Elastic IPs
  • Click on ‘Allocate new address’
  • Once the new IP address appears on the list select it and click on ‘Actions/Associate address’. By default AWS have a limit of 5 Elastic IPs but this limit can be changed through the AWS support service.

Managing training instances

There is a user called i2pc-training that can only manage training instances (start, stop and reboot). The steps are:

  • Login to AWS console
  • Go to EC2 services
  • Select Instances on the left side menu
  • Select the instance and either click on ‘Actions’ or Right-click on the instance and select Instance State and the action required.

AWS console - manage training instances

Accessing training instances

Once instances are up and running they can be accessed either through ssh or guacamole.

  • Ssh: user ubuntu with pubkey i2pc-training or user scipion with password
  • Guacamole with user scipion and password on the following URL https://PUB_IP/scipion/

The first time you will be warned that connection is not secure and will have to add a security exception). Also, the first time you might get an error saying ‘connection failed’. Simply click ‘Ok’, ‘Cancel’, ‘Logout’ and try again (it has to do something with initializing a machine from an AWS AMI or snapshot and I do not know how to solve it).

This is how Desktop on the new machine will look like:

Guacamole desktop

There is a shortcut for Scipion and other for a Terminal.

The following software is installed on the machine:

  • Scipion on /usr/local/scipion (git installation branch master but moving to other branches is possible)
  • EM Packages:
    • Bsoft 1.9.0
    • Ctffind4 4.1.5
    • Eman 2.12
    • Frealing 9.07
    • Motioncor2 16.10.19
    • Motioncorr 2.1
    • Relion 1.4 and 2.0
    • resmap 1.1.5s2
    • Spider 21.13
    • Summovie 1.0.2

Chimera is not installed because it hangs out the guacamole connection.

  • Nvidia driver version 375
  • CUDA 7.5 and 8.0 under /usr/local/ (default cuda links to 7.5).
  • Guacamole 0.9.9 is installed on /services/guacamole.

The ExtendedBetagal finished project can be found under Scipion projects while project raw data is located at /data/relion13_tutorial.

To logout from guacamole session it is advised to click on ‘Ctrl-Shift-Alt’ select ‘Logout’ or ‘Disconnect’ from scipion submenu as shown below.

Guacamole desktop - disconnect

Prices on AWS for Scipion training

On AWS you pay for time you consume resources. In this case we have to pay for the time that instances are up running plus the disk space they occupy even when they are stopped. Also, we should pay for downloading data. Typical GPU instance type used for the course is p2.xlarge and it costs 1.09$/hour in AWS Ireland region. EBS volume costs 0.133$/GB/month. Downloads costs 0.11$/GB.

So, for instance, for a typical two days course in Europe (AWS Ireland region) one p2.xlarge instance with a 30 GB disk assuming that machine is up for 16 hours but it exists during 48 hours will have the following cost:

  • Instance: 16*1.09$ = 17.44$
  • EBS volume: 300.13348/720 = 0.266$
  • The cost of downloading final results (maps) which are in order of MB are irrelevant.

Total cost per machine for a 2 days course is around 18$ (tax included).

If Elastic IPs were specified they might imply extra costs. Elastic IPs are free while they are allocated to a running instance, but if the instance is stopped or terminated the IP costs 0.006$/hour, which for a 2 days course might be considered irrelevant.

EGI Federated Cloud for training

We could also use the FedCloud for training with the following considerations:

  • We normally operate through Virtual Organization enmr.eu (Westlife project) but we could also use VO training.egi.eu
  • The agreement to use the latest for GPU instances (IISAS) is:
    • 2 VMs with 6 CPU cores, 1 GPU and 12GB of RAM (in normal condition)
    • 4 VMs with 6 CPU cores, 1 GPU and 12GB of RAM (require reservation)
  • Non GPU instances are not that restricted using different providers (CESNET, IISAS)
  • They have a FedCloud Client UI (Ubuntu 16.04.02) already configured with 40 different training accounts. Every 6h, a cron job updates the proxy for each user with the PUSP generated from the robot. They can create an account for us on that UI. This is useful if we want students to start their own VMs but this is not the normal case.

FedCloud instances for training

Currently there is not a proper image for Scipion training in the FedCloud, however there are two instances running that could be cloned if needed. These machine can be accessed at:

https://scipiongpu-training.westlife.fedcloud.eu/vnc.html

This machine is not accessed through guacamole but using noVNC. It only requires password.

https://scipion-training.westlife.fedcloud.eu/scipion

Guacamole user is scipion and password ask me :).

Non-GPU machine contains exactly the same software as the AWS training image.

GPU machine uses release1.1-facilities-branch that contains the streaming demo. Both cuda 8.0 and cuda 7.5 are installed. Chimera is also installed and can be used via TurboVNC/VirtualGL.

The GPU machine is a gpu1cpu6 type on IISAS GPU-Cloud site (https://accelerated.ui.sav.sk/?page_id=355) with 1 GPU Nvidia Tesla K20 (4 GB RAM), 6 CPUs, 24 GB RAM and 80 GB disk. ExtendedBetagal workflow runs ok except Optical Flow that is not compiled with GPU support.

The non-GPU machine is a sym_mammoth type on CESNET MetaCloud site () with 32 CPUs, 32 GB RAM and 80 GB disk.

Clone this wiki locally