LA Development Guide

• Introduction
• Install basic dependencies
• Install sdkman
• Install maven
• Clone the ALA module you are interested in
• Install grails
• Basic build
• Install Intellij or similar
• Import the ALA module in Intellij
• Development configuration
• Development databases
• Patching ALA
• General ALA development/contribution recommendations
• More information

Introduction

This is an unofficial guide to setup and ALA development environment and start to contribute to ALA code base and adapt it to your node needs.

Disclaimer: Software forks and custom code changes are sometimes quite easy to do, but quite hard to maintain, even for Google. So we recommend to try to generalize your software customizations and send Pull Requests to ALA repositories, so your code can be useful to other LA nodes (and also maintained by all of us).

Install basic dependencies

sudo apt install curl git openjdk-8-jdk
sudo update-java-alternatives -s java-1.8.0-openjdk-amd64
(you may need to do sudo apt-get install icedtea-8-plugin)

Install sdkman

Install sdkman:

curl -s "https://get.sdkman.io" | bash
source "$HOME/.sdkman/bin/sdkman-init.sh"
sdk version

Install maven

Some ALA modules are build with maven (like biocache-store):

sudo apt install maven

Clone the ALA module you are interested in

In this guide we'll use ala-bie as sample:

git clone https://github.com/AtlasOfLivingAustralia/ala-bie.git
cd ala-bie

Install grails

Install the grails version that uses the ALA module you are interested in:

grep grailsVersion gradle.properties
# Doing this in our cloned module ala-bie shows that nowadays grailsVersion=3.2.11 so:

sdk install grails 3.2.11

Basic build

Let's do a basic build to download dependencies and test the repository building before modify any code:

# in ala-bie
./gradlew build
Downloading https://services.gradle.org/distributions/gradle-3.4.1-all.zip
(...)
Finished Precompiling Assets
:compileWebappGroovyPages NO-SOURCE
:compileGroovyPages
:bootRepackage
:assemble
:processTestResources NO-SOURCE
:testClasses UP-TO-DATE
:test NO-SOURCE
:check UP-TO-DATE
:build

BUILD SUCCESSFUL

Total time: 10 mins 0.015 secs

The war should be in:

ls build/libs/
ala-bie-1.4.20.war  ala-bie-1.4.20.war.original

Install Intellij or similar

Following some tutorial, install an IDE like intellij:

sudo snap install intellij-idea-community --classic

Run Intellij from Ubuntu "Activities" and choose default options, like suggested plugins.

Import the ALA module in Intellij

In our example, as ala-bie is a grails project, check the Intellij documentation about Creating Grails Application from Existing Code.

Select "Import project" in main Intellij dialog:

In your project structure select build.grandle file:

Intellij will index, compile and build our code:

Now you can run your project:

So you can see in your console something like:

2020-01-16 13:05:41.745  INFO --- [           main] au.org.ala.bie.Application               : Started Application in 8.767 seconds (JVM running for 10.183)
Grails application running at http://localhost:8080 in environment: development

and some browser tab will open pointing to your running instance: http://localhost:8080 .

In many cases this will fail because we need some LA module configuration (similar to the one created by ansible in /data/*/config/ and also access to some database (depending on the module).

Some LA modules don't require db connections (like regions, or biocache-hub) so we can point our development configuration to some production biocache-service, spatial, etc to do our development tests.

Development configuration

As we mentioned previously your LA module will look to /data for configuration properties like in production. So we need to generate some configuration.

A fast way to generate many of these configurations is to use ansible, locally with the help of the LA ansible generator:

https://github.com/living-atlases/generator-living-atlas/wiki/Generating-updated-LA-configuration-properties-files

Development databases

You need some local database (mysql, postgresql, mongo, etc) to run your LA module correctly in local, but exists other options like to have some VM, vagrant or docker environment. Also you can tunnel you db connections via ssh.

Some ALA modules are using docker to get a development database easily, for instance:

• https://github.com/AtlasOfLivingAustralia/spatial-service#running-it-locally
• https://github.com/AtlasOfLivingAustralia/image-service#running-it-locally

Other option if you have a test LA environment you can use their database. For this you can tune your mysql connections (for instance) via ssh:

ssh -L 33060:127.0.0.1:3306 ubuntu@your_la_demo -N -f

and setup that port in your development configuration:

dataSource.url=jdbc\:mysql\://localhost:33060/specieslists?autoReconnect=true&connectTimeout=0&useUnicode=true&characterEncoding=UTF-8

Don't forget to match your mysql user & password with those configured in your LA test environment.

You can use this also to tunnel solr and cassandra connections.

Patching ALA

Following our example, imagine that we want to patch the "Online Resources" section in the main page of each specie in bie service:

this is generated by: https://github.com/AtlasOfLivingAustralia/ala-bie/blob/master/grails-app/views/species/_onlineResources.gsp and nowadays has this ALA url hardcoded:

<li><a href="https://biocache.ala.org.au/occurrences/search?taxa=${tc?.taxonConcept?.nameString?.replace(" ","%20")}">ALA occurrences</a></li>

You can simply edit this URL in your ala-bie fork, put your biocache url instead:

<li><a href="https://biocache.la.site/occurrences/search?taxa=${tc?.taxonConcept?.nameString?.replace(" ","%20")}">Wakanda occurrences</a></li>

generate a new war, and deploy your custom war to your bie server.

This is the faster way to do it (in short term), but has several inconveniences:

• You have to maintain this little change over time, that is, the next time you want to update the bie service, you should update your fork and merge these kind of small changes with new ALA upstream code.
• Other LA portals need to do the same effort. Identify where the code is, modify it, generate a new war, etcetera. A waste of time if some other portal had solved this already.
• Also, it's easy to lose this change over time or to discorauge you to update your ALA software because of these customizations.

No problem should ever have to be solved twice. Eric S. Raymond, 2001

The prefered way to do this is to create a general patch (not specific to any national node) and send a Pull Request to the ala-bie ALA repository, so others nodes can get this fix in the future.

This is what @angelicalleite did in this PR. Thanks indeed!

General ALA development/contribution recommendations

When doing pull requests (PRs):

• try to follow the current code indentation and style. Avoid unnecessary refactorizations.
• try send a unique PR per functionality, fix, or enhancement so it's more easy to review and merge. Don't send big PR with different changes (FIXME explain this better).
• related with i18n, try only to add additions and changes to English typically at grails/i18n/messages.properties. Other translations are done via crowdin and later synchronized automatically.
• try to do PR to develop branches (if exists) of each module.
• Keep your local forks updated with the ALA repo by setting your git upstream value.
• Before submitting a PR make sure you rebase your local branch.

More information

See Development-Resources for more LA development wiki pages and more related information.