Skip to content

Commit de4ecfc

Browse files
VolkerHartmannVolkerHartmann
authored
Merge pull request #2 from kit-data-manager/addDocker
Add docker and travis
2 parents d044fc9 + bd34e39 commit de4ecfc

File tree

15 files changed

+530
-21
lines changed

15 files changed

+530
-21
lines changed

‎.travis.yml

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
language: java
2+
3+
dist: bionic
4+
5+
jdk:
6+
# Enable all versions which should be supported
7+
- openjdk8
8+
- openjdk11
9+
# - openjdk13
10+
11+
12+
before_install:
13+
- docker pull elasticsearch:7.9.3
14+
- docker run -d --name elasticsearch4metastore -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.9.3
15+
- sudo apt-get install --assume-yes python3 python3-setuptools python3-pip
16+
- pip3 install --upgrade pip
17+
- pip3 install xmltodict wget
18+
19+
install: true
20+
21+
script:
22+
- "./gradlew -Ptravis clean check jacocoTestReport"
23+
24+
after_success:
25+
- "./gradlew coveralls"
26+
27+
before_cache:
28+
- rm -f $HOME/.gradle/caches/modules-2/modules-2.lock
29+
- rm -fr $HOME/.gradle/caches/*/plugin-resolution/
30+
31+
cache:
32+
directories:
33+
- $HOME/.gradle/caches/

‎CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Changelog
2+
All notable changes to this project will be documented in this file.
3+
4+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
5+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6+
7+
## [Unreleased]
8+
9+
## [0.0.1] - date 2020-12-15
10+
First version supporting registering of mappings (Gemma only)
11+
and mapping of metadata documents delivered by RabbitMQ
12+
### Added
13+
- Registration of mapping documents.
14+
- Mapping of metadata documents with Gemma
15+
- Ingest to elasticsearch
16+
17+
[Unreleased]: https://github.com/kit-data-manager/indexing-service/compare/v0.0.1...HEAD
18+
[0.1.0]: https://github.com/kit-data-manager/metastore2/indexing-service/tag/v0.0.1
19+

‎Dockerfile

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# START GLOBAL DECLARATION
33
####################################################
44
ARG REPO_NAME_DEFAULT=indexing-service
5-
ARG REPO_PORT_DEFAULT=8030
5+
ARG REPO_PORT_DEFAULT=8050
66
ARG SERVICE_ROOT_DIRECTORY_DEFAULT=/spring/
77
####################################################
88
# END GLOBAL DECLARATION
@@ -39,6 +39,8 @@ ENV SERVICE_DIRECTORY=$SERVICE_ROOT_DIRECTORY_DEFAULT$REPO_NAME
3939
RUN mkdir -p /git/${REPO_NAME}
4040
WORKDIR /git/${REPO_NAME}
4141
COPY . .
42+
RUN cp settings/application-docker.properties settings/application-default.properties
43+
4244
# Build service in given directory
4345
RUN bash ./build.sh $SERVICE_DIRECTORY
4446

‎README.md

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,11 @@
44
[![Coverage Status](https://coveralls.io/repos/github/kit-data-manager/indexing-service/badge.svg?branch=master)](https://coveralls.io/github/kit-data-manager/indexing-service?branch=master)
55
![License](https://img.shields.io/github/license/kit-data-manager/indexing-service.svg)
66

7-
A standalone service which receives messages about changes in pid records, maps them to a common format and ingests them into engines like elasticsearch.
7+
:::warning
8+
Not fully tested yet!
9+
For mapping documents only Gemma is available currently!
10+
:::
11+
A standalone service which receives messages about changes in metadata documents maps them to a common format and ingests them into elasticsearch.
812

913
![Visualization of use case structure.](use-case.drawio.svg)
1014

@@ -29,19 +33,39 @@ You might want to take a look at testbed4inf, which should make it easy to satis
2933
- a RabbitMQ instance
3034
- an elasticsearch instance
3135

32-
### Setup
36+
### Setup using Docker
3337
#### Install Gemma
3438
```
3539
sudo apt-get install --assume-yes python3 python3-pip
3640
pip3 install xmltodict wget
3741
```
3842

43+
#### Install and Start Network
44+
```
45+
docker network create network4datamanager
46+
```
47+
3948
#### Install and Start Elasticsearch
4049
```
4150
docker pull elasticsearch:7.9.3
4251
docker run -d --name elasticsearch4metastore -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.9.3
4352
```
4453

54+
#### Install and Start RabbitMQ
55+
```
56+
docker run -d --hostname rabbitmq --net network4datamanager --name rabbitmq4docker -p 5672:5672 -p 15672:15672 rabbitmq:3-management
57+
```
58+
#### Managing Services
59+
To start/stop a single service just type
60+
```
61+
docker start/stop name_of_container
62+
```
63+
e.g.:
64+
```
65+
docker stop elasticsearch4metastore
66+
```
67+
68+
4569
## More information
4670

4771
## License

‎build.gradle

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -79,8 +79,8 @@ dependencies {
7979
implementation("org.json:json:20200518")
8080

8181
// datamanager
82-
implementation("edu.kit.datamanager:generic-message-consumer:0.1.2")
83-
implementation "edu.kit.datamanager:service-base:0.1.2"
82+
implementation("edu.kit.datamanager:generic-message-consumer:0.2.0")
83+
implementation "edu.kit.datamanager:service-base:0.2.0"
8484

8585
runtimeOnly 'org.apache.httpcomponents:httpclient:4.5.6'
8686

‎build.sh

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
#!/bin/bash
2+
################################################################################
3+
# Build spring boot with starter script
4+
# Usage:
5+
# bash build.sh [/path/to/installation/dir]
6+
################################################################################
7+
8+
################################################################################
9+
# Define default values for variables
10+
################################################################################
11+
# no defaults yet!
12+
13+
################################################################################
14+
# START DECLARATION FUNCTIONS
15+
################################################################################
16+
17+
################################################################################
18+
function usage {
19+
################################################################################
20+
echo "Script for creating indexing-service."
21+
echo "USAGE:"
22+
echo " $0 [/path/to/installation/dir]"
23+
echo "IMPORTANT: Please enter an empty or new directory as installation directory."
24+
exit 1
25+
}
26+
27+
################################################################################
28+
function checkParameters {
29+
################################################################################
30+
# Check no of parameters.
31+
if [ "$#" -ne 1 ]; then
32+
echo "Illegal number of parameters!"
33+
usage
34+
fi
35+
36+
# Check if argument is given
37+
if [ -z "$1" ]; then
38+
echo "Please provide a directory where to install."
39+
usage
40+
exit 1
41+
fi
42+
43+
# Check for invalid flags
44+
if [ "${1:0:1}" = "-" ]; then
45+
usage
46+
fi
47+
48+
INSTALLATION_DIRECTORY=$1
49+
50+
# Check if directory exists
51+
if [ ! -d "$INSTALLATION_DIRECTORY" ]; then
52+
# Create directory if it doesn't exists.
53+
mkdir -p "$INSTALLATION_DIRECTORY"
54+
if [ $? -ne 0 ]; then
55+
echo "Error creating directory '$INSTALLATION_DIRECTORY'!"
56+
echo "Please make sure that you have the correct access permissions for the specified directory."
57+
exit 1
58+
fi
59+
fi
60+
# Check if directory is empty
61+
if [ ! -z "$(ls -A "$INSTALLATION_DIRECTORY")" ]; then
62+
echo "Directory '$INSTALLATION_DIRECTORY' is not empty!"
63+
echo "Please enter an empty or a new directory!"
64+
exit 1
65+
fi
66+
# Convert variable of installation directory to an absolute path
67+
cd "$INSTALLATION_DIRECTORY"
68+
INSTALLATION_DIRECTORY=`pwd`
69+
cd "$ACTUAL_DIR"
70+
}
71+
72+
################################################################################
73+
function printInfo {
74+
################################################################################
75+
echo "---------------------------------------------------------------------------"
76+
echo $*
77+
echo "---------------------------------------------------------------------------"
78+
}
79+
80+
################################################################################
81+
# END DECLARATION FUNCTIONS / START OF SCRIPT
82+
################################################################################
83+
84+
################################################################################
85+
# Test for commands used in this script
86+
################################################################################
87+
testForCommands="chmod cp dirname find java javac mkdir"
88+
89+
for command in $testForCommands
90+
do
91+
type $command >> /dev/null
92+
if [ $? -ne 0 ]; then
93+
echo "Error: command '$command' is not installed!"
94+
exit 1
95+
fi
96+
done
97+
98+
################################################################################
99+
# Determine directory of script.
100+
################################################################################
101+
ACTUAL_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
102+
103+
################################################################################
104+
# Check parameters
105+
################################################################################
106+
checkParameters $*
107+
108+
################################################################################
109+
# Determine repo name
110+
################################################################################
111+
REPO_NAME=`./gradlew -q printProjectName`
112+
# Use only last line
113+
REPO_NAME=${REPO_NAME##*$'\n'}
114+
115+
printInfo "Build microservice of $REPO_NAME at '$INSTALLATION_DIRECTORY'"
116+
117+
118+
################################################################################
119+
# Build service
120+
################################################################################
121+
122+
echo Build service...
123+
./gradlew -Prelease clean build
124+
125+
126+
echo "Copy configuration to '$INSTALLATION_DIRECTORY'..."
127+
find . -name application-default.properties -exec cp '{}' "$INSTALLATION_DIRECTORY"/application.properties \;
128+
129+
echo "Copy jar file to '$INSTALLATION_DIRECTORY'..."
130+
find . -name "$REPO_NAME*.jar" -exec cp '{}' "$INSTALLATION_DIRECTORY" \;
131+
132+
echo "Create config directory"
133+
mkdir "$INSTALLATION_DIRECTORY"/config
134+
135+
echo "Create lib directory"
136+
mkdir "$INSTALLATION_DIRECTORY"/lib
137+
138+
###############################################################################
139+
# Create run script
140+
################################################################################
141+
printInfo "Create run script ..."
142+
143+
cd "$INSTALLATION_DIRECTORY"
144+
145+
# Determine name of jar file.
146+
jarFile=(`ls $REPO_NAME*.jar`)
147+
148+
echo "#!/bin/bash" > run.sh
149+
echo "################################################################################" >> run.sh
150+
echo "# Run microservice '$REPO_NAME'" >> run.sh
151+
echo "# /" >> run.sh
152+
echo "# |- application.properties - Default configuration for microservice" >> run.sh
153+
echo "# |- '$REPO_NAME'*.jar" - Microservice >> run.sh
154+
echo "# |- run.sh - Start script " >> run.sh
155+
echo "# |- lib/ - Directory for plugins (if supported)" >> run.sh
156+
echo "# |- config/ " >> run.sh
157+
echo "# |- application.properties - Overwrites default configuration (optional)" >> run.sh
158+
echo "################################################################################" >> run.sh
159+
echo " " >> run.sh
160+
echo "################################################################################" >> run.sh
161+
echo "# Define jar file" >> run.sh
162+
echo "################################################################################" >> run.sh
163+
echo jarFile=$jarFile >> run.sh
164+
echo " " >> run.sh
165+
echo "################################################################################" >> run.sh
166+
echo "# Determine directory of script." >> run.sh
167+
echo "################################################################################" >> run.sh
168+
echo 'ACTUAL_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"' >> run.sh
169+
echo 'cd "$ACTUAL_DIR"' >> run.sh
170+
echo " " >> run.sh
171+
echo "################################################################################" >> run.sh
172+
echo "# Start micro service" >> run.sh
173+
echo "################################################################################" >> run.sh
174+
echo 'java -cp ".:$jarFile" -Dloader.path="file://$ACTUAL_DIR/$jarFile,./lib/,." -jar $jarFile' >> run.sh
175+
176+
# make script executable
177+
chmod 755 run.sh
178+
179+
echo .
180+
printInfo "Now you can start the service by calling '$INSTALLATION_DIRECTORY/run.sh'"

0 commit comments

Comments
 (0)