Skip to content

Home

Jump to bottom
apryamostanov edited this page Mar 19, 2019 · 50 revisions

Infinite Technology ∞ Pigeon 🕊

Post pigeons have been (and sometimes still are) extensively used for quick delivery of paper-based messages (such as text, drawings, maps) - thus the project name: Pigeon.

Purpose

Pigeon is an end-user server application (HTTP Message Broker) designed for distribution of text messages in HTTP format.

In short

Pigeon allows to:

  • Accept a text message in a free format (it can be anything - XML, JSON, TLV, Base64, etc) from external source using "enqueue" REST API or direct DB insert by external app
  • Convert it into one or more HTTP messages with a specified body/query string parameters using one of the Plugins (written in Groovy Script)
    • HTTP message attributes such as Headers, Body, URL are controlled by the Plugin and there are no constraints on the produced output (as long as it within allowed HTTP format boundaries)
  • Send the resulting HTTP messages ansynchronously to one or more recipients (URLs) using a variety of HTTP connection and authentication mechanisms (such as AWS v4 signature)
  • If needed retry sending the message several times

Features

Extensibility

Pigeon supports programmability extension in a form of user-defined Plugins written using Groovy script.

Plugins

The below types of plugins are supported.

Output plugins

Output plugins are used to convert a free-form enqueued text message into an HTTP message.

Output plugins have full control on the resulting HTTP body, headers and URL.

Example output plugin: SMSGlobal plugin

Installation

Download jar

  1. Open latest version of pigeon-app on Bintray:

Download

  1. On the same page scroll down to the Downloads section
  2. Click on pigeon-app-<version>.jar to start the download
  3. Rename the file to pigeon.jar

Service installation

❗ This step is optional and is recommended for Production and unattended test environments.
You can directly run Pigeon using Command-line.

Windows

Install Pigeon as Windows service following instructions (using instructions: https://dzone.com/articles/spring-boot-as-a-windows-service-in-5-minutes)

Sample pigeon.xml

❗ Recommended Java Heap size parameters: at least -Xms1000m -Xmx1500m

<?xml version="1.0" encoding="UTF-8"?>
<service>
    <id>Pigeon</id>
    <name>Pigeon</name>
    <description>Pigeon Windows Service</description>
    <executable>"C:\Program Files\Java\jdk1.8.0_181\bin\java"</executable>
    <arguments>-Xms1000m -Xmx1500m -cp "pigeon.jar" -Dloader.path=lib org.springframework.boot.loader.PropertiesLauncher</arguments>	
    <logmode>rotate</logmode>
</service>

Other operating systems

Follow operating system documentation to install Pigeon jar as a service.

❗ It is needed to set loader.path to lib. Refer to Windows example. Recommended Java Heap size parameters: at least -Xms1000m -Xmx1500m

Database connector

❇ Optional Step (recommended)

By default Pigeon comes with H2 in-memory database.
While suitable for trying and certain testing cases, it is highly recommended (with H2 DB data is lost when Pigeon is restarted) to use stand-alone database.
Pigeon is using JPA which allows using a variety of JDBC connectors.

  1. Download JAR JDBC connector supporting your DBMS (e.g. mysql-connector-java-8.0.13.jar)

  2. Deploy it into same {Pigeon Home}/lib directory (create it first).

Note: refer to documentation of your DBMS/JDBC driver for more details.

Configuration

application.properties

❇ Optional Step (recommended)

❗ Refer to Spring and Spring Boot documentation for details on supported configuration in application.properties.

In the Pigeon working directory, create file application.properties.

Define the below mandatory properties (refer to sample application.properties file):

Pigeon configuration paths

pigeonConfFile=./conf/Pigeon.json
pigeonOutPluginsDir=./conf/plugins/output/
pigeonInputPluginsRestDir=./conf/plugins/input/rest/
pigeonInputPluginsHttpDir=./conf/plugins/input/http/

Database connection credentials

The below parameters are required if you have installed JDBC connector JAR.

If you are using default in-memory H2 DB (not recommended for Production/Test) - these parameters are optional.

❗ Refer to your JDBC driver documentation for details on how to configure the connections credentials.
Below example is demonstrating the case for MySQL connector only.

spring.datasource.url=jdbc:mysql://ip:port/db
spring.datasource.username=username
spring.datasource.password=password

Web Application settings

Define how Pigeon will expose its Web services:

spring.data.rest.base-path=/pigeon
server.contextPath=/pigeon
server.port = 8089

JPA Hibernate DDL mode

❗ Recommended value is update.
Data loss can happen when other values are used.

spring.jpa.hibernate.ddl-auto=update

Other settings

spring.banner.image.width=30
spring.banner.image.height=20
spring.jackson.serialization.indent_output=true

Pigeon.json

❗ Mandatory Step

Create a file Pigeon.json in the previously configured path.

Refer to Pigeon.json Wiki article for details on the file structure and configuration.

You can use the sample Pigeon.json as a start.

Bobbin.json

❇ Optional Step

Pigeon comes with a pre-defined well-structured logging configuration.
Therefore this step is required only if you need to tweak the logging configuration.
Refer to the Default Bobbin.json used by Pigeon.

Pigeon is using Bobbin - a revolutionary Java/Groovy Slf4j Logger.

For details on how to configure Bobbin.json refer to Bobbin Documentation

Bobbin.json should be placed in the Pigeon Working directory.

Carburetor.json

❇ Optional Step

Pigeon allows usage of @BlackBox annotation in the Pigeon plugins user code.

With the BlackBox the users don't have to worry about writing logging code in the Pigeon plugins - it is automatically generated.

All you need to do is to place @BlackBox annotation on the needed methods.

For more details refer to BlackBox Documentation.

The granularity of logging can be controlled using externalized configuration: Carburetor.json.

Place the Carburetor.json in the Pigeon working directory and define its contents:

{
    "defaultLevel": "EXPRESSION"
}

You can set defaultLevel to one of 5 values:

  1. NONE
  2. ERROR
  3. METHOD
  4. STATEMENT
  5. EXPRESSION

For more details refer to:

Running

Running from command-line

Refer to documentation of command-line of your operating system for exact syntax.

❗ It is needed to set loader.path to lib. Refer to Windows example. Recommended Java Heap size parameters: at least -Xms1000m -Xmx1500m

Example for Windows:

"C:\Program Files\Java\jdk1.8.0_181\bin\java" -Xms1000m  -Xmx1500m -cp "pigeon.jar" -Dloader.path=lib org.springframework.boot.loader.PropertiesLauncher

Running as a service

After Pigeon is installed as a service, you can start Pigeon service as per Operating System documentation.

Clone this wiki locally