Skip to content

Commit c6c15b1

Browse files
eduard93eduard93
authored
Article
1 parent 480b548 commit c6c15b1

File tree

1 file changed

+212
-1
lines changed

1 file changed

+212
-1
lines changed

‎README.md

Lines changed: 212 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,213 @@
11
# zlibisc
2-
Various examples of calling zlib
2+
Leveraging code from other languages: zlib
3+
4+
# Intro
5+
6+
Recently I reread [this article](https://community.intersystems.com/post/story-support-how-quest-raw-deflate-compressiondecompression-function-leads-node-callout-server) by @Bernd.Mueller. It's about calling DELFATE function from [zlib library](https://zlib.net/). In this article I'll demonstrate several different approaches to callout libraries, we'll build the same functionality (compress function) in several different languages and compare them.
7+
8+
# NodeJS
9+
10+
Let's start with NodeJS. I'm taking the code almost directly from Bernd's article, except it does not use files, but rather direct http connection to pass data. For production use, it would be better to pass request as a body and encode both request and response as base64. Still, here's the [code](https://github.com/intersystems-ru/zlibisc/blob/master/node/zlibserver.js):
11+
12+
```
13+
//zlibserver.js
14+
const express = require('express');
15+
const zlib = require('zlib');
16+
17+
var app = express();
18+
19+
app.get('/zlibapi/:text', function(req, res) {
20+
res.type('application/json');
21+
22+
var text=req.params.text;
23+
24+
try {
25+
zlib.deflate(text, (err, buffer) => {
26+
if (!err) {
27+
res.status(200).send(buffer.toString('binary'));
28+
} else {
29+
res.status(500).json( { "error" : err.message});
30+
// handle error
31+
}
32+
});
33+
}
34+
catch(err) {
35+
res.status(500).json({ "error" : err.message});
36+
return;
37+
}
38+
39+
});
40+
app.listen(3000, function(){
41+
console.log("zlibserver started");
42+
});
43+
```
44+
45+
To start it execute in OS bash (assuming `node` and `npm` installed):
46+
```
47+
cd <repo>\node
48+
npm install
49+
node ./zlibserver.js
50+
```
51+
52+
We're running on port 3000, reading input string from request and returning compressed data in response as is. On a Caché side [http request](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GNET_http) is used to interact with this api:
53+
54+
```
55+
/// NodeJS implementation
56+
/// do ##class(isc.zlib.Test).node()
57+
ClassMethod node(text As %String = "Hello World", Output response As %String) As %Status
58+
{
59+
kill response
60+
set req = ##class(%Net.HttpRequest).%New()
61+
set req.Server = "localhost"
62+
set req.Port = 3000
63+
set req.Location = "/zlibapi/" _ text
64+
set sc = req.Get(,,$$$NO)
65+
quit:$$$ISERR(sc) sc
66+
set response = req.HttpResponse.Data.Read($$$MaxStringLength)
67+
quit sc
68+
}
69+
```
70+
71+
Note, that I'm setting the third argument `set sc = req.Get(,,$$$NO)` - `reset` to zero. If you're writing interface to the outside http(s) server it's best to reuse one request object and just modify it as needed to perform new requests.
72+
73+
# Java
74+
75+
[Java Gateway](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=EJVG) allows calling arbitrary Java code. Coincidently Java has Deflate class which does exactly what we need:
76+
77+
```
78+
package isc.zlib;
79+
80+
import java.util.Arrays;
81+
import java.util.zip.Deflater;
82+
83+
public abstract class Java {
84+
85+
public static byte[] compress(String inputString) {
86+
byte[] output = new byte[inputString.length()*3];
87+
try {
88+
// Encode a String into bytes
89+
byte[] input = inputString.getBytes("UTF-8");
90+
91+
// Compress the bytes
92+
93+
Deflater compresser = new Deflater();
94+
compresser.setInput(input);
95+
compresser.finish();
96+
int compressedDataLength = compresser.deflate(output);
97+
compresser.end();
98+
output = Arrays.copyOfRange(output, 0, compressedDataLength);
99+
100+
} catch (java.io.UnsupportedEncodingException ex) {
101+
// handle
102+
}
103+
104+
105+
return output;
106+
}
107+
}
108+
```
109+
110+
The problem with this implementation is that it returns `byte[]` which becomes a Stream on Caché side. I have tried to return a string, but hadn't been able to found how to form proper binary string from `byte[]`. If you have any ideas please leave a comment.
111+
To run it place jar from [releases page](https://github.com/intersystems-ru/zlibisc/releases) into `<instance>/bin` folder, load ObjectScript code into your instance and execute:
112+
113+
```
114+
write $System.Status.GetErrorText(##class(isc.zlib.Utils).createGateway())
115+
write $System.Status.GetErrorText(##class(isc.zlib.Utils).updateJar())
116+
```
117+
118+
Check `createGateway` method before running the command. Second argument `javaHome` assumes that `JAVA_HOME` environment variable is set. If it does not, specify home of Java 1.8 JRE as a second argument. After installing run this code to get compressed `text`:
119+
120+
```
121+
set gateway = ##class(isc.zlib.Utils).connect()
122+
set response = ##class(isc.zlib.Java).compress(gateway, text)
123+
```
124+
125+
126+
# C
127+
128+
An InterSystems [Callout library](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BGCL_library) is a shared library that contains your custom Callout functions and the enabling code that allows Caché to use them.
129+
130+
Here's our Callout library:
131+
132+
```
133+
#define ZF_DLL
134+
135+
// Ugly Windows hack
136+
#ifndef ulong
137+
typedef unsigned long ulong;
138+
#endif
139+
140+
#include "string.h"
141+
#include "stdio.h"
142+
#include "stdlib.h"
143+
#include "zlib.h"
144+
#include <cdzf.h>
145+
146+
int Compress(char* istream, CACHE_EXSTRP retval)
147+
{
148+
ulong srcLen = strlen(istream)+1; // +1 for the trailing `\0`
149+
ulong destLen = compressBound(srcLen); // estimate size needed for the buffer
150+
char* ostream = malloc(destLen);
151+
int res = compress(ostream, &destLen, istream, srcLen);
152+
CACHEEXSTRKILL(retval);
153+
if (!CACHEEXSTRNEW(retval,destLen)) {return ZF_FAILURE;}
154+
memcpy(retval->str.ch,ostream,destLen); // copy to retval->str.ch
155+
return ZF_SUCCESS;
156+
}
157+
158+
ZFBEGIN
159+
ZFENTRY("Compress","cJ",Compress)
160+
ZFEND
161+
```
162+
163+
To run it place `dll` or `so` files from [releases page](https://github.com/intersystems-ru/zlibisc/releases) into `<instance>/bin` folder. Repository also contains build scripts for Windows and Linux, execute them to build your own version.
164+
Linux prerequisites: `apt install build-essential zlib1g zlib1g-devel`
165+
Windows prerequisites: [WinBuilds](http://win-builds.org/doku.php)
166+
167+
To interact with callout library execute:
168+
```
169+
set path = ##class(isc.zlib.Test).getLibPath() //get path to library file
170+
set response = $ZF(-3, path, "Compress", text) // execute function
171+
do $ZF(-3, "") //unload library
172+
```
173+
174+
# System
175+
176+
A little unexpected in an article about callout mechanisms, but Caché also has built-in [Compress](https://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?PAGE=CLASS&LIBRARY=%25SYS&CLASSNAME=%25SYSTEM.Util#METHOD_Compress) (and Decompress function). Call it with: `set response = $extract($SYSTEM.Util.Compress(text), 2, *-1)`
177+
Remember that searching the docs or asking the questions here on the community may save you some time.
178+
179+
# Comparison
180+
181+
I have run simple tests (1Kb text, 1 000 000 iterations) on Linux and Windows and got these results.
182+
183+
Windows:
184+
185+
| Method | Callout | System | Java | Node |
186+
|--------------|---------|--------|---------|----------|
187+
| Time | 22,77 | 33,41 | 152,73 | 622,51 |
188+
| Speed (Kb/s) | 43912 | 29927 | 6547 | 1606 |
189+
| Overhead, % | -/- | 46,73% | 570,75% | 2633,90% |
190+
191+
Linux:
192+
193+
| Method | Callout | System | Java | Node |
194+
|--------------|---------|--------|----------|----------|
195+
| Time | 76,3541 | 76,499 | 147,2436 | 953,7311 |
196+
| Speed (Kb/s) | 13097 | 13072 | 6791 | 1049 |
197+
| Overhead, % | -/- | 0,19% | 92,84% | 1149,09% |
198+
199+
To run tests load code and call: `do ##class(isc.zlib.Test).test(textLength, iterations)`
200+
201+
202+
# Conclusion
203+
204+
With InterSystems products, you can easily leverage existing code in other languages. However, choosing correct implementation is not always easy, you need to take several metrics into account, such as development speed, performance, and maintainability. Do you need to run on different operating systems? Finding answers to these questions can help you decide on the best implementation plan.
205+
206+
# Links
207+
208+
- [Repo](https://github.com/intersystems-ru/zlibisc/)
209+
- [Binaries](https://github.com/intersystems-ru/zlibisc/releases)
210+
- [Http requests](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GNET_http)
211+
- [Java Gateway](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=EJVG)
212+
- [Callout library](https://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=BGCL_library)
213+
- [Compress function](https://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?PAGE=CLASS&LIBRARY=%25SYS&CLASSNAME=%25SYSTEM.Util#METHOD_Compress)

0 commit comments

Comments
 (0)