Initial drop

Author: ibmmqmet

CLA.md

@@ -0,0 +1,32 @@
+IBM Contributor License Agreement
+=================================
+
+Version 1.0.0   January 14, 2014
+
+In order for You (as defined below) to make intellectual property Contributions (as defined below) now or in the future to IBM GitHub repositories,
+You must agree to this Contributor License Agreement ("CLA").
+
+Please read this CLA carefully before accepting its terms. By accepting the CLA, You are agreeing to be bound by its terms.
+If You submit a Pull Request against an IBM repository on GitHub You must include in the Pull Request a statement of Your acceptance of this CLA.
+
+As used in this CLA:
+(i) "You" (or "Your") shall mean the entity that is making this Agreement with IBM;
+(ii)"Contribution" shall mean any original work of authorship, including any modifications or additions to an existing work, that is submitted by You to IBM for inclusion in,
+or documentation of, any of the IBM GitHub repositories;
+(iii)  "Submit" (or "Submitted")  means any form of communication sent to IBM (e.g. the content You post in a GitHub Issue or submit as part of a GitHub Pull Request).
+
+This agreement applies to all Contributions You Submit. 
+
+This CLA, and the license(s) associated with the particular IBM GitHub repositories You are contributing to, provides a license to Your Contributions to IBM and downstream consumers,
+but You still own Your Contributions, and except for the licenses provided for in this CLA, You reserve all right, title and interest in Your Contributions.
+
+IBM requires that each Contribution You Submit now or in the future comply with the following four commitments.
+
+1) You will only Submit Contributions where You have authored 100% of the content.
+2) You will only Submit Contributions to which You have the necessary rights. This means that if You are employed You have received the necessary permissions from Your employer to make the
+Contributions.
+3) Whatever content You Contribute will be provided under the license(s) associated with the particular IBM GitHub repository You are contributing to.
+4) You understand and agree that IBM GitHub repositories and Your contributions are public, and that a record of the contribution (including all personal information You submit with it)
+is maintained indefinitely and may be redistributed consistent with the license(s) involved.
+You will promptly notify the Eclipse Foundation if You become aware of any facts or circumstances that would make these commitments inaccurate in any way.
+To do so, please create an Issue in the appropriate GitHub repository.

README.txt

@@ -0,0 +1,148 @@
+README for mqsmfcsv
+===================
+
+This package has been created to simplify the job
+of doing your own analysis of MQ SMF records from a z/OS system,
+by making it easy to import formatted records into
+a database or spreadsheet.
+
+The program takes the data from a z/OS file, and runs on
+a Windows or Unix system, creating standard CSV
+(comma-separated value) output files.
+
+Unlike packages such as SupportPac MP1B, there is no interpretation
+at all of the SMF data. That can be done by your own spreadsheet or
+database programs and macros.
+
+This makes the code for the program
+very easy to understand; all the structures in the SMF records
+are printed directly.
+
+
+Usage
+=====
+Collect the MQ SMF data in the usual way.
+Dump that data to a z/OS file using a tool such as ...
+Then download that file to
+
+The output files are written to a directory that you name
+on the command line. There is one file for each type of
+element - QPST, WTID etc. The files contain a line of
+column headers, followed by the actual data with each
+field separated by a comma. Spreadsheets such as LibreOffice
+or Excel can then import the data directly.
+
+No attempt is made to explain the column headers; these come
+straight from the field names. More information can be found in
+the csqdsmfc.h header file or the product documentation.
+
+Date and time are split into separate columns; the time
+includes a microsecond portion which is separated using ',' as using '.'
+as the decimal point seems to confuse the automatic importers on
+several spreadsheets. There is no information directly available about
+timezone offsets, so it is possible that the timestamps could be either
+GMT (CUT) or localtime, depending on system configurations.
+
+Durations are split into separate columns for
+seconds and microseconds. This is shown in the column headings.
+
+Options
+=======
+
+  printf("Usage: mqsmfcsv [-o <output dir>] [-a] [ -d <level> ]\n");
+  printf("         [ -i <input file> [-m <max records>] [-r ] [-t <ticker>]\n");
+  printf("  -a Append to existing files if they exist. Default is overwrite\n");
+  printf("  -d <level> Debug by dumping binary records (level = 1 or 2)\n");
+  printf("  -i <Input file>  Default is to read from stdin\n");
+  printf("  -m <Max records> End after formatting M records. Default to process all\n");
+  printf("  -o <Directory>   Where to put output files\n");
+  printf("  -r               Do not print '=' on numeric-looking strings\n");
+  printf("  -t <Ticker>      Print progress message every T records\n");
+
+Design and Development
+======================
+
+Binaries are provided for some platforms. For other
+platforms, you will need to compile the code yourself.
+Familiarity with Makefiles and compiling C programs is
+assumed.
+
+First, you need to get a copy of the MQ product-provided
+header file on z/OS that defines the structures for SMF
+elements. This is called csqdsmfc.h and can be found on your
+z/OS installation. Download a copy of that to your system.
+
+Using the provided Makefiles as a template, set the
+appropriate compilation options for your platform.
+For Unix platforms, the "M" script may be a convenient
+way to override options in the Makefile, as most Unix
+platforms are likely to be very similar in how to build
+programs.
+
+The Makefiles take the z/OS header file and make some
+minor transformations so that it is suitable for
+compiling on other platforms.
+
+Note that the mqsmfcsv program needs to be compiled with
+fixed datatype sizes - in particular, "int" is 32-bit, and
+"long long" is used to represent a 64-bit value. Generally
+this means that the program itself is compiled as 32-bit.
+
+The current version of this program was developed based on the
+MQ V8 SMF structures; it will run against data
+generated by older queue managers
+but will not compile if an older version of csqdsmfc.h is used.
+
+Each data structure is formatted by its own function, contained in
+the print*.c modules. Most of these are very straightforward to
+understand. Some common macros defined in mqsmf.h make it
+easy to add each field from the structure; they also simplify handling
+common fields that need to be printed on every type of SMF element.
+
+A few special cases exist, such as handling bitfields or arrays. But
+again, the code where these are used ought to be obvious, and can be
+used as strategies for handling new types of SMF element.
+
+Sample data
+===========
+The package includes a tiny amount of sample data that can be used to
+demonstrate the formatting.
+
+
+
+downloading ...
+
+
+Sequential, Variable Length Records
+
+The following is a sample FTP script to retrieve (or GET) a sequential file with variable-length records from a Mainframe system to a Windows system. This FTP script is for using at the client or Windows system.
+
+userid
+password
+CD ..
+PWD
+BINARY
+QUOTE SITE RDW
+GET mainframe.dataset.name drive:\directory\filename.ext
+QUIT
+
+The file must be transferred in BINARY. This will maintain the EBCDIC-encoding and the mainframe
+numeric formats for packed-decimal, binary and zoned-decimal data.
+
+One of the following FTP statements must be used prior to a GET or PUT statement in order to
+download the Record-Descriptor-Word (RDW) and the possible Block-Descriptor-Word (BDW).  The BDW and
+RDW are in binary format so it is critical to download in binary even if  the records in the file
+are all text strings. Prior to a GET use one of the following statements.
+QUOTE SITE RDW
+   or
+LITERAL SITE RDW
+
+If the FTP process is to be executed on the mainframe the GET statement will need to be replaced
+with a PUT and the QUOTE or LITERAL statement will need to be replaced with the following statement.
+
+LOCSITE RDW
+
+Once the file is transferred it is a mirror of the mainframe format with EBCDIC-encoding.  Each
+record is preceded by the Record Descriptor Word (RDW) and possible Block Descriptor Word (BDW) and
+must be converted to a Micro Focus formatted sequential file with variable length records. This
+process is described in the following sections of this document.

bin/aix/convH

@@ -0,0 +1,17 @@
+#!/bin/ksh
+
+plat=`uname`
+
+if [ "$plat" = "AIX" ]
+then
+  flags="-q32 "
+  cc="xlc"
+else
+  flags="-m32 "
+  cc="gcc"
+fi
+
+export PLATFLAGS=$flags CC=$cc
+make -e -f Makefile.unix $*
+
+

bin/aix/mqsmfcsv

@@ -0,0 +1,51 @@
+
+CC=cc
+CFLAGS= -I. 
+PLATFLAGS=
+SRC = mqsmf.c \
+      smfDate.c \
+      smfConv.c \
+      printDEBUG.c \
+      printQ5ST.c    \
+      printQCST.c    \
+      printQCTADP.c  \
+      printQCTDNS.c  \
+      printQCTDSP.c  \
+      printQCTSSL.c  \
+      printQCCT.c  \
+      printQESD.c  \
+      printQEST.c  \
+      printQIST.c  \
+      printQJST.c  \
+      printQLST.c  \
+      printQMAC.c  \
+      printQMST.c  \
+      printQPST.c  \
+      printQSGM.c  \
+      printQSPH.c  \
+      printQSRS.c  \
+      printQSST.c  \
+      printQTST.c  \
+      printWQ.c    \
+      printWTAS.c  \
+      printWTID.c
+
+HDR = mqsmfstruc.h \
+      mqsmf.h
+
+mqsmfcsv: $(SRC) $(HDR) Makefile.unix dummy
+	$(CC) $(PLATFLAGS) -o $@ $(SRC) $(CFLAGS)
+	-rm -f out/*
+	-rm -f SMF-*
+	-mkdir out >/dev/null 2>&1 || true
+	$@ -i data/TEST116.dat  -o out -m 200 -d 2
+	$@ -i data/TEST115.dat  -o out -a  -d 2
+
+mqsmfstruc.h: csqdsmfc.h convH
+	convH < csqdsmfc.h > $@
+
+convH: convH.c
+	$(CC) -o $@ convH.c
+
+
+dummy:

bin/linux/convH

@@ -0,0 +1,52 @@
+CFLAGS=-nologo /D_CRT_SECURE_NO_WARNINGS /Zp1 /J
+SRC = mqsmf.c \
+      smfDate.c \
+      smfConv.c \
+      printDEBUG.c \
+      printQ5ST.c    \
+      printQCST.c    \
+      printQCCT.c  \
+      printQCTADP.c\
+      printQCTDNS.c\
+      printQCTDSP.c\
+      printQCTSSL.c\
+      printQESD.c  \
+      printQEST.c  \
+      printQIST.c  \
+      printQJST.c  \
+      printQLST.c  \
+      printQMAC.c  \
+      printQMST.c  \
+      printQPST.c  \
+      printQSGM.c  \
+      printQSPH.c  \
+      printQSRS.c  \
+      printQSST.c  \
+      printQTST.c  \
+      printWQ.c    \
+      printWTAS.c  \
+      printWTID.c
+
+HDR = mqsmfstruc.h \
+      mqsmf.h
+
+INDIR=y:\\metaylor\\mf\\L\\smf
+# INDIR=c:\\temp
+OUTDIR=c:\\temp\\smf
+
+mqsmfcsv.exe: $(SRC) $(HDR) Makefile.win dummy
+	rm -f $@
+	$(CC) $(SRC) -I. /Fe$@ $(CFLAGS)
+	$(INDIR)\\test
+	cd $(INDIR)
+	del /q *.obj
+
+mqsmfstruc.h: csqdsmfc.h convH
+	convH < csqdsmfc.h > $@
+
+convH: convH.c
+	$(CC) /Fe$@ convH.c
+
+
+dummy:
+

bin/linux/mqsmfcsv

@@ -0,0 +1,69 @@
+/*
+ * Copyright (c) 2016 IBM Corporation and other Contributors.
+ *
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ *
+ * Contributors:
+ *   Mark Taylor - Initial Contribution
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+/*
+ * This is a very simple filter to take the product-provided
+ * header file from z/OS and strip a few things that are
+ * problematic on other platform compilers.
+ *
+ * It means we don't need to keep a variant copy
+ * of the header in sync when new versions come out.
+ *
+ * In particular, get rid of the z/OS format for
+ * the pragmas, and also the trigraph at the top
+ * of the file.
+ *
+ * If the contents of the header file change, this
+ * filter may need to be updated to match.
+ */
+int main(int argc, char **argv)
+{
+  char line[256] = {0};
+  char *c;
+
+  /******************************************************/
+  /* Need a portable pragma that works on all platforms */
+  /******************************************************/
+  printf("#pragma pack(1)\n");
+
+  /******************************************************/
+  /* Then copy stdin to stdout, stripping just a few    */
+  /* lines.                                             */
+  /******************************************************/
+  do
+  {
+     c = fgets(line,sizeof(line)-1,stdin);
+     if (c)
+     {
+       if (strstr(line,"pragma"))
+         continue;
+       /****************************************************/
+       /* Can't just put "??=" in here as it's converted   */
+       /* into a trigraph and no longer matches.           */
+       /****************************************************/
+       else if (strstr(line,"\?\?="))
+         continue;
+       else
+       {
+          printf(line);
+          if (line[strlen(line)-1] != '\n')
+            printf("\n");
+       }
+     }
+  } while (c);
+
+  fflush(stdout);
+  return 0;
+}

bin/win/convH.exe

@@ -0,0 +1,3 @@
+PATH=%PATH%;c:\program files (x86)\microsoft visual studio 11.0\vc
+vcvarsall
+nmake -f Makefile.win