Merge.

Author: ianamason

.gitignore

@@ -1 +1,2 @@
 *.pyc
+.idea

.travis.yml

@@ -0,0 +1,21 @@
+language: python
+python:
+  - "3.3"
+  - "2.7"
+  - "2.6"
+
+# command to install dependencies
+install:
+  - sudo apt-get update
+  - sudo apt-get install llvm-3.4 clang-3.4 libapr1-dev libaprutil1-dev
+  - sudo apt-get install dragonegg llvm-3.0 llvm-gcc
+  - export WLLVM_HOME=`pwd`
+
+# command to run tests
+script:
+  # build apache with clang (i.e. httpd-2.4.12) 
+  - ${WLLVM_HOME}/.travis/apache_clang.sh
+  # build apache with gcc and dragonegg (i.e. httpd-2.4.12) 
+  - ${WLLVM_HOME}/.travis/apache_dragonegg.sh
+  
+

.travis/apache_clang.sh

@@ -0,0 +1,30 @@
+#!/bin/bash -x
+# Make sure we exit if there is a failure
+set -e
+
+
+export PATH=/usr/lib/llvm-3.4/bin:${WLLVM_HOME}:${PATH}
+export LLVM_COMPILER=clang
+export WLLVM_OUTPUT=WARNING
+
+wget http://www.csl.sri.com/~iam/httpd-2.4.12.tar.gz
+tar xfz httpd-2.4.12.tar.gz
+mv httpd-2.4.12 apache_clang
+cd apache_clang
+CC=wllvm ./configure
+make
+extract-bc httpd
+
+
+if [ -s "httpd.bc" ]
+then
+    echo "httpd.bc built."
+else
+    exit 1
+fi
+
+
+
+
+
+

.travis/apache_dragonegg.sh

@@ -0,0 +1,34 @@
+#!/bin/bash -x
+# Make sure we exit if there is a failure
+set -e
+
+export dragonegg_disable_version_check=true
+
+export PATH=/usr/lib/llvm-3.0/bin:${WLLVM_HOME}:${PATH}
+export LLVM_COMPILER=dragonegg
+export LLVM_GCC_PREFIX=llvm-
+export LLVM_DRAGONEGG_PLUGIN=/usr/lib/gcc/x86_64-linux-gnu/4.6/plugin/dragonegg.so
+
+export WLLVM_OUTPUT=WARNING
+
+wget http://www.csl.sri.com/~iam/httpd-2.4.12.tar.gz
+tar xfz httpd-2.4.12.tar.gz
+mv httpd-2.4.12 apache_dragonegg
+cd apache_dragonegg
+CC=wllvm ./configure
+make
+extract-bc httpd
+
+
+if [ -s "httpd.bc" ]
+then
+    echo "httpd.bc built."
+else
+    exit 1
+fi
+
+
+
+
+
+

NOTES.txt

@@ -67,7 +67,7 @@ to your shell startup (in '~/.profile' for bash):
 
     export LLVM_HOME=/usr/local/llvm-3.3
 
-Get LLVM and Clang version 3.5:
+Get LLVM and Clang version 3.3:
 
     svn co http://llvm.org/svn/llvm-project/llvm/branches/release_33 llvm
     cd llvm/tools

README-freeBSD.md

@@ -1,51 +1,62 @@
+
+[![Build Status](https://travis-ci.org/SRI-CSL/whole-program-llvm.svg?branch=master)](https://travis-ci.org/SRI-CSL/whole-program-llvm)
+
+
 Introduction
 ============
 
-This is a small python-based wrapper around a GCC-compatible compiler
-to make it easy to build whole-program (or whole-library) LLVM bitcode
-files.  The idea is that it first invokes the compiler as normal to
-build a real object file.  It then invokes a bitcode compiler to
-generate the corresponding bitcode, recording the location of the
-bitcode file in an ELF section of the actual object file.
-
-When object files are linked together, the contents of non-special ELF
-sections are just concatenated (so we don't lose the locations of any
-of the constituent bitcode files).
-
-This package contains an extra utility, extract-bc, to read the
-contents of this ELF section and link all of the bitcode into a single
-whole-program bitcode file. This utility can also be used on built
-native static libraries to generate LLVM bitcode archives.
-
-This two-phase build process is slower and more elaborate than normal,
-but in practice is necessary to be a drop-in replacement for gcc in
-any build system.  Approaches using the LTO framework in gcc and the
-gold linker plugin work for many cases, but fail in the presence of
-static libraries in builds.  This approach has the distinct advantage
-of generating working binaries, in case some part of a build process
-actually requires that.
-
-Currently, this package only works using clang or the dragonegg plugin
-with gcc 4.5 (with the required patch for dragonegg).
+This project, WLLVM, provides tools for building whole-program (or
+whole-library) LLVM bitcode files from an unmodified C or C++
+source package. It currently runs on `*nix` platforms such as Linux,
+FreeBSD, and Mac OS X.
+
+WLLVM provides python-based compiler wrappers that work in two
+steps. The wrappers first invoke the compiler as normal. Then, for
+each object file, they call a bitcode compiler to produce LLVM
+bitcode. The wrappers also store the location of the generated bitcode
+file in a dedicated section of the object file.  When object files are
+linked together, the contents of the dedicated sections are
+concatenated (so we don't lose the locations of any of the constituent
+bitcode files). After the build completes, one can use an WLLVM
+utility to read the contents of the dedicated section and link all of
+the bitcode into a single whole-program bitcode file. This utility
+works for both executable and native libraries.
+
+This two-phase build process is necessary to be a drop-in replacement
+for gcc or g++ in any build system.  Using the LTO framework in gcc
+and the gold linker plugin works in many cases, but fails in the
+presence of static libraries in builds.  WLLVM's approach has the
+distinct advantage of generating working binaries, in case some part
+of a build process requires that.
+
+WLLVM works with either clang or the gcc dragonegg plugin.
 
 Usage
 =====
 
-There are three environment variables that must be set to use this
-wrapper script:
+WLLVM includes two python executables: `wllvm` for compiling C code
+and `wllvm++` for C++, and an auxiliary tool `extract-bc`.
 
- * `LLVM_COMPILER` should be set to 'dragonegg' or 'clang'.
+Three environment variables must be set to use these wrappers:
+
+ * `LLVM_COMPILER` should be set to either `dragonegg` or `clang`.
  * `LLVM_GCC_PREFIX` should be set to the prefix for the version of gcc that should
    be used with dragonegg.  This can be empty if there is no prefix.  This variable is
    not used if `$LLVM_COMPILER == clang`.
  * `LLVM_DRAGONEGG_PLUGIN` should be the full path to the dragonegg plugin.  This
    variable is not used if `$LLVM_COMPILER == clang`.
 
-Once the environment is set up, just use wllvm and wllvm++ as your C
+Once the environment is set up, just use `wllvm` and `wllvm++` as your C
 and C++ compilers, respectively.
 
+
 In addition to the above environment variables the following can be optionally used:
 
+ * `LLVM_CC_NAME` can be set if your clang compiler is not called `clang` but
+    something like `clang-3.7`. Similarly `LLVM_CXX_NAME` can be used to describe
+    what the C++ compiler is called. Note that in these sorts of cases, the environment
+    variable `LLVM_COMPILER` should still be set to `clang` not `clang-3.7` etc.
+
  * `LLVM_COMPILER_PATH` can be set to the absolute path to the folder that
    contains the compiler and other LLVM tools such as `llvm-link` to be used.
    This prevents searching for the compiler in your PATH environment variable.
@@ -54,8 +65,31 @@ In addition to the above environment variables the following can be optionally u
    variable.
    Example `LLVM_COMPILER_PATH=/home/user/llvm_and_clang/Debug+Asserts/bin`.
 
-Example building bitcode module
-===============================
+* `WLLVM_CONFIGURE_ONLY` can be set to anything. If it is set, `wllvm`
+   and `wllvm++` behave like a normal C or C++ compiler. They do not
+   produce bitcode.  Setting `WLLVM_CONFIGURE_ONLY` may prevent
+   configuration errors caused by the unexpected production of hidden
+   bitcode files.
+
+
+Building a bitcode module with clang
+====================================
+
+    export LLVM_COMPILER=clang
+
+    tar xf pkg-config-0.26.tar.gz
+    cd pkg-config-0.26
+    CC=wllvm ./configure
+    make
+
+    # Produces pkg-config.bc
+    extract-bc pkg-config
+
+A gentler set of instructions on building apache can be found
+[here.](https://github.com/SRI-CSL/whole-program-llvm/blob/master/tutorial.md)
+
+Building a bitcode module with dragonegg
+========================================
 
     export LLVM_COMPILER=dragonegg
     export LLVM_GCC_PREFIX=llvm-
@@ -69,8 +103,9 @@ Example building bitcode module
     # Produces pkg-config.bc
     extract-bc pkg-config
 
-Example building bitcode archive
-================================
+
+Building bitcode archive
+========================
 
     export LLVM_COMPILER=clang
     tar -xvf bullet-2.81-rev2613.tgz
@@ -82,11 +117,38 @@ Example building bitcode archive
     # Produces src/LinearMath/libLinearMath.bca
     extract-bc src/LinearMath/libLinearMath.a
 
-Example building an Operating System
+
+Building an Operating System
+============================
+
+To see how to build freeBSD 10.0 from scratch check out this 
+[guide.](../master/README-freeBSD.md)
+
+
+Configuring without building bitcode
 ================================
 
-To see how to build freeBSD 10.0 from scratch check out the guide 
-[here.](../master/README-freeBSD.md)
+
+    WLLVM_CONFIGURE_ONLY=1 CC=wllvm ./configure
+    CC=wllvm make
+
+
+Building a bitcode archive then extracting the bitcode
+========================
+
+    export LLVM_COMPILER=clang
+    tar xvfz jansson-2.7.tar.gz
+    cd jansson-2.7
+    CC=wllvm ./configure
+    make
+    mkdir bitcode
+    cp src/.libs/libjansson.a bitcode
+    cd bitcode
+    extract-bc libjansson.a
+    llvm-ar x libjansson.bca
+    ls -la
+   
+    
 
 Debugging
 =========
@@ -104,6 +166,18 @@ For example
 
     export WLLVM_OUTPUT=DEBUG
 
+
+Sanity Checking
+=========
+
+Too many environment variables? Try doing a sanity check:
+
+```
+wllvm-sanity-checker
+```
+it might point out what is wrong.
+
+
 License
 =======
 

README.md

@@ -10,13 +10,18 @@
 # (which was created in the first compilation phase by the real
 # compiler).  We'll link this together at a later stage.
 
-import os
+from __future__ import absolute_import
 import sys
-from utils import *
+import os
+
+sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
+
+from driver.utils import *
 from subprocess import *
-from popenwrapper import Popen
-import logconfig
+from driver.popenwrapper import Popen
 
+import logging
+logging.basicConfig()
 
 class BCFilter(ArgumentListFilter):
     def __init__(self, arglist):
@@ -48,18 +53,15 @@ except AttributeError as e:
     logging.error('Output file argument not found.\nException message: ' + str(e))
     sys.exit(1)
 
-bcfilename = '.{0}.bc'.format(filename)
-bcPath = os.path.join(dirs, bcfilename)
-fakeAssembler = [llvmAssembler, infile, '-o', bcPath]
+fakeAssembler = [llvmAssembler, infile, '-o', argFilter.outFileName]
+
 asmProc = Popen(fakeAssembler)
 realRet = asmProc.wait()
 
 if realRet != 0:
     logging.error('llvm-as failed')
     sys.exit(realRet)
 
-attachBitcodePathToObject(bcPath, argFilter.outFileName)
-
 sys.exit(realRet)