Skip to content

Commit e8198ad

Browse files
committed
Ok so the thing is now a pip package.
1 parent 9b64b8a commit e8198ad

File tree

10 files changed

+118
-70
lines changed

10 files changed

+118
-70
lines changed

README.md

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,20 @@ of a build process requires that.
3131

3232
WLLVM works with either clang or the gcc dragonegg plugin.
3333

34-
Tutorial
35-
====
34+
Installation
35+
============
36+
37+
WLLVM is now a pip package. You can just do:
38+
39+
pip install wllvm
40+
41+
42+
If you want to develop or use the development version:
43+
44+
git clone https://github.com/SRI-CSL/whole-program-llvm
45+
cd wllvm
46+
pip install -e .
3647

37-
See the tutorial markdown files for detailed instructions on how to compile apache with wllvm on Ubuntu.
3848

3949
Usage
4050
=====
@@ -79,6 +89,9 @@ In addition to the above environment variables the following can be optionally u
7989
bitcode files.
8090

8191

92+
93+
94+
8295
Building a bitcode module with clang
8396
====================================
8497

@@ -92,6 +105,9 @@ Building a bitcode module with clang
92105
# Produces pkg-config.bc
93106
extract-bc pkg-config
94107

108+
Tutorials
109+
=========
110+
95111
A gentler set of instructions on building apache in a vagrant Ubuntu 14.04 can be found
96112
[here,](https://github.com/SRI-CSL/whole-program-llvm/blob/master/doc/tutorial.md) and
97113
for Ubuntu 16.04

README.rst

Lines changed: 32 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
Introduction
2-
============
1+
Introduction to WLLVM
2+
=====================
33

44
This project, WLLVM, provides tools for building whole-program (or
55
whole-library) LLVM bitcode files from an unmodified C or C++
@@ -19,59 +19,62 @@ the bitcode into a single whole-program bitcode file. This utility
1919
works for both executable and native libraries.
2020

2121
This two-phase build process is necessary to be a drop-in replacement
22-
for gcc or g++ in any build system. Using the LTO framework in gcc
22+
for ``gcc`` or ``g++`` in any build system. Using the LTO framework in gcc
2323
and the gold linker plugin works in many cases, but fails in the
2424
presence of static libraries in builds. WLLVM's approach has the
2525
distinct advantage of generating working binaries, in case some part
2626
of a build process requires that.
2727

28-
WLLVM works with either clang or the gcc dragonegg plugin.
28+
WLLVM works with either ``clang`` or the ``gcc dragonegg`` plugin.
2929

30-
Tutorial
31-
====
32-
33-
See the tutorial markdown files for detailed instructions on how to compile apache with wllvm on Ubuntu.
3430

3531
Usage
36-
=====
32+
-----
3733

38-
WLLVM includes two python executables: `wllvm` for compiling C code
39-
and `wllvm++` for C++, and an auxiliary tool `extract-bc`.
34+
WLLVM includes two python executables: ``wllvm`` for compiling C code
35+
and ``wllvm++`` for C++, and an auxiliary tool ``extract-bc``.
4036

4137
Three environment variables must be set to use these wrappers:
4238

43-
* `LLVM_COMPILER` should be set to either `dragonegg` or `clang`.
44-
* `LLVM_GCC_PREFIX` should be set to the prefix for the version of gcc that should
39+
* ``LLVM_COMPILER`` should be set to either ``dragonegg`` or ``clang``.
40+
* ``LLVM_GCC_PREFIX`` should be set to the prefix for the version of gcc that should
4541
be used with dragonegg. This can be empty if there is no prefix. This variable is
46-
not used if `$LLVM_COMPILER == clang`.
47-
* `LLVM_DRAGONEGG_PLUGIN` should be the full path to the dragonegg plugin. This
48-
variable is not used if `$LLVM_COMPILER == clang`.
42+
not used if ``$LLVM_COMPILER == clang``.
43+
* ``LLVM_DRAGONEGG_PLUGIN`` should be the full path to the dragonegg plugin. This
44+
variable is not used if ``$LLVM_COMPILER == clang``.
4945

50-
Once the environment is set up, just use `wllvm` and `wllvm++` as your C
46+
Once the environment is set up, just use ``wllvm`` and ``wllvm++`` as your C
5147
and C++ compilers, respectively.
5248

5349

5450
In addition to the above environment variables the following can be optionally used:
5551

56-
* `LLVM_CC_NAME` can be set if your clang compiler is not called `clang` but
57-
something like `clang-3.7`. Similarly `LLVM_CXX_NAME` can be used to describe
58-
what the C++ compiler is called. Note that in these sorts of cases, the environment
59-
variable `LLVM_COMPILER` should still be set to `clang` not `clang-3.7` etc.
60-
We also pay attention to the environment variables `LLVM_LINK_NAME` and `LLVM_AR_NAME` in an
61-
analagous way, since they too get adorned with suffixes in various Linux distributions.
52+
* ``LLVM_CC_NAME`` can be set if your clang compiler is not called ``clang`` but
53+
something like ``clang-3.7``. Similarly ``LLVM_CXX_NAME`` can be used to describe
54+
what the C++ compiler is called. Note that in these sorts of cases, the environment
55+
variable ``LLVM_COMPILER`` should still be set to ``clang`` not ``clang-3.7`` etc.
56+
We also pay attention to the environment variables ``LLVM_LINK_NAME`` and ``LLVM_AR_NAME`` in an
57+
analagous way, since they too get adorned with suffixes in various Linux distributions.
6258

63-
* `LLVM_COMPILER_PATH` can be set to the absolute path to the folder that
64-
contains the compiler and other LLVM tools such as `llvm-link` to be used.
59+
* ``LLVM_COMPILER_PATH`` can be set to the absolute path to the folder that
60+
contains the compiler and other LLVM tools such as ``llvm-link`` to be used.
6561
This prevents searching for the compiler in your PATH environment variable.
6662
This can be useful if you have different versions of clang on your system
6763
and you want to easily switch compilers without tinkering with your PATH
6864
variable.
69-
Example `LLVM_COMPILER_PATH=/home/user/llvm_and_clang/Debug+Asserts/bin`.
65+
Example ``LLVM_COMPILER_PATH=/home/user/llvm_and_clang/Debug+Asserts/bin``.
7066

71-
* `WLLVM_CONFIGURE_ONLY` can be set to anything. If it is set, `wllvm`
72-
and `wllvm++` behave like a normal C or C++ compiler. They do not
73-
produce bitcode. Setting `WLLVM_CONFIGURE_ONLY` may prevent
67+
* ``WLLVM_CONFIGURE_ONLY`` can be set to anything. If it is set, ``wllvm``
68+
and ``wllvm++`` behave like a normal C or C++ compiler. They do not
69+
produce bitcode. Setting ``WLLVM_CONFIGURE_ONLY`` may prevent
7470
configuration errors caused by the unexpected production of hidden
7571
bitcode files.
7672

7773

74+
Documentation
75+
-------------
76+
77+
More detailed documentation as well as some tutorials can be found
78+
here:
79+
80+
https://github.com/SRI-CSL/whole-program-llvm

wllvm/as

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,17 @@
11
#!/usr/bin/env python
2-
#
3-
# This is the assembler phase. This variant is only invoked during
4-
# the second compilation where we are building bitcode. The compiler
5-
# has already been instructed to generate LLVM IR; the compiler then
6-
# tries to assemble it into an object file. The standard assembler
7-
# doesn't understand LLVM bitcode, so we interpose and use the llvm-as
8-
# command to build a bitcode file. We leave the bitcode in place, but
9-
# record its full absolute path in the corresponding object file
10-
# (which was created in the first compilation phase by the real
11-
# compiler). We'll link this together at a later stage.
2+
"""
3+
This is the assembler phase.
4+
5+
This variant is only invoked during
6+
the second compilation where we are building bitcode. The compiler
7+
has already been instructed to generate LLVM IR; the compiler then
8+
tries to assemble it into an object file. The standard assembler
9+
doesn't understand LLVM bitcode, so we interpose and use the llvm-as
10+
command to build a bitcode file. We leave the bitcode in place, but
11+
record its full absolute path in the corresponding object file
12+
(which was created in the first compilation phase by the real
13+
compiler). We'll link this together at a later stage.
14+
"""
1215

1316
from __future__ import absolute_import
1417
import sys

wllvm/checker.py

Lines changed: 26 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22
import os
33
import subprocess as sp
44
import errno
5+
"""
6+
Module support for the wllvm-sanity-checker tool.
7+
8+
The wllvm-sanity-checker tool examines the users
9+
environment to see if it makes sense from the
10+
wllvm point of view. Useful first step in trying to
11+
debug a failure.
12+
"""
513

614
explain_LLVM_COMPILER = """
715
@@ -72,6 +80,15 @@ def __init__(self):
7280
self.path = path if path else ''
7381

7482
def check(self):
83+
"""Performs the environmental sanity check.
84+
85+
Performs the following checks in order:
86+
87+
1. Check that the OS is supported.
88+
2. Checks that the compiler settings make sense.
89+
3. Checks that the needed LLVM utilities exists.
90+
"""
91+
7592
if not self.checkOS():
7693
print 'I do not think we support your OS. Sorry.'
7794
return 1
@@ -86,12 +103,14 @@ def check(self):
86103

87104

88105
def checkOS(self):
106+
"""Returns True if we support the OS."""
89107
return (sys.platform.startswith('freebsd') or
90108
sys.platform.startswith('linux') or
91109
sys.platform.startswith('darwin'))
92110

93111

94112
def checkSwitch(self):
113+
"""Checks the correctness of the LLVM_COMPILER env var."""
95114
compiler_type = os.getenv('LLVM_COMPILER')
96115
if compiler_type == 'clang':
97116
return (1, '\nGood, we are using clang.\n')
@@ -102,7 +121,7 @@ def checkSwitch(self):
102121

103122

104123
def checkClang(self):
105-
124+
"""Checks for clang and clang++."""
106125
cc_name = os.getenv('LLVM_CC_NAME')
107126
cxx_name = os.getenv('LLVM_CXX_NAME')
108127

@@ -113,7 +132,7 @@ def checkClang(self):
113132

114133

115134
def checkDragonegg(self):
116-
135+
"""Checks for gcc, g++ and the dragonegg plugin."""
117136
if not self.checkDragoneggPlugin():
118137
return False
119138

@@ -128,6 +147,7 @@ def checkDragonegg(self):
128147

129148

130149
def checkDragoneggPlugin(self):
150+
"""Checks for the dragonegg plugin."""
131151
plugin = os.getenv('LLVM_DRAGONEGG_PLUGIN')
132152

133153
if not plugin:
@@ -148,6 +168,7 @@ def checkDragoneggPlugin(self):
148168

149169

150170
def checkCompiler(self):
171+
"""Determines the chosen compiler, and checks it."""
151172
(code, comment) = self.checkSwitch()
152173

153174
if code == 0:
@@ -165,7 +186,7 @@ def checkCompiler(self):
165186

166187

167188
def checkCompilers(self, cc, cxx):
168-
189+
"""Tests that the compilers actually exist."""
169190
(ccOk, ccVersion) = self.checkExecutable(cc)
170191
(cxxOk, cxxVersion) = self.checkExecutable(cxx)
171192

@@ -192,6 +213,7 @@ def checkCompilers(self, cc, cxx):
192213

193214

194215
def checkExecutable(self, exe, version_switch='-v'):
216+
"""Checks that an executable exists, and is executable."""
195217
cmd = [exe, version_switch]
196218
try:
197219
compiler = sp.Popen(cmd, stdout=sp.PIPE, stderr=sp.PIPE)
@@ -210,6 +232,7 @@ def checkExecutable(self, exe, version_switch='-v'):
210232

211233

212234
def checkAuxiliaries(self):
235+
"""Checks for the archiver and linker."""
213236
link_name = os.getenv('LLVM_LINK_NAME')
214237
ar_name = os.getenv('LLVM_AR_NAME')
215238

wllvm/extraction.py

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,13 @@
3434
bitCodeArchiveExtension='bca'
3535
moduleExtension='bc'
3636

37-
# Use objdump on the provided binary; parse out the fields
38-
# to find the given section. Return the size and offset of
39-
# that section (in bytes)
4037
def getSectionSizeAndOffset(sectionName, filename):
38+
"""Returns the size and offset of the section, both in bytes.
39+
40+
Use objdump on the provided binary; parse out the fields
41+
to find the given section. Parses the output,and
42+
extracts thesize and offset of that section (in bytes).
43+
"""
4144
objdumpCmd = ['objdump', '-h', '-w', filename]
4245
objdumpProc = Popen(objdumpCmd, stdout=sp.PIPE)
4346

@@ -64,8 +67,8 @@ def getSectionSizeAndOffset(sectionName, filename):
6467
logging.warning('Could not find "{0}" ELF section in "{1}", so skipping this entry.'.format(sectionName,filename))
6568
return None
6669

67-
# Read the entire content of an ELF section into a string
6870
def getSectionContent(size, offset, filename):
71+
"""Reads the entire content of an ELF section into a string."""
6972
with open(filename, mode='rb') as f:
7073
f.seek(offset)
7174
d = ''
@@ -81,13 +84,12 @@ def getSectionContent(size, offset, filename):
8184
# nulls.
8285
return d.replace('\0', '')
8386

84-
# os independent abstraction (darwin version)
85-
#use otool to extract the segment and section
86-
#iam: "xxd -r" would just refuse to play nice, so it got tossed.
87-
# if any one can get
88-
# otool -X -s __WLLVM __llvm_bc inputFile | xxd -r
89-
# to work this can be simplified.
9087
def extract_section_darwin(inputFile):
88+
"""Extracts the section as a string, the darwin version.
89+
90+
Uses otool to extract the section, then processes it
91+
to a usable state.
92+
"""
9193

9294
otoolCmd = ['otool', '-X', '-s', darwinSegmentName, darwinSectionName, inputFile]
9395
otoolProc = Popen(otoolCmd, stdout=sp.PIPE)
@@ -108,9 +110,8 @@ def extract_section_darwin(inputFile):
108110
logging.error('{0} contained no {1} segment'.format(inputFile, darwinSegmentName))
109111
return contents
110112

111-
# os independent abstraction (*nix version)
112-
#analagous procedure for linux (relying on getSectionSizeAndOffset and getSectionContent)
113113
def extract_section_linux(inputFile):
114+
"""Extracts the section as a string, the *nix version."""
114115
val = getSectionSizeAndOffset(elfSectionName, inputFile)
115116
if val is None:
116117
return []

wllvm/extractor.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,7 @@
2121
2222
"""
2323

24-
import sys, os
25-
26-
sys.path.append(os.path.abspath(os.path.join(__file__, '..')))
24+
import sys
2725

2826
from extraction import *
2927

wllvm/sanity.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,7 @@
66
77
"""
88

9-
import sys, os
10-
11-
sys.path.append(os.path.abspath(os.path.join(__file__, '..')))
9+
import sys
1210

1311
from checker import *
1412

wllvm/version.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,17 @@
33
# this is the all important version number used by pip.
44
#
55
#
6-
wllvm_version='1.0.0.dev0'
6+
"""
7+
Version History:
8+
9+
1.0.0 - 8/2/2016 initial birth as a pip package.
10+
1.0.1 - 8/2/2016 the rst gets a make over, and doc strings
11+
became more pervasive.
12+
13+
14+
15+
16+
17+
"""
18+
wllvm_version='1.0.1'
719

wllvm/wllvm.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,6 @@
1111
"""
1212

1313
import sys
14-
import os
15-
16-
sys.path.append(os.path.abspath(os.path.join(__file__, '..')))
1714

1815
from utils import *
1916

wllvm/wllvmpp.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@
99
"""
1010

1111
import sys
12-
import os
13-
14-
sys.path.append(os.path.abspath(os.path.join(__file__, '..')))
1512

1613
from utils import *
1714
import logconfig

0 commit comments

Comments
 (0)