Quality Testing
Unit testing is not implemented yet. It should at least monitor error rate on a range of corpora.
Bugs can be reported using the "Issues" tab. Please follow the ReportingBugs guidelines.
Here is a graph of the relative evolution of time, memory and error rate according to svn release.
The automated test tool downloads each release, then configures and compiles icsiboost. It then performs 100 rounds of training on the adult dataset followed by predictions and computes the resulting error rate. The executable is monitored for memory and time.
All values are normalized between 0 and 1, only releases with actual changes and that run sucessfully are shown.
First, edit test_all_releases.sh and set the highest release number to try (you can get that from svn info|grep Revision).
mkdir test_all
cd test_all
cp $ICSIBOOST/src/test_*.sh .
wget http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.names
wget http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data
wget http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test
./test_all_releases.sh
./test_analysis.sh
The script tries to download each release, compile it, trains a model for the adult dataset and computes performance on that dataset. Then, the analysis gathers the results and generates all.csv, a file containing on each line:
release;error-rate;train-time;train-memory;test-time;test-memory
Many other measures are gathered during testing, look at the scripts to know more about them.