How to Install?

  • Note: If you want to reproduce our experiments, please follow the instructions in the previous section.

Requirements

  • RealPaver 0.4
    • Download this file: realpaver-0.4.tar.gz
    • Run "tar -xzvf realpaver-0.4.tar.gz" under the directory of your choice
    • Make sure your environment satisfies the requirements listed under "HARDWARE AND SOFTWARE REQUIREMENTS" in README file. Dependencies not listed there include flex,bison.
    • Run "./configure; make" (if 'configure' fails, try running only make)
  • Java 1.7+
  • Ant (we use 1.9, but older versions should work too)
  • (Optional) If you want to run our scripts:
    • Basic linux environment (coreutils)
    • Python 2
    • GNU Parallel
    • R
  • (Optional) To reproduce the results from Mathematica:
    • Mathematica 10

Setup

  • Download qCORAL here
  • Uncompress the file somewhere
  • Build the project with ant (ant clean; ant) or import the project with Eclipse and build it from scratch
  • Set the location of the RealPaver executable in "scripts/variables"

Organization

This artifact is organized as follows:

  • src/ - Source code for qCORAL
  • docs/ - Additional documentation for qCORAL (like the grammar for constraints)
  • libs/ - Libraries used by qCORAL
  • inputs/ - Benchmarks and other input files used in our experiments. The benchmarks used in our FSE'15 paper are located in the inputs/dwqcoral folder. See the README there for more info about the organization of the subjects.
  • scripts/ - Scripts to run our experiments.
  • output/ - The output of the scripts mentioned above is sent to here.
  • raw-results/ - The raw results from our experiments. See the README there for more information about the files, or the replication page to see how to reproduce our experiments.

How to Interface with qCORAL?

From the command-line

  • From the main directory (not the script directory!): 
 ./run_qcoral.sh [$LIST_OF_OPTIONS] $INPUT_FILE

If you don't have bash installed (e.g. Windows without cygwin), you will need to inspect the script and build the invocation to the qCORAL main class manually (set classpath, etc.).

Example: 

 ./run_qcoral --mcIterativeImprovement --mcProportionalBoxSampleAllocation \
   --mcSeed 123456 --mcMaxSamples 500000 --mcInitialPartitionBudget 50000 \
   --mcTargetVariance 1E-20 --mcSamplesPerIncrement 10000 \
   inputs/dwqcoral/normal/example-cart-12

This will run qCORAL with:

  • incremental sampling, 'sensivity' heuristic (--mcIterativeImprovement)
  • Proportional sampling within boxes (--mcProportionalBoxSampleAllocation) (see section 4.1, 'ICP-based stratified sampling' from our FSE'15 paper)
  • RNG seeded to 123456 (--mcSeed)
  • Global sampling budget = 500k (--mcMaxSamples)
  • Initial bootstrap budget = 50k (--mcInitialPartitionBudget)
  • Stop criteria: global variance < 1E-20 (--mcTargetVariance)
  • Samples per iteration step = 10k (--mcSamplesPerIncrement)

Please see the list of command-line options.

The output should be something like: 

 [inputs/dwqcoral/normal/example-cart-12]
 [qCORAL-iterative] Iterative algorithm was chosen. Partitioning, caching and interval solving arenow enabled
 [qCORAL] Samples per partition (45 total): 1111
 [qCORAL-iterative] initial run done. 49815 samples were spent. 450185 remaining; step=10000
 [qCORAL-iterative] samples=59815 mean=1.007026056585862 var=7.02460402411572E-4 time=4.019631617
 [qCORAL-iterative] samples=69815 mean=0.9941482000183062 var=6.548812797198114E-4 time=4.157834385
 [qCORAL-iterative] samples=79815 mean=0.9923352920636449 var=6.284002143018238E-4 time=4.439707941
 ...
 [qCORAL-iterative] samples=489815 mean=0.9997801102948921 var=6.466456756358703E-5 time=10.149530116
 [qCORAL-iterative] samples=499815 mean=1.000027574254487 var=6.340004635291763E-5 time=10.268969397
 [qCORAL-iterative] samples=500000 mean=1.000002330888982 var=6.337042764638242E-5 time=10.277147533
 [qCORAL:results] samples=500000, mean=1.000002e+00, variance=6.337043e-05, time=10.277397, stdev=7.960554e-03

From a Java program

qCORAL does not have a API yet, but you can invoke the analysis directly using the method coral.counters.refactoring.Runner.runIterativeAnalysis() . For more details on how to call and configure qCORAL directly from Java, please see the source.

How to extend qCORAL?

Adding new operations/data types to constraints

Check the symlib package. qCORAL uses a recursive descent parser to parse constraints, which shouldn't be too hard to extend. There are various types of visitors in coral.util.visitors that you may find useful.

If you want to add support to a new probability distribution, take a look at coral.counters.DistributionAwareQCoral.parseVariable().

Customizing the analysis

Most of the relevant code is in the coral.counters package. Start looking at coral.counters.refactoring.Runner - this is the recommended (and latest) driver for qCORAL. However, this class is focused on running the iterative algorithm described on our FSE'15 paper, and some older features are not available. If you want to use the discretization algorithm, or use Mathematica (and the NProbability function) to compute probabilities, you will need to use the coral.counters.DistributionAwareQCoral class. See the scripts/runDWCounter.sh script for more information.