If you want to write your own first wrapper, we would like to show you how we would proceed to set up a wrapper for the SAT solver Spear based on our 'generic wrapper' written in Python.
Our generic wrapper handles already the following points and you do not have to worry about it: * wrapping and execution of a target algorithm (e.g., Spear) with the runsolver to limit the usage of memory and runtime * recognizes timeouts and memouts of target algorithms * provides a mapping from parameter name to parameter value * prints the result for SMAC (or ParamILS) in the appropriated format
The file genericWrapper.py is an abstract class where only two methods are open for implementation, i.e., get_command_line_args() and process_results() (details are covered later on).
First, we have to prepare a directory with all required files.
You will find three files:
The empty wrapper is a simple class in python which inherits the framework of the 'AbstractWrapper' defined in genericWrapper.py. It has a lot of boilerplate text to explain what is happening:
#!/usr/bin/python
# encoding: utf-8
from genericWrapper import AbstractWrapper
class EmptyWrapper(AbstractWrapper):
def get_command_line_args(self, runargs, config):
'''
Returns the command line call string to execute the target algorithm (here: Spear).
Args:
runargs: a map of several optional arguments for the execution of the target algorithm.
{
"instance": <instance>,
"specifics" : <extra data associated with the instance>,
"cutoff" : <runtime cutoff>,
"runlength" : <runlength cutoff>,
"seed" : <seed>
}
config: a mapping from parameter name to parameter value
Returns:
A command call list to execute the target algorithm.
'''
cmd = "<TODO: fill in your binary> %s" %(runargs["instance"])
# TODO: add the paramters in <config> to your cmd
return cmd
def process_results(self, filepointer, out_args):
'''
Parse a results file to extract the runs status (SUCCESS/CRASHED/etc) and other optional results.
Args:
filepointer: a pointer to the file containing the solver execution standard out.
out_args : a map with {"exit_code" : exit code of target algorithm}
Returns:
A map containing the standard AClib run results. The current standard result map as of AClib 2.06 is:
{
"status" : <"SUCCESS"/"SAT"/"UNSAT"/"TIMEOUT"/"CRASHED"/"ABORT">,
"runtime" : <runtime of target algrithm>,
"quality" : <a domain specific measure of the quality of the solution [optional]>,
"misc" : <a (comma-less) string that will be associated with the run [optional]>
}
ATTENTION: The return values (i.e., status and runtime) will overwrite the measured results of the runsolver (if runsolver was used).
'''
resultMap = {}
#TODO: parse the output of your solver which can be found in the filepointer <filepointer>
return resultMap
if __name__ == "__main__":
wrapper = EmptyWrapper()
wrapper.main()
So what happens here:
To create the spearWrapper.py, we simply copied the emptyWrapper.py first:
cp emptyWrapper.py spearWrapper.py
Then we have to make some small changes in spearWrapper.py (see code below)
#!/usr/bin/python
# encoding: utf-8
from genericWrapper import AbstractWrapper
class SpearWrapper(AbstractWrapper):
def get_command_line_args(self, runargs, config):
'''
Returns the command line call string to execute the target algorithm (here: Spear).
Args:
runargs: a map of several optional arguments for the execution of the target algorithm.
{
"instance": <instance>,
"specifics" : <extra data associated with the instance>,
"cutoff" : <runtime cutoff>,
"runlength" : <runlength cutoff>,
"seed" : <seed>
}
config: a mapping from parameter name to parameter value
Returns:
A command call list to execute the target algorithm.
'''
binary_path = "./spear"
cmd = "%s --seed %d --model-stdout --dimacs %s" %(binary_path, runargs["seed"], runargs["instance"])
for name, value in config.items():
cmd += " -%s %s" %(name, value)
return cmd
def process_results(self, filepointer, out_args):
'''
Parse a results file to extract the runs status (SUCCESS/CRASHED/etc) and other optional results.
Args:
filepointer: a pointer to the file containing the solver execution standard out.
out_args : a map with {"exit_code" : exit code of target algorithm}
Returns:
A map containing the standard AClib run results. The current standard result map as of AClib 2.06 is:
{
"status" : <"SUCCESS"/"SAT"/"UNSAT"/"TIMEOUT"/"CRASHED"/"ABORT">,
"runtime" : <runtime of target algrithm>,
"quality" : <a domain specific measure of the quality of the solution [optional]>,
"misc" : <a (comma-less) string that will be associated with the run [optional]>
}
ATTENTION: The return values will overwrite the measured results of the runsolver (if runsolver was used).
'''
import re
data = filepointer.read()
resultMap = {}
if (re.search('s SATISFIABLE', data)) or (re.search('s UNSATISFIABLE', data)):
resultMap['status'] = 'SUCCESS'
return resultMap
if __name__ == "__main__":
wrapper = SpearWrapper()
wrapper.main()
The generic wrapper framework uses runsolver from Olivier Roussel to limit the runtime and memory. examples/generic-wrapper/ has already an precompiled version of the runsolver. However, we strongly recommend to rebuild the runsolver because runsolver uses OS dependent system calls. You can find the sources of runsolver at
http://www.cril.univ-artois.fr/~roussel/runsolver/
Our generic wrapper has some own options:
python emptyWrapper.py --help
For instance, an important option is '--runsolver-path RUNSOLVER' to specify where the wrapper can find the binary of the runsolver.
SMAC calls a wrapper in the following way:
<algo> <instance> <specifics> <runtime cutoff> <runlength> <seed> [solver parameters]
So a possible call of our Spear wrapper could look like:
python spearWrapper.py --runsolver-path runsolver/runsolver example_scenarios/spear/instances/train/qcplin2006.10085.cnf "" 30.0 2147483647 1234 -sp-var-dec-heur 16 -sp-learned-clause-sort-heur 5
with:
Our example only touches the surface of what is possible with a wrapper based on our generic wrapper. For instances, as you can find in the docstring of process_results(), you can return a lot more detailed information, i.e., runtime of your algorithm, quality of the solution and some misc information for logging purposes.
ATTENTION: Every information returned by process_results() will overwrite information extracted in the runsolver output, i.e., timeouts or memouts, and runtime.
Furthermore, we strongly recommend to add some functionality in process_results() which verifies the output. SMAC will probably try configurations of your target algorithm, which you never considered before in any experiment. In our experience, bugs are often exposed in such configurations. In the best case, your target algorithm will only crash (which is also not favorable for the configuration process and can decrease the performance of the configuration process). In the worst case, SMACs selects a final incumbent with a bug.
For example, in the case of a SAT solver, the returned assignment of a satisfiable instance can be easily verified with an additional function which has to be integrated in process_results(). (This is an easy example and not the most efficient code):
def _verify_SAT(self, model, solver_output):
with open(self._instance) as fp:
for line in fp:
if line.startswith("c"):
continue
if line.startswith("p"):
continue
clause = map(int, line.split(" ")[:-1])
satisfied = False
for lit in clause:
if lit in model:
satisfied = True
if not satisfied:
return False
return True