191 lines
7.6 KiB
TeX
191 lines
7.6 KiB
TeX
%************************************************
|
|
\chapter{Experiment Resources}%
|
|
\label{ch:benchmarks}
|
|
%************************************************
|
|
|
|
\noindent{}All experiments included in this thesis were conducted on a dedicated node in a computational cluster.
|
|
The machine operates using a \textbf{Intel Xeon 8260} \gls{cpu}, which has 24 non-hyperthreaded cores, and has access to \textbf{268.55 GB} of \gls{ram}.
|
|
Each experimental test was given exclusive access to a single \gls{cpu} core and access to sufficient \gls{ram}.
|
|
|
|
\section{Experimental Design}%
|
|
\label{sec:bench-env}
|
|
|
|
To promote the verification and recreation of our experiments, we have publish our experimental design in the form of several bundles.
|
|
Each bundle contains the exact versions of the software, models, and data that were used.
|
|
They are accompanied with instructions to perform the experiments.
|
|
|
|
The experiments are split into three bundles, each testing a different system.
|
|
Each part is published as a \gls{git} repository on GitHub.
|
|
|
|
\paragraph{MiniZinc Prototype} The design for the experiments of the \minizinc{} prototype implementation can be found in:
|
|
|
|
\begin{center}
|
|
\url{file://TODO}
|
|
\end{center}
|
|
|
|
This repository contains the flattening, recursive function, and the incremental flattening benchmarks.
|
|
|
|
\paragraph{Half Reification} The design for all the \gls{half-reif} experiments is located in:
|
|
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/half-reif-benchmarks}
|
|
\end{center}
|
|
|
|
This repository contains both the benchmark for the \gls{chuffed} propagators of \gls{half-reif} constraints and the \minizinc{} Challenge 2019 \& 2020 benchmark.
|
|
|
|
\paragraph{Restarting Meta-Search} Finally, the design of the experiments for restart based \gls{meta-search} can be found in:
|
|
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/on-restart-benchmarks}
|
|
\end{center}
|
|
|
|
This repository contains the benchmarks for both \gls{gecode} and \gls{chuffed}.
|
|
|
|
|
|
\section{Software}%
|
|
\label{sec:bench-soft}
|
|
|
|
Central to the experiments in this thesis are two kinds of programs: programs that rewrite a high-level \constraint{} model into a \solver{} specification and the \solvers{} that given a specification search for solutions.
|
|
We generally test the rewriting programs, in particular the \minizinc{} flattener, but use the \solvers{} to evaluate the effect of changes to these rewriting programs.
|
|
|
|
\subsection{MiniZinc Flattener}
|
|
|
|
In this thesis we use three different versions of the \minizinc\ flattening tool:
|
|
|
|
\begin{itemize}
|
|
|
|
\item The officially released MiniZinc flattener (version 2.5.5) \autocite{minizinc-2021-minizinc}.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/MiniZinc/libminizinc/tree/2.5.5}
|
|
\end{center}
|
|
|
|
\item A prototype for the language architecture designed as part of this thesis.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{file://TODO}
|
|
\end{center}
|
|
|
|
\item The official MiniZinc flattener (version 2.5.5) \autocite{minizinc-2021-minizinc}, adjusted with the restart-based mechanisms.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/libminizinc/tree/feature/on_restart}
|
|
\end{center}
|
|
|
|
\end{itemize}
|
|
|
|
\subsection{MiniZinc Solvers}
|
|
|
|
In this thesis we use the following \solvers{}:
|
|
|
|
\paragraph{Chuffed} is an open source \gls{lcg} \solver{} \cite{chuffed-2021-chuffed}.
|
|
In this thesis we use three different versions of the \gls{chuffed} \solver{}:
|
|
|
|
\begin{itemize}
|
|
|
|
\item The official development version of \gls{chuffed}.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/chuffed/chuffed/tree/develop}
|
|
\end{center}
|
|
|
|
\item An extension of \gls{chuffed} that includes propagators for \glspl{half-reif} of the \mzninline{all_different} and \mzninline{element} \constraints{}.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/chuffed/tree/feature/imp_globals}
|
|
\end{center}
|
|
|
|
\item An extension of \gls{chuffed} that includes propagators for restart-based meta-search.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/chuffed/tree/feature/on_restart}
|
|
\end{center}
|
|
|
|
\end{itemize}
|
|
|
|
\paragraph{Coin-OR Branch-and-Cut} is an open source \gls{mip} \solver{} \autocite{forrest-2020-cbc}.
|
|
In this thesis we use version 2.10.5 of this solver.
|
|
Its source code is available at
|
|
|
|
\begin{center}
|
|
\url{https://github.com/coin-or/Cbc/tree/releases/2.10.5}
|
|
\end{center}
|
|
|
|
\paragraph{Gecode} is an open source \gls{cp} solver \autocite{gecode-2021-gecode}.
|
|
In this thesis we use four different versions of the \gls{gecode} solver:
|
|
\begin{itemize}
|
|
|
|
\item The official release of \gls{gecode}, version 6.3.0.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Gecode/gecode/tree/release-6.3.0}
|
|
\end{center}
|
|
|
|
\item An extension of \gls{gecode} that includes propagators for restart-based meta-search.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/gecode/tree/feature/on_restart}
|
|
\end{center}
|
|
|
|
\item An extension of \gls{gecode} that is able to record the search process of restart-based meta-search.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/gecode/tree/feature/on_restart_record}
|
|
\end{center}
|
|
|
|
\item An extension of \gls{gecode} that is able to replay the search process of restart-based meta-search.
|
|
The source code for this version can be found here:
|
|
\begin{center}
|
|
\url{https://github.com/Dekker1/gecode/tree/feature/on_restart_replay}
|
|
\end{center}
|
|
|
|
\end{itemize}
|
|
|
|
\paragraph{IBM CPLEX} is a proprietary \gls{mip} \solver{} \autocite{cplex-2020-cplex}.
|
|
In this thesis we use \gls{cplex} version 20.1 under an academic license in our experiments.
|
|
|
|
\paragraph{OpenWBO} is a open source \gls{maxsat} \solver{} \autocite{martins-2014-openwbo}.
|
|
In this thesis we use version \jip{xxx} in our experiments.
|
|
The source code for this version can be found here:
|
|
|
|
\begin{center}
|
|
\url{https://github.com/sat-group/open-wbo}
|
|
\end{center}
|
|
|
|
|
|
\section{MiniZinc Models}%
|
|
\label{sec:bench-models}
|
|
|
|
\paragraph{MiniZinc Challenge}
|
|
|
|
\paragraph{Prize Collecting Path} This problem was introduced in by Feydy et al.\ in the original paper on \gls{half-reif} \autocite{feydy-2011-half-reif}.
|
|
In this thesis it is used to benchmark the implementation of a propagator for the \gls{half-reif} of the \mzninline{element} constraint.
|
|
This experiment is shown in \cref{sec:half-experiments}.
|
|
\Cref{lst:bench-prize} shows a \minizinc{} model that can be used to solve the problem.
|
|
The data and original model are available at:
|
|
|
|
\begin{center}
|
|
\url{https://people.eng.unimelb.edu.au/pstuckey/half/}
|
|
\end{center}
|
|
|
|
\begin{listing}
|
|
\mznfile{assets/mzn/prize.mzn}
|
|
\caption{\label{lst:bench-prize} A \minizinc{} model for the Prize Collecting Path problem}
|
|
\end{listing}
|
|
|
|
\paragraph{QCP-Max} This problem was introduced in by Feydy et al.\ in the original paper on \gls{half-reif} \autocite{feydy-2011-half-reif}.
|
|
In this thesis it is used to benchmark the implementation of a propagator for the \gls{half-reif} of the \mzninline{all_different} constraint.
|
|
This experiment is shown in \cref{sec:half-experiments}.
|
|
\Cref{lst:bench-qcpmax} shows a \minizinc{} model that can be used to solve the problem.
|
|
The data and original model are available at:
|
|
|
|
\begin{center}
|
|
\url{https://people.eng.unimelb.edu.au/pstuckey/half/}
|
|
\end{center}
|
|
|
|
\begin{listing}
|
|
\mznfile{assets/mzn/qcp_max.mzn}
|
|
\caption{\label{lst:bench-qcpmax} A \minizinc{} model for the QCP-Max problem}
|
|
\end{listing}
|
|
|