Complete MiniZinc Models section in experiment resources appendix

README.md

@@ -15,4 +15,4 @@ This is the repository containing Jip J. Dekker's PhD thesis: "A Modern Architec
 | 5   | Incremental Processing                   | 🚧 Received feedback from Peter |
 | 6   | Conclusions                              | ⏳ Sent to Guido                |
 | A1  | MiniZinc Syntax                          | 📖 No planned work              |
-| A2  | Experiment Resources                     | 📝 To be completed              |
+| A2  | Experiment Resources                     | 📖 Awaiting final changes       |

assets/packages.tex

@@ -5,6 +5,7 @@
 \usepackage{hyperref}
 \hypersetup{bookmarksnumbered}
 \usepackage{xspace}
+\usepackage{multicol}
 
 % Fonts
 \usepackage{fontspec}

chapters/3_rewriting.tex

@@ -763,7 +763,7 @@ When the intermediate variables become unused, they will be removed using the no
 \jip{TODO:\ Aggregation is also part of the background. How can we make this more
 	specific to how it works in \nanozinc{}.}
 
-\section{Experiments}\label{sec:4-experiments}
+\section{Experiments}\label{sec:rew-experiments}
 
 We have created a prototype implementation of the architecture presented in the preceding sections.
 It consists of a compiler from \minizinc\ to \microzinc{}, and an incremental \microzinc\ interpreter producing \nanozinc{}.
@@ -773,10 +773,10 @@ We will release our implementation under an open-source license and can make it
 The implementation is not optimised for performance yet, but was created as a faithful implementation of the developed concepts, in order to evaluate their suitability and provide a solid baseline for future improvements.
 In the following we present experimental results on basic flattening performance as well as incremental flattening and solving that demonstrate the efficiency gains that are possible thanks to the new architecture.
 
-We selected 20 models from the annual \minizinc\ challenge and compiled 5 instances of each model to \flatzinc{}, using the current \minizinc\ release version 2.5.5 and the new prototype system.
-In both cases we use the standard \minizinc\ library of global constraints (\ie\ we decompose those constraints rather than using solver built-ins, in order to stress-test the flattening).
+We selected 17 models from the annual \minizinc\ challenge and compiled 5 instances of each model to \flatzinc{}, using the current \minizinc\ release version 2.5.5 and the new prototype system.
+In both cases we use the standard \minizinc\ library of global constraints (\ie\ we decompose those constraints rather than using solver primitives, in order to stress-test the flattening).
 We measured pure flattening time, \ie\ without time required to parse and type check in version 2.5.5, and without time required for compilation to \microzinc\ in the new system (compilation is usually very fast).
-Times are averages of 10 runs.\footnote{All models obtained from \url{https://github.com/minizinc/minizinc-benchmarks}: \texttt{\justify{}accap, amaze, city-position, community-detection, depot-placement, freepizza, groupsplitter, kidney-exchange, median-string, multi-knapsack, nonogram, nside, problem, rcpsp-wet, road-cons, roster, stack-cuttingstock, steelmillslab, train, triangular, zephyrus}.}
+Times are averages of 10 runs.
 
 \Cref{sfig:4-compareruntime} compares the flattening time for each of the 100 instances.
 Points below the line indicate that the new system is faster.

chapters/3_rewriting_preamble.tex

@@ -29,7 +29,7 @@ The prototype is still very experimental, but preliminary results suggest the ne
 
 This chapter is organised as follows.
 \Cref{sec:4-micronano} introduces the \microzinc\ and \nanozinc\ languages, the new intermediate representation we propose that enables more efficient flattening.
-\Cref{sec:4-simplification} describes how we can perform various processing and simplification steps on this representation, and in \cref{sec:4-experiments} we report on the experimental results of the prototype implementation.
+\Cref{sec:4-simplification} describes how we can perform various processing and simplification steps on this representation, and in \cref{sec:rew-experiments} we report on the experimental results of the prototype implementation.
 
 \begin{figure}
 	\centering

chapters/5_incremental.tex

@@ -512,7 +512,7 @@ The system can therefore support solvers with different levels of an incremental
 \end{itemize}
 
 
-\section{Experiments}\label{sec:6-experiments}
+\section{Experiments}\label{sec:inc-experiments}
 
 We have created a prototype implementation of the architecture presented in the preceding sections.
 It consists of a compiler from \minizinc\ to \microzinc{}, and an incremental \microzinc\ interpreter producing \nanozinc{}.

chapters/5_incremental_preamble.tex

@@ -41,6 +41,5 @@ The rest of the chapter is organised as follows.
 \Cref{sec:6-modelling} discusses the declarative modelling of restart-based \gls{meta-search} algorithms that can be modelled directly in a \cml{}.
 \Cref{sec:6-solver-extension} introduces the method to compile these \gls{meta-search} specifications into efficient solver-level specifications that only require a small extension of existing \glspl{solver}.
 \Cref{sec:6-incremental-compilation} introduces the alternative method that extends the \gls{constraint-modelling} infrastructure with an interface to add and remove constraints from an existing model while avoiding recompilation.
-\Cref{sec:6-experiments} reports on the experimental results of both approaches.
-Finally, \Cref{sec:6-conclusion} presents the conclusions.
+\Cref{sec:inc-experiments} reports on the experimental results of both approaches.
 

chapters/A2_benchmark.tex

@@ -156,7 +156,114 @@ The source code for this version can be found here:
 \section{MiniZinc Models}%
 \label{sec:bench-models}
 
-\paragraph{MiniZinc Challenge} 
+In the following paragraphs, we introduce the origins of the models used in the experiments of this thesis.
+
+\paragraph{MiniZinc Challenge} Most of the models used in the experiments in this thesis stem from the \minizinc{} Challenge \autocite{stuckey-2010-challenge,stuckey-2014-challenge}.
+The \minizinc{} Challenge is a yearly competition, held since 2008, in which \minizinc{} \solvers{} compete against each other in solving 100 \minizinc{} instances.
+\Glspl{solver} are scored based on their performance solving each instance: time to solution, quality of solution, and proof of optimality.
+The 100 instances stem from 20 \minizinc{} models, 10 of which are guaranteed to have never been used in earlier challenges.
+
+A repository of all \minizinc{} models previously used in the challenges and accompanying data files can be found here:
+
+\begin{center}
+	\url{https://github.com/minizinc/minizinc-benchmarks}
+\end{center}
+
+The flattening experiment presented in \cref{sec:rew-experiments} use a selection of seventeen models originating from the \minizinc{} challenge.
+These models can identified using the following labels:
+
+\begin{multicols}{3}
+	\ttfamily
+	\begin{itemize}
+		\item accap
+		\item amaze
+		\item city-position
+		\item community-detection
+		\item depot-placement
+		\item freepizza
+		\item groupsplitter
+		\item kidney-exchange
+		\item multi-knapsack
+		\item nonogram
+		\item nside
+		\item rcpsp-wet
+		\item road-cons
+		\item roster
+		\item stack-cuttingstock
+		\item steelmillslab
+		\item triangular
+	\end{itemize}
+\end{multicols}
+
+The large scale \gls{half-reif} experiment presented in \cref{sec:half-experiments} uses the \minizinc{} instances from 2019 and 2020.
+The 2019 \minizinc{} Challenge included the models identified by the following labels:
+
+\begin{multicols}{3}
+	\ttfamily
+	\begin{itemize}
+		\item accap
+		\item amaze
+		\item code-generator
+		\item fox-geese-corn 
+		\item groupsplitter
+		\item hrc
+		\item kidney-exchange
+		\item {\scriptsize{} liner-sf-repositioning}
+		\item lot-sizing
+		\item median-string
+		\item multi-knapsack
+		\item nside
+		\item ptv
+		\item rcpsp-wet-diverse
+		\item rotating-workforce
+		\item stack-cuttingstock
+		\item steelmillslab
+		\item stochastic-vrp
+		\item triangular
+		\item zephyrus
+	\end{itemize}
+\end{multicols}
+
+\noindent{}The 2020 \minizinc{} Challenge included the models identified by the following labels:
+
+\begin{multicols}{3}
+	\ttfamily
+	\begin{itemize}
+		\item bnn-planner
+		\item cable\_tree\_wiring
+		\item code-generator
+		\item {\scriptsize{} collaborative-construction}
+		\item gbac
+		\item hoist-benchmark
+		\item is
+		\item lot-sizing
+		\item {\scriptsize{} minimal-decision-sets}
+		\item p1f-pjs
+		\item pentominoes
+		\item pillars-and-planks
+		\item racp
+		\item radiation
+		\item sdn-change 
+		\item skill-allocation
+		\item soccer-computational
+		\item stable-goods
+		\item tower\_challenge
+		\item whirlpool
+	\end{itemize}
+\end{multicols}
+
+The incremental experiments presented in \cref{sec:inc-experiments} are based on four models from the \minizinc{} Challenge:
+
+\begin{multicols}{2}
+	\ttfamily
+	\begin{itemize}
+		\item gbac
+		\item radiation
+		\item steelmillslab
+		\item rcpsp-wet
+	\end{itemize}
+\end{multicols}
+
 
 \paragraph{Prize Collecting Path} This problem was introduced in by Feydy et al.\ in the original paper on \gls{half-reif} \autocite{feydy-2011-half-reif}.
 In this thesis it is used to benchmark the implementation of a propagator for the \gls{half-reif} of the \mzninline{element} constraint.