Method Benchmarks and Measures

mini_proj/report/references.bib

@@ -0,0 +1,17 @@
+@article{Kotsiantis2007,
+abstract = {Supervised machine learning is the search for algorithms that reason from externally supplied instances to produce general hypotheses, which then make predictions about future instances. In other words, the goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various supervised machine learning classification techniques. Of course, a single article cannot be a complete review of all supervised machine learning classification algorithms (also known induction classification algorithms), yet we hope that the references cited will cover the major theoretical issues, guiding the researcher in interesting research directions and suggesting possible bias combinations that have yet to be explored.},
+author = {Kotsiantis, Sotiris B.},
+doi = {10.1115/1.1559160},
+file = {:home/kelvin/.local/share/data/Mendeley Ltd./Mendeley Desktop/Downloaded/Kotsiantis - 2007 - Supervised machine learning A review of classification techniques.pdf:pdf},
+isbn = {1586037803},
+issn = {09226389},
+journal = {Informatica},
+keywords = {algorithms analysis classifiers computational conn,classifiers,data mining techniques,intelligent data analysis,learning algorithms},
+mendeley-groups = {CS Proj/ML,CS Proj,Thesis,Thesis/ML},
+pages = {249--268},
+title = {{Supervised machine learning: A review of classification techniques}},
+url = {http://books.google.com/books?hl=en{\&}lr={\&}id=vLiTXDHr{\_}sYC{\&}oi=fnd{\&}pg=PA3{\&}dq=survey+machine+learning{\&}ots=CVsyuwYHjo{\&}sig=A6wYWvywU8XTc7Dzp8ZdKJaW7rc{\%}5Cnpapers://5e3e5e59-48a2-47c1-b6b1-a778137d3ec1/Paper/p800{\%}5Cnhttp://www.informatica.si/PDF/31-3/11{\_}Kotsiantis - S},
+volume = {31},
+year = {2007}
+}
+

mini_proj/report/waldo.tex

@@ -32,8 +32,60 @@
 
 		\section{Background}
 
+      This paper is mad \cite{Kotsiantis2007}.
+
 		\section{Methods}
 
+      % Kelvin Start
+      \subsection{Benchmarking}\label{benchmarking}
+
+      In order to benchmark the Neural Networks, the performance of these
+      algorithms are evaluated against other Machine Learning algorithms. We
+      use Support Vector Machines, K-Nearest Neighbours (\(K=5\)), Gaussian
+      Naive Bayes and Random Forest classifiers, as provided in Scikit-Learn.
+
+      \subsection{Performance Metrics}\label{performance-metrics}
+
+      To evaluate the performance of the models, we record the time taken by
+      each model to train, based on the training data and statistics about the
+      predictions the models make on the test data. These prediction
+      statistics include:
+
+      \begin{itemize}
+      \tightlist
+      \item
+        \textbf{Accuracy:}
+        \[a = \dfrac{|correct\ predictions|}{|predictions|} = \dfrac{tp + tn}{tp + tn + fp + fn}\]
+      \item
+        \textbf{Precision:}
+        \[p = \dfrac{|Waldo\ predicted\ as\ Waldo|}{|predicted\ as\ Waldo|} = \dfrac{tp}{tp + fp}\]
+      \item
+        \textbf{Recall:}
+        \[r = \dfrac{|Waldo\ predicted\ as\ Waldo|}{|actually\ Waldo|} = \dfrac{tp}{tp + fn}\]
+      \item
+        \textbf{F1 Measure:} \[f1 = \dfrac{2pr}{p + r}\] where \(tp\) is the
+        number of true positives, \(tn\) is the number of true negatives,
+        \(fp\) is the number of false positives, and \(tp\) is the number of
+        false negatives.
+      \end{itemize}
+
+      Accuracy is a common performance metric used in Machine Learning,
+      however in classification problems where the training data is heavily
+      biased toward one category, sometimes a model will learn to optimize its
+      accuracy by classifying all instances as one category. I.e. the
+      classifier will classify all images that do not contain Waldo as not
+      containing Waldo, but will also classify all images containing Waldo as
+      not containing Waldo. Thus we use, other metrics to measure performance
+      as well.
+
+      \emph{Precision} returns the percentage of classifications of Waldo that
+      are actually Waldo. \emph{Recall} returns the percentage of Waldos that
+      were actually predicted as Waldo. In the case of a classifier that
+      classifies all things as Waldo, the recall would be 0. \emph{F1-Measure}
+      returns a combination of precision and recall that heavily penalises
+      classifiers that perform poorly in either precision or recall.
+    % Kelvin End
+
 		\section{Results}
 
 		\section{Discussion and Conclusion}