1. Travelling Salesman Problem
The traveling salesman problem (TSP) seeks to find the shortest route between a set of points
(cities) that a salesperson must visit exactly once before returning their starting city.
The goal is to minimise the distance travelled by the salesperson, as a proxy measure for
costs incurred. Our studies have focused on Euclidean TSPs, and the performance of two wellknown
variants of the LinKernighan heuristic: one that exploits cluster structure in a TSP instance (set of cities),
and one that doesn't. The instance space analysis supports the intuition that the heuristic that
exploits cluster structure is more powerful, except on instances where little cluster structure exists.
The test instances used to build the instance space include randomly generated 100node TSPs, and instances
that have been evolved to be hard or easy for the two heuristics, as described in the papers below.
Research Publications 
Downloads 
Instance Space Analysis 
1. SmithMiles, K. A., van Hemert, J., Lim, Y., “Understanding TSP Difficulty by Learning from Evolved Instances”, Proceedings of Learning and Intelligent Optimization, LION 4, Lecture Notes in Computer Science, vol. 6073, pp. 266280, 2010.
2. SmithMiles, K. and van Hemert, J. “Discovering the Suitability of Optimisation Algorithms by Learning from Evolved Instances”, Annals of Mathematics and Artificial Intelligence, vol. 61, no. 2, pp. 87104, 2011.
3. SmithMiles, K. and Tan, T. “Measuring Algorithm Footprints in Instance Space”, in Proceedings of the 2012 IEEE Congress on Evolutionary Computation, pp. 34463453, 2012

Instances
Metadata
Feature Extraction Code


2. Graph Coloring Problem
In graph theory, graph or vertex colouring aims to colour the vertices of a graph
such that no two adjacent vertices share the same color. Our 2014 study considered the
benchmark graphs commonly used in computational studies
of graph colouring algorithms, as well as a collection of eight stateoftheart heuristics.
The instance space reveals pockets where the "onaveragebest" performing algorithm is not as well suited as other
algorithms, and the unique strengths and weaknesses of each algorithm can be visualised. Our extended study in 2015,
focusing on 100node graphs,
showed how new graphs could be evolved to fill the instance space, challenging the conclusions that are formed
when only graphs from existing generators are studied.
Research Publications 
Downloads 
Instance Space Analysis 
SmithMiles, K. A., Baatar, D., Wreford, B. and Lewis, R., “Towards Objective Measures of Algorithm Performance across Instance Space”, Computers and Operations Research, vol. 45, pp. 1224, 2014.

Metadata
Code for Feature Selection


SmithMiles, K. A. and Bowly, S., "Generating New Test Instances by Evolving in Instance Space", Computers & Operations Research, vol. 63, pp. 102113, 2015. 
Instances
Metadata
Code for Feature Selection

Coming Soon 
3. Knapsack Problem
Given a set of items, each with a weight and a value, the Knapsack Problem aims to determine the number of
each item to select so that the total value of the knapsack is as large as possible, while the total weight
meets the maximum capacity constraint. Our studies have focused on the 01 knapsack problem, where each
item is either selected or not, and multiple copies of each item are not considered. We have constructed
an instance space of well known benchmark instances aiming to show where the hard instances lie, in response to
efforts to generate harder knapsack instances.
4. Linear Programming
Linear programming is an optimization technique for finding the optimal decision variables that maximise or
minimise a linear objective function subject to a set of linear inequality or equality constraints. Our
studies have focused on understanding how the various strategies used in simplex algorithms are influenced by
the properties of the constraint set. We have also developed methods for generating feasible bounded instances to
enable a more diverse set of instances to be considered than existing benchmark problems offer.
Research Publications 
Downloads 
Instance Space Analysis 

Coming Soon 
Coming Soon 
5. Mixed Integer Programming
Mixed integer programming extends linear programming by constraining some variables to only integer or binary values.
Our studies have focused on generating small instances which are challenging for branch variable selection methods.
Specifically, we evolve instances which show a large discrepancy in the size of the branch and bound tree generated by
different branch variable selection methods.
Research Publications 
Downloads 
Instance Space Analysis 

Coming Soon 
Coming Soon 
6. BlackBox Optimization
The goal in a continuous optimization problem is to maximize or minimize a function whose input and output
variables are real numbers, subject to some constraints. Often, these problems lack an algebraic
expression, have noncalculable derivatives, exhibit uncertainty or noise, or may involve timeconsuming
simulations or experiments. All that is known is a blackbox output response to a given input. Therefore,
sampling the inputs and obtaining the output response is the only way to acquire information about the problem.
This approach is known as blackbox optimization (BBO). Our studies have focused on both singleobjective and
multiobjective BBO problems. For singleobjective BBO we have constructed an instance space of the wellstudied COCO
benchmarks, and then filled the instance space by evolving new test instances exhibiting a wide variety of problem features
and landscapes. Our ongoing work in multiobjective BBO is developing features that can characterise the difficulties of
MOO problems, and the robustness of existing test instance benchmarks.
Research Publications 
Downloads 
Instance Space Analysis 
Muñoz, M. A. and SmithMiles, K. A., "Performance analysis of continuous blackbox optimization algorithms via footprints in instance space", Evolutionary Computation, vol. 25, no. 4, pp. 529554, 2017.

Metadata
Instances
Code for Feature Selection


M.A. Muñoz and K. SmithMiles. Generating new spacefilling test instances for continuous blackbox optimization. Evolutionary Computation, vol. 28, no. 3, pp. 379404, 2020.

Instances
2D Contour Plot Images
Code for Feature Selection

Coming Soon

6. Job Shop Scheduling
The EarlyTardy (E/T) Scheduling Problem is motivated by JustinTime production, which required delivery
of goods to be made at the time required. Both early and late production are discouraged, as early production
incurs holding costs, and late delivery means a loss of customer goodwill. The problem we consider is the single
machine, distinct due date, early/tardy scheduling problem, where each job has an earliness and tardiness penalty
and due date. Once a job is dispatched on the machine, it runs to completion with no
interruptions permitted. The objective is to minimise the total penalty produced when each job is scheduled as
close as possible to its due date. We evaluate the performance of two of the simplest and most commonly used dispatching
heuristics for the E/T scheduling problem:Earliest Due Date and Shortest Processing Time heuristics. The instance space analysis
provides visual confirmation of the relationships discovered in a rulebased machine learning analysis in the 2009 paper below, and
identifies the conditions under which each heuristic is better suited.
Research Publications 
Downloads 
Instance Space Analysis 
SmithMiles, K. A., James, R. J. W., Giffin, J. W. and Tu, Y., “A Knowledge Discovery Approach
to Understanding Relationships between Scheduling Problem Structure and Heuristic Performance”,
Selected papers of the 3rd International Conference on Learning and Intelligent Optimization,
Lecture Notes in Computer Science, vol. 5851, pp. 89103, 2009. 
Metadata


7. Timetabling
Udine Timetabling, also known as the curriculumbased course timetabling problem, was the subject of
track 3 of the 2007 International Timetabling Competition (ITC2007). 21 realworld instances from the
University of Udine were available. In a series of studies below, we have taken the top 2 performing
algorithms in ITC2007  simulated annealing with constraint propagation (SACP) and tabu search over weighted
constraint satisfaction problem (TSCS) and examined their performance on instances from a variety of sources.
Besides the 21 competition realworld instances, we used instances from a random generator of Burke et al. (2010),
and instances we generated by using machine learning to adapt the parameters of this random generator to create
instances that are i) realworld like, and ii) more discriminating of unique algorithm performance. A total of
8199 instances from these three classes are contained in the metadata, along with 32 timetabling features.
Applying instance space analysis now to these previous studies supports the findings in the papers below
(which used decision trees and selforganising maps at that time). There are features such as slack that
determine unique algorithm behaviour, other features such as properties of the teacher connectivity graph that
determine tied behaviours, and features that explain the key differences between the randomly generated instances
and realworld instances.
Research Publications 
Downloads 
Instance Space Analysis 
SmithMiles, K. A.,
and Lopes, L. B., “Generalising Algorithm Performance in Instance Space: A timetabling
case study”, Proceedings of the 5th Learning and Intelligent Optimization conference, LION 5,
Lecture Notes in Computer Science, vol. 6683, pp. 524539, Springer, 2011.
Lopes, L. and SmithMiles, K., “Generating Applicable Synthetic Instances
for Branch Problems”, Operations Research, vol. 61, no. 3, pp. 563577, 2013.
Lopes, L. B. and SmithMiles, K. A., “Pitfalls in Instance Generation for Udine Timetabling”, Proceedings of the 4th Learning and Intelligent Optimization conference, LION 4, Lecture Notes in Computer Science, vol. 6073, pp. 299302, Springer, 2010.

Metadata
Summary of Instance Space Analysis

