Analysis: Context handling for predictors

[spli]

Some analysis algorithms may be applicable to all iamges and datasets. Others are designed to work on either a particular type of image, or require careful selection of a training dataset for optimal performance.

This means we need to store and manage the context of each classifier/predictor. A single algorithm may have multiple contexts, which will may include:

• List of the training images which defines the context.
• Feature weights, which indicate which features were used for the training process (this may be a subset of the complete set), and weights for each of these features.
• Additional parameters which are automatically determined during training.
• Performance information during training. Some of these fields are common between classifiers (e.g. continuous: MSE, Pearman/Spearman? coefficients/p-values, discrete: error rates, confusion matrix), some will be algorithm specific.
• Individual image predictions. Useful for checking performance, comparing classifiers, or to look for outliers.
• Also additional image specific prediction data such as confidence measures.
• Test set predictions. This may or may not be the same as the training set.
• Cross-validation performance during training. Particularly useful if the classifier is directly used to generate results for publication.
• Externally defined algorithm specific parameters.

We'd also like a way of characterising the contexts to reduce the complexity faced by a user when selecting which algorithm/context. For example, if an algorithm is designed for colour images there is no point in trying it on greyscale images.