Wednesday, October 19, 2016

Statistical Measures

Azure ML Evaluation results often include some statistical measures that need some explanation. 
Here is a brief summary:-

For Classifiers

MeasureDescription
True PositiveA count of the number of positive outcomes that the algorithm predicted correctly (TP)
True NegativeA count of the number of negative outcomes that the algorithm predicted correctly (TN)
False PositiveA count of the number of positive outcomes that the algorithm predicted incorrectly (FP)
False NegativeA count of the number of negative outcomes that the algorithm predicted incorrectly (FN)
PrecisionThe proportion of predicted positives that are classified correctly: TP/(TP+FP)
RecallThe proportion of actual positives which are classified correctly: TP/(TP+FN)
AccuracyThe proportion of all values classified correctly: (TP+TN)/(TP+TN+FP+FN). Accuracy is not a reliable metric for the real performance of a classifier.
F1 ScoreThe F1 score is the harmonic mean of precision and recall: F1 = 2 * ((precision/recall)/(precision+recall)). The F1 Score is a good metric for the real performance of a classifier since it includes both precision and recall.
AUCArea Under the Curve: This is the are under the Receiver Operating Characteristic (ROC) curve. This is the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance. The AUC is a good metric for the real performance of a classifier since it includes both precision and recall.

For Regression

MeasureDescription
Negative LoglikelihoodThe Negative Loglikelihood is a measure of the variance of the actual data from the predicted values. A regression model attempts to reach the lowest Negative Loglikelihood. A low value indicates a well trained model.
Mean Absolute ErrorA low value indicates a well trained model.
Root Mean Squared ErrorA low value indicates a well trained model.
Relative Absolute ErrorA low value indicates a well trained model.
Relative Squared ErrorA low value indicates a well trained model.
Coefficient of Determination (R2)A statistical measure of how well the regression line approximates the real data points. The coefficient of determination ranges from 0 to 1. An R2 of 1 indicates that the regression line perfectly fits the data, but low values can be entirely normal.

For Clustering

It's not clear to me yet what best indicates a well trained Clustering model.
MeasureDescription
Average Distance to Cluster CenterThe average closeness of all points in a cluster to the centroid of that cluster.
Average Distance to Other CenterThe average closeness of all points in a cluster to the centroid of all clusters.
Number of PointsThe number of points in that cluster
Maximal Distance To Cluster CenterThe sum of the distances between each point and the centroid of that point’s cluster.

No comments:

Post a Comment