Statistical Measures
Azure ML Evaluation results often include some statistical measures that need some explanation.
Here is a brief summary:-
For Classifiers
Measure | Description |
---|---|
True Positive | A count of the number of positive outcomes that the algorithm predicted correctly (TP) |
True Negative | A count of the number of negative outcomes that the algorithm predicted correctly (TN) |
False Positive | A count of the number of positive outcomes that the algorithm predicted incorrectly (FP) |
False Negative | A count of the number of negative outcomes that the algorithm predicted incorrectly (FN) |
Precision | The proportion of predicted positives that are classified correctly: TP/(TP+FP) |
Recall | The proportion of actual positives which are classified correctly: TP/(TP+FN) |
Accuracy | The proportion of all values classified correctly: (TP+TN)/(TP+TN+FP+FN). Accuracy is not a reliable metric for the real performance of a classifier. |
F1 Score | The F1 score is the harmonic mean of precision and recall: F1 = 2 * ((precision/recall)/(precision+recall)). The F1 Score is a good metric for the real performance of a classifier since it includes both precision and recall. |
AUC | Area Under the Curve: This is the are under the Receiver Operating Characteristic (ROC) curve. This is the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance. The AUC is a good metric for the real performance of a classifier since it includes both precision and recall. |
For Regression
Measure | Description |
---|---|
Negative Loglikelihood | The Negative Loglikelihood is a measure of the variance of the actual data from the predicted values. A regression model attempts to reach the lowest Negative Loglikelihood. A low value indicates a well trained model. |
Mean Absolute Error | A low value indicates a well trained model. |
Root Mean Squared Error | A low value indicates a well trained model. |
Relative Absolute Error | A low value indicates a well trained model. |
Relative Squared Error | A low value indicates a well trained model. |
Coefficient of Determination (R2) | A statistical measure of how well the regression line approximates the real data points. The coefficient of determination ranges from 0 to 1. An R2 of 1 indicates that the regression line perfectly fits the data, but low values can be entirely normal. |
For Clustering
It's not clear to me yet what best indicates a well trained Clustering model.
Measure | Description |
---|---|
Average Distance to Cluster Center | The average closeness of all points in a cluster to the centroid of that cluster. |
Average Distance to Other Center | The average closeness of all points in a cluster to the centroid of all clusters. |
Number of Points | The number of points in that cluster |
Maximal Distance To Cluster Center | The sum of the distances between each point and the centroid of that point’s cluster. |
No comments:
Post a Comment