Statistical Measures
Azure ML Evaluation results often include some statistical measures that need some explanation.
Here is a brief summary:-
For Classifiers
| Measure | Description |
|---|---|
| True Positive | A count of the number of positive outcomes that the algorithm predicted correctly (TP) |
| True Negative | A count of the number of negative outcomes that the algorithm predicted correctly (TN) |
| False Positive | A count of the number of positive outcomes that the algorithm predicted incorrectly (FP) |
| False Negative | A count of the number of negative outcomes that the algorithm predicted incorrectly (FN) |
| Precision | The proportion of predicted positives that are classified correctly: TP/(TP+FP) |
| Recall | The proportion of actual positives which are classified correctly: TP/(TP+FN) |
| Accuracy | The proportion of all values classified correctly: (TP+TN)/(TP+TN+FP+FN). Accuracy is not a reliable metric for the real performance of a classifier. |
| F1 Score | The F1 score is the harmonic mean of precision and recall: F1 = 2 * ((precision/recall)/(precision+recall)). The F1 Score is a good metric for the real performance of a classifier since it includes both precision and recall. |
| AUC | Area Under the Curve: This is the are under the Receiver Operating Characteristic (ROC) curve. This is the probability that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative instance. The AUC is a good metric for the real performance of a classifier since it includes both precision and recall. |
For Regression
| Measure | Description |
|---|---|
| Negative Loglikelihood | The Negative Loglikelihood is a measure of the variance of the actual data from the predicted values. A regression model attempts to reach the lowest Negative Loglikelihood. A low value indicates a well trained model. |
| Mean Absolute Error | A low value indicates a well trained model. |
| Root Mean Squared Error | A low value indicates a well trained model. |
| Relative Absolute Error | A low value indicates a well trained model. |
| Relative Squared Error | A low value indicates a well trained model. |
| Coefficient of Determination (R2) | A statistical measure of how well the regression line approximates the real data points. The coefficient of determination ranges from 0 to 1. An R2 of 1 indicates that the regression line perfectly fits the data, but low values can be entirely normal. |
For Clustering
It's not clear to me yet what best indicates a well trained Clustering model.
| Measure | Description |
|---|---|
| Average Distance to Cluster Center | The average closeness of all points in a cluster to the centroid of that cluster. |
| Average Distance to Other Center | The average closeness of all points in a cluster to the centroid of all clusters. |
| Number of Points | The number of points in that cluster |
| Maximal Distance To Cluster Center | The sum of the distances between each point and the centroid of that point’s cluster. |
No comments:
Post a Comment