paddlets.metrics.metrics

class MSE(mode: str = 'normal')[source]

Bases: Metric

Mean Squared Error.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Mean squared error regression loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

Mean squared error regression loss. A non-negative floating point value (the best value is 0.0).

Return type

float

class MAE(mode: str = 'normal')[source]

Bases: Metric

Mean Absolute Error.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Mean absolute error regression loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

Mean absolute error regression loss. A non-negative floating point value (the best value is 0.0).

Return type

float

class LogLoss(mode: str = 'normal')[source]

Bases: Metric

Log loss or cross-entropy loss.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Log loss or cross-entropy loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) labels.

  • y_score (np.ndarray) – Predicted probabilities.

Returns

Log loss or cross-entropy loss.

Return type

float

class QuantileLoss(q_points: List[float] = [0.1, 0.5, 0.9], quantile_level: Optional[Union[ndarray, List[float]]] = None, mode: str = 'prob')[source]

Bases: Metric

Quantile loss, following the article: Bayesian Intermittent Demand Forecasting for Large Inventories .

A quantile of q=0.5 will give half of the mean absolute error as it is calcualted as max(q * (y-y_pred), (1-q) * (y_pred-y)).

Parameters

  • q_points (List[float]) – The quantile points of interest, the default value is None. In the evaluation of the prediction, while q_points is specified, output a dict which contains each quantile result respect to quantile points.

  • mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

metric_fn(y_true: ndarray, y_pred_sample: ndarray) → Union[float, dict][source]

Quantile loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) labels.

  • y_score (np.ndarray) – Predicted quantiles.

Returns

Quantile loss.

Return type

float

class ACC(mode: str = 'anomaly')[source]

Bases: Metric

Accuracy_score.

Parameters

mode (str) – Supported metric modes, only anomaly is valid value.

_NAME

Metric name.

Type

str

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Accuracy_score.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

accuracy_score. A non-negative floating point value (the best value is 1.0).

Return type

float

class Precision(mode: str = 'anomaly')[source]

Bases: Metric

Precision_score.

Parameters

mode (str) – Supported metric modes, only anomaly is valid value.

_NAME

Metric name.

Type

str

metric_fn(y_true: ndarray, y_score: ndarray, **kwargs) → float[source]

Precision_score.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

precision_score. A non-negative floating point value (the best value is 1.0).

Return type

float

class Recall(mode: str = 'anomaly')[source]

Bases: Metric

Recall_score.

Parameters

mode (str) – Supported metric modes, only anomaly is valid value.

_NAME

Metric name.

Type

str

metric_fn(y_true: ndarray, y_score: ndarray, **kwargs) → float[source]

Recall_score.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

recall_score. A non-negative floating point value (the best value is 1.0).

Return type

float

class F1(mode: str = 'anomaly')[source]

Bases: Metric

F1_score.

Parameters

mode (str) – Supported metric modes, only anomaly is valid value.

_NAME

Metric name.

Type

str

metric_fn(y_true: ndarray, y_score: ndarray, **kwargs) → float[source]

F1_score.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

f1_score. A non-negative floating point value (the best value is 1.0).

Return type

float

class MetricContainer(metrics: Union[List[str], List[Metric]], prefix: str = '')[source]

Bases: object

Container holding a list of metrics.

Parameters

  • metrics (List[str]|List[Metric]) – List of metric or metric names.

  • prefix (str) – Prefix of metric names.

_prefix

Prefix of metric names.

Type

str

_metrics

List of metric instance.

Type

List[Metric]

_names

List of metric names associated with eval_name.

Type

List[str]