paddlets.metrics.metrics

class MSE(mode: str = 'normal')[source]

Bases: Metric

Mean Squared Error.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Mean squared error regression loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

Mean squared error regression loss. A non-negative floating point value (the best value is 0.0).

Return type

float

class MAE(mode: str = 'normal')[source]

Bases: Metric

Mean Absolute Error.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Mean absolute error regression loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) target values.

  • y_score (np.ndarray) – Estimated target values.

Returns

Mean absolute error regression loss. A non-negative floating point value (the best value is 0.0).

Return type

float

class LogLoss(mode: str = 'normal')[source]

Bases: Metric

Log loss or cross-entropy loss.

Parameters

mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

_NAME

Metric name.

Type

str

_MAXIMIZE

Identify optimization direction.

Type

bool

metric_fn(y_true: ndarray, y_score: ndarray) → float[source]

Log loss or cross-entropy loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) labels.

  • y_score (np.ndarray) – Predicted probabilities.

Returns

Log loss or cross-entropy loss.

Return type

float

class QuantileLoss(q_points: Optional[List[float]] = None, quantile_level: Union[ndarray, List[float]] = array([0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18, 0.19, 0.2, 0.21, 0.22, 0.23, 0.24, 0.25, 0.26, 0.27, 0.28, 0.29, 0.3, 0.31, 0.32, 0.33, 0.34, 0.35, 0.36, 0.37, 0.38, 0.39, 0.4, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48, 0.49, 0.5, 0.51, 0.52, 0.53, 0.54, 0.55, 0.56, 0.57, 0.58, 0.59, 0.6, 0.61, 0.62, 0.63, 0.64, 0.65, 0.66, 0.67, 0.68, 0.69, 0.7, 0.71, 0.72, 0.73, 0.74, 0.75, 0.76, 0.77, 0.78, 0.79, 0.8, 0.81, 0.82, 0.83, 0.84, 0.85, 0.86, 0.87, 0.88, 0.89, 0.9, 0.91, 0.92, 0.93, 0.94, 0.95, 0.96, 0.97, 0.98, 0.99, 1.0]), mode: str = 'prob')[source]

Bases: Metric

Quantile loss, following the article: Bayesian Intermittent Demand Forecasting for Large Inventories .

A quantile of q=0.5 will give half of the mean absolute error as it is calcualted as max(q * (y-y_pred), (1-q) * (y_pred-y)).

Parameters

  • q_points (List[float]) – The quantile points of interest, the default value is None. In the evaluation of the prediction, while q_points is specified, output a dict which contains each quantile result respect to quantile points.

  • quantile_level (np.ndarray, List[float]) – The quantile level of interest. In validation phase, it is used to measure validation loss.

  • mode (str) – Supported metric modes, only normal and prob are valid values. Set to normal for non-probability use cases, set to prob for probability use cases.

metric_fn(y_true: ndarray, y_pred_sample: ndarray) → Union[float, dict][source]

Quantile loss.

Parameters

  • y_true (np.ndarray) – Ground truth (correct) labels.

  • y_score (np.ndarray) – Predicted quantiles.

Returns

Quantile loss.

Return type

float

class MetricContainer(metric_names: List[str], prefix: str = '')[source]

Bases: object

Container holding a list of metrics.

Parameters

  • metric_names (List[str]) – List of metric names.

  • prefix (str) – Prefix of metric names.

_prefix

Prefix of metric names.

Type

str

_metrics

List of metric instance.

Type

List[Metric]

_names

List of metric names associated with eval_name.

Type

List[str]