paddlets.utils.backtest

backtest(data: ~paddlets.datasets.tsdataset.TSDataset, model: ~paddlets.models.base.Trainable, start: ~typing.Optional[~typing.Union[~pandas._libs.tslibs.timestamps.Timestamp, int, str, float]] = None, predict_window: int = 1, stride: int = 1, metric: ~typing.Optional[~paddlets.metrics.base.Metric] = None, return_score: bool = True, reduction: ~typing.Optional[~typing.Callable[[~numpy.ndarray], float]] = <function mean>) → Union[TSDataset, List[TSDataset], float][source]

Backtest A repeated forecasting and validating process. It first use data with the length of predict_window, and then moves the end of the training set forward by stride time steps. By default, Backtest will generate a TSdataset with length (data_length - model.skip_chunk_len) as output. If set predict_window != stride Backtest will generate a List of TSdataset as output If set return_score=True, A metric (given by the metric function) is then evaluated on the forecast and the actual values. It will returns a mean-of all these metric scores by default.

Parameters

  • data (TSDataset) – The TSdataset to use to successively evaluate the historical forecasts

  • model (Trainable) – The fitted model to use to successively evaluate the historical forecasts start(pd.Timestamp|int|None): The first prediction time, at which a prediction is computed for a future time.

  • predict_window (int) – The predict window for the prediction.

  • stride (int) – The number of time steps between two consecutive predict window.

  • metric (Metric) – A function that takes two TSdataset instances as inputs and returns an error value.

  • return_score (bool) – If set return_score=True, A metric (given by the metric function) is then evaluated on the forecast and the actual values. It will returns a mean-of all these metric scores by default.

  • reduction (Callable[[np.ndarray]|None) – A function used to combine the individual error scores obtained when predict_window ！= stride. If explicitely set to None, the method will return a list of the individual error scores instead. Set to np.mean by default.

Returns

Union[TSDataset,List[TSdataset],float]

Raises

ValueError –