paddlets.utils.backtest

backtest(data: ~paddlets.datasets.tsdataset.TSDataset, model: ~paddlets.models.base.Trainable, start: ~typing.Optional[~typing.Union[~pandas._libs.tslibs.timestamps.Timestamp, int, str, float]] = None, predict_window: ~typing.Union[None, int] = None, stride: ~typing.Union[None, int] = None, metric: ~typing.Optional[~paddlets.metrics.base.Metric] = None, return_predicts: bool = False, reduction: ~typing.Optional[~typing.Callable[[~numpy.ndarray], float]] = <function mean>, verbose: bool = True) → Union[float, Tuple[float, Union[TSDataset, List[TSDataset]]]][source]

Backtest A repeated forecasting and validating process. It first use data with the length of predict_window, and then moves the end of the training set forward by stride time steps. By default, Backtest will generate a TSdataset with length (data_length - model.skip_chunk_len) as output. If set predict_window != stride Backtest will generate a List of TSdataset as output

Parameters

  • data (TSDataset) – The TSdataset to use to successively evaluate the historical forecasts

  • model (Trainable) – The fitted model to use to successively evaluate the historical forecasts

  • start (pd.Timestamp|int|None) – The first prediction time, at which a prediction is computed for a future time.

  • predict_window (int|None) – The predict window for the prediction.

  • stride (int|None) – The number of time steps between two consecutive predict window.

  • metric (Metric) – A function that takes two TSdataset instances as inputs and returns an error value.

  • return_predicts (bool) – If set return_predicts=True, the predict results will return additionaly.

  • reduction (Callable[[np.ndarray]|None) – A function used to combine the individual error scores obtained when predict_window ！= stride. If explicitely set to None, the method will return a list of the individual error scores instead. Set to np.mean by default.

  • verbose (bool) – Turn on Verbose mode,set to true by default.

Returns

Return score by default, If set return_predicts=True, the predict results will return additionaly.

Return type

float|(float, TSDataset|List[TSDataset])

Raises

ValueError –