paddlets.utils.validation

cross_validate(data: ~paddlets.datasets.tsdataset.TSDataset, estimator: ~paddlets.models.base.Trainable, splitter: ~paddlets.datasets.splitter.SplitterBase = <paddlets.datasets.splitter.ExpandingWindowSplitter object>, use_backtest: bool = True, predict_window: ~typing.Optional[int] = None, stride: ~typing.Optional[int] = None, metric: ~typing.Optional[~paddlets.metrics.base.Metric] = None, return_score=True, reduction: ~typing.Optional[~typing.Callable[[~numpy.ndarray], float]] = <function mean>, verbose: bool = False) → Union[float, List[dict]][source]

Cross validate Evaluate forecaster using timeseries cross-validation.

Parameters

  • data (TSDataset) – The TSdataset to use in cross validate.

  • estimator (Trainable) – The model use in cross validate.

  • splitter (SplitterBase) – CV splitter, use 5-folds ExpandingWindowSplitter by default.

  • use_backtest (bool) – Use backtest or not, set to True by default, use recursive predict if False.

  • predict_window (int) – The size of predict window for the prediction.

  • stride (int) – The number of time steps between two consecutive predict window.

  • metric (Metric) – A function that takes two TSdataset instances as inputs and returns an error value.

  • return_score (bool) – Compute and return score on test set. Will return a list of dict if set to False

  • reduction (Callable[[np.ndarray]|None) – A function used to combine the individual error scores obtained when predict_window ！= stride. If explicitely set to None, the method will return a list of the individual error scores instead. Set to np.mean by default.

  • verbose (bool) – Verbose mode.

Returns

float, list[dict]

Raises

ValueError –

fit_and_score(train_data: ~typing.Union[~paddlets.datasets.tsdataset.TSDataset, ~typing.List[~paddlets.datasets.tsdataset.TSDataset]], valid_data: ~typing.Union[~paddlets.datasets.tsdataset.TSDataset, ~typing.List[~paddlets.datasets.tsdataset.TSDataset]], estimator: ~paddlets.models.base.Trainable, use_backtest: bool = True, predict_window: ~typing.Optional[int] = None, stride: ~typing.Optional[int] = None, metric: ~typing.Optional[~paddlets.metrics.base.Metric] = None, return_score=True, return_estimator=True, return_predicts=True, reduction: ~typing.Optional[~typing.Callable[[~numpy.ndarray], float]] = <function mean>, verbose: bool = False) → dict[source]

Fit and score

Parameters

  • train_data (Union[TSDataset, List[TSDataset]]) – Train dataset .

  • valid_data (Union[TSDataset, List[TSDataset]]) – Valid dataset .

  • estimator (Trainable) – The model use in cross validate.

  • use_backtest (bool) – Use backtest or not.

  • predict_window (int) – The predict window for the prediction.

  • stride (int) – The number of time steps between two consecutive predict window.

  • metric (Metric) – A function that takes two TSdataset instances as inputs and returns an error value.

  • verbose (bool) – verbose mode.

  • return_score (bool) – Compute and return score on test set.

  • return_estimator (bool) – Whether to return the fitted estimator.

  • reduction (Callable[[np.ndarray]|None) – A function used to combine the individual error scores obtained when predict_window ！= stride. If explicitely set to None, the method will return a list of the individual error scores instead. Set to np.mean by default.

Returns

dict

Raises

ValueError –