paddlets.models.anomaly.dl._anomaly_transformer.attention

class TriangularCausalMask(batch_size: int, length: int)[source]

Bases: object

Triangular Causal Mask.

Parameters

  • batch_size (int) – Number of samples per batch.

  • length (int) – Length of samples per data.

_nn

Dynamic graph LayerList.

Type

paddle.nn.Sequential

class AnomalyAttention(win_size: int, mask_flag: bool = True, scale: Optional[bool] = None, attention_dropout: float = 0.0, output_attention: bool = False)[source]

Bases: Layer

Anomaly Attention:

For the prior-association, a learnable Gaussian kernel to calculate the prior with the relative temporal distance. For the series-association branch is to learn the associations from raw series.

Parameters

  • win_size (int) – The size of the loopback window, i.e. the number of time steps feed to the model.

  • mask_flag (bool) – Whether to use attn_mask.

  • scale (int|None) – It can scale the dot products.

  • attention_dropout (float) – Dropout regularization parameter.

  • output_attention (bool) – Whether to output series, prior and sigma.

_nn

Dynamic graph LayerList.

Type

paddle.nn.Sequential

forward(queries: Tensor, keys: Tensor, values: Tensor, sigma: Tensor, attn_mask: Callable[[...], Tensor]) → Tensor[source]

The prior-association result from Gaussian kernel branch. the series-association result from self attention branch.

Parameters

  • queries (paddle.Tensor) – The query projection layer.

  • keys (paddle.Tensor) – The key projection layer.

  • values (paddle.Tensor) – The value projection layer.

  • sigma (paddle.Tensor) – A learnable scale parameter for the Gaussian kernel, making ther prior-associations adapt the various time series patterns.

  • attn_mask (Callable[..., paddle.Tensor]|None) – Whether to use mask in ecoder.

Returns

Output of AnomalyAttention. series(paddle.Tensor): The series-association from Gaussian kernel branch. prior(paddle.Tensor): The prior-association from self attention. sigma(paddle.Tensor): A learnable scale parameter for the Gaussian kernel.

Return type

V(paddle.Tensor)

class AttentionLayer(attention: Callable[[...], Tensor], d_model: int, n_heads: int, d_keys: Optional[int] = None, d_values: Optional[int] = None)[source]

Bases: Layer

AttentionLayer for anomaly transformer.

Parameters

  • attention (Callable[..., paddle.Tensor]) – Attention layers in anomaly transformer.

  • d_model (int) – The expected feature size for the input of the anomaly transformer.

  • n_heads (int) – The number of heads in multi-head attention.

  • d_keys (int) – The feature size in key.

  • d_values (int) – The feature size in value.

_nn

Dynamic graph LayerList.

Type

paddle.nn.Sequential

forward(queries: Tensor, keys: Tensor, values: Tensor, attn_mask: Callable[[...], Tensor]) → Tensor[source]

The series-association and the prior-association forward.

Parameters

  • queries (paddle.Tensor) – The query projection layer tensor.

  • keys (paddle.Tensor) – The key projection layer tensor.

  • values (paddle.Tensor) – The value projection layer tensor.

  • sigma (paddle.Tensor) – A learnable scale parameter for the Gaussian kernel.

  • attn_mask (Callable[..., paddle.Tensor]) – Whether to use mask in ecoder.

Returns

pred of model. series(paddle.Tensor): The series-association output tensor. prior(paddle.Tensor): The prior-association output tensor. sigma(paddle.Tensor): A learnable scale parameter for the Gaussian kernel.

Return type

self.out_projection(out)(paddle.Tensor)