Skip to content

[ENH] Implement the TemporalConvolutionalNetwork in aeon/networks #2931

Open
@lucifer4073

Description

@lucifer4073

Describe the feature or idea you want to propose

Introduce a robust and modular implementation of Temporal Convolutional Networks (TCN) for sequence modeling in aeon.forecasting.deep_learning. The architecture follows the foundational design proposed in the paper "An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling" by Bai et al. (2018), and implements the TCN using Keras/TensorFlow, adapted from the original PyTorch version locuslab/TCN.

The design is aimed at preserving the original model's key ideas:

  • Dilated convolutions to exponentially expand the receptive field.
  • Residual connections for improved stability during training.
  • Weight-normalized Conv1D logic re-implemented through customized Conv1D layers in Keras.

Describe your proposed solution

I have implemented this TCN architecture as a Keras-based class TemporalConvolutionalNetwork, inheriting from BaseDeepLearningNetwork. The following private methods abstract key functionalities of the model:

  • _conv1d_with_variable_padding(...): Handles variable causal padding logic akin to padding + chomp used in the Torch Chomp1d module.
  • _chomp_1d(...): Ensures strict causality by slicing excess padding.
  • _temporal_block(...): Builds individual residual blocks with two Conv1D layers, activations, dropout, and a residual connection with optional 1x1 projection.
  • _temporal_conv_net(...): Stacks temporal blocks with exponentially increasing dilation factors (i.e., d=2^i) to create a deep, memory-efficient model.
  • build_network(...): Orchestrates the overall architecture using the private components.

Example snippet (from _tcn.py)

def _temporal_block(self, x, n_inputs, n_outputs, kernel_size, stride, dilation, padding, dropout, training):
    out = self._conv1d_with_variable_padding(x, n_outputs, kernel_size, padding, stride, dilation)
    out = self._chomp_1d(out, padding)
    ...
    if n_inputs != n_outputs:
        res = self._conv1d_with_variable_padding(x, n_outputs, 1, 0, 1, 1)
    result = tf.keras.layers.Add()([out, res])
    return tf.keras.layers.ReLU()(result)

The structure mirrors the reference PyTorch TemporalBlock:

class TemporalBlock(nn.Module):
    ...
    self.conv1 = weight_norm(nn.Conv1d(...))
    self.chomp1 = Chomp1d(padding)
    ...
    self.net = nn.Sequential(...)
    self.downsample = nn.Conv1d(...) if n_inputs != n_outputs else None

Describe alternatives you've considered, if relevant

N/A

Additional context

N/A

Metadata

Metadata

Assignees

Labels

enhancementNew feature, improvement request or other non-bug code enhancement

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions