Neural Networks — colvarsfinder.nn

Author:

Wei Zhang

Year:

2022

Copyright:

GNU Public License v3

This module implements PyTorch neural network classes that represent autoencoders or eigenfunctions. These classes are used in the module colvarsfinder.core.

colvarsfinder.nn.create_sequential_nn(layer_dims, activation=Tanh())

Construct a feedforward Pytorch neural network

Parameters:

• layer_dims (list of int) – dimensions of layers

• activation – PyTorch (nonlinear) activation function

Raises:

AssertionError – if length of layer_dims is not larger than 1.

Example

from colvarsfinder.nn import create_sequential_nn
import torch

nn1 = create_sequential_nn([10, 5, 1])
nn2 = create_sequential_nn([10, 2], activation=torch.nn.ReLU())

class colvarsfinder.nn.AutoEncoder(e_layer_dims, d_layer_dims, activation=Tanh())

Autoencoder neural network

Parameters:

• e_layer_dims (list of ints) – dimensions of encoder’s layers

• d_layer_dims (list of ints) – dimensions of decoder’s layers

• activation – PyTorch activation function

Raises:

AssertionError – if e_layer_dims[-1] != d_layer_dims[0].

encoder

feedforward PyTorch neural network representing encoder

decoder

feedforward PyTorch neural network representing decoder

encoded_dim

encoded dimension

Type:

int

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(inp)

Parameters:

inp (input PyTorch tensor) –

Returns:

output of autoencoder given the input tensor inp

get_params_of_cv(cv_idx)

Parameters:

cv_idx (int) – index of collective variables

Returns:

list of pairs of name and parameters of all linear layers.

class colvarsfinder.nn.RegAutoEncoder(e_layer_dims, d_layer_dims, reg_layer_dims, K, activation=Tanh())

Neural network representing a regularized autoencoder

Parameters:

• e_layer_dims (list of ints) – dimensions of encoder’s layers

• d_layer_dims (list of ints) – dimensions of decoder’s decoder

• reg_layer_dims (list of ints) – dimensions of a regularizer’s layers

• K (int) – number of regularizers

• activation – PyTorch nonlinear activation function

Raises:

AssertionError – if e_layer_dims[-1] != d_layer_dims[0] or e_layer_dims[-1] != reg_layer_dims[0]

encoder

feedforward PyTorch neural network representing encoder

decoder

feedforward PyTorch neural network representing decoder

reg

feedforward PyTorch neural network representing regularizers, or None if K=0

encoded_dim

encoded dimension

Type:

int

num_reg

number of eigenfunctions used for regularization (K)

Type:

int

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(inp)

Returns:

values of autodecoder and regularizers given the input tensor inp

forward_ae(inp)

Parameters:

inp – PyTorch tensor

Returns:

value of autoencoder given the input tensor inp

forward_reg(inp)

Parameters:

inp – PyTorch tensor

Returns:

value of regularizers (e.g. eigenfunctions) given the input tensor inp

get_params_of_cv(cv_idx)

Parameters:

cv_idx (int) – index of collective variables

Returns:

list of pairs of name and parameters of all linear layers.

class colvarsfinder.nn.RegModel(reg_ae, cvec)

Neural network representing the eigenfunctions built from a RegAutoEncoder.

Parameters:

• reg_ae (RegAutoEncoder) – an object of class RegAutoEncoder

• cvec (list of int) – order of regularizers

Raises:

AssertionError – reg_ae doesn’t have a regularizer

encoder

feedforward PyTorch neural network representing encoder

reg

feedforward PyTorch neural network representing regularizers

encoded_dim

encoded dimension

Type:

int

num_reg

number of eigenfunctions used for regularization

Type:

int

cvec

same as input

Type:

list of int

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(inp)

return values of regularizers (reordered according to cvec) given input tensor inp

class colvarsfinder.nn.EigenFunctions(layer_dims, k, activation=Tanh())

Feedforward neural network representing eigenfunctions.

Parameters:

• layer_dims (list of ints) – dimensions of layers for each eigenfunction

• k (int) – number of eigenfunctions

• activation – PyTorch nonlinear activation function

Raises:

AssertionError – if layer_dims[-1] != 1 (since each eigenfunction is scalar-valued function).

This class defines \(k\) functions \(g_1, g_2, \dots, g_k\) and corresponds to the model used as an input paramter to define the class EigenFunctionTask. Each \(g_i:\mathbb{R}^{d_r}\rightarrow \mathbb{R}\) is represented by a feedforward neural network of the same architecture specified by layer_dims. After training, it can be concatenated to the preprocessing layer to obtain eigenfunctions, or collective variables. See Loss function for training eigenfunctions for details.

Note

The first item in the list layer_dims should equal to \(d_r\), i.e. the output dimension of the preprocessing layer, while the last item in layer_dims should be one.

eigen_funcs

PyTorch module list that contains \(k\) PyTorch feedforward neural networks of the same architecture.

Type:

torch.nn.ModuleList

forward(inp)

Parameters:

inp – PyTorch tensor. Typically it is the output of preprocessing layer, with shape \([l, d_r]\).

Returns:

PyTorch tensor of shape \([l, k]\), values of the \(k\) functions \(g_1, \cdots, g_k\) given the input tensor.

get_params_of_cv(cv_idx)

Parameters:

cv_idx (int) – index of collective variables (i.e. eigenfunctions)

Returns:

list of pairs of name and parameters of all linear layers.