src.classification.image package

Submodules

src.classification.image.cross_attention_classifier module

This file contains the CrossAttention facial emotion classifier.

class src.classification.image.cross_attention_classifier.AffinityLoss(device: device, num_class: int = 7, feat_dim: int = 512)

Bases: Module

Affinity Loss function that is supposed to increase the inter-class distances while decreasing the intra-class distances. For more details about the computation go to chapter 3.1.1 of the paper.

forward(x: Tensor, labels: Tensor) → Tensor

Forward pass through the loss function.

Parameters:

• x – features of the DAN model. Output after resnet.

• labels – Labels for the inputs to classify.

Returns:

Loss function value

class src.classification.image.cross_attention_classifier.ChannelAttention

Bases: Module

The ChannelAttention layer, which is the second part of the CrossAttentionHead.

forward(sa: Tensor) → Tensor

Forward pass through the ChannelAttention layer

Parameters:

sa – Input into the layer (should be the output of SpatialAttention)

Returns:

Output of the ChannelAttention layer

class src.classification.image.cross_attention_classifier.CrossAttentionHead

Bases: Module

Implementation of a CrossAttention Head in pytorch. Taken from the original paper without changes.

forward(x: Tensor) → Tensor

Forward pass through the AttentionHead layer.

Parameters:

x – tensor to pass through the layer

Returns:

Output of the layer

init_weights() → None

Function that initializes the weights of all layers.

class src.classification.image.cross_attention_classifier.CrossAttentionNetworkClassifier(parameters: Optional[Dict] = None)

Bases: ImageEmotionClassifier

Class that implements an emotion classifier using a multi-head cross attention network based on ResNet50. Details can be found here: https://paperswithcode.com/paper/distract-your-attention-multi-head-cross

classify(parameters: Optional[Dict] = None, **kwargs) → array

Classification method used to classify emotions from images

Parameters:

• parameters – Parameter dictionary used for classification

• kwargs – Additional kwargs parameters

Returns:

An array with predicted emotion indices

initialize_model(parameters: Dict) → None

Initializes a new and pretrained version of the CrossAttention model

load(parameters: Optional[Dict] = None, **kwargs) → None

Loading method that loads a previously trained model from disk.

Parameters:

• parameters – Parameters required for loading the model

• kwargs – Additional kwargs parameters

save(parameters: Optional[Dict] = None, **kwargs) → None

Saving method that saves a previously trained model on disk.

Parameters:

• parameters – Parameters required for storing the model

• kwargs – Additional kwargs parameters

train(parameters: Optional[Dict] = None, **kwargs) → None

Training method for CrossAttention model. Taken from https://github.com/yaoing/DAN and adapted slightly

Parameters:

• parameters – Parameter dictionary used for training

• kwargs – Additional kwargs parameters

transform_data(data: Tensor, labels: Tensor) → Tuple[Tensor, Tensor]

Transforming method that transforms the data to torch tensors in the correct format.

Parameters:

• data – Tensor containing a batch of images

• labels – Tensor containing a batch of labels

Returns:

torch tensors with images and labels

class src.classification.image.cross_attention_classifier.DAN(num_class: int = 7, num_head: int = 4, pretrained: bool = True)

Bases: Module

This class implements the “Distract Your Attention” (DAN) network, which was introduced in https://github.com/yaoing/DAN

forward(x: Tensor) → Tuple[Tensor, Tensor, Tensor]

Forward pass through the DAN network.

Parameters:

x – The tensor x to pass through the network;

Returns:

Tuple containing three elements: 0: Output of the classifier 1: Resnet output features 2: Output after the attention heads before the classifier

class src.classification.image.cross_attention_classifier.PartitionLoss

Bases: Module

Partition loss function that maximizes the variance among attention maps. Refer to chapter 3.3.1 for more details.

forward(x: Tensor) → Tensor

Forward pass through the loss function that computes the loss value

Parameters:

x – The input tensor to compute the loss for. Should contain the values of the tensors after the DAN heads.

Returns:

Loss value in a tensor.

class src.classification.image.cross_attention_classifier.SpatialAttention

Bases: Module

SpatialAttention layer that is a part of the CrossAttentionHead.

forward(x: Tensor) → Tensor

Forward pass through the SpatialAttention layer

Parameters:

x – The tensor to pass through the layer

Returns:

The output tensor

src.classification.image.efficientnet_classifier module

This file contains the EfficientNet facial emotion classifier

class src.classification.image.efficientnet_classifier.MultiTaskEfficientNetB2Classifier(parameters: Optional[Dict] = None)

Bases: ImageEmotionClassifier

Class that implements an efficient net emotion classifier.

classify(parameters: Optional[Dict] = None, **kwargs) → array

Classification method used to classify emotions from images

Parameters:

• parameters – Parameter dictionary used for classification

• kwargs – Additional kwargs parameters

Returns:

An array with predicted emotion indices

initialize_model(parameters: Dict) → None

Initializes a new and pretrained version of the EfficientNetB2 model

load(parameters: Optional[Dict] = None, **kwargs) → None

Loading method that loads a previously trained model from disk.

Parameters:

• parameters – Parameters required for loading the model

• kwargs – Additional kwargs parameters

save(parameters: Optional[Dict] = None, **kwargs) → None

Saving method that saves a previously trained model on disk.

Parameters:

• parameters – Parameters required for storing the model

• kwargs – Additional kwargs parameters

train(parameters: Optional[Dict] = None, **kwargs) → None

Training method for EfficientNet model

Parameters:

• parameters – Parameter dictionary used for training

• kwargs – Additional kwargs parameters

src.classification.image.image_emotion_classifier module

Base class for all image emotion classifiers

class src.classification.image.image_emotion_classifier.ImageEmotionClassifier(name: str = 'image', parameters: Optional[Dict] = None)

Bases: EmotionClassifier

Base class for all image emotion classifiers. Contains common functionality that concerns all image classifiers.

abstract classify(parameters: Optional[Dict] = None, **kwargs) → array

The virtual classification method for interfacing

Parameters:

• parameters – Parameter dictionary used for classification

• kwargs – Additional kwargs parameters

Returns:

An array with predicted emotion indices

abstract load(parameters: Optional[Dict] = None, **kwargs) → None

Loading method that loads a previously trained model from disk.

Parameters:

• parameters – Parameters required for loading the model

• kwargs – Additional kwargs parameters

prepare_data(parameters: Dict) → None

Function that prepares image datasets for training and stores them inside the class.

Parameters:

parameters – Parameter dictionary that contains important params. including: which_set, batch_size, weighted

prepare_training(parameters: Dict) → None

Function that prepares the training by initializing optimizer, loss, metrics and callbacks for training.

Parameters:

parameters – Training parameters

abstract save(parameters: Optional[Dict] = None, **kwargs) → None

Saving method that saves a previously trained model on disk.

Parameters:

• parameters – Parameters required for storing the model

• kwargs – Additional kwargs parameters

abstract train(parameters: Optional[Dict] = None, **kwargs) → None

Virtual training method for interfacing

Parameters:

• parameters – Parameter dictionary used for training

• kwargs – Additional kwargs parameters

src.classification.image.vgg16_classifier module

This file contains the VGG16 facial emotion classifier

class src.classification.image.vgg16_classifier.VGG16Classifier(parameters: Optional[Dict] = None)

Bases: ImageEmotionClassifier

Class that implements an emotion classifier using VGG16

classify(parameters: Optional[Dict] = None, **kwargs) → array

Classification method used to classify emotions from images

Parameters:

• parameters – Parameter dictionary used for classification

• kwargs – Additional kwargs parameters

Returns:

An array with predicted emotion indices

initialize_model(parameters: Dict) → None

Initializes a new and pretrained version of the VGG16Classifier model

Parameters:

parameters – Parameters for initializing the model.

load(parameters: Optional[Dict] = None, **kwargs) → None

Loading method that loads a previously trained model from disk.

Parameters:

• parameters – Parameters required for loading the model

• kwargs – Additional kwargs parameters

save(parameters: Optional[Dict] = None, **kwargs) → None

Saving method that saves a previously trained model on disk.

Parameters:

• parameters – Parameters required for storing the model

• kwargs – Additional kwargs parameters

train(parameters: Optional[Dict] = None, **kwargs) → None

Training method for VGG16Classifier model

Parameters:

• parameters – Parameter dictionary used for training

• kwargs – Additional kwargs parameters

Module contents

This package contains a few image emotion classifiers based on different classifier architectures.