TensorFlow: Custom Layers for Movie Preference Prediction
TensorFlow: Custom Layers for Movie Preference Prediction
TensorFlow, an open-source machine learning framework by Google, enables the development of custom neural network architectures through its tf.keras API. This article examines the implementation of custom layers using methods such as __init__, build, and call, alongside auxiliary methods like get_config and compute_output_shape. To illustrate, we present a model for predicting movie preferences based on features such as duration, IMDb rating, and action scene count, offering a practical and accessible application of TensorFlow’s capabilities.
1. Custom Layers in TensorFlow
Neural network layers encapsulate transformations of input data. The tf.keras.layers module provides predefined layers, including Dense for fully connected networks, Conv2D for convolutional operations, and LSTM for sequential data. Custom layers, created by subclassing tf.keras.layers.Layer, allow tailored computations. Key methods include:
__init__: Initializes hyperparameters and sub-layers.build: Defines trainable weights based on input shape.call: Executes the forward pass computation.get_config: Serializes layer configuration for model persistence.compute_output_shape: Specifies output dimensions.add_lossandadd_metric: Incorporate regularization and performance metrics.
The following sections detail these methods through a custom MoviePreferenceLayer designed for a binary classification task.
2. Core Methods of a Custom Layer
The MoviePreferenceLayer transforms input features to predict whether a user likes a movie, incorporating a scaling factor to emphasize certain features.
2.1 Initialization (__init__)
The __init__ method configures hyperparameters and non-trainable components.
def __init__(self, units, excitement_factor=1.0, activation=None):
super(MoviePreferenceLayer, self).__init__()
self.units = units
self.excitement_factor = excitement_factor
self.activation = tf.keras.activations.get(activation)
The method defines the number of output units, an excitement_factor to scale outputs (emphasizing features like action scenes), and an optional activation function converted via tf.keras.activations.get.
2.2 Weight Construction (build)
The build method initializes trainable weights upon receiving the input shape.
def build(self, input_shape):
self.kernel = self.add_weight(
name='kernel',
shape=(input_shape[-1], self.units),
initializer='glorot_uniform',
trainable=True
)
self.bias = self.add_weight(
name='bias',
shape=(self.units,),
initializer='zeros',
trainable=True
)
A weight matrix (kernel) of shape (input_dim, units) is created with Glorot uniform initialization to ensure stable training. A bias vector of shape (units,) is initialized to zeros.
2.3 Forward Pass (call)
The call method performs the forward pass, transforming inputs into outputs.
def call(self, inputs, training=False):
output = tf.matmul(inputs, self.kernel) + self.bias
output = output * self.excitement_factor
if self.activation is not None:
output = self.activation(output)
l1_loss = tf.reduce_sum(tf.abs(self.kernel)) * 0.01
self.add_loss(l1_loss)
self.add_metric(tf.reduce_mean(output), name='preference_score')
return output
The forward pass involves:
- Matrix multiplication (
tf.matmul) of inputs withkernel, followed by bias addition. - Scaling by
excitement_factorto amplify outputs. - Application of the specified activation function, if any.
- Computation of L1 regularization loss (
tf.reduce_sum(tf.abs(kernel)) * 0.01) to penalize large weights, added viaself.add_loss. - Tracking of the mean output as a metric (
preference_score) viaself.add_metric.
2.4 Auxiliary Methods
Additional methods support layer functionality:
def compute_output_shape(self, input_shape):
return tf.TensorShape([input_shape[0], self.units])
def get_config(self):
config = super().get_config()
config.update({
'units': self.units,
'excitement_factor': self.excitement_factor,
'activation': tf.keras.activations.serialize(self.activation)
})
return config
compute_output_shape returns the output shape, preserving the batch dimension and using units for the feature dimension. get_config serializes the layer’s hyperparameters for model saving and loading.
Other methods, such as get_weights and trainable_variables, are managed by TensorFlow for weight access and optimization.
3. Application: Predicting Movie Preferences
The task is to predict whether a user likes a movie (1) or not (0) based on three features: duration (minutes), IMDb rating (1–10), and number of action scenes (0–20). A synthetic dataset is generated to simulate user preferences, providing a straightforward yet illustrative binary classification problem.
The dataset comprises 5000 samples, with features drawn from uniform distributions: duration (90–180 minutes), IMDb rating (1–10), and action scenes (0–20). Labels are assigned based on the rule: a movie is liked if rating + 0.5 * action_scenes > duration / 10. For example, a 120-minute movie with a rating of 7 and 10 action scenes is liked if 7 + 0.5 * 10 = 12 > 120 / 10 = 12. This rule creates a learnable but nonlinear decision boundary.
Movie Preference Prediction Model
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
# Custom Layer
class MoviePreferenceLayer(tf.keras.layers.Layer):
def __init__(self, units, excitement_factor=1.0, activation=None):
super(MoviePreferenceLayer, self).__init__()
self.units = units
self.excitement_factor = excitement_factor
self.activation = tf.keras.activations.get(activation)
def build(self, input_shape):
self.kernel = self.add_weight(
name='kernel',
shape=(input_shape[-1], self.units),
initializer='glorot_uniform',
trainable=True
)
self.bias = self.add_weight(
name='bias',
shape=(self.units,),
initializer='zeros',
trainable=True
)
def call(self, inputs, training=False):
output = tf.matmul(inputs, self.kernel) + self.bias
output = output * self.excitement_factor
if self.activation is not None:
output = self.activation(output)
l1_loss = tf.reduce_sum(tf.abs(self.kernel)) * 0.01
self.add_loss(l1_loss)
self.add_metric(tf.reduce_mean(output), name='preference_score')
return output
def compute_output_shape(self, input_shape):
return tf.TensorShape([input_shape[0], self.units])
def get_config(self):
config = super().get_config()
config.update({
'units': self.units,
'excitement_factor': self.excitement_factor,
'activation': tf.keras.activations.serialize(self.activation)
})
return config
# Model
class MoviePreferenceModel(tf.keras.Model):
def __init__(self):
super(MoviePreferenceModel, self).__init__()
self.movie_layer1 = MoviePreferenceLayer(16, excitement_factor=1.2, activation='relu')
self.movie_layer2 = MoviePreferenceLayer(8, excitement_factor=1.0)
self.output_layer = tf.keras.layers.Dense(1, activation='sigmoid')
def call(self, inputs, training=False):
x = self.movie_layer1(inputs, training=training)
x = self.movie_layer2(x, training=training)
return self.output_layer(x)
# Generate synthetic movie data
np.random.seed(42)
num_samples = 5000
movie_length = np.random.uniform(90, 180, num_samples)
imdb_rating = np.random.uniform(1, 10, num_samples)
action_scenes = np.random.randint(0, 20, num_samples)
X = np.column_stack([movie_length, imdb_rating, action_scenes])
y = (imdb_rating + action_scenes * 0.5 > movie_length / 10).astype(np.int32)
# Normalize inputs
X = (X - X.mean(axis=0)) / X.std(axis=0)
# Convert to tensors
X_tensor = tf.convert_to_tensor(X, dtype=tf.float32)
y_tensor = tf.convert_to_tensor(y, dtype=tf.float32)
# Training setup
model = MoviePreferenceModel()
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
loss_fn = tf.keras.losses.BinaryCrossentropy()
@tf.function
def train_step(inputs, labels):
with tf.GradientTape() as tape:
predictions = model(inputs, training=True)
loss = loss_fn(labels, predictions)
total_loss = loss + sum(model.losses)
gradients = tape.gradient(total_loss, model.trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))
return total_loss
# Train the model
losses = []
for epoch in range(50):
loss = train_step(X_tensor, y_tensor)
losses.append(loss.numpy())
if epoch % 10 == 0:
print(f"Epoch {epoch}, Loss: {loss:.4f}")
# Evaluate
predictions = model(X_tensor)
accuracy = tf.reduce_mean(tf.cast(tf.round(predictions) == y_tensor, tf.float32))
print(f"Accuracy: {accuracy:.4f}")
# Plot loss curve
plt.plot(losses)
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.title('Movie Preference Model Loss')
plt.savefig('loss_curve.png')
plt.close()
4. Model Architecture and Training
The MoviePreferenceModel comprises two MoviePreferenceLayers followed by a Dense layer with sigmoid activation for binary classification. The first layer transforms the three input features (duration, rating, action scenes) into 16 dimensions, applying ReLU activation and a 1.2 excitement_factor. The second layer reduces to 8 dimensions without activation. The final layer outputs a probability (0–1) of liking the movie.
Training uses the Adam optimizer with a learning rate of 0.01, minimizing binary cross-entropy loss augmented by L1 regularization from each layer. The dataset is normalized to zero mean and unit variance to enhance convergence. Over 50 epochs, the model achieves approximately 85–95% accuracy, as the decision boundary is learnable but nonlinear due to the feature interactions.
The loss trajectory is visualized in the following figure, generated by saving loss_curve.png during execution:
Ensure ...
5. Conclusion
TensorFlow’s tf.keras API facilitates the creation of custom layers through methods like __init__, build, call, get_config, and compute_output_shape. The movie preference prediction model demonstrates their application in a practical binary classification task, leveraging a custom MoviePreferenceLayer to process interpretable features. For further exploration, consult the TensorFlow documentation to develop tailored neural network architectures.
Comments
Post a Comment