Example 2: Building a Neural Network with TensorFlow

In this example, we’ll build and train a simple neural network using TensorFlow. This neural network will classify data points into two categories. It demonstrates how to use TensorFlow’s high-level Keras API for building, training, and evaluating a neural network.

Problem Statement

We will classify points into two categories based on their coordinates. For simplicity, we’ll generate synthetic data with clear boundaries.

Step-by-Step Guide

1. Import Required Libraries

import numpy as np
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt

2. Generate Synthetic Data

We’ll create a dataset with two classes separated by a nonlinear boundary.

# Generate synthetic data
np.random.seed(42)
X = np.random.rand(1000, 2)  # 1000 points with 2 features
y = (X[:, 0] + X[:, 1] > 1).astype(int)  # Label: 1 if x1 + x2 > 1, else 0

# Split the data into training and testing sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

3. Build the Neural Network Model

We’ll create a neural network with the following architecture:

  • Input layer: 2 neurons (for 2 features)
  • Hidden layer: 8 neurons with ReLU activation
  • Output layer: 1 neuron with sigmoid activation
# Define the model
model = Sequential([
    Dense(8, activation='relu', input_shape=(2,)),  # Hidden layer
    Dense(1, activation='sigmoid')  # Output layer
])

# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

4. Train the Model

We’ll train the model for 50 epochs using the training data.

# Train the model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2, verbose=1)

5. Evaluate the Model

Once trained, evaluate the model on the test data to check its performance.

# Evaluate the model
test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss: {test_loss:.4f}")
print(f"Test Accuracy: {test_accuracy:.4f}")

6. Visualize Training Results

Plot the training and validation accuracy over epochs.

# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.title('Training and Validation Accuracy')
plt.legend()
plt.show()

7. Visualize Decision Boundary

To better understand how the model separates the data, visualize the decision boundary.

# Visualize the decision boundary
xx, yy = np.meshgrid(np.linspace(0, 1, 100), np.linspace(0, 1, 100))
grid = np.c_[xx.ravel(), yy.ravel()]
predictions = model.predict(grid).reshape(xx.shape)

plt.contourf(xx, yy, predictions, alpha=0.8, levels=[0, 0.5, 1], cmap='coolwarm')
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, edgecolor='k', cmap='coolwarm')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('Decision Boundary')
plt.show()

Full Code Example

import numpy as np
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

# Generate synthetic data
np.random.seed(42)
X = np.random.rand(1000, 2)
y = (X[:, 0] + X[:, 1] > 1).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Define the model
model = Sequential([
    Dense(8, activation='relu', input_shape=(2,)),
    Dense(1, activation='sigmoid')
])

# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train the model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2, verbose=1)

# Evaluate the model
test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss: {test_loss:.4f}")
print(f"Test Accuracy: {test_accuracy:.4f}")

# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.title('Training and Validation Accuracy')
plt.legend()
plt.show()

# Visualize the decision boundary
xx, yy = np.meshgrid(np.linspace(0, 1, 100), np.linspace(0, 1, 100))
grid = np.c_[xx.ravel(), yy.ravel()]
predictions = model.predict(grid).reshape(xx.shape)

plt.contourf(xx, yy, predictions, alpha=0.8, levels=[0, 0.5, 1], cmap='coolwarm')
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, edgecolor='k', cmap='coolwarm')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('Decision Boundary')
plt.show()

Key Insights

  1. Binary Classification: The sigmoid activation ensures outputs are in the range [0, 1], suitable for binary classification.
  2. Visualization: Decision boundary plots help assess how well the model separates classes.
  3. Overfitting: Monitor the difference between training and validation accuracy to detect overfitting.

Leave a Comment