In this example, we’ll build and train a simple neural network using TensorFlow. This neural network will classify data points into two categories. It demonstrates how to use TensorFlow’s high-level Keras API for building, training, and evaluating a neural network.
Problem Statement
We will classify points into two categories based on their coordinates. For simplicity, we’ll generate synthetic data with clear boundaries.
Step-by-Step Guide
1. Import Required Libraries
import numpy as np
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt
2. Generate Synthetic Data
We’ll create a dataset with two classes separated by a nonlinear boundary.
# Generate synthetic data
np.random.seed(42)
X = np.random.rand(1000, 2) # 1000 points with 2 features
y = (X[:, 0] + X[:, 1] > 1).astype(int) # Label: 1 if x1 + x2 > 1, else 0
# Split the data into training and testing sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
3. Build the Neural Network Model
We’ll create a neural network with the following architecture:
- Input layer: 2 neurons (for 2 features)
- Hidden layer: 8 neurons with ReLU activation
- Output layer: 1 neuron with sigmoid activation
# Define the model
model = Sequential([
Dense(8, activation='relu', input_shape=(2,)), # Hidden layer
Dense(1, activation='sigmoid') # Output layer
])
# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
4. Train the Model
We’ll train the model for 50 epochs using the training data.
# Train the model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2, verbose=1)
5. Evaluate the Model
Once trained, evaluate the model on the test data to check its performance.
# Evaluate the model
test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss: {test_loss:.4f}")
print(f"Test Accuracy: {test_accuracy:.4f}")
6. Visualize Training Results
Plot the training and validation accuracy over epochs.
# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.title('Training and Validation Accuracy')
plt.legend()
plt.show()
7. Visualize Decision Boundary
To better understand how the model separates the data, visualize the decision boundary.
# Visualize the decision boundary
xx, yy = np.meshgrid(np.linspace(0, 1, 100), np.linspace(0, 1, 100))
grid = np.c_[xx.ravel(), yy.ravel()]
predictions = model.predict(grid).reshape(xx.shape)
plt.contourf(xx, yy, predictions, alpha=0.8, levels=[0, 0.5, 1], cmap='coolwarm')
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, edgecolor='k', cmap='coolwarm')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('Decision Boundary')
plt.show()
Full Code Example
import numpy as np
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
# Generate synthetic data
np.random.seed(42)
X = np.random.rand(1000, 2)
y = (X[:, 0] + X[:, 1] > 1).astype(int)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Define the model
model = Sequential([
Dense(8, activation='relu', input_shape=(2,)),
Dense(1, activation='sigmoid')
])
# Compile the model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# Train the model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_split=0.2, verbose=1)
# Evaluate the model
test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss: {test_loss:.4f}")
print(f"Test Accuracy: {test_accuracy:.4f}")
# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.title('Training and Validation Accuracy')
plt.legend()
plt.show()
# Visualize the decision boundary
xx, yy = np.meshgrid(np.linspace(0, 1, 100), np.linspace(0, 1, 100))
grid = np.c_[xx.ravel(), yy.ravel()]
predictions = model.predict(grid).reshape(xx.shape)
plt.contourf(xx, yy, predictions, alpha=0.8, levels=[0, 0.5, 1], cmap='coolwarm')
plt.scatter(X_test[:, 0], X_test[:, 1], c=y_test, edgecolor='k', cmap='coolwarm')
plt.xlabel('Feature 1')
plt.ylabel('Feature 2')
plt.title('Decision Boundary')
plt.show()
Key Insights
- Binary Classification: The sigmoid activation ensures outputs are in the range [0, 1], suitable for binary classification.
- Visualization: Decision boundary plots help assess how well the model separates classes.
- Overfitting: Monitor the difference between training and validation accuracy to detect overfitting.