Published on: 2024-04-15
Description: Building a Neural Network Library in Rust
Written by: Fadi Atieh
#rust
In early 2024, I embarked on a project to develop a neural network library in Rust, with Python bindings for accessibility. This project was not just about the end product, but an exploration into the depths of neural network internals and the Rust programming language. This blog post chronicles the technical evolution of this project, highlighting key insights and learnings along the way.
The goal was to construct a neural network library that leverages Rust’s performance and safety features while offering the usability of Python through bindings. The library’s core components include a matrix manipulation module, neural network architecture, and an optimizer for training models.
By April 3, 2024, the project reached a pivotal milestone with the completion of backward propagation testing for the fully connected layers. This was a crucial step in ensuring that the network could adjust weights efficiently during training.
#[test]
fn test_fully_connected_layer_backward() {
let mut layer = FullyConnectedLayer {
weights: matrix::Matrix::new(vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0], (2, 3)),
biases: matrix::Matrix::new(vec![1.0, 2.0], (2, 1)),
weights_gradient: None,
biases_gradient: None,
forward_pass_input: None,
};
let input = matrix::Matrix::new(vec![1.0, 2.0, 3.0], (3, 1));
let output = layer.forward(&input);
assert_eq!(output.get_shape(), (2, 1));
assert_eq!(output.get_value((0, 0)), 15.0);
assert_eq!(output.get_value((1, 0)), 34.0);
let gradient = matrix::Matrix::new(vec![1.0, 2.0], (2, 1));
let backward_output = layer.backward(&gradient);
assert_eq!(backward_output.get_shape(), (3, 1));
}
The focus on backward propagation highlighted the importance of precise matrix operations. Implementing and testing backward propagation demanded an in-depth understanding of the mathematical underpinnings of neural networks, particularly the chain rule of calculus applied across matrices.
By April 12, 2024, the neural network was integrated with Python, marking a significant enhancement in usability. This step involved the creation of Python bindings using the pyo3 and maturin libraries, enabling seamless interfacing with Python-based data science tools.
[dependencies]
maturin = "..."
pyo3 = "..."
Integrating Rust with Python required a deep dive into foreign function interfaces (FFI). The choice of maturin and pyo3 provided a robust solution for building Python extensions, demonstrating the power of Rust in creating performant libraries with cross-language compatibility.
By May 19, 2024, the project culminated in a fully functional neural network library with demo images illustrating its capabilities. The library could handle both linear and non-linear function fitting, confirmed by Jupyter notebook demos.
This neural network library is implemented in Rust with Python bindings. It was created as an educational project to deepen my understanding of neural network internals and Rust programming.
The project underscored the significance of comprehensive testing and documentation. Through automation and clear documentation, the library was not only functional but also user-friendly, catering to both Rust and Python developers.
This project was a journey into the intricacies of neural networks and the Rust programming language. It reinforced the importance of robust testing, documentation, and the potential of Rust in creating high-performance, safe software. The integration with Python expanded its reach, making it accessible to a broader audience. As I look forward to future enhancements, the foundational insights gained here will undoubtedly guide the way.
For those interested in exploring the library, visit the GitHub repository and try out the Jupyter notebook demos to see the neural network in action.