A Hardware Implementation of a Mixture of Experts
By Antony Savich, PhD, May 2008 - 2011
Abstract:
Recent applications of Artificial Neural Networks (ANN) have increased the size, performance and
difficulty requirements of single networks used to solve complex problems of today. Previous research
focused on optimizing single ANN performance and hardware requirements for accelerated training cycles and
use of cheaper implementation platforms (both on generic processors and more recently on reconfigurable
hardware). After training these single networks were inflexible to change, and in general it was
demonstrated difficult to use the complete parallelism of single ANN algorithms and fit a large flexible
fully-parallel network design into the confines of a single chip solution.
This research focuses on realizing solutions for complex problems using, instead of a single large ANN,
a subdivision of the solution into a group of smaller networks in a Mixture of Experts configuration,
with the flexibility of modularly adding new functionality after initial training process. The goal of
this research is to demonstrate the ability of such a solution to have a performance advantage over current
implementations with single network topologies by using smaller faster networks, and flexibility
advantage over monolithically trained networks in adapting to new conditions by expanding the
problem/solution set. This will be accomplished through the implementation of a fully functional
single FPGA Mixture of Experts system and its subsequent performance analysis on various sample problems.
