Computer Architectures for Mobile Computer Vision Systems

Mobile vision is enabling many new applications such as face recognition and augmented
reality. However, the performance of mobile processors is limiting the capability of mobile vision
computing.

This dissertation presents an in-depth analysis of mobile computer vision applications and proposes
novel hardware and software optimizations with the goal to increase mobile computer vision processing
capability. We present the Michigan Visual Sonification System, a new mobile vision application that provides
navigational aid to the visually impaired. The development of this application gives insights into
the nature of mobile vision applications including the tradeoffs between performance and energy on
mobile processors. We then present MEVBench, a mobile vision benchmark suite that we built to determine
the computational characteristics of various mobile vision kernels. This analysis exposes the vector
reduction operations, the imbalanced task or thread parallelism and the 2D spatial locality in memory
accesses, all which we exploit in the pursuit of highly efficient mobile vision architectures.

Armed with a deeper understanding of computer vision processing, the core of this thesis focuses on software
and hardware based optimization to improve the efficiency of mobile vision processing. We begin
the optimization with a software optimization known as Single Eigenvector Solver (SEVS), an algorithm
that reduces the computation for augmented reality applications. We begin the hardware optimizations
with EFFEX, a heterogeneous multicore architecture that utilizes vector reduction functional units and a
2D memory controller to improve the efficiency of feature extraction. We close with the Efficient Vision
Architecture (EVA). EVA expands the EFFEX architecture by adding more custom accelerators for vision
operations beyond feature extraction. It also utilizes the tile cache to allow for both 1D and 2D spatial
locality in cache accesses. Overall, this dissertation demonstrates that an application specific approach
to processor design can create a flexible programmable design with significant efficiency improvements
in mobile vision performance when compared to currently available mobile processors. These works
enable the development of richer more capable mobile vision systems.