Distributed redundant representations in man-made and biological sensing systems (bibtex)
by
Abstract:
The ability of a man-made or biological system to understand its environment is limited by the methods used to process sensory information. In particular, the data representation is often a critical component of such systems. Neural systems represent sensory information using distributed populations of neurons that are highly redundant. Understanding the role of redundancy in distributed systems is important both to understanding neural systems and to efficiently solving many modern signal processing problems. This thesis makes contributions to understanding redundant representations in distributed processing systems in three specific areas. First, we explore the robustness of redundant representations by generalizing existing results regarding noise-reduction to Poisson process modulation. Additionally, we characterize how the noise-reduction ability of redundant representation is weakened when we enforce a distributed processing constraint on the system. Second, we explore the task of managing redundancy in the context of distributed settings through the specific example of wireless sensor and actuator networks (WSANs). Using a crayfish reflex behavior as a guide, we develop an analytic WSAN model that implements control laws in a completely distributed manner. We also develop an algorithm to optimize the system resource allocation by adjusting the number of bits used to quantize messages on each sensor-actuator communication link. This optimal power scheduling yields several orders of magnitude in power savings over uniform allocation strategies that use a fixed number of bits on each communication link. Finally, we explore the flexibility of redundant representations for sparse approximation. Neuroscience and signal processing both need a sparse approximation algorithm (i.e., representing a signal with few non-zero coefficients) that is physically implementable in a parallel system and produces smooth coefficient time-series for time-varying signals (e.g., video). We present a class of \emphlocally competitive algorithms (LCAs) that minimize a weighted combination of mean-squared error and a coefficient cost function. LCAs produce coefficients with sparsity levels comparable to centralized algorithms while being more realistic for physical implementation. The resultant LCA coefficients for video sequences are more regular (i.e., smoother and more predictable) than the coefficients produced by existing algorithms.
Reference:
Distributed redundant representations in man-made and biological sensing systemsC.J. Rozell. Ph.D. thesis, Rice University, May 2007.
Bibtex Entry:
@PhdThesis{rozell.07,
  author = 	 {Rozell, C.J.},
  title = 	 {Distributed redundant representations in man-made and
biological sensing systems},
  school = 	 {Rice University},
  year = 	 2007,
month = {May},
  address =	 {Houston, TX},
  url = {http://siplab.gatech.edu/pubs/crozell_PhDThesis_final.pdf},
  abstract={The ability of a man-made or biological system to understand its
environment is limited by the methods used to process sensory
information. In particular, the data representation is often a
critical component of such systems. Neural systems represent sensory
information using distributed populations of neurons that are highly
redundant. Understanding the role of redundancy in
distributed systems is important both to understanding neural systems
and to efficiently solving many modern signal processing problems.

This thesis makes contributions to understanding redundant
representations in distributed processing systems in three specific
areas. First, we explore the robustness of redundant representations
by generalizing existing results regarding noise-reduction to
Poisson process modulation. Additionally, we characterize how the
noise-reduction ability of redundant representation is weakened when
we enforce a distributed processing constraint on the system.

Second, we explore the task of managing redundancy in the context of
distributed settings through the specific example of wireless sensor
and actuator networks (WSANs).  Using a crayfish reflex behavior as a
guide, we develop an analytic WSAN model that implements control laws
in a completely distributed manner.  We also develop an algorithm to
optimize the system resource allocation by adjusting the number of
bits used to quantize messages on each sensor-actuator communication
link.  This optimal power scheduling yields several orders of
magnitude in power savings over uniform allocation strategies that use
a fixed number of bits on each communication link.

Finally, we explore the flexibility of redundant representations for
sparse approximation.  Neuroscience and signal processing both need a
sparse approximation algorithm (i.e., representing a signal with few
non-zero coefficients) that is physically implementable in a parallel
system and produces smooth coefficient time-series for time-varying
signals (e.g., video). We present a class of \emph{locally competitive
algorithms} (LCAs) that minimize a weighted combination of
mean-squared error and a coefficient cost function. LCAs produce
coefficients with sparsity levels comparable to centralized algorithms
while being more realistic for physical implementation. The resultant
LCA coefficients for video sequences are more regular (i.e., smoother
and more predictable) than the coefficients produced by existing
algorithms.}
}
Powered by bibtexbrowser