pfc mostly done -- just needs computational description

Author: rcoreilly

citedrefs.json

@@ -5,9 +5,11 @@ bibfile = "ccnlab.json"
 
 **Distributed representations** are simply the population of many individual [[neuron detector]]s, each of which is detecting something different. The aggregate pattern of output activity ("detection alarms") across this population of detectors can encode a wealth of information in _parallel_, without having to decide on a specific individual feature that is most relevant at the present moment.
 
-Thus, distributed representations are a key part of the overall strategy to combat the [[curse of dimensionality]] in the universal computational process of [[search]]. Also see the discussion of [[combinatorial vs conjunctive]] representations for additional dimensions along which distributed representations can vary, and the computational implications thereof.
+Because individual neurons are always integrating the inputs from many other neurons (e.g., roughly 10,000 in the [[neocortex]]), it is the population activity distributed across many neurons that will determine how other neurons respond: an individual neuron provides just one "vote" in this larger "democratic" system. Therefore, it is essential to understand the population-level activity patterns and not just the individual neural responses, to really understand what is being [[representation|represented]] in a given area.
 
-In the context of mental [[categorization]] to provide more abstract and efficient ways of representing behaviorally-relevant information, distributed representations can capture the amorphousness of a mental category, because it isn't just one single discrete factor that goes into it. There are many factors, each of which plays a role.
+At a computational level, distributed representations are a key part of the overall strategy to combat the [[curse of dimensionality]] in the universal computational process of [[search]]. Also see the discussion of [[combinatorial vs conjunctive]] representations for additional dimensions along which distributed representations can vary, and the computational implications thereof.
+
+In the context of [[categorization]] to provide more abstract and efficient ways of representing behaviorally-relevant information, distributed representations can capture the amorphousness of a mental category, because it isn't just one single discrete factor that goes into it. There are many factors, each of which plays a role.
 
 In other words, categorization is highly **polymorphous**: any given input can be categorized in many different ways at the same time. There is no such thing as _the_ appropriate level of categorization for any given thing. A chair can also be _furniture,_ _art,_ _trash,_ _firewood,_ _doorstopper,_ _plastic_ and any number of other such things.
 
@@ -60,7 +62,7 @@ One critical lesson from this is that:
 
 > any functional analysis based on the response properties of individual neurons should be regarded with a great deal of suspicion!
 
-Instead, the resounding message from the perspective of distributed representations is that only _population-level_ analysis that takes into account the responses across many neurons in a given area can provide an accurate picture of what that  area is really representing. For example, _representational similarity analysis_ (RSA) provides a useful framework for comparing how similar the overall pattern of neural activity is for different types of inputs ([[@KriegeskorteMurBandettini08]]).
+Instead, the resounding message from the perspective of distributed representations is that, as emphasized at the outset, only _population-level_ analysis that takes into account the responses across many neurons in a given area can provide an accurate picture of what that  area is really representing. For example, _representational similarity analysis_ (RSA) provides a useful framework for comparing how similar the overall pattern of neural activity is for different types of inputs ([[@KriegeskorteMurBandettini08]]).
 
 Interestingly, the use of _decoder_ models to analyze neural population activity is actually subject to many of the concerns as the analysis of individual neurons. This is because the diversity of neural responding within a given area generally allows a decoder trained to detect a particular pattern to perform above chance, even when the overall similarity structure of neural coding in a given area is not strongly aligned with that pattern.
 
@@ -75,4 +77,3 @@ The opposite of a distributed representation is a **localist** representation, w
 
 Even though the evidence for such neurons is fascinating (including other neurons for Bill Clinton and Jennifer Aniston), it does little to change our basic understanding of how the vast majority of neurons in the cortex respond. Clearly, when an image of Halle Berry is viewed, a huge number of neurons at all levels of the cortex will respond, so the overall representation is still highly distributed. But it does appear that, amongst all the different ways of categorizing such inputs, there are a few highly selective "grandmother" neurons! One other outstanding question is the extent to which these neurons actually do show graded responses to other inputs --- there is some indication of this in the figure, and more data would be required to really test this more extensively.
 
-

content/distributed representations.md

@@ -1,8 +1,8 @@
-The web of interconnected [[neuron]]s in the brain is known as a **network**, and there are a number of important [[computational cognitive neuroscience#emergent phenomena]] that arise in these networks, associated with different patterns of connectivity present in the [[neocortex]].
+The web of interconnected [[neuron]]s in the brain is known as a **network**, and there are a number of important [[computational cognitive neuroscience#emergent phenomena]] that arise in these networks, associated with different patterns of connectivity present in the [[neocortex]]. We can only really understand brain function in terms of these networks and populations of neurons.
 
 The _feedforward_ flow of neural activity from lower layers of neurons that are closer to the sensory inputs up to higher layers can be understood in terms of [[categorization]], where detailed sensory patterns are systematically transformed into more abstract categories that provide a more efficient [[linear algebra|basis]] for behavior.
 
-Thus, the network level of analysis is focused on the nature of _representations_, i.e., the properties of the patterns of neuron firing in a given area of the network. [[Distributed representations]] are critical for efficient encoding of high-dimensional information, as elaborated in the discussion of [[combinatorial-vs-conjunctive]] encodings. These different types of representations have important implications for [[generalization]] performance: how well the system can process novel inputs and situations.
+Thus, the network level of analysis is focused on the nature of [[representation]]s, i.e., the properties of the patterns of neuron firing in a given area of the network. [[Distributed representations]] are critical for efficient encoding of high-dimensional information, as elaborated in the discussion of [[combinatorial-vs-conjunctive]] encodings. These different types of representations have important implications for [[generalization]] performance: how well the system can process novel inputs and situations.
 
 [[Bidirectional connectivity]] enables additional emergent phenomena to arise, including [[attractor dynamics]] (e.g., top-down imagery and ambiguity resolution), and multiple [[constraint satisfaction]], which involves finding a representation that best satisfies constraints from the current inputs and prior learned knowledge encoded in synaptic weights. The constraint satisfaction process is an efficient, parallel form of [[search]] through representational space, providing [[optimized representations]] relative to purely feedforward activation flow.
 

content/media/fig_guo_etal_2018_alm_pt_ct_loops.png

@@ -3,7 +3,7 @@ Categories = ["Computation", "Activation", "Axon"]
 bibfile = "ccnlab.json"
 +++
 
-The use of **optimized representations** means that the system computes dynamically updated novel representations based on [[constraint satisfaction]] and/or [[error backpropagation#backpropagation to activations]] as the basis for processing inputs and driving cognition and behavior. The use of [[bidirectional connectivity]] in [[Axon]] automatically generates optimized representations, where the [[GeneRec]] analysis shows that the activity states automatically compute the _error gradient_ in the activation states of neurons.
+The use of **optimized representations** means that the system computes dynamically updated novel [[representation]]s based on [[constraint satisfaction]] and/or [[error backpropagation#backpropagation to activations]] as the basis for processing inputs and driving cognition and behavior. The use of [[bidirectional connectivity]] in [[Axon]] automatically generates optimized representations, where the [[GeneRec]] analysis shows that the activity states automatically compute the _error gradient_ in the activation states of neurons.
 
 There is a tradeoff in computation expended for generating these optimized representations on each trial, versus optimizing the number of trials for learning, using a much faster mode of per-trial processing, such as the standard single-iteration feedforward pass in a standard [[abstract neural network]]. Given the biologically-realistic approach taken in the [[Axon]] model, which uses bidirectional connectivity to perform [[error-driven learning]], these models are effectively committed to at least around 200 iterations worth of constraint-satisfaction processing per trial.