**3. Generalised Overarching SHIA fast Fourier transform (GOSH-FFT)**

In this section we propose a new fast Fourier transform that, as we will see later, possesses the potential to be implemented in cortical architecture and thereby address the cortical convolution conundrum. Associated with SHIA, as described in Section 2.2, is a Cooley-Tucky type fast Fourier transform, named Generalised Overarching SHIA Fast Fourier Transform (GOSH-FFT). This novel fast Fourier transform employs the transform *M*10, as described in Section 2.2, as the critical mechanism that turns a Fourier transform into a fast Fourier transform.

#### 3.0.0.11

Suppose an image is represented on a SHIA of size 7*n*, where *<sup>n</sup>* <sup>≥</sup> 0. Let *<sup>m</sup>* <sup>≥</sup> 0 such that *nmodulom* <sup>=</sup> 0. Let *<sup>k</sup>* <sup>=</sup> *<sup>n</sup>* <sup>÷</sup> *<sup>m</sup>* and *<sup>M</sup>* denote *<sup>M</sup>*10*m*, the composite of *<sup>M</sup>*<sup>10</sup> ◦ *<sup>M</sup>*<sup>10</sup> ◦ ... m times. For(i:0:k)


A special case of GOSH-FFT was initially described in (Sheridan, 2007), with *m* = 1. The significance of the initial work was that it demonstrated the intrinsic connection between the Fourier transform and primitive image transformations of translation, rotation and scaling. It also turns out that another special case of GOSH-FFT, when *n* = 2*m*, will play a critical role in the core hypothesis of this paper. This special case, named Particular SHIA FFT (PaSH-FFT), is illustrated in Fig. 3.

A complete statement of Algorithm 3.0.0.11 is written in MatLab-like pseudo-code and can be found in Section 7 along with a mathematical proof that GOSH-FFT delivers a Fourier transform.

#### **4. Cortical implementation of contextual modulation**

In Section 1, we reviewed state of the art models of contextual modulation and concluded that these models implied the cortical convolution conundrum. We further motivate this conundrum by observing that as a consequence of Equation 3, a convolution of the entire visual field requires every minicolumn in Layer 2/3 of area V1 to receive an input from every other minicolumn of that layer. As there just are not enough connections to convolve the visual field in one cortical step, the convolution problem reduces to one of determining, given the known connectivity, what is the sequence of intermediate neurons an initial input must pass 8 Will-be-set-by-IN-TECH

A well known result to researchers in the field of signal processing is the Convolution Theorem, which relates convolution in the spatial domain to convolution in the frequency domain. For two functions, *f*(*x*) and *g*(*x*), let *F*(*x*) and *G*(*x*) represent the Fourier transform

In other words, the convolution of two functions in the spatial domain can be achieved by the

In this section we propose a new fast Fourier transform that, as we will see later, possesses the potential to be implemented in cortical architecture and thereby address the cortical convolution conundrum. Associated with SHIA, as described in Section 2.2, is a Cooley-Tucky type fast Fourier transform, named Generalised Overarching SHIA Fast Fourier Transform (GOSH-FFT). This novel fast Fourier transform employs the transform *M*10, as described in Section 2.2, as the critical mechanism that turns a Fourier transform into a fast Fourier

Suppose an image is represented on a SHIA of size 7*n*, where *<sup>n</sup>* <sup>≥</sup> 0. Let *<sup>m</sup>* <sup>≥</sup> 0 such that *nmodulom* <sup>=</sup> 0. Let *<sup>k</sup>* <sup>=</sup> *<sup>n</sup>* <sup>÷</sup> *<sup>m</sup>* and *<sup>M</sup>* denote *<sup>M</sup>*10*m*, the composite of *<sup>M</sup>*<sup>10</sup> ◦ *<sup>M</sup>*<sup>10</sup> ◦ ... m times.

A special case of GOSH-FFT was initially described in (Sheridan, 2007), with *m* = 1. The significance of the initial work was that it demonstrated the intrinsic connection between the Fourier transform and primitive image transformations of translation, rotation and scaling. It also turns out that another special case of GOSH-FFT, when *n* = 2*m*, will play a critical role in the core hypothesis of this paper. This special case, named Particular SHIA FFT (PaSH-FFT),

A complete statement of Algorithm 3.0.0.11 is written in MatLab-like pseudo-code and can be found in Section 7 along with a mathematical proof that GOSH-FFT delivers a Fourier

In Section 1, we reviewed state of the art models of contextual modulation and concluded that these models implied the cortical convolution conundrum. We further motivate this conundrum by observing that as a consequence of Equation 3, a convolution of the entire visual field requires every minicolumn in Layer 2/3 of area V1 to receive an input from every other minicolumn of that layer. As there just are not enough connections to convolve the visual field in one cortical step, the convolution problem reduces to one of determining, given the known connectivity, what is the sequence of intermediate neurons an initial input must pass

2. Perform a discrete Fourier transform over a sequence of sub images of size 7*m*;

**4. Cortical implementation of contextual modulation**

*f*(*x*) ∗ *g*(*x*) *F*(*x*)*G*(*x*) (4)

of *f*(*x*) and *g*(*x*) respectively. The Convolution Theorem states that,

**3. Generalised Overarching SHIA fast Fourier transform (GOSH-FFT)**

multiplication of the functions in the frequency domain.

transform. 3.0.0.11

For(i:0:k)

1. Apply *M* to the input;

is illustrated in Fig. 3.

transform.

3. Apply the inverse of *M<sup>i</sup>* locally.

(c) Intermediate step (d) Inverse Fourier transform

Fig. 3. Displays the results of applying the special case of GOSH-FFT, that is PaSH-FFT, to image of Fig. 2 (a), with n=4 and m=2. The four sub figures display intermediate results of PaSH-FFT: (a) on completion of first iteration of PaSH-FFT to Fig. 2; (b) Fourier transform on completion of second iteration; (c) on completion of first iteration of inverse PaSH-FFT; (d) Inverse PaSH-FFT on completion of second iteration.

through before being output as a convolved value. With the cortical convolution conundrum thus fully formulated, in this section we will establish a specification of a sufficient sequence of steps to address the issue. This specification will unfold in three steps. First, we will discuss how the SHIA transform *M*10 manifests in cortical architecture. We will then employ this manifestation to demonstrate how neural circuitry accommodates PaSH-FFT. Lastly, we will show how the cortical manifestation of PaSH-FFT supports long-range convolution.

#### **4.1 Cortical manifestation of M10**

A critical component of the fast Fourier transform, PaSH-FFT, is the transform *M*10. Consequently, it is an imperative of our argument that the redistribution properties of *M*10 be accounted for in the neural circuitry of the visual system. To this end we now argue that the required effects of *M*10 are accounted for by the long-range properties of patchy connections

**4.2 Cortical manifestation of PaSH-FFT**

Transform Supports a Convolution Model of Visual Perception

4.2.0.15

4.2.0.16

4.2.0.17

The next step in accounting for global convolution in cortical circuitry is to explore how PaSH-FFT manifests itself in cortical architecture. The raw data, at the lowest level of PaSH-FFT, are complex numbers that must be multiplied and added. The first issue to address is to justify our assumption that the operations being performed by a neuron could be represented as complex arithmetical operations on complex numbers. Specifically, PaSH-FFT requires that a neuron can be regarded as a mechanism capable of representing and manipulating complex numbers in accordance with the arithmetical operations of addition and multiplication. There are many ways in which to interpret neuronal function in terms of complex addition and multiplication. The model presented by (MacLennan, 1999) is adequate for the purposes of this paper, where it is shown how the representation of complex numbers can be encoded as the rate and relative phase of axonal impulse. From this encoding, complex multiplication is associated with the strength of a synaptic connection as the signal passes through it and complex addition is associated with the summing of the neuronal inputs. Thus at the lowest level of computation in our model, we assume that the operation being

<sup>191</sup> Cortical Specification of a Fast Fourier

performed by a neuron can be represented as complex addition and multiplication.

integration" actually occurs in the neuronal system remains open.

In area V1, each neuron makes use of information available to it in real time. There is evidence that contextual information is projected to widespread regions in V1 in an anticipatory manner. Since the spatial changes in the visual field tend to be predictable from previous visual inputs, anticipatory contextual inputs can arrive in time to be integrated in an adaptive manner with ongoing feedforward input. In order to express the properties of widespread contextual integration in a more formal manner, however, we will use the mathematical convenience of assuming that each of the distinct mathematical processes to be described occurs in a step-wise fashion. This more constrained approach allows not only each distinct part of the process to be formulated, but also formulates the inter-relationships between the various sub-processes. Although it is claimed that this approach is appropriate for the purposes of this paper, it must be acknowledged that the question of how such "contextual

At the finest scale of connectivity via short-range intrinsic connections, each neuron of a local map is treated as if it were connected to every other neuron minicolumn of that local map. While this is not literally true, considerations of poly-synaptic interactions at this local scale, and the real-time, anticipatory nature of visual processing means that it is a reasonable approximation of the functional connectivity. Consequently, we can assume that each neuron in a local map can sum the outputs of all other neurons in that local map which have been multiplied by unique complex numbers. We call such a collection of parallel computations a

This paper now needs to discuss three types of local computations, each of which is determined by the interpretation of the input signal and the collection of synaptic strengths which weight the input signal. If the input signal is the spatial domain and the weights are associated with a set of primitive roots of unity, then the resulting local computation is a Fourier transform, denoted *F*. (See Equation 1 for a definition of a Fourier transform.) If the

local computation. See Fig. 5, which is a schematic diagram of a local computation.

between columns of Layer 2/3 and similarly patchy extra-striate feedback connections to area V1.

## 4.1.0.12

It has been argued that the orientation pinwheel comprises a unitary organisational structure or local map in layer 2/3 of area V1 (Hubel & Wiesel, 1974); (Bartfeld & Grinvald, 1992); (Blasdel, 1992). When four pinwheels are reflected about their common borders, a saddle point arises at the centre of the four pinwheels. See Fig. 4.

Fig. 4. Displays a schematic diagram of the pinwheel like structures of visual area V1, extracted from Figure 10 page 43 of Bruce et al. (2003).

#### 4.1.0.13

In the macaque, the preferred response properties of V1 neurons can be influenced by activity from a wide extent of the visual field. A review of contextual modulation in the monkey demonstrated contextual modulation in V1 from long-ranges in the visual field (Alexander & Wright, 2006). The review was compiled from a number of experimental paradigms, including visual stimulation with long lines while the neuron's receptive field is occluded (Fiorani et al., 1992), surround only textures (Rossi et al., 2001) and colour patches placed distally to the neuron's receptive field (Wachtler et al., 2003). It was shown that the maximum range of contextual modulation measurable in V1 approaches a large extent of the visual field relative to a neuron's receptive field size or the local cortical magnification factor. Some experimental paradigms, such as the curve tracing effect (Roelfsema & Lamme, 1998); (Khayat et al., 2004), relative luminance (Kinoshita & Komatsu, 2001), and texture defined boundaries (Lee et al., 1998) show excitatory contextual modulation with 'tuning curves' that are flat out to the maximum distance tested. The functional connectivity that underlies this long-range contextual modulation in the monkey is likely to involve cortico-cortical feedback from higher visual areas working in concert with long-range intrinsic patchy connectivity. In the monkey, the feedback connections to V1 from higher visual areas incorporate inputs from a very large extent of the visual field (Angelucci et al., 2002); (Lund et al., 2003).

#### 4.1.0.14

In the analysis that follows, the combination of patchy intrinsic connections and patchy feedback connections are therefore assumed to enable transfer of visual information at ranges approaching the global scale of the visual field. Moreover, we assume that the quantity and distribution of these connections are adequate to deliver the effects of transform *M*10 at the scale of the visual field.
