**4. Parallel and distributed processes of tracking mean field dynamics of sparse connectivity**

This section discusses the case of sparse interconnection among processing units. In the case, a processing connects only with processing units in a small neighborhood. Sparsely interconnected processing units are partitioned to *K* clusters such that the cutting size of interconnections crossing distinct clusters is minimized. This formulates a typical problem of *K*-set partition to a sparse graph. Mean field dynamics for *K*-set graph partition has been proposed in [6]. As argued previously, parallel and synchronous computations by recurrent multilayer perceptrons can be obtained for tracking mean field dynamics of resolving *K*-set graph partition. Let {*Sk*}*<sup>K</sup> <sup>k</sup>*=<sup>1</sup> be the partitioned *K* clusters of sparsely interconnected

10.5772/57217

297

http://dx.doi.org/10.5772/57217

S2

Tracking Mean Field Dynamics by Synchronous Computations of Recurrent Multilayer Perceptrons

3 **c**

S3

*<sup>K</sup>* <sup>≪</sup> *<sup>N</sup>*. Figure 14 shows the flow chart of the proposed

S1

The size of nodes in *Sk* is |*Sk*| = *<sup>N</sup>*

corresponding recurrent computations.

is given in Appendix C.

**5. Conclusions**

*K* clusters.

SK

tracking mean field dynamics sparse connectivity. The idea follows parallel and distributed processes. This approach decomposes a large system to several sparsely connected small systems, updates mean activations inside each small system synchronously and updates decomposed systems sequentially. Suppose that each *Sk* has the same number of nodes.

approach. The halting condition states to compare the stability *χ*(**v**) with a threshold. An example with *N* = 12 for illustrating decomposition of a sparse system to three small systems

This paper has proposed a novel approach for tracking mean field dynamics by synchronous computations of recurrent multilayer perceptrons. The strategy is to introduce time delays and auxiliary variables and constructs equivalent recursive relations. This strategy essentially constructs recurrent multilayer perceptrons for tracking densely coupled mean field dynamics. The proposed approach is also extended to deal with large-scale sparsely interconnected mean field dynamics. In the beginning, all processing units are partitioned into *K* clusters by solving graph partition. The task is decomposed to *K* subtasks of synchronous computations and different clusters are sparsely connected by outer-inputs. The work combines synchronous updating inside each cluster with sequential updating among

Numerical simulations show that the proposed approach has successfully translated mean field equations of solving the graph bisection problem to a system of post-nonlinear recursive functions, and verified the consistency between the original mean field equations and

**Figure 13.** Partition of all nodes into *K* clusters to attain dense interconnection in each cluster.

K**c**

<sup>1</sup> **<sup>c</sup>** <sup>2</sup> **<sup>c</sup>**

**Figure 12.** The histograms of cutsize obtained by 50 executions of synchronous update and asynchronous update.

processing units and **c***<sup>k</sup>* be the outer-input of processing units in *Sk*. **c***<sup>k</sup>* contains nonzero elements if there exists a processing unit in *Sk* that is connected with units not in *Sk* and those nonzero elements are determined by mean activations of processing units outside *Sk*. After *K*-set graph partition, all nodes are reindexed according to {*Sk*}*<sup>K</sup> <sup>k</sup>*=1. Ideally, there is dense connectivity among processing units inside each *Sk* and sparse connectivity among {*Sk*}*<sup>K</sup> <sup>k</sup>*=<sup>1</sup> through {**c***k*}*<sup>K</sup> <sup>k</sup>*=<sup>1</sup> as illustrated in Figure 13.

In each cluster *Sk* when there is a processing unit connecting to processing units outside *Sk* according to the approach in section 2, all processing units inside *Sk* are evaluated directly by synchronous computations for fixed **c***k*. The approach which combines synchronous update of mean activations in side each *Sk* and sequential update among {*Sk*}*<sup>K</sup> <sup>k</sup>*=<sup>1</sup> is proposed for

**Figure 13.** Partition of all nodes into *K* clusters to attain dense interconnection in each cluster.

tracking mean field dynamics sparse connectivity. The idea follows parallel and distributed processes. This approach decomposes a large system to several sparsely connected small systems, updates mean activations inside each small system synchronously and updates decomposed systems sequentially. Suppose that each *Sk* has the same number of nodes. The size of nodes in *Sk* is |*Sk*| = *<sup>N</sup> <sup>K</sup>* <sup>≪</sup> *<sup>N</sup>*. Figure 14 shows the flow chart of the proposed approach. The halting condition states to compare the stability *χ*(**v**) with a threshold. An example with *N* = 12 for illustrating decomposition of a sparse system to three small systems is given in Appendix C.
