Update on ripple-spreading (or wave-spreading) representation scheme: Plans by the end of this year

Just had some talks with Ezequiel, Lionel and Patricia, discussing how to push ahead the study on the ripple-spreading representation scheme. Basically, we have two lines to follow.

 

The first line is to study the potential of using this ripple-spreading representation scheme to model random networks. A good thing about this ripple-spreading model of random networks is: it may be a deterministic model as required, i.e., once a set of ripple-spreading related parameters are fixed, the resulted network topology will be unique. This is unlike many other complex network properties, such as degree distribution and preferential attachment, which can only roughly determine the category of the resulted network, but can not give a unique topology. Another thing is, this ripple-spreading model can easily integrate spatial factors, as well as temporal factors if required, into the network topology. Actually, one of Lionel’s suggestions is we should compare the ripple-spreading model with his spatial model of random networks, in order to better understand the mechanism and features of the ripple-spreading model. Last but not least, the new model is expected to be very friendly to the design of genetic algorithms (GAs) for optimizing network topology, because, owing to the deterministic ripple-spreading model of random networks, rather than to record the entire network connection information, a chromosome only needs to represent a few ripple-spreading related parameters, and, rather than to evolve the network topology directly, the GA only needs to evolve these parameters. This will probably result in a higher memory-efficiency, less feasibility problems, and a simpler design of evolutionary operators. Actually, to design a highly efficient GA to optimize network topology is an important motivation for studying the ripple-spreading model.

 

In the first stage, we will study a simple model with a few key ripple-spreading related parameters, such as epicentres of initial stimulating ripples, threshold of nodes, and amplifying factor of nodes. Two behaviours (being activated and being linked) and three sub-models (deterministic sub-model, semi-deterministic sub-model, and stochastic sub-model) will be initially investigated. Theoretical study and statistic analysis of the features of the model and the effects of the parameters will be the focus. We probably start with a fixed distribution of nodes, and then move onto random distribution of nodes. To do this work, I will collaborate with Ezequiel and Lionel, and also Seth, if you would like to : ), closely in the next few months.

 

Potential papers: I’m now working on a conference paper for the WCCI2008. This paper will explain the basic idea of the ripple-spreading model of random networks and the associated GA for network topology optimization. When we finish the work of the first stage, we plan to write a paper on the statistic features of the simple ripple-spreading model.

 

The second line is to apply the idea of combining binary GAs with a problem casting process based on the ripple-spreading scheme to a wide range of combinatory optimization problems. Of course those ATC (air traffic control) problems will be our first choice. Actually, work is now going on or will start soon to design some effective ripple-spreading GAs for aircraft arrival sequencing, airport gate assignment and airline route network management. Another case study of applying this ripple-spreading GA is to work with Patricia to see if it is of any use for constructing neural-networks.

 

Potential papers: There could be another conference paper for the WCCI2008, which will talk about the preliminary study of applying the ripple-spreading GA to the airport gate assignment problem. Probably, Patricia and I will do a co-authored paper on her study on the design of neural-networks.

‘Liquid State’ Networks in Space

Paper 2 - Peter, Chris, Ezequiel, Seth and Patricia

Echo/Liquid state networks are a neural network paradigm involving a large, random network receiving inputs relevant to a task. The network’s random dynamics expand this input into a high dimensional space (hence the term liquid, as in the complex ripples formed when a liquid is perturbed). A simple, linear output mapping from this random ‘resevoir’ of dynamics has been shown to be sufficient to perform highly non-linear processing on the input data.

Typically, these random networks have been embedded within a space (via a spatial constraint on their topology), although this appears not to have been systematically studied within the liquid state literature. We would therefore eventually like to examine how spatially organized dynamics can affect the capabilities of a neural network reservoir. For a first step, however, we will be carrying out a more traditional Evolutionary Robotics type of task, whilst applying just one component of the echo state network formalism. This entails restricting the network dynamics to just one steady state attractor, forcing it to perform its behaviour using only its transient dynamics around that attractor. Like water, in the absence of input or perturbation, the network will return to a steady state eventually.

By offering an experimental example which shows the capacity for a single-attractor network to produce multiple behavioural modes, the work will demonstrate an interesting counter-example to the more traditional notion that different forms of activity must correspond to different attractor states. It will also provide a basis for the investigation of other parts of the Echo State network paradigm, and how they can be of use to the investigation of the spatially embedded dynamics of neural phenomena in sensorimotor, closed-loop systems.
This could be targeted to a conference such as Simulation of Adaptive Behavior, which has a deadline in January.

The following reference provides an overview of the ESN paradigm:
H. Jaeger(2001): The “echo state” approach to analysing and training recurrent neural networks. GMD Report 148, German National Research Center for Information Technology (pdf).

Paper 5. Using wave-spreading for encoding large networks

People: Patricia, Xiaobing, Seth & Phil

1) Paper motivation and main message

Our first idea is to develop an efficient non-conventional scheme, namely wave-spreading scheme, to connect nodes in a non-spatial GasNet (NSGasNet) and apply this novel non-spatial GasNet to a real problem which requires larger networks than the problems that have been addressed so far.

2) Target journal

It will depend on the focus of the paper. For instance, if we decide to apply the novel network to an optimization task involving electric power systems networks, then we could try a power systems journal. If we decide for another task, then maybe we could try a NN, complex system or evolutionary computation journal.

3) Background reading, lit survey

Literature on GasNets, Network Flows, Graph theory and Complex Network Theory.
Depending on the application task we could also include readings on power systems distribution networks.

4) Paper structure

Will depend on the scope. But a tentative one could be:
Introduction; Description of the problem to be tackled; Non-Spatial GasNet modelling; Wave-spreading coding scheme; New gas network modelling; Simulation and analyses.

5) Division of labour

Xiaobing may focus on the wave-spreading coding scheme, as well as maybe develop a GA to optimize the NSGasNet network.
Patricia could be in charge of mapping the problem into a GasNet structure and also helping Xiaobing to create a variant of the wave-spreading coding scheme.

6) Timeline

Immediate action would be literature search, discuss and confirm the possibility of applying the wave-spreading coding scheme to the NSGasNet and work out a plan and timetable.

A first tentative timeline could be:
Oct/2007 to Dec/2007
- coding, simulation and analyses.
Jan/2008 to Feb/2008
- write up a paper, and submit somewhere.

Any comments or suggestions are welcome!

Modularity in spatial networks

Hi Lionel, I managed to track down that paper that re-interprets the neural complexity measure in terms of binary graph properties. I am still working through it.  It talks about the small-world property near the end and covers a lot of the stuff I wanted to do. It also seems to suggest that complexity will be higher in a random graph than in a lattice which is the exact opposite of my paper…phweh!  Anyway the reference is

“Topologial approach to neural complexity” M De Lucia et. al. in Physica Review E, 71 , 016114 (2005)

Chris

PS

I will upload some of the orginal papers on the measure later

Paper 4. Wave-spreading representation scheme

First stab at organizing work for this paper.

1. Motivation and main message

The idea of the paper is to present the wave-spreading method as a potentially useful technique for spatial representation of data amenable for use in GAs and other stochastic optimization techniques.

1. Test the wave-spreading representation scheme on the TSP and some ATC problems such as aircraft arriving and gate assignment. (Partly done)
2. Based on the tests, develop a generic framework about how to apply
the new representation scheme to various problems. Question: When is the method appropriate? What are its limitations?

3. (Possibly in different paper) Use the wave-spreading idea to construct random networks, analyze the network properties, compare with other random network modelling methods, and design an effective and efficient GA to optimize the network topology.
4. Use the new random network modelling method to study some real complex network systems. To this end, will keep the mind open to further discussions and collaborations with other teams.
- Target journal

Depending on emphasis on the points above…

1. Case studies on the TSP and the ATC problems may target journals such as Computation & Operations Research, journal of Operations Research Society, IEEE Transaction on Evolutionary Computation, and IEEE Transaction on Intelligent Transportation Systems. There could be a few case study papers.

2. The work on the generic framework may target journals on operations
research or artificial intelligence.
3. The work on random network modelling may target journals on complex
systems and physics, such as Physics Review or Complexity?

4. The generic presentation of the technique should perhaps be targetted for an evolutionary computation journal (EC, IEEE Trans EC, etc).

Paper structure

1. Case study paper: Introduction; Problem formulation; Why this technique? Wave-spreading representation scheme; Design of evolutionary algorithms;
Simulations, analyses and conclusions.
2. Paper on the generic framework (rough idea): Survey existing representation methods; Explain the ripple-spreading representation scheme; Mathematically describe the new representation scheme; Theoretically study the properties of the new representation scheme; Summarize and analyze the results of some applications; Conclusions.

- Division of labour
Xiaobing may work on most designing, coding, simulating, analyzing and writing
tasks. Regular discussions with Lionel and Ezequiel, particularly, need to work together with Lionel on random network modelling and mathematic analyses.

- Timeline

1. From now on to next Feb: finish at least two case studies on the wave-spreading representation scheme. Work on the generic framework and explore the idea of using the wave-spreading method to model random networks.

Review paper on complex spatial networks

my idea is to provde a chapter for a review paper on complex spatial networks. this would include on our side a chapter on cities and networks which i am planning to wite within the next few months. the collaborative side would be then to include this chapter on a wider review where each relevnt subject from each group could be incorporated. this would be like the review paper by Bocarretti et al which I have seent around some time ago. i’ll be sending the chapter son

thanks

List of short-term meso-collaborations

This is a list of possible papers emerging from discussions at the New Forest meeting. The list defines some working directions for collaborations that should now be taken by the people involved.

Updates on each item will be posted by the people involved.

FOR THOSE NOT PRESENT AT THE MEETING: Please, go through the list and send your comments and additions.

1. Is firing really necessary? Signal propagation across IF networks.
Bringing together signal propagation studies.
Spatial element: Role of space in transition between regimes.
People: Steve, Chris, Ezequiel.

2. ECHO-state machines in space
use “ECHO-like” dynamics for evolutionary task
Comparative with spatial embedding situation.
People: Chris, Peter, Patricia

3. Modularity in spatial networks
Information-theoretic measures of modularity and network theoretical community structure measures.
People: Chris, Lionel, Ezequiel, Netta, Seth, Rui.

4. Wave-spreading representation scheme
Why and when does it work?
Comparative studies, plus analysis, looking for limitations
People: Xiaobing, Ezequiel, Lionel

5. Using wave-spreading for encoding large networks
comparative study of different encodings in large network task.
People: Patricia, Xiaobing

6. Deconstructing Artificial Neuromodulation in terms of Evolvability.
What contributes to the evolvability in GasNets (space, timescales, multiplicative interactions)? Analysis of fitness landscapes.
People: Chris, Patricia, Steve, Lionel

7. Position paper in neuromodulation.
Analysis of ideas in neuromodulation.
People: Chris

8. Applying complex network techniques to data on citations
Exploration.
People: Rui, Lionel

9. Repackaging Lionel’s framework in non-technical format
Extracting most interesting results, derive non-trivial implications
People: Lionel, Ezequiel, Seth

10. Dynamics and transformation and re-configuration (history-dependence) in spatial networks
People: Ezequiel, Seth (Mariusz), Steve.

Visits:

Chris to Leeds, Brighton
Seth to Brighton
Rui to Brighton
Patricia to Southampton
Peter to Southampton

Steve to Brighton

Patricia to Leeds

People named should start discussions immediately about practical aspects of pursuing the papers. In particular, they should resolve the following issues:

- Paper motivation and main message

- Target journal

- Background reading, lit survey

- Paper structure

- Division of labour

- Timeline

In a week’s time there should be 10 new posts describing these items!

Research statement of CASA

We are interested in social systems at geographical scales, specially in city and regions. We are particularly interested in networks which are embedded in geographical space. Our work goes in two directions: on one side we analyse large datasets and on the other we develop models to explain behaviour on these datasets. I have also worked in social networks, as a means to develop analytical and computational techniques which we could feedback into networks which are geographical in nature. The current toolbox is data mining, statistics and simulation and we are pursuing several collaborations to explore these directions. More information from our web pages: http://www.casa.ucl.ac.uk/

Steve’s research interests and current work

My work focuses on biologically constrained neural network models and I specialize in investigating the phenomenological properties of the unitary parts which provide functionality in the network as a whole. In particular I consider how adaptation, regulation and plasticity may be combined to generate robust networks capable of carrying out cognitive tasks. I am particularly interested in the processing and learning of sequential information, and have a passing aquaintanceship with the evolution of language faculty in the brain.

Currently (July 07) I’m writing up a paper concerning how the detailed time course of communication between model neurons effects the propensity of a network to globally oscillate. In another paper in prep, this result is being used to demonstrate how, unlike in existing computational models, several coupled networks may avoid entering an oscilatory state in keeping with established biological results. This work is also being integrated with a previous more abstract study (see project papers section) concerning the effects of cell homeostasis on global network properties. 

SECSE Publications, Steve Womble

 HOMEOSTATIC REGULATION OF NEURONAL EXCITABILITY, CORRELATED PATTERNS AND GLOBAL ATTRACTORS, CNS 2006 (Edinburgh).  An extended abstract (includes main results) of conference presentation, at the annual computational neuroscience society meeting.