High Capacity Associative Memories and Small World Networks

Models of associative memory usually have full connectivity or if diluted, random symmetric connectivity. In contrast biological neural systems have predominantly local, non-symmetric connectivity. Here we investigate sparse networks of threshold units, trained with the perception learning rule. The units are arranged in a small world network, with short path-lengths but cliquish connectivity. The connectivity may be symmetric or non-symmetric. The results show that the small-world networks with non-symmetric weights perform well as associative memories. It is also shown that in highly dilute networks with random connectivity, it is symmetry of the weights, rather than symmetry of the connectivity matrix, that causes poor performance.