Lambda as a Complexity Control in Negative Correlation Learning.

Abstract

The parameter in negative correlation learning (NC) controls the degree of co-operation between individual networks. This paper looks at the way the choice of in the NC algorithm affects the complexity of the function NC can fit, and shows that it acts as a complexity control allowing smooth adjustment of the network between one large back-propagated network, and many independent networks individually trained before combination. The effect of the base complexity of the individual networks, and the number of networks that are trained co-operatively, on the algorithms overall complexity are also empirically investigated. Empirical results are presented from 4 different datatsets. The way in which the performance changes as parameters change, and the point at which over-fitting sets in give us information about the complexity of the model the algorithm can fit at those parameter settings.