In the previous message but one, I said that there would be three important principles enunciated. In the last message I argued that people who have low error-covariance with respect to each other are also likely to be compatible with each other. I hope you gave that some thought. It seems likely to me that people so carefully matched for estimation and decision-making purposes would have some important relationship when meeting socially, and I say this would be a good one.
The problem of actually matching such people is complicated by the fact that bipartite matching is an O(3) problem, that is to say that the amount of processing required increases as the cube, the third power, of the number of points. Even an O(2) approximation would take too much processing if large numbers of people are to be matched. What is wanted is a linear-time algorithm. Genetic, evolutionary or steepest descent algorithms are possible linear-time approximations, but a problem with this is the intransitivity of human affections. If you bring A closer to B because they could be happy together, and also bring A closer to C, because they could be happy together, then you would also move B closer to C, when perhaps they are completely incompatible.
There is also a tendency for this kind of algorithm to bring all points together, towards what might called a centre of mass.
It seems that the latter problem can be solved by identifying the centre of mass and just pushing points away from it. Points are moved towards other points, and thereby tend to move towards the centre of mass, but can be explicitly pushed away from it, forcing them to move “sideways”, “orbiting” the centre of mass, rather than falling into it.
The first problem is also solvable. We can define points not as individuals, but as social environments. If the small network of individuals A is compatible with the small network of individuals B, and is also compatible with the small network of individuals, C, then an evolutionary algorithm will tend to bring together B and C. But these two small social neighbourhoods are in fact likely be mutually attracted. The thing that attracted person A1 to B1 and to C1 cannot keep the whole sets B and C from being attracted, since there will probably be some Bx who is attracted to Cy.
For example, consider two small opera companies in the same city, each with a single lead contralto, Bertha and Brunhilde.
They might both be in love with a baritone from the first company, Alfredo. What is the chance that Bertha and Brunhilde will be best friends. That is most unlikely. Now consider the whole of company A and the whole of company B. Petty loyalties and jealousies aside, there is an excellant chance that some soprano, Alicia, from one company will love a bass, Bruno, from the second.
Individual members of both companies will have good reason to be compatible, they will have shared interests, some shared experiences, often with complementary abilities. A man from one company might happily be singing duets with a woman from the other, all the while Bertha and Brunhilde are glowering at each other.
Having solved the intransitivity problem, we can then treat it as the kind of simpler problem in which pushing these points out from the centre can work.
There is a big IF here. This assumes that the problem of grouping people into small social networks has been solved. Well, it doesn’t have to be completely solved. Whatever approximation we have can be improved by the above process, iteratively.
An excellent choice for a local neighbourhood is a set of people bound together by minimal error-covariance.
This suggests an iterative solution, in which good neighbourhoods are found, then used as a basis for an improved approximation.