Suggestions from Others
I am sometimes challenged by people who say that if these ideas are any good then I should be trying to make money with them instead of posting them up here. I think somebody could make money with these ideas, but not me. I’m an ideas person; anything to do with money makes me uncomfortable.
In the kind of society I dream about people could easily find compatible people to work with, and in that society maybe I could find partners to handle all the business things, leaving me to do the research. But we don’t live in that kind of society yet, and my priority is not to make money but to help move us all from here to there — to help bring about that kind of society.
One of my correspondents suggested that I should translate my ideas into software and sell the software. I actually spend a lot of my time writing software, but it is research-grade, proof-of-concept software, at best, and definitely not ready for prime time. When it is more or less ready I intend to post it up here for anyone who wants it, under the GNU public license.
That license explicitly permits people who want to enhance the software and sell the enhanced version to do so, and therefore there will be an opportunity for enterprising individuals to make some money from these ideas, something that I don’t object to at all.
Lots of people make money by adding value to something that is basically free. Linux is free, and comes with full source code, under the GNU public license, but it is also the foundation of quite a few businesses, such as Red Hat in Canada, which make a living from packaging, porting, and setting up the software.
CYGNUS, which I think stands for “Cygnus, your GNU user support”, or something like that, a recursive acronym , is a business entirely devoted to the support of free software from the Free Software Foundation.
This question recently came up in another context, where there was some question about paid and volunteer staff working together. This brought to mind some of my experiences from student days.
I was a reporter on the Peak, the SFU student newspaper for many years, covering university senate meetings and miscellaneous news, and I remember being quite disturbed my first year when I found out that the editor was paid a salary — the rest of us were all volunteers and that didn’t seem fair. For one thing it seemed to violate the laws of supply and demand : any number of us would have served as editors for nothing!
But over the years I came to care more about the quality of the paper than for strict fairness, and I realized that the supply of good responsible editors is rather limited — lots of people want power, or a soapbox, but few just want to do a good job. I served on the editor-selection committee for several years, so I got to see how this worked. Putting out a really good paper seemed to require paying the editor, and so I eventually came around to the view that we were doing the right thing by paying for the good editors we wanted.
To me this whole question arises because of our poor social technology.
Ideally society should make the best possible use of each person in it and make sure each of them is paid appropriately. Currently our society does NOT make very good use of people, and the mechanism for paying or supporting them is quite poor, but I think all that will change. I want it to change, and I want it to change soon, so I intend to do everything I can to help change it. That’s why I have put up all these web pages and written so much about social technology.
I think any use of these ideas will help change the world, and that includes business use, so I have no objections at all to business people taking an interest.
But exactly what is it that might interest them?
Consider a simple matching problem that might just occur in the real world, even in a business. Suppose we had a business with 1000 employees. A (somewhat artificial, but plausible) problem would be to match each person with one ideal team-mate or co-worker.
The data I’d like to have, to work from, would be a long set of test scores or questionnaire answers for each person, which could then be used to generate a dense 1000 x 1000 weight matrix. The matrix could be used by a fast weighted bipartite matching algorithm such as one of Goldberg’s.
But I anticipate it being very hard to collect that kind of data, so I’ve written programs that take in more common forms of data, such as who-has-worked-with-whom, and how successfully, how much e-mail do two people exchange, and various other things that could be mined from project management records.
The results of data collection will normally be sparse matrices, containing rather noisy data. For example, to collect data for the current 1000 employees one may need to mine project records including data on many former employees, and some current employees may be represented only by data involving former ones.
But there are a lot of ways to regularize, normalize, estimate, and reduce an assortment of noisy data into something more manageable, which I can eventually turn into a dense matrix of estimates, which can then be evaluated in many rather obvious ways (and a few more subtle ones such as Kalman filtering).
The whole purpose of this would be to produce suggested matchings, including probability estimates, which could be used to guide team creation, and the success or failure of these matches would be fed back into the process.
There are ways of extending this process to very large matrices with special properties, such as block or band or tridiagonal forms, and that allows me to dream of doing it for society as a whole, which I insist is not just a fantasy but a future possibility.
I should remark that my own work on social networks is based entirely on simulations with generated data or whatever I can find on the net, mostly in machine learning archives. So while I think I do have techniques that should work in the real world, I can’t prove they will, yet.
Also, my work on social networks is only one of a few projects that often share data and programs, so some of the software I think could be used for social network applications has only been tested on data from an entirely different application such as semantic network analysis.
Another part of what I do involves specifying criteria for matching co-workers, and the chief criterion is error-covariance-minimization.
That is to say, my suggested ideal pairing of two co-workers is one in which the two of them have contra-variant error behaviour patterns, so that working as a team they can produce better decisions than either of them could alone.
I am sure you recognize that decisions made by a committee very often poor ones, but this is not necessarily so, and many attempts have been made to change things so that people working together could produce better decisions as a result of their cooperation. The most famous of these attempts was the RAND Corporation’s work on the Delphi Method, which made careful use of written submissions and anonymity to try to force objective evaluation of each other’s ideas.
But the Delphi Method ignores error-covariance . If you put together a committee of people who make similar mistakes, you will get a committee more likely to make those mistakes than any individual on it.
So, ideally the data in my 1000 x 1000 dense weight matrix for matching co-workers should represent the probability of the pair’s errors cancelling out. This goes back to the information theory of Shannon and Weaver, and their discussion of the ability to synthesize reliable communication channels from redundant noisy channels — when it comes to decision makers and information or knowledge workers, employees can be thought of as communications channels between the organization and the rest of society.
One example I use in a yet unposted-page is the familiar problem of estimating the future prices of stock, bonds, or commodities.
An analyst who produces such estimates is something of a channel or connection to the whole rest of the social network. There can be little doubt that all of the information necessary to predict a stock price is out there, somewhere, in the global social network, and by whatever means he uses, the analyst is attempting to extract that information from the network.
So, suppose you put together a committee of two analysts, or four, or 256 — would you get a better result? Would that committee be a synthetic channel of greater reliability than any individual analyst? Not today, it wouldn’t, because nobody knows how to make a committee work better than the people in it. But Shannon and Weaver’s work suggests (I’m tempted to say “proves”) that it is possible. I think it is possible, not only in theory, but in practice, and it all boils down to estimating error-covariance.
I think what I’ve just described is all possible, and not even very difficult. Prospective businessmen wanting to do something with these ideas need not even wait for my own software to be posted — I think there is enough information contained in these web pages to drive a R&D project, and I’d also be glad to volunteer suggestions for further research.
Copyright © 1998 Douglas P. Wilson
Copyright © 2009 Douglas Pardoe Wilson
Other relevant content:
Please see these web pages:
The main Social Technology page.
Find Compatibles , the key page, with the real solution to all other problems explained
Technological Fantasies , a page about future technology
Social Tech a page about Social Technology, technology for social purposes. I think I was the first person to use this phrase on the Internet, quite a long time ago.
Roughly corresponding to these web pages are the following blogs :
Social Technology the main blog, hosted on this site, with posts imported from the following blogger.com blogs, which still exist and are useable.
Find Compatibles devoted to matching people with friends, lovers, jobs, places to live and so on, but doing so in ways that will actually work, using good math, good algorithms, good analysis.
Technological Fantasies devoted to future stuff, new ideas, things that might be invented or might happen, such as what is listed above and below.
Sex-Politics-Religion is a blog about these important topics, which I have been told should never be mentioned in polite conversation. Alright that advice does seem a bit dated, but many people are still told not to bring up these subjects around the dinner table.
I believe I was the first person on the Internet to use the phrase Social Technology — years before the Web existed.
Those were the good old days, when the number of people using the net exceeed the amount of content on it, so that it was easy to start a discussion about such an upopular topic. Now things are different. There are so many web pages that the chances of anyone finding this page are low, even with good search engines like Google. Oh, well.
By Social Technology I mean the technology for organizing and maintaining human society. The example I had most firmly in mind is the subject of Find Compatibles , what I consider to be the key page, the one with the real solution to all other problems explained.
As I explained on my early mailing lists and later webpages, I find that social technology has hardly improved at all over the years. We still use representative democracy, exactly the same as it was used in the 18th century. By contrast, horse and buggy transporation has been replaced by automobiles and airplanes, enormous changes.
In the picture below you will see some 18th century technology, such as the ox-plow in the middle of the picture. How things have changed since then in agricultural technology. But we still use chance encounters, engagements and marriages to organize our home life and the raising of children.
I claim that great advances in social technology are not only possible but inevitable. I have written three novels about this, one preposterously long, 5000 pages, another merely very very long, 1500 pages. The third is short enough at 340 pages to be published some day. Maybe. The topic is still not interesting to most people. I will excerpt small parts of these novels on the web sometime, maybe even post the raw text for the larger two.
This site includes many pages dating from 1997 to 2008 which are quite out of date. They are included here partly to show the development of these ideas and partly to cover things the newer pages do not. There will be broken links where these pages referenced external sites. I’ve tried to fix up or maiintain all internal links, but some will probably have been missed. One may wish to look at an earlier version of this page , rather longer, and at an overview of most parts of what can be called a bigger project.
Type in this address to e-mail me. The image is interesting. See Status of Social Technology
Copyright © 2007, 2008, 2009, Douglas Pardoe Wilson
I have used a series of e-mail address over the years, each of which eventually became out of date because of a change of Internet services or became almost useless because of spam. Eventually I stuck with a Yahoo address, but my inbox still fills up with spam and their spam filter still removes messages I wanted to see. So I have switched to a new e-mail service. Web spiders should not be able to find it, since it is hidden in a jpeg picture. I have also made it difficult to reach me. The picture is not a clickable link. To send me e-mail you must want to do so badly enough to type this address in. That is a nuisance, for which I do apologize, but I just don’t want a lot of mail from people who do not care about what I have to say.
Copyright © 2009 Douglas Pardoe Wilson