Social Technology for a Better World

Author in 1998
when this page was written”

On my old home page I asked readers to imagine a future world which would be better for everyone:

Imagine a future world in which it is easy to find a good job.

Imagine a future in which it is easy to find a truly compatible spouse or sexual partner. Imagine a world in which it is easy to find good long-term friends.

I went on to argue that such a world would be an affluent one, without poverty, and one with almost no crime, since people with good jobs, good friends, and a compatible spouse are much less likely to commit crimes.

All true, I maintain, but a bit too much for most people’s imaginations, so I’ll have to add a lot of explanation and present my concrete plans.

What I have sketched above relates to something I call “pure social technology”. People who follow the stock markets hear a lot about “information technology” and may have been told that the dominant technology of the new millenium will be some form of information technology involving the internet. I certainly don’t deny the importance of information technology or its social side effects, but I think the future belongs to pure social technoloyg, technology that is intrinsically social, not merely a side effect of better communication and transportation.

In our twentieth-century culture people somehow find jobs, spouses, and friends — or don’t. Far too many people never find a good job, and are either on welfare, begging on streetcorners, or stuck in a dead-end job they hate. Far too many people never find a truly compatible spouse and are either lonely and embittered, move from one unsatisfactory relationship to another, or are stuck in a marriage with someone they’ve grown to hate.

This is not a satisfactory situation. It is unacceptable. I believe society must provide a reliable method to guarantee that each person has a good job — one they will enjoy and learn from, or at very least one that compensates them well for their time and effort. I believe that society must provide a reliable method to guarantee that each person has a truly compatible spouse or sexual partner. And I believe that society must have appropriate mechanisms for finding or growing good friendships.

Much more about social technology will follow here and on other pages, but I think it important to sketch out the plan I’m working from and hint at the technical details. A key part of this plan is the creation of a non-profit society to prototype new social technology and promote its use. If I’m right about the eventual impact of this new technology it will transform our world beyond recognition, perhaps making the business sector less about making money and more about serving human needs, but in the next few years, as this process gets underway many businesses will be set up to exploit social technology and many people will makes lots of money doing so.

Let me be clear about my own role in this. My own list of personal heroes does not include Bill Gates or anyone else who has made a large fortune from exploiting information technology — instead it includes people like Richard Stallman of the Free Software Foundation and Linus Torvald, who wrote the Linux kernel and started the remarkable global development effort that made Linux a success. So I am trying to create a non-profit society, SocialTechnology.ca, to do research and develop free software aimed at social change. It will also promote social technology and help people make use of it. I have no objection to people making money from social technology and plan to encourage people who want to do that, but I am not one of them myself.

(In fact I am totally incompetant when it comes to money and couldn’t make a penny from social technology even if I spent the rest of my life trying).

Last year I started the SocialTechnology mailing list, and I began this year by registering the SocialTechnology.ca domain name and putting up a preliminary home page for the (future) organization.

I spent much of last year writing about social technology and putting up web pages about these ideas, but this year my emphasis has shifted to data collection and the development of new software.

Social scientists have been collecting social survey data for many years, but it has been used mainly for research by those same social scientists, who have written many academic papers about their findings. Now it is time to make proper use of all that data, making it the foundation of a new high-tech industry.

Social technology includes many things, but on this page I want to concentrate on my own plan for pure social technology, the creation and maintainance of social structure — the realization of that utopian future world I’ve written about, or something like it.

Because this is a plan meant to be taken seriously and acted upon, it is somewhat technical in places — the kind of people I most want to communicate with will want to see some technical details up front. If that doesn’t include you, please jump to a non-technical overview page.

The first technical point to make concerns the underlying mathematics, and the word ‘mathematics’ will immediately make some people turn away from this page. But please note that the eventual software products will hide the mathematics from the end users (unless they want to see it — anything I produce will be open-source). To write this software people will need to know some Discrete Mathematics, particularly Graph Theory and combinatorial optimization, but the end-users won’t need to know what a triangle is.

The word ‘optimization’ is again a provocative word which will displease many people for whom it suggests “Big Brother”, centralized organization, and all the other negative connotations of what was once called “Social Engineering”. I emphasis the phrase ‘Social Technology’ instead because I envision the creation of tools and techniques, and explicitly reject all authoritarian attempts to “optimize” or “engineer” society.

Some of the tools and techniques to be created will indeed use combinatorial optimization methods, and in a larger sense the development of social structure is clearly a combinatorial optimization problem, but nothing authoritarian is intended. On the contrary the new social technology will permit society to be much less centralized and will encourage new freedoms.

Discrete mathematics is involved simply because people have only a limited capacity for social interaction. Today most people have one or perhaps two jobs, if married have only a single spouse, and live in a household with an integral number of people. Friendship is less restrictive and may shade from slight acquaintances to much closer bonds, but it is still best modelled with links between people forming a graph called a sociogram.

(Insert GIF of suitable graph here! Something like this, perhaps:

A B C O O O———–>O D E F

but with many more vertices and edges).

Sociologists have been using sociograms for many years, and discrete mathematicians have been working with similar looking graphs for many years, but somehow the obvious connection has never materialized.

— Sociologists describe small sets of mutual friends as “cliques”, and indeed discrete mathematicians talk about “clique detection algorithms” for finding mutually interconnected vertices.

— A familiar combinatorial optimization problem is the bipartite matching problem, which mathematicians sometimes refer to as the assignment problem, since bipartite matching algorithms could be used to match workers to job assignments.

— Another familiar mathematical problem is the so-called “stable marriage problem” which addresses the problem of creating a stable solution to a bipartite matching problem amongst mutually attracting couples.

But still, the obvious connection has never materialized. Combinatorial optimization is not used to match workers to jobs or generate stable marriages. I can assure you, having spend many years trying, that it is very difficult to get anyone to seriously consider applying such mathematical techniques for anything involving real human beings.

“But wait”, you might ask, “what about computer dating?”

Someone always asks that.

A “dating service”, whether computerized or not, is something I don’t like at all, for several reasons:

— most dating services are scams aimed at making money from other people’s lonely desperation

— dating services involve a small pool of people, but the quality of matching depends on the pool size (there may be a lot of fish in the sea, but not many in pothole-size puddles)

— as far as I know most dating services that claim to use computers don’t use computers with any real understanding of either human personality or discrete mathematics, and certainly don’t use social survey data

— dating is a terribly high-pressure experience that makes true communication between people unlikely

— my own personal experiences with dating are mostly unpleasant memories, even though I already knew the people I dated

— most important, to me, but hardest to explain, dating services attempt to find people “best matches”, equivalent to the use of a “greedy” algorithm and that kind of algorithm just doesn’t work for bipartite matching.

People who truly understand combinatorial optimization recognize that most good algoriths are equivalent to “local search” in an appropriate space. Defining the space is the key problem — it is really a kind of phase space, not unlike those used in thermodynamics, a space of possible configurations, which may indeed occur sequentially. Local search is the search for a configuration which is very similar to the current one, but better. If you find it, then you search for another one, also similar-but-better.

What does this mean in the real world? I can give you my answer, and of course I will give you my answer, if I can persuade you to listen, but the real solution will involve lot of discussion and careful analysis. All I am seeking to do right now is get your attention. I want you to participate in this discussion and analysis — I don’t want you to pay too much heed to my own ideas.

For me, the answer is a progressive enrichment of each person’s social environment, so that each person is brought together with more and more compatible people over time, and can therefore spend less and less time with incompatible people (or working at unsuitable jobs).

I am not just talking about potential “dates”, I mean people of all ages, potential friends, potential co-workers, and so on for all possible types of social relationship.

Here is a little exercise for the reader: Take a piece of paper and write down all the people you spend time with on any regular basis, family, friends, rivals, supervisors, co-workers, and so on. Write down how many minutes or hours you might spend with them in a typical week or month. And beside each number, write a quick guess at how compatible you are with that person — on a scale from 1 to 10.

In the ideal world I dream about anyone you spend more than a few minutes with should get a high number, or at least a passing grade. I can remember a few glorious months when that seemed almost true for me. It wasn’t real or didn’t last. “Six billion people in this world, and I have to spend every day working with him??!!” “Billions of women in this world, and I end up in bed with her!!??” Oh, the nightmares!

Elsewhere in these pages you will find some discussion of a simulation of the world economy. That’s also social technology, and yes I am working on such a simulation, but for me it is all part of the same problem.

To properly simulate the economy, you need to simulate human society, and to simulate human society you need to be able to make predictions about human social behaviour. In fact we can predict human behaviour quite well already, on average — on a statistical basis. We can make decent predictions about groups of people. It’s just individual people who defy prediction, partly because we all want to defy prediction.

I can’t change that, and certainly don’t want to change it. Who would want to live around completely predictable people? But that’s OK. Statistical predictability is just fine. I never want “society” to tell Dick he must marry Jane, but I’d be quite happy to see society making a series of suggestions that are pretty good on average, especially if none of these suggestions involved making a date with some stranger.

People should meet in groups. Indeed that is how people do meet, most of the time. The best relationships don’t start with high-pressure dates, and I don’t think high-pressure job interviews are a good mechanism either. Over time people should meet many highly compatible people, in a social setting — in groups, including many potential spouses and many potential employers.

Yes, people meet in groups all the time today, and they do meet many potential spouses and perhaps potential employers. But they don’t meet many compatible ones or appropriate ones. That’s what has to change.

If truth be told, our society is filled with social technology, tools and techniques for a social purpose, but it is filled with very “low-tech” social technology. The typical “computer dating service” of today is paleolithic social technology. Even the high-priced headhunters who find expert employees for high-tech companies are offering something from the old stone age.

You might ask what right I have to say this. Good question! Let me turn the question around and ask what right they have to claim their services work — have they proof? How do they measure it?

I’m not talking about something to be taken on faith, I am talking about something that can be tested and measured. The best way to illustrate this is by considering employees hired to make decisions with immediate financial consequences. Suppose, for example, the employees need to make guesses about the future prices of stocks, bonds, or commodities.

Some people seem to be very good at this and make a lot of money. Others loose money, and also their jobs. But rather than thinking about individual talents, what about pairs or small groups of people working together. Two heads are better than one, aren’t they?

Apparently not. Decisions made by committees are often worse than those made by any one person working alone. I am sure you recognize that decisions made by a committee very often poor ones, but this is not necessarily so — in theory individual errors could cancel out, making for better decisions. Many attempts have been made to change things so that people working together could produce better decisions as a result of their cooperation. The most famous of these attempts was the RAND Corporation’s work on the Delphi Method, which made careful use of written submissions and anonymity to try to force objective evaluation of each other’s ideas.

But the Delphi Method ignores error-covariance. If you put together a committee of people who make similar mistakes, you will get a committee more likely to make those mistakes than any individual on it.

I’ve written a lot about this but rather than boring you with my arguments I’ll just state it as a requirement. To count as high-technology, any method for matching people with co-workers must meet the committee-test: working together the people must make better decisions than any of the the people would working alone.

That something that can easily be measured. In the example above it can be measured in terms of dollars. A committee trading stocks, bonds, or commodities should not only make more money than any individual working alone, it should make more money than the total for all of them working individually.

To put it in a context I’m more familiar with, Brook’s Law should fail — adding more people to a late software development should NOT make it later!

Is this possible? I claim it is. It is quite easy to measure people’s error-tendencies: just ask them to make a lot of guesses or estimates and compare them with reality. We know that people have quite consistent and long-lasting error patterns. The simplest case is simply to match people in pairs, and for that we have adequate bipartite matching algorithms.

Let me try that again with less tech-speak. Get “the computer” to match up two people who meet this simple criterion: when they agree they are right, when they disagree they are BOTH wrong. Like this:

Allan’s Guess Bill’s Guess Cecily’s Guess Reality ———————————————————— Acme $1.00 $2.00 $1.25 $1.50 Better $2.00 $2.00 $2.00 $2.00 Central $1.00 $1.00 $1.00 $1.00 Delta $2.50 $1.50 $1.75 $2.00

If you had to hire one person, it would be Cecily, whose guesses are better than either Allan or Bill. But she didn’t guess right all the time, and there is a better answer. Think about hiring both Allan and Bill, because when they agree they are right, and when they disagree the one is high and the other is low, so the average of their guesses is bang on.

That’s the whole thing in a nutshell. Of course it doesn’t have to be right and wrong, doesn’t have to involve decisions, and certainly doesn’t have involve money or any business situation. But whether it is love or money, there’s a powerful idea here.

An old idea, of course, and not mine — it goes back to Shannon and Weaver, and the synthesis of reliable channels from redundant unreliable ones. Using combinatorial optimization to do the matching is a little more original in the sense that I clearly remember thinking it out for myself — but again it is probably an old idea.

A lot of good old ideas are out there just waiting to be applied.

I’m not looking for credit for figuring all this out, I’m just looking ahead to that better society, the one with love and money for all of us. A lot of people are broke and lonely, but none of us need be.

In the wrong social context almost any one of us could end up hungry, homeless and without a friend in the world. And in the right social context even the saddest specimens of humanity could flourish and contribute something of worth. Am I mad to think these thoughts?

What I’m doing is empirically based, and I’m not going to impose my definition of compatibility on anyone, but rather try to meet theirs.

The empirical data comes largely from longitudinal social surveys, such as the Wisconsin Longitudinal Study, (WLS) which followed thousands of high school seniors from the Class of 1957 for twenty years at least, some of them to the present day. Other studies have covered much larger numbers of people, and are still underway, but the WLS is especially interesting because it covers such a long period of time.

The WLS includes high school performance, intelligence, personality information and much more, — thousands of questions altogether, and recorded further education, marriages, and jobs, together with specific questions about job satisfaction, salaries, standard of living, and so on.

From these various studies we have a lot of empirical information about people, and it can be used to estimate compatibility and the appropriateness of jobs. It’s specific enough about friendships and marriages that I’m sure it can be used for finding potential friends and potential spouses, and it is extremely good in the areas of higher education and jobs.

To make use of this data we need to compare other people with those surveyed in the WLS. That isn’t easy, and it’s not completely reliable, but I’m not promising perfect answers, just suggestions that people need to check carefully.

I will be putting up some web pages about this process, but basically (in techspeak, sorry) it involves factor analysis to produce coordinates for representing people and jobs as vectors in a space of many dimensions, then using nearest-neighbour and Bayesian measures of vector association as data for bipartite and other matching algorithms.

(I’ll expand that into English on a web page!)

Suppose we look at two 17-year-olds from the WLS who were very similar in intelligence, interests, personality, self-discipline, and overall health. If the job one choose after high school was unsatisfactory and while the other’s was the start of a long and happy career, we can inductively estimate that the former should have looked for a job like the latter had. That information will have some relevence to two similar young people today, and can be combined with a lot of other data from other studies so that we can produce sensible suggestions for them to follow (or not, as they choose).

Also, if two people of today are very similar in personality and interests to two people from the WLS who have been close friends for the past four decades, we ought to suggest they spend a bit of time getting to know one another.

And if two people are very similar to a couple from the WLS who married briefly but got divorced after a lot of fighting, we ought to give them some warning. But if they are very similar to a couple still happily married 42 years later, we should suggest they do get to know each other.

It is not up to me or any of us to say whether compatibility is the ability to get along for a long time, to have an intense relationship for a short time, or any other definition we might choose. There is is enough data to support a variety of interpretaions and we should use it to help people find what they want to find, individually.

Of course the results will be only suggestions, but I think they will be good suggestions that people can trust. We need to warn people to only trust them so far — people need to make their own decisions based on their own experiences, and shouldn’t place too much trust in what a piece of software tells them.

There is much more to this than the simplified version given above, but if anything the technical details make using the data easier and make the results more reliable. I’ll be posting other messages with various technical details — this is just a quick introduction to the idea of using social survey data: the idea of using what social scientists have collected as the basis for our social technology.

> Define this in computer language: > How often are couples able to resolve > disagreements in a satisfactory manner for > both of them? How would you know this before > they meet?

I’ll be putting up source code for various programs when it’s ready.

It’s all about comparing people today with people studied in the past and in ongoing studies. Several studies asked thousands of people almost exactly the question you posed — how often are they able to resolve disagreements.

> I think the same applies for friend-relationships. > Would people be happy to leave their local > pond for the possibility of “better” friendship/ > spouse?

Leaving is too drastic, the key idea is local search which means making only small changes in social environments — spending a little bit of time with a few new friends so that gradually over months or years people grow a new social environment, perhaps still seeing their old friends, but less often. The same applies to jobs — we need to set up a situation in which people can make gradual transitions from one job to another by holding the two as part-time jobs for a while.

I’ll make an exception to that rule for sexual relationships of any intimacy, discouraging people from having two part-time spouses, but part time is OK in the “pre-intimacy” stage of mate selection.

> It seems that in the past in small communities people > found both friend and spouse – though probably > the expectations were miles different. people > became isolated/enstranged in the large masses > of the cities, but I think more to do with > the working/living practices, than the numbers.

I think numbers have a lot to do with it. I use a compatibility scale based on the base-10 logarithm of the pool size, (that is to say the number of zeroes on the number). On average, people who choose the most compatible person from a pool of 1000 would get what I call compatibility level 3 — from the 3 zeroes in 1000. There is actually a fokelore factoid in the social science community which says that you pick out one good friend from each thousand people you meet.

It is possible to meet a thousand people and get to know them quite well, and so people in villages of 1000 people will probably find themselves a level 3 friendship. But in a city of a million it is very hard to get to know 1000 people anywhere near as well as you would if isolated in a small town with those people, so I say it is very difficult to find even a level 3 friendship in a city where a level 6 (1-in-a-million) or better is theoretically possible.

The same thing applies to jobs, where I use the same logarithmic scale. I’m not aiming at perfect matches, but I think level 6 is possible for all of us with a bit of technological help. I think we could each get that 1-in-a-million job and the 1-in-a-million friend, and many other equally compatible social relationships.

Please look at dpwilson/scale.html for more information about this scale and how it is applied.

> 2. If you picture a no-profit, thus non-capitalist > society, as you mention at some point, there is > no employer-employee, but co-worker relationships.

There is some ambiguity in the terms here, but I can argue for both interpretations. Yes, I envision a non-capitalist society at some point in time, and I can go along with calling that a non-profit society, in the sense of non-exploitive society — yes. Surely the exploitation of one person by another must end some day. But when I used the phase “non-profit society” I was trying to describe a non-profit organization, or non-profit association, an NGO, to develop and promote social technology.

I can also agree that some day the employee-employer relationship will disappear, though people may still have co-workers who perform some sort of supervisory tasks — but even that may not be necessary.

I do put a lot of emphasis on compatible co-workers, and I can agree that finding what we now call a job will someday come down to finding compatible co-workers and suitable tasks to carry out with them.

> 3. You still have to have a high level economical > integration for a high-technology society you > envisage – and indeed we need, to keep 6–>10 > billion people going. …

I envision a very high level of interpersonal integration, and I think that will bring about a high level of economic integration. Remember that I am envisioning people matched with very compatible co-workers who can collectively make good decisions, better decisions that any of them alone. So society should run efficiently and be sustainable in all ways.

But be careful with the phrase “high-technology society”. I’m talking about high-SOCIAL-technology, or SOCIAL high-technology, one in which social structures and relations are as good as possible, but that does not mean I favour high-technology in all aspects of society. I try to remain neutral on that. Will the future have high-tech automobiles, fusion power plants, and all the rest of it? Only if the people want these things. The well-integrated, stable, happy, and very democratic society of the future could choose a simpler life style and may very well do so.

That’s up to them. They will make the right decision, whatever it is.

Let me quote myself from my (old) home page:

Traditional futurists have described worlds with humanoid robots and flying automobiles, but the world I dream about is one in which love and friendship are abundant, along with truly satisfying work in good jobs that are easy to find and keep.

> … You have to decide on > a more universal democratic structure to make > the larger community to be able to cooperate > effectively.

I don’t disagree with that, but I think the integrated social network will be intrinsically democratic. You might take a look at my page on social power structures, dpwilson/structures.html for an overview. I don’t think we will need the kind of democratic institutions we have now, though I think we should keep them around as a safety-device, not to mention historical artifact and tourist attraction.

I hope that answers your questions, Eva. For your information, my political tendencies are rather anarchistic, in a totally non-violent way, and I much prefer Kropotkin’s “Mutual Aid” to Marx’s “Das Capital”.

Kropotkin places enormous emphasis on voluntary cooperation, and I agree, but it isn’t easy to cooperate with many people and very hard to base a society on that. So I add the insight that there are “lots of fish in the sea” — lots of people we CAN cooperate with, because they are very compatible with us. And so, bringing about global voluntary cooperation and mutual aid becomes a question of helping each person to find the others he or she can and will cooperate with easily.

Copyright © 1998 Douglas P. Wilson    

Copyright © 2009   Douglas Pardoe Wilson

Other relevant content:

New: Social Technology through Diagrams

New: Social Techs novel online

New: Social Technology Blog

New: Social Technology Wiki

Please see these web pages:

The main Social Technology page.

Find Compatibles , the key page, with the real solution to all other problems explained

Technological Fantasies , a page about future technology

Social Tech a page about Social Technology, technology for social purposes.  I think I was the first person to use this phrase on the Internet, quite a long time ago.

Roughly corresponding to these web pages are the following blogs :

Social Technology the main blog, hosted on this site, with posts imported from the following blogger.com blogs, which still exist and are useable.

Find Compatibles devoted to matching people with friends, lovers, jobs, places to live and so on, but doing so in ways that will actually work, using good math, good algorithms, good analysis.

Technological Fantasies devoted to future stuff, new ideas, things that might be invented or might happen, such as what is listed above and below.

Sex-Politics-Religion is a blog about these important topics, which I have been told should never be mentioned in polite conversation.  Alright that advice does seem a bit dated, but many people are still told not to bring up these subjects around the dinner table.

I believe I was the first person on the Internet to use the phrase Social Technology — years before the Web existed.

Those were the good old days, when the number of people using the net exceeed the amount of content on it, so that it was easy to start a discussion about such an upopular topic.  Now things are different.  There are so many web pages that the chances of anyone finding this page are low, even with good search engines like Google.   Oh, well.

By Social Technology I mean the technology for organizing and maintaining human society.  The example I had most firmly in mind is the subject of  Find Compatibles , what I consider to be the key page, the one with the real solution to all other problems explained.

As I explained on my early mailing lists and later webpages, I find that social technology has hardly improved at all over the years.   We still use representative democracy, exactly the same as it was used in the 18th century.  By contrast, horse and buggy transporation has been replaced by automobiles and airplanes, enormous changes.

In the picture below you will see some 18th century technology, such as the ox-plow in the middle of the picture.  How things have changed since then in agricultural technology.  But we still use chance encounters, engagements and marriages to organize our home life and the raising of children.  

I claim that great advances in social technology are not only possible but inevitable.  I have written three novels about this, one preposterously long, 5000 pages, another merely very very long, 1500 pages.  The third is short enough at 340 pages to be published some day.  Maybe.  The topic is still not interesting to most people.   I will excerpt small parts of these novels on the web sometime, maybe even post the raw text for the larger two.

This site includes many pages dating from 1997 to 2008 which are quite out of date.  They are included here partly to show the development of these ideas and partly to cover things the newer pages do not.  There will be broken links where these pages referenced external sites.  I’ve tried to fix up or maiintain all internal links, but some will probably have been missed.   One may wish to look at an earlier version of this page , rather longer, and at an overview of most parts of what can be called a bigger project.

Type in this address to e-mail me.  The image is interesting.  See Status of Social Technology

Copyright © 2007, 2008, 2009, Douglas Pardoe Wilson

I have used a series of e-mail address over the years, each of which eventually became out of date because of a change of Internet services or became almost useless because of spam.  Eventually I stuck with a Yahoo address, but my inbox still fills up with spam and their spam filter still removes messages I wanted to see.  So I have switched to a new e-mail service.  Web spiders should not be able to find it, since it is hidden in a jpeg picture.   I have also made it difficult to reach me.  The picture is not a clickable link.  To send me e-mail you must want to do so badly enough to type this address in.  That is a nuisance, for which I do apologize, but I just don’t want a lot of mail from people who do not care about what I have to say.


Copyright © 2009   Douglas Pardoe Wilson

This entry was posted in Old Pages. Bookmark the permalink.

Leave a Reply