The Focus of Collective Attention and the Long-Run Future — Stefan Schubert

“One question that’s been much discussed within effective altruism is: whom do we care about? How wide is our circle of moral concern? Do we find those who are close to us in space and time more morally important?”
does ‘care’ matter tho?

“what problems do we give the most attention to? Which problems do we discuss the most, and think about the most? Problems in our own country, or problems in distant countries? Present problems, or problems which may come to affect the distant future?”

“That means that our collective attention is a partially separate driver of our decisions, which is worth thinking about. (Cf. my post on Attention Theory.) And, as we will see, these other determinants contribute to neglect of the long-run future. Thus lack of moral concern for the long-run future isn’t the only reason why we neglect it.”

“Problems here and now, that require immediate action, tend to be much more psychologically salient than questions that are further away in space and time. Homelessness in your local town, and national injustices, tend to be more psychologically salient than more distant issues. Because those questions are so salient, they tend to get more attention, and be more discussed. Even people who sincerely believe that we should be impartial with regards to temporal and spatial distance have that tendency, to varying degrees.”

“This effect is strengthened further by group-level dynamics. I think that questions that interest almost everyone get more attention than questions that interest only a smaller group of people, even if they interest that group a great deal. When everyone else is discussing something, you know that you’ll get a lot of attention if you weigh in on it. You might almost feel compelled to have a view on it. And questions that concern the here and now are more likely to get everyone’s attention. So this mechanism strengthens the focus on them even more.

The salience factor puts the long-run future at a particular disadvantage. Spatially distant issues, such as big natural catastrophes, can be pretty salient. They can get the whole world’s attention. The same isn’t true of the distant future to the same extent, not the least because we can’t directly observe the beneficiaries of future-oriented actions.”

“You might be very concerned with some group in principle, but feel paralysed by uncertainty. It’s so unclear how to help them that you lose interest in the cause, and focus on other issues. This factor also disadvantages the long-run future. Many feel that it’s unclear how to influence the long-run future, and that feeling probably makes them less inclined to discuss and think about it.”

“Clear conflict lines between different political groups (“polarisation”) is likely another determinant of our collective attention. When there is no real political conflict, a question might get less discussed, even if it’s seen as morally important. And so it gets less attention. That could also contribute to some distant issues getting less attention”

“So we see that there are several factors that contribute to the long-run future receiving little collective attention. It’s not just our lack of moral concern. It’s also that it’s not salient, that many feel at a loss regarding how to influence it, and that it’s not polarising.”

“It’s important, however, to increase our attention to the long-run future in the right way. It would be bad if longtermist issues became polarised, for instance. Possibly there are smaller risks to making smaller circles of key decision-makers and intellectuals attend more to longtermist issues, than to making a central point of the more general political debate. Such issues merit more thinking and research.”


“"It clearly isn't designed to be a scientist. Your mental capacity is extremely limited. You have to undergo all kinds of unnatural training to get your brain even half suited for this kind of work - and for that reason, it's hard work. You live just long enough to start figuring things out before your brain starts deteriorating. And then, you die."”

“. "But wouldn't it be great," he says, "if you could enhance your abilities via artificial intelligence, and extend your lifespan, and improve on the human condition?"”

“He is a rare mixture of visionary and engineer, equally comfortable speculating on the fate of the planet or using a soldering iron, microchips, and stepper motors to build high-tech versions of his childhood dancing man. More than that, though, he's our most gung-ho advocate of technology as a tool to transform human beings and make us more than we are - within our lifetimes, if we want it.”

“For instance, to find out how the mind works, Moravec suggests severing a volunteer's corpus callosum (the nerve bundle linking the two hemispheres of the human brain) and interposing a computer to monitor thought traffic. After the computer has had time to learn the code, it can start inserting its own input, helping solve difficult math problems, suggesting new ideas, even offering friendly advice.”

“another scenario for anyone who'd like to escape the constrictions of dull old human biology: a futuristic robot surgeon peels away the brain of a conscious patient, using sensors to analyze and simulate the function of every neuron in each slice. As Moravec puts it, "Eventually your skull is empty, and the surgeon's hand rests deep in your brainstem. Though you haven't lost consciousness, your mind has been removed from the brain and transferred to a machine."”

“Joseph Weizenbaum, professor emeritus of computer science at MIT, complains that Moravec's book Mind Children: The Future of Robot and Human Intelligence is as dangerous as Mein Kampf. Respected mathematician Roger Penrose has written a long essay for The New York Review of Books in which he twice uses the word "horrific" to describe some of Moravec's concepts. Book reviewer Poovan Murugesan denounces Moravec as "a loose cannon of fast ideas" who suffers from "irresponsible optimism”

“in the words of award-winning science fiction author Vernor Vinge, who is also an associate professor of mathematical sciences at San Diego State University, "Moravec puts the rest of the technological optimists to shame. He is beyond their wildest extremes." But, Vinge adds hastily, "I mean this as praise!"

How seriously should we take Moravec's ideas? He is widely respected as a pioneer in robotics, but where is the line dividing his painstakingly practical research from his unfettered speculation? Why does he insist that breaking the boundaries of being human is important not just for himself, but for everyone - and why does he seem so crazy-cheerful about the whole thing?”

“His office is next door to the "high bay," a big lab displaying the results of previous Robotics Institute projects, including a huge, multilegged "walker" that was sent down into the cone of an active volcano, and a Pontiac minivan that can drive itself at speeds up to 60 mph. The van has already found its way from Pittsburgh to Washington, DC, with minimal human supervision, under the legal fiction that its four onboard Sparcstations and their mechanical interface are "an advanced form of cruise control."”

“He hides out in a small, undistinguished, modern office with a couple of computers, a few file cabinets, a refrigerator, a microwave oven, and a lot of books. This is where he pursues his immediate goal: designing and programming a domestic robot that can navigate freely in cluttered home environments. It is the next logical step, he says, toward truly intelligent machines that we will not only tolerate but love - even as they threaten to displace us as the dominant form of life on Earth.”

“Moravec's early work in robotics was plagued by setbacks. "I spent most of the 1970s," he recalls, "trying to teach a robot to find its way across a room. After 10 years, in 1979, I finally had one that could get where it was going three times out of four - but it took five hours to travel 90 feet." He chuckles like a fond father recalling the first incompetent steps of his baby boy.”

“The problem of moving through a three-dimensional world is hideously complex, as Moravec indicates, while counting off the tasks on his fingers: "Our robot used multiple images of the same scene, taken from different points of view, in order to infer distance and construct a sparse description of its surroundings. It used statistical methods to resolve mismatching errors. It planned obstacle-avoidance paths. And then it had to decide how to actually turn its motors and wheels."”

“By "brittleness" Moravec means that the system tended to fail suddenly and catastrophically. "Accidental conspiracies of sensory miscues would lead it to a wrong conclusion while being sure that it was right. In practical terms, it could misidentify the surrounding objects and run into a wall."”

“Like Wile E. Coyote in a Road Runner cartoon, trying to run into the mouth of a tunnel painted on a rockface?

"Precisely!" he laughs again, sounding genuinely happy, as he does whenever he describes the lovably fallible behavior of his creations.”

“In 1984, using US$10 Polaroid ultrasonic range finders instead of expensive video cameras, he created a new commercial robot that analyzed maps of the surrounding space rather than just objects in it. The result, to his surprise, was a system that could navigate reliably and relatively swiftly.”

“"Today's best robots can think at insect level," he says as we return to his office. He explains that state-of-the-art mobile robots orient themselves by sensing special markers placed on floors, walls, or ceilings. Insects behave the same way: ants follow pheromone trails, lightning bugs look for each other's flashes, and moths navigate with reference to the moon.”

“Robots that orient themselves with markers have found some application in industry - transporting pallets and cleaning floors - but they offer few advantages over the older systems that follow hidden guide wires. As a result, the market is very limited. "In fact," says Moravec, "the market barely exists at all. So, what we're shooting at now is a robot with the intelligence of a small vertebrate - the smallest fish you can imagine. It will no longer depend on navigational points; it will build a relatively dense representation of volumes of space."”

“We walk into a windowless space no larger than an average living room. There are a couple of video monitors, workbenches littered with tools, pale beige walls, and a vinyl floor. The robot stands in the center of the room: an ugly little four-wheeled truck the size of a go-cart. But Moravec exudes pleasure and affection as he guides his toy out of the workshop, into the hall, and back again.”

“Moravec estimates that these systems will need an onboard computer capable of 500 million instructions per second. The first IBM PCs managed 0.3 mips; a modern Pentium-based PC reaches 200 mips; and it's reasonable to expect that 500-mips processors will be affordable by the turn of the century.”

“Once robots find a niche doing dull, repetitive jobs, Moravec sees an ever-expanding market. "The next step will be adding an arm and improving the sensor resolution so that they can find and manipulate objects. The result will be a first generation of universal robots, around 2010, with enough general competence to do relatively intricate mechanical tasks such as automotive repair, bathroom cleaning, or factory assembly work."

By "universal" Moravec means the robot will tackle many different jobs in the same way a Nintendo system plays many different games. Plug in one cartridge, and the robot will know how to change the oil in your car. Plug in another, and it will know how to patrol your property and challenge intruders.”

“"It will tackle tasks in various ways," says Moravec, "keep statistics on how well each alternative has succeeded, and choose the approach that worked best. This means that it can learn and adapt. Success or failure will be defined by separate programs that will monitor the robot's actions and generate internal punishment and reward signals, which will actually shape its character - what it likes to do and what it prefers not to do."”

“By 2030, according to Moravec, we should have a third-generation universal robot that emulates higher-level thought processes such as planning and foresight. "It will maintain an internal model not only of its own past actions, but of the outside world," he explains. "This means it can run different simulations of how it plans to tackle a task, see how well each one works out, and compare them with what it's done before." An onlooker will have the eerie sense that it's imagining different solutions to a problem, developing its own ideas”

“On the plus side, each time a robot learns a fact or masters a skill, it will be able to pass its knowledge to other robots as quickly and easily as sending a program over the Net. This way, the task of understanding the world can be divided among thousands or millions of robot minds. As a result, the machines will soon develop a deeper knowledge base than any single person can hope to possess. Within a short space of time, robots that are linked in this way will no longer need our help to show them how to do anything.”

“Meanwhile, they will be smart enough to interact with us on a human level. "Their world model will include psychological attributes," Moravec says, "which means, for instance, that a robot will express in its internal language a logical statement such as 'I must be careful with this item, because it is valuable to my owner, and if I break it, my owner will be angry.' This means that if the robot's internal processes are translated into human terms, you will hear a description of consciousness - especially if the robot applies psychological attributes to its own actions, as in 'I don't like to bump into things,' which is a compact way of saying that the robot gets an internal negative reinforcement signal whenever it collides with something, or imagines a collision."”

“issue. He believes a robot that understands human behavior can be programmed to act as if it is conscious, and can also claim to be conscious. If it says it's conscious, and it seems conscious, how can we prove that it isn't conscious?”

“In this case, though, Moravec's conclusion is less radical than it seems - because when many jobs are broken down into tasks, they require a relatively limited degree of "humanness." Even today, we have expert systems that offer advice based on a large number of facts in a field such as medicine or geology. Imagine this expertise gradually broadening to include subjects such as corporate law, mechanical design, profitability, and efficiency. Decisions in these areas are all made logically from sets of facts, which means that if the facts are completely spelled out, a machine intelligence should be able to deal with them.”

“Thus a corporation can literally become automated from the bottom up: first the assembly lines, then bookkeeping, product design, and planning. Even management can be taken over by computers that are able to learn from past performance. Ultimately, a corporation will consist of a diverse mix of robots, some mobile, some fixed, some large and powerful, some microscopic, all interacting with speed and versatility that is completely beyond human abilities.”

“"Back in the 1970s I made some overoptimistic assumptions about the rate of progress of computers. I thought that using an array of cheap microcomputers, we might achieve human equivalence by the mid-1980s. Then I did a slightly more careful calculation around 1978 and decided it would take another 20 years, requiring a supercomputer. But then I started getting serious, writing articles and essays, and I thought I should do the calculations more rigorously. So I collected 100 data points of previous computer progress, I did the best calculation I could, I compared the human retina with computer vision applications, and I plotted it all out."”

“When robots are doing all the work, no one will earn any money. How can an economy flourish when all the consumers are penniless?”

“Today, he points out, people who retire are supported via wealth that is ultimately created by industry. As industry becomes more efficient, there will be more wealth, allowing people to retire earlier. When industry is totally automated and hyper-efficient, it will create so much wealth that retirement can begin at birth. "We'll levy a tax on corporations," Moravec says, "and distribute the money to everyone as lifetime social-security payments."”
😳😳😳 — is it not weird that retirement is dependant on returns? (idk if it makes sense or not?)

“He assumes these business entities will follow programs written by us, compelling them to obey laws and pay their taxes. But the programming will also encourage robot-controlled corporations to compete with each other.”

“"It is unstable," he agrees. "Everything will depend on the way in which we create it. Crafting these machines and the corporate laws that control them is going to be the most important thing humanity ever does. You know, each age has an activity in which the best minds get involved. Crafting the laws, and their implementation, will be the thing to do in the 21st century."”

“If the job is done right, he predicts a world of comfort, health, and boundless plenty - at least for a while. Human beings will be like slave owners whose servants never complain, need no supervision, and are constantly eager to please.”

“Unlike human beings, robots don't need to breathe air, aren't disoriented by zero gravity, and can be easily shielded from harmful radiation. There are vast mineral resources in the asteroid belt, where there will be no regulations regarding pollution, noise, or safety. Robot factories located in space would be able to manufacture products with maximum efficiency and then drop them down into Earth's gravity well. Alternatively, they could conduct hazardous research and radio the encrypted results back to their parent corporation on Earth.

Only a small "seed colony" of robots would be needed to set up an off-world operation. Using local mineral ores and solar energy, robots could build everything they required - including copies of themselves.”

“In this scenario, everything is still being controlled by the parent corporations, which are still being controlled by us. Therefore, the off-world operations should present no problems. "But now suppose a company goes out of business," Moravec says, "leaving its research division in space, where there's no supervision. The result is self-sustaining, superintelligent wildlife."”

“Moravec feels that in a short period of time, all the local materials will be plundered and converted into machines, and all available solar energy will be used to power them.

The result will be a dense, interacting swarm of competing entities - although, he says, the competition will be relatively benign. Warfare among robots will be rare because "fighting wastes energy, and a third entity can eat the pieces."”

“Robots will be motivated to make themselves as small as possible, conserving raw materials to build better brains. "As a result, you end up with the whole mess forming a cyberspace where entities try to outsmart each other by causing their way of thinking to be more pervasive. Here's an ecology where all the dead-matter activity has been squeezed out and almost everything that happens is meaningful. You have this sphere of cyberspace with a robot shell, expanding outward toward Earth."”

“Since space-based machine intelligences will be free to develop at their own pace, they will quickly outstrip their cousins on Earth and eventually will be tempted to use the planet for their own purposes. "I don't think humanity will last long under these conditions," Moravec says. But, ever the optimist, he believes that "the takeover will be swift and painless."

Why? Because machine intelligence will be so far advanced, so incomprehensible to human beings, that we literally won't know what hit us. Moravec foresees a kind of happy ending, though, because the cyberspace entities should find human activity interesting from a historical perspective.

We will be remembered as their ancestors, the creators who enabled them to exist.

As Moravec puts it, "We are their past, and they will be interested in us for the same reason that today we are interested in the origins of our own life on Earth."”

“Machine intelligences of the far future will develop from our initial programming, just as a child grows from its parents' DNA. Consequently, even when robots are smarter than we are, they should retain many of our priorities and values.”

“Assuming the artificial intelligences now have truly overwhelming processing power, they should be able to reconstruct human society in every detail by tracing atomic events backward in time. "It will cost them very little to preserve us this way," he points out. "They will, in fact, be able to re-create a model of our entire civilization, with everything and everyone in it, down to the atomic level, simulating our atoms with machinery that's vastly subatomic. Also," he says with amusement, "they'll be able to use data compression to remove the redundant stuff that isn't important."”
wut how does this tracing work

“our current "reality" could be nothing more than a simulation produced by information entities.”

“"Of course." Moravec shrugs and waves his hand as if the idea is too obvious. "In fact, the robots will re-create us any number of times, whereas the original version of our world exists, at most, only once. Therefore, statistically speaking, it's much more likely we're living in a vast simulation than in the original version. To me, the whole concept of reality is rather absurd. But while you're inside the scenario, you can't help but play by the rules. So we might as well pretend this is real - even though the chance things are as they seem is essentially negligible."

And so, according to Hans Moravec, the human race is almost certainly extinct, while the world around us is just an advanced version of SimCity.”

“The vision he has described exists for him as a unified whole; it takes him only about an hour to describe it clearly and fluently from beginning to end. For him it seems entirely pleasurable: a destiny that grows out of his own work and affirms his own values.”

“This has created uncertainty and discontent - as he readily admits. "We all agree," he says, "that the world is a bit screwed up. The reason for this is rather obvious. We have a Stone Age brain, but we don't live in the Stone Age anymore. We were fitted by evolution to live in tribal villages of up to 200 relatives and friends, finding and hunting our food. We now live in cities of millions of strangers, supporting ourselves with unnatural tasks we have to be trained to accomplish, like animals who have been forced to learn circus tricks."”

“Moravec adamantly believes that reversing the evolution of technology would create an even bigger disaster. "Most of us would starve," he says. He suggests the opposite approach: that we try to catch up with technology by accelerating our own evolution. "We can change ourselves," he says, "and we can also build new children who are properly suited for the new conditions. Robot children."”

“"No. In fact, I am biologically incapable of it. I contracted testicular cancer as I was finishing my PhD; it didn't affect me very much, it didn't really hurt, I noticed a growth, but I still had my thesis to write and my orals to do, and the whole thing seemed very unreal. There were two surgeries, one minor, one major - with my intestines out in a bag to get at the lymph nodes. I came through it in sparkling condition, aged around 30. But a side effect is that I'm basically infertile."”

“"Not at all. Long before the cancer, I was already obsessively committed to robots for whatever neurotic reason. That was where I wanted to spend my energy. I met my wife in the hospital when I was getting chemotherapy in 1980. She already had two children, so I inherited them as stepchildren."

Does his wife share any of his feelings about machines?

He laughs. "At the moment, my wife is a biblical scholar."”

“"My father was an engineer in Czechoslovakia and had a business making and selling electrical goods during the war. When the Russians arrived in 1944, he became a refugee. He left the country on a tricycle with 50 kilos of tools and 50 kilos of food. He met my mother in Austria, which is where I was born. He had an electrical store, where he'd hand wind transformers to convert battery-operated radios so they'd run on house current. We relocated to Canada in 1953."”

“In person, Moravec seems diffident and gentle; he doesn't drive a car because, he says, he's uneasy with so much potentially dangerous mass in his control. He likes living in Pittsburgh because his home is a short walk from his office, and he seems to feel little need to venture outside this simple life.

Yet as a child he enjoyed fantasies about superheroes and supervillains, and as an adult he talks casually of totally rebuilding human society. He refers to his new book, for which he's currently seeking a publisher, as "a kind of speculative long-term business plan for humanity," and in it he speaks condescendingly of "Earth's small-minded biological natives."”

“"People such as myself," he says, "may have a little bit of influence, but we're like mosquitoes pushing at a rolling boulder. Progress is inflicted on people in the same way that natural evolution is inflicted on people. It really is evolution; it's the selection and growth of information, transmitted from one generation to the next."”

“"All I'm suggesting is that we give people a choice. In the next decade, people will either buy their housecleaning robot or not buy it. And I think they'll want to buy it. Then they'll have the choice of upgrading to one that learns, and I think they'll want that, too. Then they'll have the choice of a robot that claims it's conscious, a really nice entity that talks like a person, seems to understand you, and has nothing but your best interests at heart - because that's how it's programmed. And then the fourth generation will take that personality and add intelligence. It will be a constant help to you; it will explain why something that you want to do isn't what you should do - because it loves you. I think people will like these machines and will quickly get used to them."”

“"But I don't consider it a demise," Moravec retorts, still insisting that his vision is wholly positive. "The robots will be a continuation of us, and they won't mean our extinction any more than a new generation of children spells the extinction of the previous generation of adults. In any case, in the long term, the robots are much more likely to resurrect us than our biological children are."”

“Personally, I suspect he likes the idea of radical change because he's an intensely intelligent man who is easily bored by the everyday world. He finds it impossible to believe that it makes sense to continue, as human beings, in our exact same form. "Do we really want more of what we have now?" he asks, sounding incredulous. "More millennia of the same old human soap opera? Surely we have played out most of the interesting scenarios already in terms of human relationships in a trivial framework. What I'm talking about transcends all that. There'll be far more interesting stories. And what is life but a set of stories?"”

“On a long-term basis, Moravec points out, our planet may not be a hospitable place to live. Huge climatic shifts may occur (as they did during the ice ages). Our sun may become unstable. The world may be ravaged by incurable diseases. Our entire ecology could be destroyed by a large meteor or comet. "Sooner or later," he says, "something big will come along that we cannot deal with. But by changing ourselves in the most fundamental way, we will be able to survive such catastrophes."”