Moderator Nicholas Thompson, CEO of the U.S. publisher The Atlantic introduces Nita A. Farahany, professor of law and philosophy at Duke University. He cheerfully promises that the audience will be enlightened and excited by her talk.
The presentation “Ready for Brain Transparency?” begins on the screen with a colorful cartoon full of happy people. The main character gloats in the office about how productive she has been today and checks the recording of her brain waves to make sure everything is okay.
But when she gets briefly distracted by amorous thoughts about her office counterpart, she gets nervous because her boss, who can also see her brainwaves, might suspect her of violating the ban on sexual relations in the office. So she quickly returns her thoughts to work.
Promptly she receives the good news from her boss that she will get a bonus because of her excellent brainwave measurements during the last quarter.
For most people, it is a horror scenario, which is sketched-out here in a cheerful tone with colorful pictures as if everything were just fine.
The next day, however, a colleague is arrested for allegedly committing online fraud. The brainwave recordings of all colleagues are confiscated to find any accomplices. Our main character gets nervous, because she has been discussing in secret a possible startup with the culprit. Her brainwaves could thus create a false impression with the authorities.
After the dark side of the new technology has been briefly hinted at in this light-hearted way via a comic strip, Professor Farahany, who could easily pass for a professional entertainer, is now free to concentrate entirely on the positive side. She asks “What do you think, are you ready for this future?”, just to make it clear that this future is already with us, as all the technology shown already exists.
What follows is a vivid description of what is already possible in terms of decoding thoughts and feelings, even with small wearable devices, and what will be possible in the near future.
Farahany praises brainwave monitoring in the workplace because it can prevent accidents. The wearable technology notices early on if someone is getting distracted or tired and sounds the alarm. She touts:
“We as a society should want that. We should want a technology that enables us to be safer, that enables us all to exist in an environment where commercial drivers or individuals who need to be wide awake are as wide awake as they should be. Because, if they’re not, the consequences are desasterous.”
Brainwave monitoring already reality for many employees
More than 5,000 companies already require their employees to wear helmets that monitor brainwaves, Farahany reports, giving an example from – of course – China, the motherland of total surveillance.
You might be shuddering,” she says, as she herslelf did at first, but she askes for “a much more nuanced discussion” about technological mind-reading. Because:
“If done well, neurotechnology has extraordinary potential. If done poorly, it could become the worst oppressive technology we have ever introduced in society. We still have the chance to get it right.”
Farahany reports that workers at Amazon and other companies hate so-called bossware. Bossware are wearables for the workplace that measure bodily fun8ctions or give controlling signals. Without hesitation, she adds: “They reject it, even though it makes their lives better.” Farahany thinks she knows better than the people concerned what constitutes a good life for them. Self-determination, apparently, is not necessary for a good life.
Bossware enforeces greater efficiency
After the Corona pandemic, 80% of companies surveyed say they use some form of bossware to control productivity, Farahany tells us, and that they have every reason to do so, with nine out of 10 employees admitting to occasionally “wasting time” at work. Life time that is not used productively to increase the output of the machine – whatever it may be – is wasted time.
We’ve already become so accustomed to Fitbits, Apple Watches and other wearables recording our bodily functions, she reminds us, that most of us don’t think there is much to worry about. However, she advocates refraining from using this technology to manipulate our behvaour directly. She wants it put in the hands of employees as a way to help them stay focused at work.
Or better yet, a robot in the factory may detect when its human colleagues’ stress level is rising and “reduce the frequency with which it gives them tasks.” This is what Farahany calls “Cognitive Ergonomics”, which is “the healthier workplace of the future, a workplace that adapts to our abilities, slows down when we need it to.” She doesn’t mention that it also speeds up when it realizes that the person it controls can work faster without breaking down right away.
In summary, she notes that technological surveilla8nce of mind and emotion can be a powerful tool to make our lives better, but also a powerful tool of oppression. Her solution:
“We can mke a choice to use it well. We can make a choce to make it something that empowers individuals, that helps them gain insight into their own mental health and well-being, increase their own productivity and well-being, and (…) unlock the potential for humanity.”
Who is “we”? This question remains open. She is talking to the Davos business elite. One may assume that “we” is them and her. The corporate bosses should therefore be allowed to decide whether the new technology is used for the greater well-being of employees or to control them and squeeze maximum performance out of them.
Humans as the cogs of a machine
The kind of pleasant shudder that Farahany projects on the screen to laugh it away is likely to be felt only in the Davos scene. Cold shivers will go down the spines of normal people, instead.
After these cold shivers hve subsided, I ask myself what view of man and the world underlies this narrative. I find that t is precisely the machine ideology that underlies our society. That technology is openly approaching the center of humanity and preparing to take control of it, according to this presentation, offers an opportunity to better understand the anti-human assumptions and structures of our society.
In this ideology, man is part of the gears of an almost infinitely productive machine that has long since become an end in itself. What it produces and for whom does not matter. The main thing is that it runs like clockwork and becomes ever more efficient. Man’s physical and mental weaknesses, his unproductive needs and goals, must hinder the production process as little as possible. Social values beyond an empty concept of freedom and self-determination do not exist.
Any technology that enables man to be a better, more productive cog in the machine is to be welcomed. There is hardly a better way to illustrate this than with a robot that optimally adjusts the speed at which it assigns tasks to its subservient auxiliary human to the latter’s ability.
Most of the time, this ideology, which instrumentalizes humans, comes along more covertly. The systems puts people under pressure to perform, then leaves it up to them to optimize themselves and gives them tools to do so, from fitness trackers to brainwave monitors.
Perverse progress
When a technique is presented that is well on its way to creating something that almost all people oppose, the impulse of a sane, responsible person would be to stop this. But a critical reaction did not take place in Davos, where government leaders and ministers, party politicians and corporate executives listened devoutly and discussed the details calmly at the end.
The conclusion of “technology ethicist” Farahany was: We’re developing this further. Because it’s technically challenging, and because we can. We assume that it will be fine. Because – in our technocratic world view – we think everything is good that leads to the machine working more smoothly and becoming more efficient, and because we assume that technocrats at the levers of power will mean well and will do it right.
This is exactly how we deal with technology in general. What can be invented and developed must be developed and introduced. To prevent it or to want to steer it in a human-friendly direction would be technophobia. Technolgy disrups and society adapts is the rule of the game.
Disruption has perverseely been charged with a positive connotation. It is meant to express that an economic or social structure is destroyed by an economic or technological innovation and rebuilt to fit the new technology. People with their preferences and their needs, which have shaped what exists, are seen as disruptive factors for the application of a new technology. The technocratic fixation on efficiency for efficiency’s sake could hardly be more exaggerated than in the worship of disruption.
The Davos technocrats and their academic and political auxiliaries simply laugh away the horror of the implicit loss of humanity. What is human, the unproductive (in the machine sense), the irrational has to give. The new technology will prevail, no matter what the cost, since the costs cannot be measured in units which the technocrats regard as relevant..
I invite you to watch the half-hour video in the light of the image of man as a cog of a machine to be optimized and ask yourself: Is this an off-the-wall fantasy of a wacky philosophy professor or a plausible description of where we are heading, if we continue on our current path?
Trust the technocrats!
What the technology ethicist offers to avoid the outcome of a highly repressive technology is a joke at which she herself laughs the loudest (see below). She assumes that within our lifetime many healthy people will be using implanted brainwave technology. Then, she says, people will be able to read thoughts. The allegedly unstoppable and rapid technological progress will far outpace governments’ ability to regulate it, she says.
But it would be enough, she claims, if corporations developed and observed best practices:
“We have to decide to move forward in a way that is best for humanity, using the technology in ways that enbale us on a pathway forward, rather than oppress us. I think, that’s a possibility that we can still choose. I hope that it is one that you join me chosing.”
This “we”, i.e. the elite of corporate capitalism gathered in Davos, which she seems to identivy with, would not have to wait for the government to develop and enforce a right to mental privacy:
“We can simply act as if this right already exists. When a company chooses to monitor employees’ brain waves, it chooses to monitor them only for signs of fatigue, not amorours feelings. They engage in data minimization and follow best practices that respect the autonomy of the individual. They will not alter their thought processes to make them more productive, recognizing that they have a right to mental freedom.”
In a light-hearted manner, moderator Thompson makes it clear to all, that this is cheap talk, to reassure those who have qualms, and give cues to those who need to justify the development and use of this creepy technology. He injects: “As a CEO, I’ll say this: I’m sure all CEOs will use this technology completely responsibly.”
The audience laughs, Farahany laughs the longest and loudest.
Stop playing along!
The fact that this presentation could be presented to the economic, media and political elite and made available to the general public without causing an outcry of indignation and serious consequences shows how deeply the technocratic view of man as a means to the end of greater machine output has already permeated society.
This is why it is so important, that we make ourselves and others aware of the full extent of this problem. For this attitude is at the root of much that is going wrong in our society. It permeates and corrupts many efforts to make the world a better place. First and foremost, but of course not only, climate change policy. Any ideology that subordinates humans and nature to a goal, be it productivity ideology, communist ideology, or CO2 ideology, can only make the world worse.
Every reformer should therefore ask himself whether he perhaps understands society too much as a machine, for example as an economic-social machine whose CO2 emissions must be optimized so that an imagined climate machine does not get out of sync.
Further, we should withdraw from the cult of self-optimization and listen to our body and mind instead of a device that compares us to some external standard and urges us to increase the chord. Those who ask us to “invest” in our “human capital” should be invited to rethink their view of humanity. We should ignore the carrots they hold out in front of us.
Do we really want to bend over backwards for a promotion that, if successful, promises more stress and more work for a little more money – and for a transitory sense of achievement until we revert to comparisons with the salary and status of those above us? Career advancement only promotes happiness in life if we can take pride in what we are doing. If we don’t, we should either look for another job, become self-employed, ideally together with others. A cooperative will often be the best form to realize this, be it a production cooperative, or a consumption cooperative or, best of all, both combined.
Alternatively, we should at least focus on optimizing our work-life balance by working to rule.
If we critically question which needs are our own and which have been implanted in us by profit-oriented companies, we can create a little more financial freedom for ourselves to do what we want and not what the “job market” demands.
Let us live for what constitutes being human, not for being a more productive cog of a machine! A good life requires, in variable weighting and without claim to completeness, a self-determined and varied spiritual and social life, a task to which we dedicate ourselves gladly, the good feeling if we are needed and can do something for others, feeling rooted in a homeland, with its nature and its people.
Technocrats consider much of this to be backward and a hindrance to technical progress, and capitalists consider it a hindrance to profit maximization. On the “labor-market”, it is only possible to a limited extent to implement these ideals. But we should at least be aware of how we would prefer to live, so that we can focus our energy and attention on what gets us closer to this ideal. The more we do that, the more the world of work will have to adapt.
If our production and consumption were organized in units where consumers and working people were in charge, instead of anonymous capital “givers” and return takers living somewhere else in the world, our companies would focus on satisfying real needs instead of creating ones that they can then satisfy in a profit-making and environmentally damaging way. Working people would know what they are working for and would not have to be motivated by carrots and sticks. They could let their very own interest in production in harmony with the natural and social environment become effective.
We are still a long way from being able to do this on a large scale. The capitalist machine has taken over our society and our thinking far too extensively to bring about some quick and profound change. But we should develop and nurture the ideal in our minds and hearts. The more we strive for such a world and implement components of it, the more people will believe, that a better system is possible. Without many more people believing that, fundamental change for the better will not be possible.
