How can scientific research and technological development be slowed down?  And who oversees this, and how can we prevent new AI-related technologies from ultimately causing serious harm to people and the environment? The debate that has been raging in the media for the past few days regarding the risks associated with the overly rapid and unchecked development of artificial intelligence (some are even speaking of a full-blown ‘apocalyptic alarm’ and ‘humanity at risk’) must be taken seriously. This has captured the attention of public opinion, starting with the political sphere, the business sector, and the scientific community, on a recurring issue that was already evident throughout the 20th century, and which is becoming increasingly topical as time goes by. So, what are the limits of capitalism and the market? And what are the limits to the expansion of the dominance of technology?

It is worth heeding the warnings given by great 20th-century philosophers, such as Heidegger and Husserl, regarding the risks of humankind being dominated by technology. And the ethical concerns of scientists such as Niels Bohr regarding the use of atomic energy as a terrible instrument of war and a weapon of mass destruction (it is particularly enlightening to re-read the exchanges between Bohr and his favourite pupil, Werner Heisenberg, reconstructed by Michael Frayn in ‘Copenhagen’, one of the most thought-provoking plays of the last twenty years, with over a thousand performances at the Royal National Theatre in London alone). We should also pay close attention to Magnificent Humanitas, Pope Leo XIV’s recent encyclical ‘on the protection of the human person in the age of artificial intelligence’.

Avoiding the pitfalls of both the doomsayers and the overly optimistic, it is in fact a matter of acknowledging the consequences of the creative capacity for computation and autonomous development of the systems that drive sophisticated machines, whose operation and expansion are occurring at a pace far exceeding humanity’s current ability to understand their workings and effects. And to be aware of, and therefore take responsibility for, the effects of such disruptive technologies on daily life, on the economic and financial world, on work, on the balance of the environment (the growing consumption of water and energy), and on the conduct of wars.

AI offers extraordinary benefits in scientific and medical research, the prevention and treatment of previously incurable diseases, industrial productivity, the monitoring of environmental and atmospheric phenomena, and the organisation of daily life and services in major cities. However, the faster algorithms generate autonomous thought and decision-making capabilities in machines, the greater the risk that those decisions will be devastating. Technology is not neutral, and the interests linked to it are enormous.

The warning about the risks, which has been circulating in scientific and industrial circles for some time, has now reached the top echelons of Big Tech companies in the last few days.

Following statements by leading Anthropic researcher Jacob Coxon (‘AI could kill us all within a decade’), many figures in the industry, including Anthropic’s CEO Dario Amodei, OpenAI’s Sam Altman, and Elon Musk, have emphasised the gravity of the situation, which could soon slip beyond fragile controls and lead to disaster. ‘ Slow down’ is therefore their watchword, and Altman insists that we must ‘manage the pace of development of cutting-edge technologies’.

This is a significant development.Until now, in fact, all the Big Tech companies have shown themselves to be averse to rules, controls and regulations imposed by states and independent organisations, pushing the development of systems and innovations to the absolute limit, whilst also driven by the prospect of spectacular gains for stock market investors (with all the consequent risks of speculative bubbles destined sooner or later to burst, with devastating effects on all the world’s financial markets). They have found a highly sympathetic audience in the US government, starting with Donald Trump’s White House, who have accused public critics of being scaremongers, but if OpenAI and Anthropic are now sounding the alarm, it means the risk is escalating and the spectre of disaster is by no means a remote possibility.  To understand this better, consider the statements made by Musk, the figurehead of the most reckless high-tech ventures: ‘Those warnings about the dangers of AI are apocalyptic, but Dario Amodei is right’.  Amodei was one of the first to highlight the dangers.

There is yet another issue to bear in mind: Altman, Amodei and other major high-tech groups are on the verge of making a further appeal for funds from  a market that is already far too heavily exposed to technology shares. Nevertheless, the limelight remains an interesting prospect for those seeking funds, even in the name of safe investments.

A perceptive observation comes from Luciano Floridi, a professor at Yale and one of the world’s leading thinkers on the relationship between philosophy, technology and ethics: ‘I hear scandalous alarmism; the real danger is oligopoly’, that is, a system in which ‘artificial intelligence acts but does not understand’, left solely in the hands of Big Tech, without any oversight from politicians, governments or well-informed public opinion (La Stampa, 12 September). Regarding the threshold beyond which an AI system should be considered too risky to develop and deploy, Floridi clarifies that the answer lies in three questions: ‘Who is liable if the system causes harm? Who can verify this from the outside? And who can switch it off?’ Technical and political answers,  not corporate ones,  are required, just as we already do with many other things, from medicines to aeroplanes.

Floridi has faith in Europe, saying: ‘It has the right framework to regulate the issue. It has already lost the battle over large language models to the US and China, and it must focus on open-source models. If they are developed under stricter rules, they will be more reliable’.

Rules and  controls,  not bureaucracy,  as there is a risk of with the EU’s AI Act. And, of course, there are international implications. The fact that the EU is regulating AI independently while the rest of the world is driven solely by innovation and profit still leaves the risk threshold far too high.

‘The problem with AI is the human mind’, explains Giuliano Noci, a professor at the Politecnico di Milano (IlSole24Ore, 12 September). ‘The problem isn’t just the engine; it’s the road on which we’re driving it, who’s holding the steering wheel and  what happens if we forget the steering wheel.’

Technical issues point to ethical, cultural and political choices, so it is good that the debate has begun and it must not be stifled or overshadowed by the interests of power, finance or anything that jeopardises our future.

(photo Getty Images)