Artificial Intelligence Raises the Stakes

Artificial intelligence is now raising the stakes again.

AI can help criminals produce persuasive communications in almost flawless language, imitate voices, create synthetic photographs and video, construct fake identities, automate conversations and conduct fraud at a scale that previously required enormous human effort. The technical and linguistic barriers that once limited many forms of fraud are disappearing.

We therefore have an extraordinary imbalance developing.

On one side may be a child, an elderly person, a busy parent or an ordinary employee who has perhaps seconds to decide whether something appearing on a screen, arriving in an inbox or speaking through a telephone is genuine.

On the other side may be automated infrastructure, stolen personal information, professional criminal organisations, artificial intelligence, synthetic identities and systems capable of attempting the same attack against thousands or millions of people simultaneously.

That is not a fair contest.

And simply telling people to become more “cyber aware” is no longer an adequate answer.

The Weakest Path Is Enough

There is an important technical distinction that needs to be made here. Not every scam begins with an operating-system vulnerability.

Not every stolen identity results from an unpatched computer.

Social engineering frequently attacks the person rather than the software, and many attacks begin through credentials, applications, communications platforms or compromised accounts rather than an underlying operating system.

But separating these problems too neatly misses the much larger point.

The attacker does not care which component of the digital environment fails.

• A software vulnerability may provide an entry point.

• A compromised account may provide identity.

• A stolen session may provide access.

• A manipulated person may provide credentials.

• A malicious application may provide persistence.

• Social media may provide information about the victim.

• Artificial intelligence may provide the persuasion.

The criminal simply searches for the weakest available path.

Security professionals may classify all of those as different attack vectors. For the person on the receiving end, the distinction is largely academic.

Someone got through.

Beyond Patch Tuesday

That is why the extraordinary number of vulnerabilities being discovered inside the foundations of our digital environment matters far beyond Patch Tuesday.

A vulnerability is not merely a technical defect sitting quietly inside a CVE database. Under the right circumstances, it can become one part of an attack chain that eventually reaches a real human being.

Behind that screen may be a pensioner. Behind that account may be a teenager. Behind that identity may be somebody’s child.

We therefore need to be extremely careful about normalising a model in which the answer is simply another patch.

Patching is essential. Of course it is. When a vulnerability is discovered it must be fixed, and it must be fixed quickly.

But patching is remediation after weakness has already entered the ecosystem.

There is an enormous difference between saying, “We discovered this problem and fixed it quickly,” and building a digital world in which the number and complexity of problems have become so great that continuous remediation is accepted as the permanent natural condition of computing.

A multinational organisation with a security operations centre may be able to manage that reality.

Somebody’s home cannot.

Ordinary people do not have vulnerability-management teams, threat-intelligence feeds, security architects or analysts investigating anomalous behaviour in the middle of the night. They have a laptop, a phone, perhaps a tablet, an internet connection and an entirely reasonable assumption that the technology they purchased is fundamentally safe.

That assumption matters because digital technology is no longer optional infrastructure.

• Children learn through it.
• Families communicate through it.
• People bank through it.
• Older people manage pensions and savings through it. Medical information travels through it.
• Governments provide services through it.
• Our photographs, identities, relationships, finances, locations and conversations increasingly exist inside it.

We would never knowingly design physical public infrastructure on the assumption that every citizen must become a structural engineer in order to remain safe.

Yet in digital infrastructure, we repeatedly place the final line of defence on the individual.

We tell the user not to click, not to trust, to update everything, to recognise the fake and to notice when something does not look quite right. Then, when an attack succeeds, there is often an unspoken suggestion that the victim should somehow have known better.

I believe we need to reverse that thinking.

Stop Designing for Perfect Users

The ordinary person should be the last person we expect to compensate for insecure architecture.

Security needs to move toward systems that assume human beings will occasionally make mistakes, because human beings always have and always will. A child will eventually click the wrong thing. An elderly person may trust somebody who sounds completely convincing. A tired employee will eventually open the wrong attachment. Somebody will reuse a password. Somebody will believe a voice on the telephone because it sounds exactly like their son or daughter.

In the age of artificial intelligence, even intelligent, cautious and technically competent people are going to be deceived.

The objective therefore cannot be to manufacture perfect users.

The objective has to be to build digital environments that remain resilient when imperfect human beings use them.

That requires a fundamentally different philosophy.

Technology should increasingly be capable of:

• Understanding context

• Recognising abnormal behaviour

• Detecting when identity and behaviour no longer correspond

• Questioning consequential actions

• Limiting authority

• Isolating suspicious activity

• Intervening before one human mistake becomes a financial, personal or organisational catastrophe.

AI as Part of the Solution

This is where artificial intelligence can become part of the solution rather than simply another source of risk.

The same technology that allows attackers to operate at machine speed can allow defenders to operate at machine speed. But intelligence alone is not enough. An AI system capable of observing everything and acting without meaningful control simply creates another form of risk.

What we need are intelligent systems in which observation, detection, reasoning, decision and action are governed; where authority is explicit, contextual and bounded; where consequential actions can be challenged, contained or denied; where evidence is preserved; and where control remains with the organisation or individual rather than disappearing into an opaque technological system.

And that brings this discussion to something much larger than cybersecurity.