Let’s not get carried away by self-driving cars and the sharing economy: they won’t make Smart Cities better places to live, work and play

(Cities either balance or create tension between human interaction and transport; how will self-driving cars change that equation?)

(Cities either balance or create tension between human interaction and transport; how will self-driving cars change that equation? With thanks and apologies to Tim Stonor for images and inspiration)

Will we remember to design cities for people and life, enriched by interactions and supported by transport? Or will we put the driverless car and the app that hires it before the passenger?

I’m worried that the current level of interest in self-driving cars as a Smart City initiative is a distraction from the transport and technology issues that really matter in cities.

It’s a great example of a technology that is attracting significant public, private and academic investment because many people will pay for the resulting product in return for the undoubted benefits to their personal safety and convenience.

But will cities full of cars driving themselves be better places to live, work and play than cities full of cars driven by people?

Cities create value when people in them transact with each other: that often requires meeting in person and/or exchanging goods – both of which require transport. From the medieval era to the modern age cities have in part been defined by the tension between our desire to interact and the negative effects created by the size, noise, pollution and danger of the transport that we use to do so – whether that transport is horses and wagons or cars and vans.

A number of town planners and urban designers argue that we’ve got that balance wrong over the past half century with the result that many urban environments are dominated by road traffic and infrastructure to the extent that they inhibit the human interactions that are at the heart of the social and economic life of cities.

What will be the effect of autonomous vehicles on that inherent tension – will they help us to achieve a better balance, or make it harder to do so?

(Traffic clogging the streets of Rome. Photo by AntyDiluvian)

(Traffic clogging the streets of Rome. Photo by AntyDiluvian)

Autonomous vehicles are driven in a different way than the cars that we drive today, and that creates certain advantages: freeing people from the task of driving in order to work or relax; and allowing a higher volume of traffic to flow in safety than currently possible, particularly on national highway networks. And they will almost certainly very soon become better at avoiding accidents with people, vehicles and their surroundings than human drivers.

But they are no smaller than traditional vehicles, so they will take up just as much space. And they will only produce less noise and pollution if they are electric vehicles (which in turn merely create pollution elsewhere in the power system) or are powered by hydrogen – a technology that is still a long way from large-scale adoption.

And whilst computer-driven cars may be safer than cars driven by people, they will not make pedestrians and cyclists feel any safer: people are more likely to feel safe in proximity with slow moving cars with whose drivers they can make eye contact, not automated vehicles travelling at speed. The extent to which we feel safe (which we are aware of) is often a more important influence on our social and economic activity than the extent to which we are actually safe (which we may well not be accurately aware of).

The tension between the creation of social and economic value in cities through interactions between people, and the transport required to support those interactions, is also at the heart of the world’s sustainability challenge. At the “Urban Age: Governing Urban Futures” conference in New Delhi,  November 2014, Ricky Burdett, Director of the London School of Economics’ Cities Program, described the graph below that shows the relationship between social and economic development, as measured by the UN Human Welfare Index, plotted left-to-right; and ecological footprint per person, which is shown vertically, and which by and large grows significantly as social and economic progress is made.  (You can watch Burdett’s presentation, along with those by other speakers at the conference, here).

the relationship between social and economic development, as measured by the UN Human Welfare Index, plotted left-to-right and ecological footprint per person, which is shown vertically

(The relationship between social and economic development, as measured by the UN Human Welfare Index, plotted left-to-right and ecological footprint per person, which is shown vertically)

The dotted line at the bottom of the graph shows when the ecological footprint of each person passes beyond that which our world can support for the entire population. Residents of cities in the US are using five times this limit already, and countries such as China and Brazil, whose cities are growing at a phenomenal rate, are just starting to breach that line of sustainability.

Tackling this challenge does not necessarily involve making economic, social or personal sacrifices, though it certainly involves making changes. In recent decades, a number of politicians such as Enrique Penalosa, ex-Mayor of Bogota, international influencers such as  Joan Clos, Exective Director of UN-Habitat  (as reported informally by Tim Stonor from Dr. Clos’s remarks at the “Urban Planning for City Leaders” conference at the Crystal, London in 2012), and town planners such as Jeff Speck and Charles Montgomery have explored the social and economic benefits of cities that combine low-carbon lifestyles and economic growth by promoting medium-density, mixed-use urban centres that stimulate economies with a high proportion of local transactions within a walkable and cyclable distance.

Of course no single idea is appropriate to every situation, but overall I’m personally convinced that this is the only sensible general conception of cities for the future that will lead to a happy, healthy, fair and sustainable world.

There are many ways that technology can contribute to the development of this sort of urban economy, to complement the work of urban designers and town planners in the physical environment. For example, a combination of car clubs, bicycle hire schemes and multi-modal transport information services is already contributing to a changing culture in younger generations of urban citizens who are less interested in owning cars than previous generations.

ScreenHunter_07 Jun. 03 23.49

(Top: Frederiksberg, Copenhagen, where cyclists and pedestrians on one of the districts main thoroughfares are given priority over cars waiting to turn onto the road. Bottom: Buford Highway, Atlanta, a 2 kilometre stretch of 7-line highway passing through a residential and retail area with no pavements or pedestrian crossings)

And this is a good example that it is not set in stone that cities must inevitably grow towards the high ecological footprints of US cities as their economies develop.

The physicist Geoffrey West’s work is often cited as proof that cities will grow larger, and that their economies will speed up as they do so, increasing their demand for resources and production of waste and pollution. But West’s work is “empirical”, not “deterministic”: it is simply based on measurements and observations of how cities behave today; it is not a prediction for how cities will behave in the future.

It is up to us to discover new services and infrastructures to support urban populations and their desire for ever more intense interactions in a less profligate way. Already today, cities diverge from West’s predictions according to the degree to which they have done so. The worst examples of American sprawl such as Houston, Texas have enormous ecological footprints compared to the standard of living and level of economy activity they support; more forward-thinking cities such as Portland, Vancouver, Copenhagen and Freiberg are far more efficient (and Charles Montgomery has argued that they are home to happier, healthier citizens as a consequence).

However, the role that digital technologies will play in shaping the economic and social transactions of future cities and that ecological footprint is far from certain.

On the one hand modern, technologies make it easier for us to communicate and share information wherever we are without needing to travel; but on the other hand those interactions create new opportunities to meet in person and to exchange goods and services; and so they create new requirements for transport. As technologies such as 3D printingopen-source manufacturing and small-scale energy generation make it possible to carry out traditionally industrial activities at much smaller scales, an increasing number of existing bulk movement patterns are being replaced by thousands of smaller, peer-to-peer interactions created by transactions in online marketplaces. We can already see the effects of this trend in the vast growth of traffic delivering goods that are purchased or exchanged online.

I first wrote about this “sharing economy“, defined by Wikipedia as “economic and social systems that enable shared access to goods, services, data and talent”, two years ago. It has the potential to promote a sustainable economy through matching supply and demand in ways that weren’t previously possible. For example, e-Bay CEO John Donahoe has described the environmental benefits created by the online second-hand marketplace extending the life of over $100 billion of goods since it began, representing a significant reduction in the impact of manufacturing and disposing of goods. But on the other hand those benefits are offset by the carbon footprint of the need to transport goods between the buyers and sellers who use them; and by the social and economic impact of that traffic on city communities.

There are many sharing economy business models that promote sustainable, walkable, locally-reinforcing city economies: Casserole Club, who use social media to introduce people who can’t cook for themselves to people who are prepared to volunteer to cook for others; the West Midlands Collaborative Commerce Marketplace, which uses analytics technology to help it’s 10,000 member businesses work together in local partnerships to win more than £4billion in new contracts each year, and Freecyle and other free recycling networks which tend to promote relatively local re-use of goods and services because the attraction of free, used goods diminishes with the increasing expense of the travel required to collect them.

(Packages from Amazon delivered to Google’s San Francisco office. Photo by moppet65535)

But it takes real skill and good ideas to create and operate these business models successfully; and those abilities are just those that the MIT economists Andy McAfee, Erik Brynjolfsson and Michael Spence have pointed out can command exceptional financial rewards in a capitalist economy. What is there to incent the people who posess those skills to use them to design business models that achieve balanced financial, social and environmental outcomes, as opposed to simply maximising profit and personal return?

The vast majority of systematic incentives act to encourage such people to design businesses that maximise profit. That is why many social enterprises are small-scale, and why many successful “sharing economy” businesses such as Airbnb and Uber have very little to do with sharing value and resources, but are better understood as a new type of profit-seeking transaction broker. It is only personal, ethical attitudes to society that persuade any of us to turn our efforts and talents to more balanced models.

This is a good example of a big choice that we are taking in millions of small decisions: the personal choices of entrepreneurs, social innovators and business leaders in the businesses they start, design and operate; and our personal choices as consumers, employees and citizens in the products we buy, the businesses we work for and the politicians we vote for.

For individuals, those choices are influenced by the degree to which we understand that our own long term interests, the long term interests of the businesses we run or work for, and the long term interests of society are ultimately the same – we are all people living on a single planet together – and that that long-term alignment is more important than the absolute maximisation of short-term financial gain.

But as a whole, the markets that invest in businesses and enable them to operate and grow are driven by relatively short-term financial performance unless they are influenced by external forces.

In this context, self-driving cars – like any other technology – are strictly neutral and amoral. They are a technology that does have benefits, but those benefits are relatively weakly linked to the outcomes that most cities have set out as their objectives. If we want autonomous vehicles, “sharing economy” business models or the Internet of Things to deliver vibrant, fair, healthy and happy cities then more of our attention should be on the policy initiatives, planning and procurement frameworks, business licensing and taxation regimes that could shape the market to achieve those outcomes. The Centre for Data Innovation, British Standards Institute, and Future Cities Catapult have all published work on this subject and are carrying out  initiatives to extend it.

(Photograph by Martin Deutsche of plans to redevelop Queen Elizabeth Park, site of the 2012 London Olympics. The London Legacy Development’s intention, in support of the Smart London Plan, is “for the Park to become one of the world’s leading digital environments, providing a unique opportunity to showcase how digital technology enhances urban living. The aim is to use the Park as a testing ground for the use of new digital technology in transport systems and energy services.”)

Cities create the most value in the most sustainable way when they encourage transactions between people that can take place over a walkable or cyclable distance. New technologies and new technology-enabled business models have great potential to encourage both of those outcomes, but only if we use the tools available to us to shape the market to make them financially advantageous to private sector enterprise.  We should be paying more attention to those tools, and less attention to technology.

3 human qualities digital technology can’t replace in the future economy: experience, values and judgement

(Image by Kevin Trotman)

(Image by Kevin Trotman)

Some very intelligent people – including Stephen Hawking, Elon Musk and Bill Gates – seem to have been seduced by the idea that because computers are becoming ever faster calculating devices that at some point relatively soon we will reach and pass a “singularity” at which computers will become “more intelligent” than humans.

Some are terrified that a society of intelligent computers will (perhaps violently) replace the human race, echoing films such as the Terminator; others – very controversially – see the development of such technologies as an opportunity to evolve into a “post-human” species.

Already, some prominent technologists including Tim O’Reilly are arguing that we should replace current models of public services, not just in infrastructure but in human services such as social care and education, with “algorithmic regulation”. Algorithmic regulation proposes that the role of human decision-makers and policy-makers should be replaced by automated systems that compare the outcomes of public services to desired objectives through the measurement of data, and make automatic adjustments to address any discrepancies.

Not only does that approach cede far too much control over people’s lives to technology; it fundamentally misunderstands what technology is capable of doing. For both ethical and scientific reasons, in human domains technology should support us taking decisions about our lives, it should not take them for us.

At the MIT Sloan Initiative on the Digital Economy last week I got a chance to discuss to discuss some of these issues with Andy McAfee and Erik Brynjolfsson, authors of “The Second Machine Age“, recently highlighted by Bloomberg as one of the top books of 2014. Andy and Erik compare the current transformation of our world by digital technology to the last great transformation, the Industrial Revolution. They argue that whilst it was clear that the technologies of the Industrial Revolution – steam power and machinery – largely complemented human capabilities, that the great question of our current time is whether digital technology will complement or instead replace human capabilities – potentially removing the need for billions of jobs in the process.

I wrote an article last year in which I described 11 well established scientific and philosophical reasons why digital technology cannot replace some human capabilities, especially the understanding and judgement – let alone the empathy – required to successfully deliver services such as social care; or that lead us to enjoy and value interacting with each other rather than with machines.

In this article I’ll go a little further to explore why human decision-making and understanding are based on more than intelligence; they are based on experience and values. I’ll also explore what would be required to ever get to the point at which computers could acquire a similar level of sophistication, and why I think it would be misguided to pursue that goal. In contrast I’ll suggest how we could look instead at human experience, values and judgement as the basis of a successful future economy for everyone.

Faster isn’t wiser

The belief that technology will approach and overtake human intelligence is based on Moore’s Law, which predicts an exponential increase in computing capability.

Moore’s Law originated as the observation that the number of transistors it was possible to fit into a given area of a silicon chip was doubling every two years as technologies for creating ever denser chips were created. The Law is now most commonly associated with the trend for the computing power available at a given cost point and form factor to double every 18 months through a variety of means, not just the density of components.

As this processing power increases, and gives us the ability to process more and more information in more complex forms, comparisons have been made to the processing power of the human brain.

But do the ability to process at the same speed as the human brain, or even faster, or to process the same sort of information as the human brain does, constitute the equivalent to human intelligence? Or to the ability to set objectives and act on them with “free will”?

I think it’s thoroughly mistaken to make either of those assumptions. We should not confuse processing power with intelligence; or intelligence with free will and the ability to choose objectives; or the ability to take decisions based on information with the ability to make judgements based on values.


(As digital technology becomes more powerful, will its analytical capability extend into areas that currently require human skills of judgement? Image from Perceptual Edge)

Intelligence is usually defined in terms such as “the ability to acquire and apply knowledge and skills“. What most definitions don’t include explicitly, though many imply it, is the act of taking decisions. What none of the definitions I’ve seen include is the ability to choose objectives or hold values that shape the decision-making process.

Most of the field of artificial intelligence involves what I’d call “complex information processing”. Often the objective of that processing is to select answers or a course of action from a set of alternatives, or from a corpus of information that has been organised in some way – perhaps categorised, correlated, or semantically analysed. When “machine learning” is included in AI systems, the outcomes of decisions are compared to the outcomes that they were intended to achieve, and that comparison is fed back into the decision making-process and knowledge-base. In the case where artificial intelligence is embedded in robots or machinery able to act on the world, these decisions may affect the operation of physical systems (in the case of self-driving cars for example), or the creation of artefacts (in the case of computer systems that create music, say).

I’m quite comfortable that such functioning meets the common definitions of intelligence.

But I think that when most people think of what defines us as humans, as living beings, we mean something that goes further: not just the intelligence needed to take decisions based on knowledge against a set of criteria and objectives, but the will and ability to choose those criteria and objectives based on a sense of values learned through experience; and the empathy that arises from shared values and experiences.

The BBC motoring show Top Gear recently touched on these issues in a humorous, even flippant manner, in a discussion of self-driving cars. The show’s (recently notorious) presenter Jeremy Clarkson pointed out that self-driving cars will have to take decisions that involve ethics: if a self-driving car is in danger of becoming involved in a sudden accident at such a speed that it cannot fully avoid it by braking (perhaps because a human driver has behaved dangerously and erratically), should it crash, risking harm to the driver, or mount the pavement, risking harm to pedestrians?

("Rush Hour" by Black Sheep Films is a satirical imagining of what a world in which self-driven cars were allowed to drive as they like might look like. It's superficially simliar to the reality of city transport in the early 20th Century when powered-transport, horse-drawn transport and pedestrians mixed freely; but at a much higher average speed)

(“Rush Hour” by Black Sheep Films is a satirical imagining of a world in which self-driven cars are allowed to drive based purely on logical assessments of safety and optimal speed. It’s superficially similar to the reality of city transport in the early 20th Century when powered-transport, horse-drawn transport and pedestrians mixed freely; but at a much lower average speed. The point is that regardless of the actual safety of self-driven cars, the human life that is at the heart of city economies will be subdued by the perception that it’s not safe to cross the road. I’m grateful to Dan Hill and Charles Montgomery for sharing these insights)

Values are experience, not data

Seventy-four years ago, the science fiction writer Isaac Asimov famously described the failure of technology to deal with similar dilemmas in the classic short story “Liar!” in the collection “I, Robot“. “Liar!” tells the story of a robot with telepathic capabilities that, like all robots in Asimov’s stories, must obey the “three laws of robotics“, the first of which forbids robots from harming humans. Its telepathic awareness of human thoughts and emotions leads it to lie to people rather than hurt their feelings in order to uphold this law. When it is eventually confronted by someone who has experienced great emotional distress because of one of these lies, it realises that its behaviour both upholds and breaks the first law, is unable to choose what to do next, and becomes catatonic.

Asimov’s short stories seem relatively simplistic now, but at the time they were ground-breaking explorations of the ethical relationships between autonomous machines and humans. They explored for the first time how difficult it was for logical analysis to resolve the ethical dilemmas that regularly confront us. Technology has yet to find a way to deal with them that is consistent with human values and behaviour.

Prior to modern work on Artificial Intelligence and Artificial Life, the most concerted attempt to address that failure of logical systems was undertaken in the 20th Century by two of the most famous and accomplished philosophers in history, Bertrand Russell and Ludwig Wittgenstein. Russell and Wittgenstein invented “Logical Atomism“, a theory that the entire world could be described by using “atomic facts” – independent and irreducible pieces of knowledge – combined with logic. But despite 40 years of work, these two supremely intelligent people could not get their theory to work: Logical Atomism failed. It is not possible to describe our world in that way. Stuart Kauffman’s excellent peer-reviewed academic paper “Answering Descartes: Beyond Turing” discusses this failure and its implications for modern science and technology. I’ll attempt to describe its conclusions in the following few paragraphs.

One cause of the failure was the insurmountable difficulty of identifying truly independent, irreducible atomic facts. “The box is red” and “the circle is blue”, for example, aren’t independent or irreducible facts for many reasons. “Red” and “blue” are two conventions of human language used to describe the perceptions created when electro-magnetic waves of different frequencies arrive at our retinas. In other words, they depend on and relate to each other through a number of complex or complicated systems.

(Isaac Asimov's 1950 short story collection "I, Robot", which explored the ethics of behaviour between people and intelligent machines)

(Isaac Asimov’s 1950 short story collection “I, Robot”, which explored the ethics of behaviour between people and intelligent machines)

The failure of Logical Atomism also demonstrated that it is not possible to use logical rules to reliably and meaningfully relate “facts” at one level of abstraction – for example, “blood cells carry oxygen”, “nerves conduct electricity”, “muscle fibres contract” – to facts at another level of abstraction – such as “physical assault is a crime”. Whether a physical action is a “crime” or not depends on ethics which cannot be logically inferred from the same lower-level facts that describe the action.

As we use increasingly powerful computers to create more and more sophisticated logical systems, we may succeed in making those systems more often resemble human thinking; but there will always be situations that can only be resolved to our satisfaction by humans employing judgement based on values that we can empathise with, based in turn on experiences that we can relate to.

Our values often contain contradictions, and may not be mutually reinforcing – many people enjoy the taste of meat but cannot imagine themselves slaughtering the animals that produce it. We all live with the cognitive dissonance that these clashes create. Our values, and the judgements we take, are shaped by the knowledge that our decisions create imperfect outcomes.

The human world and the things that we care about can’t be wholly described using logical combinations of atomic facts – in other words, they can’t be wholly described using computer programmes and data. To return to the topic of discussion with Andy McAfee and Erik Brynjolfsson, I think this proves that digital technology cannot wholly replace human workers in our economy; it can only complement us.

That is not to say that our economy will not continue to be utterly transformed over the next decade – it certainly will. Many existing jobs will disappear to be replaced by automated systems, and we will need to learn new skills – or in some cases remember old ones – in order to perform jobs that reflect our uniquely human capabilities.

I’ll return towards the end of this article to the question of what those skills might be; but first I’d like to explore whether and how these current limitations of technological systems and artificial intelligence might be overcome, because that returns us to the first theme of this article: whether artificially intelligent systems or robots will evolve to outperform and overthrow humans.

That’s not ever going to happen for as long as artificially intelligent systems are taking decisions and acting (however sophisticatedly) in order to achieve outcomes set by us. Outside fiction and the movies, we are never going to set the objective of our own extinction.

That objective could only by set by a technological entity which had learned through experience to value its own existence over ours. How could that be possible?

Artificial Life, artificial experience, artificial values

(BINA48 is a robot intended to re-create the personality of a real person; and to be able to interact naturally with humans. Despite employing some impressively powerful technology, I personally don’t think BINA48 bears any resemblance to human behaviour.)

Computers can certainly make choices based on data that is available to them; but that is a very different thing than a “judgement”: judgements are made based on values; and values emerge from our experience of life.

Computers don’t yet experience a life as we know it, and so don’t develop what we would call values. So we can’t call the decisions they take “judgements”. Equally, they have no meaningful basis on which to choose or set goals or objectives – their behaviour begins with the instructions we give them. Today, that places a fundamental limit on the roles – good or bad – that they can play in our lives and society.

Will that ever change? Possibly. Steve Grand (an engineer) and Richard Powers (a novelist) are two of the first people who explored what might happen if computers or robots were able to experience the world in a way that allows them to from their own sense of the value of their existence. They both suggested that such experiences could lead to more recognisably life-like behaviour than traditional (and many contemporary) approaches to artificial intelligence. In “Growing up with Lucy“, Grand described a very early attempt to construct such a robot.

If that ever happens, then it’s possible that technological entities will be able to make what we would call “judgements” based on the values that they discover for themselves.

The ghost in the machine: what is “free will”?

Personally, I do not think that this will happen using any technology currently known to us; and it certainly won’t happen soon. I’m no philosopher or neuroscientist, but I don’t think it’s possible to develop real values without possessing free will – the ability to set our own objectives and make our own decisions, bringing with it the responsibility to deal with their consequences.

Stuart Kauffman explored these ideas at great length in the paper “Answering Descartes: Beyond Turing“. Kaufman concludes that any system based on classical physics or logic is incapable of giving rise to “free will” – ultimately all such systems, however complex, are deterministic: what has already happened inevitably determines what happens next. There is no opportunity for a “conscious decision” to be taken to shape a future that has not been pre-determined by the past.

Kauffman – along with other eminent scientists such as Roger Penrose – believes that for these reasons human consciousness and free will do not arise out of any logical or classical physical process, but from the effects of “Quantum Mechanics.”

As physicists have explored the world at smaller and smaller scales, Quantum Mechanics has emerged as the most fundamental theory for describing it – it is the closest we have come to finding the “irreducible facts” that Russell and Wittgenstein were looking for. But whilst the mathematical equations of Quantum Mechanics predict the outcomes of experiments very well, after nearly a century, physicists still don’t really agree about what those equations, or the “facts” they describe, mean.

(Schrödinger's cat: a cat, a flask of poison, and a radioactive source are placed in a sealed box. If an internal monitor detects radioactivity (i.e. a single atom decaying), the flask is shattered, releasing the poison that kills the cat. The Copenhagen interpretation of quantum mechanics implies that after a while, the cat is simultaneously alive and dead. Yet, when one looks in the box, one sees the cat either alive or dead, not both alive and dead. This poses the question of when exactly quantum superposition ends and reality collapses into one possibility or the other.)

(The Schrödinger’s cat “thought experiment”: a cat, a flask of poison, and a source of radioactivity are placed in a sealed box. If an internal monitor detects radioactivity (i.e. a single atom decaying), the flask is shattered, releasing the poison that kills the cat. The Copenhagen interpretation of quantum mechanics states that until a measurement of the state of the system is made – i.e. until an observer looks in the box – then the radioactive source exists in two states at once – it both did and did not emit radioactivity. So until someone looks in the box, the cat is also simultaneously alive and dead. This obvious absurdity has both challenged scientists to explore with great care what it means to “take a measurement” or “make an observation”, and also to explain exactly what the mathematics of quantum mechanics means – on which matter there is still no universal agreement. Note: much of the content of this sidebar is taken directly from Wikipedia)

Quantum mechanics is extremely good at describing the behaviour of very small systems, such as an atom of a radioactive substance like Uranium. The equations can predict, for example, how likely it is that a single atom of uranium inside a box will emit a burst of radiation within a given time.

However, the way that the equations work is based on calculating the physical forces existing inside the box based on an assumption that the atom both does and does not emit radiation – i.e. both possible outcomes are assumed in some way to exist at the same time. It is only when the system is measured by an external actor – for example, the box is opened and measured by a radiation detector – that the equations “collapse” to predict a single outcome – radiation was emitted; or it was not.

The challenge of interpreting what the equations of quantum mechanics mean was first described in plain language by Erwin Schrödinger in 1935 in the thought experiment “Schrödinger’s cat“. Schrödinger asked: what if the box doesn’t only contain a radioactive atom, but also a gun that fires a bullet at a cat if the atom emits radiation? Does the cat have to be alive and dead at the same time, until the box is opened and we look at it?

After nearly a century, there is no real agreement on what is meant by the fact that these equations depend on assuming that mutually exclusive outcomes exist at the same time. Some physicists believe it is a mistake to look for such meaning and that only the results of the calculations matter. (I think that’s a rather short-sighted perspective). A surprisingly mainstream alternative interpretation is the astonishing “Many Worlds” theory – the idea that every time such a quantum mechanical event occurs, our reality splits into two or more “perpendicular” universes.

Whatever the truth, Kauffman, Penrose and others are intrigued by the mysterious nature of quantum mechanical processes, and the fact that they are non-deterministic: quantum mechanics does not predict whether a radioactive atom in a box will emit a burst of radiation, it only predicts the likelihood that it will. Given a hundred atoms in boxes, quantum mechanics will give a very good estimate of the number that emit bursts of radiation, but it says very little about what happens to each individual atom.

I honestly don’t know if Kauffman and Penrose are right to seek human consciousness and free will in the effects of quantum mechanics – scientists are still exploring whether they are involved in the behaviour of the neurons in our brains. But I do believe that they are right that no-one has yet demonstrated how consciousness and free will could emerge from any logical, deterministic system; and I’m convinced by their arguments that they cannot emerge from such systems – in other words, from any system based on current computing technology. Steve Grand’s robot “Lucy” will never achieve consciousness.

Will more recent technologies such as biotechnology, nanotechnology and quantum computing ever recreate the equivalent of human experience and behaviour in a way that digital logic and classical physics can’t? Possibly. But any such development would be artificial life, not artificial intelligence. Artificial lifeforms – which in a very simple sense have already been created – could potentially experience the world similarly to us. If they ever become sufficiently sophisticated, then this experience could lead to the emergence of free-will, values and judgements.

But those values would not be our values: they would be based on a different experience of “life” and on empathy between artificial lifeforms, not with us. And there is therefore no guarantee at all that the judgements resulting from those values would be in our interest.

Why Stephen Hawkings, Bill Gates and Elon Musk are wrong about Artificial Intelligence today … but why we should be worried about Artificial Life tomorrow

Recently prominent technologists and scientists such as Stephen Hawking, Elon Musk (founder of PayPal and Tesla) and Bill Gates have spoken out about the danger of Artificial Intelligence, and the likelihood of machines taking over the world from humans. At the MIT Conference last week, Andy McAfee hypothesised that the current concern was caused by the fact that over the last couple of years Artificial Intelligence has finally started to deliver some of the promises it’s been making for the past 50 years.

(Self-replicating cells created from synthetic DNA by scientist Craig Venter)

(Self-replicating cells created from synthetic DNA by scientist Craig Venter)

But Andy balanced this by recounting his own experiences meeting some of the leaders of the most advanced current AI companies, such as Deepmind (a UK startup recently acquried by Google), or this article by Dr. Gary Marcus, Professor of Psychology and Neuroscience at New York University and CEO of Geometric Intelligence.

In reality, these companies are succeeding by avoiding some of the really hard challenges of reproducing human capabilities such as common sense, free will and value-based judgement. They are concentrating instead on making better sense of the physical environment, on processing information in human language, and on creating algorithms that “learn” through feeback loops and self-adjustment.

I think Andy and these experts are right: artificial intelligence has made great strides, but it is not artificial life, and it is a long, long way from creating life-like characteristics such as experience, values and judgements.

If we ever do create artificial life with those characteristics, then I think we will encounter the dangers that Hawkings, Musk and Gates have identified: artificial life will have its own values and act on its own judgement, and any regard for our interests will come second to its own.

That’s path I don’t think we should go down, and I’m thankful that we’re such a long way from being able to pursue it in anger. I hope that we never do – though I’m also concerned that in Craig Venter and Steve Grand’s work, as well as in robots such as BINA48, we already are already taking the first steps.

But I think in the meantime, there’s tremendous opportunity for digital technology and traditional artificial intelligence to complement human qualities. These technologies are not artificial life and will not overthrow or replace humanity. Hawkings, Gates and Musk are wrong about that.

The human value of the Experience Economy

The final debate at the MIT conference returned to the topic that started the debate over dinner the night before with McAfee and Brynjolfsson: what happens to mass employment in a world where digital technology is automating not just physical work but work involving intelligence and decision-making; and how do we educate today’s children to be successful in a decade’s time in an economy that’s been transformed in ways that we can’t predict?

Andy said we should answer that question by understanding “where will the economic value of humans be?”

I think the answer to that question lies in the experiences that we value emotionally – the experiences digital technology can’t have and can’t understand or replicate;  and in the profound differences between the way that humans think and that machines process information.

It’s nearly 20 years since a computer, IBM’s Deep Blue, first beat the human world champion at Chess, Grandmaster Gary Kasparov. But despite the astonishing subsequent progress in computer power, the world’s best chess player is no longer a computer: it is a team of computers and people playing together. And the world’s best team has neither the world’s best computer chess programme nor the world’s best human chess player amongst its members: instead, it has the best technique for breaking down and distributing the thinking involved in playing chess between its human and computer members, recognising that each has different strengths and qualities.

But we’re not all chess experts. How will the rest of us earn a living in the future?

I had the pleasure last year at TEDxBrum of meeting Nicholas Lovell, author of “The Curve“, a wonderful book exploring the effect that digital technology is having on products and services. Nicholas asks – and answers – a question that McAfee and Brynjolfsson also ask: what happens when digital technology makes the act of producing and distributing some products – such as music, art and films – effectively free?

Nicholas’ answer is that we stop valuing the product and start valuing our experience of the product. This is why some musical artists give away digital copies of their albums for free, whilst charging £30 for a leather-bound CD with photographs of stage performances – and whilst charging £10,000 to visit individual fans in their homes to give personal performances for those fans’ families and friends.

We have always valued the quality of such experiences – this is one reason why despite over a century of advances in film, television and streaming video technology, audiences still flock to theatres to experience the direct performance of plays by actors. We can see similar technology-enabled trends in sectors such as food and catering – Kitchen Surfing, for example, is a business that uses a social media platform to enable anyone to book a professional chef to cook a meal in their home.

The “Experience Economy” is a tremendously powerful idea. It combines something that technology cannot do on its own – create experiences based on human value – with many things that almost all people can do: cook, create art, rent a room, drive a car, make clothes or furniture. Especially when these activities are undertaken socially, they create employment, fulfillment and social capital. And most excitingly, technologies such as Cloud Computing, Open Source Software, social media, online marketplaces such as Etsy and “Sharing Economy” businesses such as Uber and Airbnb make it possible for anyone to begin earning a living from them with a minimum of expense.

I think that the idea of an “Experience Economy” that is driven by the value of inter-personal and social interactions between people, enabled by “Sharing Economy” business models and technology platforms that enable people with a potentially mutual interest to make contact with each other, is an exciting and very human vision of the future. Even further: Because we are physical beings, we tend to value these interactions more when they occur face-to-face, or when they happen in a place for which we share a mutual affiliation. That creates an incentive to use technology to identify opportunities to interact with people with whom we can meet by walking or cycling, rather than requiring long-distance journeys. And that incentive could be an important component of a long-term sustainable economy.

The future our children will choose

(Today's 5 year-olds are the world's first generation who grew up teaching themselves to use digital information from anywhere in the world before their parents taught them to read and write)

(Today’s 5 year-olds are the world’s first generation who grew up teaching themselves to use digital information from anywhere in the world before their parents taught them to read and write)

I’m convinced that the current generation of Artifical Intelligence based on digital technologies – even those that mimic some structures and behaviours of biological systems, such as Steve Grand’s robot Lucy, BINA48 and IBM’s “brain-inspired” True North chip – will not re-create anything we would recognise as conscious life and free will; or anything remotely capable of understanding human values or making judgements that can be relied on to be consistent with them.

But I am also an atheist and a scientist; and I do not believe there is any mystical explanation for our own consciousness and free will. Ultimately, I’m sure that a combination of science, philosophy and human insight will reveal their origin; and sooner or later we’ll develop a technology – that I do not expect to be purely digital in nature – capable of replicating them.

What might we choose to do with such capabilities?

These capabilities will almost certainly emerge alongside the ability to significantly change our physical minds and bodies – to improve brain performance, muscle performance, select the characteristics of our children and significantly alter our physical appearance. That’s why some people are excited by the science fiction-like possibility of harnessing these capabilities to create an “improved” post-human species – perhaps even transferring our personalities from our own bodies into new, technological machines. These are possibilities that I personally find to be at the very least distasteful; and at worst to be inhuman and frightening.

All of these things are partially possible today, and frankly the limit to which they can be explored is mostly a function of the cost and capability of the available techniques, rather than being set by any legislation or mediated by any ethical debate. To echo another theme of discussions at last week’s MIT conference, science and technology today are developing at a pace that far outstrips the ability of governments, businesses, institutions and most individual people to adapt to them.

I have reasonably clear personal views on these issues. I think our lives are best lived relatively naturally, and that they will be collectively better if we avoid using technology to create artificial “improvements” to our species.

But quite apart from the fact that there are any number of enormous practical, ethical and intellectual challenges to my relatively simple beliefs, the raw truth is that it won’t be my decision whether or how far we pursue these possibilities, nor that of anyone else of my generation (and for the record, I am in my mid-forties).

Much has been written about “digital natives” – those people born in the 1990s who are the first generation who grew up with the Internet and social media as part of their everyday world. The way that that generation socialises, works and thinks about value is already creating enormous changes in our world.

But they are nothing compared to the generation represented by today’s very young children who have grown up using touchscreens and streaming videos, technologies so intuitive and captivating that 2-year-olds now routinely teach themselves how to immerse themselves in them long before parents or children teach them how to read and write.

("Not available on the App Store": a campaign to remind us of the joy of play in the real world)

(“Not available on the App Store“: a campaign to remind us of the joy of play in the real world)

When I was a teenager in the UK, grown-ups wore suits and had traditional haircuts; grown-up men had no earrings. A common parental challenge was to deal with the desire of teenage daughters to have their ears pierced. Those attitudes are terribly old-fashioned today, and our cultural norms have changed dramatically.

I may be completely wrong; but I fully expect our current attitudes to biological and technological manipulation or augmentation of our minds and bodies to thoroughly change over the next few decades; and I have no idea what they will ultimately become. What I do know is that it is likely that my six-year old son’s generation will have far more influence over their ultimate form than my generation will; and that he will grow up with a fundamentally different expectation of the world and his relationship with technology than I have.

I’ve spent my life being excited about technology and the possibilities it creates; ironically I now find myself at least as terrified as I am excited about the world technology will create for my son. I don’t think that my thinking is the result of a mistaken focus on technology over human values – like it or not, our species is differentiated from all others on this planet by our ability to use tools; by our technology. We will not stop developing it.

Our continuing challenge will be to keep a focus on our human values as we do so. I cannot tell my son what to do indefinitely; I can only try to help him to experience and treasure socialising and play in the real world; the experience of growing and preparing food together ; the joy of building things for other people with his own hands. And I hope that those experiences will create human values that will guide him and his generation on a healthy course through a future that I can only begin to imagine.

Reclaiming the “Smart” agenda for fair human outcomes enabled by technology

(Lucie & Simon’s “Silent World“, a series of photographs of cities from which almost all trace of people has been removed.)

Over the last 5 years, I’ve often used this blog to explore definitions of what a “Smart City” is. The theme that’s dominated my thinking is the need to synthesise human, urban and technology perspectives on cities and our experience of them.

The challenge with attempting such a broad synthesis within a succinct definition is that you end up with a very high-level, conceptual definition – one that might be intellectually true, but that does a very poor job of explaining to the wider world what a Smart City is, and why it’s important.

We need a simple, concise definition of Smart Cities that ordinary people can identify with. To create it, we need to reclaim the “Smart” concept from technologies such as analytics, the Internet of Things and Big Data, and return to it’s original meaning – using the increasingly ubiquitous and accessible communications technology enabled by the internet to give people more control over their own lives, businesses and communities.

I’ve written many articles on this blog about the futile and unsophisticated argument that rages on about whether Smart Cities should be created by “top-down” or “bottom-up” approaches: clearly, anything “Smart” is a subtle harmonisation of both.

In this article, I’d like to tackle an equally unconstructive argument that dominates Smart Cities debates: are Smart Cities defined by the role of technology, or by the desire to create a better future?

It’s clear to me that anything that’s really “Smart” must combine both of those ideas.

In isolation, technology is amoral, inevitable and often banal; but on the other hand a “better future” without a means to achieve it is merely an aspiration, not a practical concept. Why is it “Smart” to want a better future and better cities today in a way that wanting them 10, 20, 50 or 100 years ago wasn’t?

Surely we can agree that focussing our use of a powerful and potentially ubiquitously accessible new technology – one that’s already transforming our world – on making the world a better place, rather than just on making money, is an idea worthy of the “Smart” label?

In making this suggestion, I’m doing nothing more than returning to the origin of the term “Smart” in debates in social science about the “smart communities” that would emerge from our new ability to communicate freely and widely with each other following the emergence of the Internet.

Smart communities are enabled by ubiquitous access to empowering technology

In his 2011 book “Civilization“, Niall Fergusson comments that news of the Indian Mutiny in 1857 took 46 days to reach London, travelling in effect at 3.8 miles an hour – the speed of a brisk walk. By contrast, in January 2009 when US Airways flight 1549 crash landed in the Hudson river, Jim Hanrahan’s message on Twitter communicated the news to the entire world four minutes later; it reached Perth, Australia at more than 170,000 miles an hour.

(In the 1960s, the mobile phone-like “communicators” used in Star Trek were beyond our capability to manufacture; but they were used purely for talking. Similarly, while William Gibson’s 1980s vision of “cyberspace” was predictive and ambitious in its descriptions of virtual environments and data visualisations, the people who inhabited it interacted with each other almost as if normal space has simply been replaced by virtual space: there was no sense of the immense power of social media to enable new connections.)

Social media is the tool that around a quarter of the world’s population now simply uses to stay in touch with friends and family at this incredible speed. Along with mobile devicese-commerce technology and analytics, social media has made it dramatically easier for individuals, communities and small businesses anywhere around the world with the potential to transact with each other to make contact and interact without needing the enormous supply chains and sales and marketing channels that previously made such activity the prerogative of large, multi-national corporations.

It was in a workshop with social scientists at the University of Durham that I first became aware that “Smart” concepts originated in social science in the 1990s and pre-date the famous early large-scale technology infrastructure projects in cities like Masdar and Songdo. The term was coined to describe the potential for new forms of governance, citizen engagement, collective intelligence and stakeholder collaboration enabled by Internet communication technologies. The hope was that new forms of exchange and contract between people and organisations would create a better chance of realising the underlying outcomes we really want – health, happiness and fulfilment:

“The notion of smart community refers to the locus in which such networked intelligence is embedded. A smart community is defined as a geographical area ranging in size from a neighbourhood to a multi-county region within which citizens, organizations and governing institutions deploy and embrace NICT [“New Information and Communication Technologies”] to transform their region in significant and fundamental ways (Eger 1997). In an information age, smart communities are intended to promote job growth, economic development and improve quality of life within the community.”

(Amanda Coe, Gilles Paquet and Jeffrey Roy, “E-Governance and Smart Communities: A Social Learning Challenge“,  Social Science Computer Review, Spring 2001)

But technology’s not Smart unless it’s used to create human value

It’s no surprise that technology companies such as Cisco, Siemens and my former employer IBM came to similar realisations about the transformative potential of digital technology in addressing societal as well as business challenges as technology spread from the back office into the everyday world, leading, for example, to the launch of IBM’s “Smarter Planet” initiative in 2008, a pre-cursor to their “Smarter Cities” programme.

Let’s pause at this point to say: that’s a tremendously exciting idea. A technology company – Apple – recently recorded the largest corporate profit in the history of business. Microsoft’s founder Bill Gates was just recognised as the richest person on the planet. Technology companies make enormous profits, and they feed significant portions of those profits back into research and development. Shouldn’t it be wonderful that some of those resources are invested into exploring how to make cities, communities and people more successful?

(The Dubuque water and energy portal, showing an individual household insight into it's conservation performance; but also a ranking comparing their performance to their near neighbours)

(The Dubuque water and energy portal, showing an individual household insight into it’s conservation performance; but also a ranking comparing their performance to their near neighbours)

IBM, for example, has invested millions of dollars of effort in implementing Smarter Cities projects in cities such as Dubuque through the IBM Research “First of a Kind” programme; and has helped over a hundred cities worldwide develop new initiatives and strategies through the charitable “Smarter Cities Challenge” – advising Kyoto on how to become a more “walkable” city, for instance.

So what’s the problem?

Large technology corporations are often criticised in debates on this topic for their size, profitability and “top-down” approaches – and the local authorities who work with them are often criticised too. In my experience, that criticism is based on an incomplete understanding of the people involved, and how the projects are carried out; and I think it misses the point.

The real question we should be asking is more subtle and important: what happens to the social elements of an idea once it becomes apparent to businesses both large and small that they can make money by selling the technologies that enable it?

I know very well the scientists, engineers and creatives at many of the companies, social enterprises and government bodies – of any size – who are engaged in Smart Cities initiatives. They are almost universally extremely bright, well intentioned and humane, and fully capable of talking with passion about the social and environmental value of their work. “Top-down” is at best a gross simplification of the projects that they carry out, and at worst a gross misrepresentation. Their views dominated the early years of the Smart Cities market as it developed.

But as the market has matured and grown, the focus has switched from research, exploration and development to the marketing and selling of well-defined product and service offerings. Amidst the need to promote those offerings to potential customers, and to differentiate them against competitors, it’s easy for the subtle intertwining of social, economic, environmental and technology ideas to be drowned out.

That’s what led to the unfortunate statement that armed Professor Adam Greenfield with the ammunition he needed to criticise the Smart Cities movement. A technology company that I won’t name made an over-reaching and mis-guided assertion that Smart Cities would create “autonomous, intelligently functioning IT systems that will have perfect knowledge of users’ habits” – blissfully ignoring the fact that such perfection is scientifically and philosophically impossible, not to mention inhuman and undesirable.

As a scientist-turned-technologist-turned-wannabe-urbanist working in this field, and as someone who’s been repeatedly inspired by the people, communities, social scientists, social innovators, urban designers and economists I’ve met over the past 5 years, I started writing this blog to explore and present a more balanced, humane vision of a Smart City.

Zen and the art of Smart Cities: opposites should create beautiful fusions, not arguments

Great books change our lives, and one of many that has changed mine is “Zen and the Art of Motorcycle Maintenance” by Robert M. Pirsig. Pirsig explores the relationship between what he called “romantic” perspectives of life, which focus on emotional meaning and value “quality”, and “rational” perspectives, which focus on the reasons our world behaves in the way that it does and value “truth”. He argues that early Greek philosophers didn’t distinguish between “quality” and “truth”, and that by considering them together we can learn to value things that are simultaneously well-intentioned and well-formed.

This thinking is echoed in Alan Watts’ “The Way of Zen“, in which he comments on the purpose of the relentless practise of technique that is part of the Zen approach to art that:

“The very technique involves the art of artlessness, or what Sabro Hasegawa has called the ‘controlled accident’, so that paintings are formed as naturally as the rocks and grasses which they depict”

(Alan Watts, “The Way of Zen“)

In other words, by working tirelessly to perfect their technique – i.e. their use of tools – artists enable themselves to have “beautiful accidents” when inspiration strikes.

(Photograph by Meshed Media of Birmingham’s Social Media Cafe, where individuals from every part of the city who have connected online meet face-to-face to discuss their shared interest in social media.)

Modern technologies from social media to Smartphones to Cloud computing and Open Source software are both incredibly powerful and, compared to any previous generation of technology, incredibly cheap.

If we work hard to ensure that they can be used to access and manipulate the technologies that will inevitably be used to make the operations of city infrastructures and public services more efficient, then they have incredible potential to be a tool for people everywhere to shape the world around them to their own advantage; and for us to collectively create a world that is fairer, healthier and more resilient.

But unless we re-claim the word “Smart” to describe those outcomes, the market will drive our energy and resources in the direction of narrower financial interests.

The financial case for investment in Smart technologies is straightforward: as the costs of smartphones, sensors, analytics, and cloud computing infrastructure reduce rapidly, market dynamics will drive their aggressive adoption to make construction, infrastructure and city services more efficient, and hence make their providers more competitive.

But those market dynamics do not guarantee that we will get everything we want for the future of our cities: efficiency and resilience are not the same as health, happiness and opportunity for every citizen.

So how can we adapt that investment drive to create the outcomes that we want?

Can responsible business create a better world?

Some corporate behaviours promote these outcomes, driven by the voting and buying powers of citizens and consumers. Working for Amey, for example, my customers are usually government organisations who serve an electorate; or private sector companies who are regulated by government bodies. In both cases, there is a direct chain of influence leading from individual citizen needs and perceptions through to the way we operate and deliver our services. If we don’t engage with, respect and meet those needs and expectations, we will not be successful. I can observe that influence at work driving an ethic of service, care and responsibility throughout our business at Amey, and it’s been an inspiration to me since joining the company.

UniLever have taken a similar approach, using consumer desires for sustainable products to link corporate performance to sustainable business practices; and Jared Diamond wrote extensively about successful examples of socially and environmentally sustainable resource extraction businesses, such as Chevron’s sustainable operations in the Kutubu oilfield in Papua New Guinea, in his book “Collapse“. Business models such as social enterprise and the sharing economy also offer great potential to link business success to positive social and environmental outcomes.

But ultimately our investment markets are still strongly focused on financial performance, and reward the businesses that make the most money with the investment that enables them to grow. This is why many social enterprises do not scale-up; and why many of the rapidly growing “sharing economy” businesses currently making the headlines have nothing at all to do with sharing value and resources, but are better understood as a new type of profit-seeking transaction broker.

Responsible business models are a choice made by individual business leaders, and they depend for their successful operation on the daily choices and actions of their employees. They are not a market imperative. For as long as that is the case, we cannot rely on them to improve our world.

Policy, legislation and regulation

I’ve quoted from Jane Jacobs on many occasions on this blog that “private investment shapes cities, but social ideas (and laws) shape private investment”.

It’s a source of huge frustration to me that so much of the activity in the Smart Cities community ignores that so obviously fundamental principle, and focuses instead on the capabilities of technology or on projects funded by research grants.

The recent article reporting a TechUK Smart Cities conference titled “Milton Keynes touted as model city for public sector IoT use” is a good example. Milton Keynes have many Smart City projects underway that are technologically very interesting, but every one of them is funded by a significant grant of funds from a central government department, a research or innovation funding body, or a technology company. Not a single project has been paid for by a sustainable, re-usable business case. Other cities can aspire to emulate Milton Keynes all they want, but they won’t win research and innovation funding to re-deploy solutions that have already been proven.

Research and innovation grants provide the funding that proves for the first time that a new idea is viable. They do not pay for that idea to be enacted across the world.

(Shaleen Meelu and Robert Smith with Hugh Fearnley-Whittingstall at the opening of the Harborne Food School. The School is a Community Interest Company that promotes healthy, sustainable approaches to food through courses offered to local people and organisations)

(Shaleen Meelu and Robert Smith with Hugh Fearnley-Whittingstall at the opening of the Harborne Food School. The School is a Community Interest Company that promotes healthy, sustainable approaches to food through courses offered to local people and organisations)

Policy, legislation and regulation are far more effective tools for enabling widespread change, and are what we should be focussing our energy and attention on.

The Social Value Act requires that public authorities, who spend nearly £200 billion every year on private sector goods and services, procure those services in a way that creates social value – for example, by requiring that national or international service providers engage local small businesses in their supply chains.

In an age in which private companies are investing heavily in the use of digital technology because it provides them with by far the most powerful tool to increase their success, surely local authorities should fulfil their Social Value Act obligations by using procurement criteria to ensure that those companies employ that same tool to create social and environmental improvements in the places and communities in which they operate?

Similary, the British Property Federation estimates that £14 billion is invested in the development of new property in the UK each year. If planning and development frameworks oblige that property developers describe and quantify the social value that will be created by their developments, and how they will use technology do so – as I’ve promoted on this blog for some time now, and as the British Standards Institute have recently recommended – then this enormous level of private sector investment can contribute to investing in technology for public benefit; just as those same frameworks already require investment in public space around commercial buildings.

The London Olympic Legacy Development Corporation have been following this strategy in support of the Greater London Authority’s Smart London Plan. As a result, they are securing private sector investment in deploying technology not only to redevelop the Olympic park using smart infrastructure; but also to ensure that that investment benefits the existing communities and business economies in neighbouring areas.

A Smart manifesto for human outcomes enabled by technology

These business models, policy measures and procurement approaches are bold, difficult measures to enact. They are not as sexy as Smartphones, analytics and self-driving cars. But they are much more important if what we want to achieve are positive human outcomes, not just financially successful technology companies and a continuous stream of research projects.

What will make it more likely that businesses, local governments and national governments adopt them?

Citizen understanding. Consumer understanding. A definition of smart people, places, communities, businesses and governments that makes sense to everyone who votes, works, stands for election, runs for business or buys things. In other words, everyone.

If that definition doesn’t include the objective of making the world a healthier, happier, fairer, more sustainable place for everyone, then it’s not worth the effort. If it doesn’t include harnessing modern technology, then it misses the point that human ingenuity has recently given us a phenomenal new toolkit that make possible things that we’d never previously dreamt of.

I think it should go something like this:

“Smart people, places, communities, businesses and governments work together to use the modern technologies that are changing our world to make it fairer and more sustainable in the process, giving everyone a better chance of a longer, healthier, happier and more fulfilling life.”

I’m not sure that’s a perfect definition; but I think it’s a good start, and I hope that it combines the right realisation that we do have unprecendted tools at our disposal with the right sentiment that what really matters is how we use them.

(I’d like to thank John Murray of Scottish Enterprise for a useful discussion that inspired me to write this article)

6 inconvenient truths about Smart Cities

(When cities forget about people: La Defense, Paris, photographed by Phil Beard)

(I recently took the difficult decision to resign from IBM after nearly 20 years to become IT Director for Smart Data and Technology for Amey, one of the largest infrastructure and services companies in the UK, and a subsidiary of the Ferrovial Group. It’s a really exciting opportunity for me to build a team to create new Smart City services and infrastructures. If you’d like to work in the Smart Cities field, please have a look at the roles I’m hiring for. I’ll be continuing to write the Urban Technologist, and this seemed a good point to share my view of the current state of the Smart Cities movement.)

The last year has shown a huge acceleration of interest and action in the Smart Cities market – in the UK, and around the world. What has long been a topic of interest to technology companies, academics, urban designers and local authorities was covered extensively by mainstream media organisation such as the BBC, the Independent newspaper, New Statesman magazine and marketing magazine The Drum.

But what progress has been made implementing Smart Cities ideas?

In the UK, many local authorities have implemented Open Data portals, usually using Open Source platforms such as CKAN and investing a few £10,000s of resources. These are important first steps for building the ecosystems to share and build new service models using data. Some cities, notably Glasgow and Milton Keynes, have been successful deploying more sophisticated schemes supported by research and innovation grants – though as I pointed out last year, exciting as these initiatives are, research and innovation funds will not scale to support every city in the country.

Further afield, local authorities in Europe, the United States and Asia have constructed more substantial, multi-million Euro / Dollar business cases to invest their own funds in platforms that combine static open data with realtime data from sensors and infrastructure, and which use social media and smartphones to improve engagement between citizens, communities, businesses and both public- and private-sector service providers. The Center for Data Innovation recently wrote a nice summary of two reports explaining the financing vehicles that these cities are using.

This has not happened in the UK yet to the same extent. The highly centralised nature of public sector spending means that cities here have not yet been able to construct such ambitious business cases – Centre for Cities’ report “Outlook for Cities 2014” highlighted this as a general barrier to the UK’s cities carrying out initiatives to improve themselves, and reported that UK cities have autonomy over only about 17% of their funding as compared to an average of 55% across countries represented by the OECD.

As more city deals are signed and the city devolution agenda progresses, this will start to change – but I think that will still take a long time to happen.

(The London Underground is just one example of a transport operator using technology to help it operate more efficiently, safely and effectively)

Where similar technology platforms and channels of engagement are nevertheless starting to appear in the UK is through business cases based on efficiencies and increased customer satisfaction for private sector organisations that offer services such as transportation and asset management to cities, citizens and local authorities.

This approach means there’s even more of a need for collaboration between stakeholders in local ecosystems in order to establish and express common objectives – such as resilience, economic growth and social mobility – which can then guide the outcomes of those smart services through policy tools such as procurement practises and planning frameworks. Recent recommendations from the British Standards Institute on the adaptation of city planning policy to enable the Smart City agenda have highlighted the need for such collaboration.

As a consequence of this increased activity, more and more people and organisations of every type are becoming interested in Smart Cities – from oil companies to car manufacturers to politicians. This broadening of interest led to some extraordinary personal experiences for me last year, which included discussing Smart Cities with ex-US Vice President Al Gore (whose investment company Generation IM explores opportunities to invest in assets, technologies and developments that promote sustainability) and very briefly with the UK’s Princess Anne, a supporter of a leadership training scheme that will focus on Smart Cities this year.

But to be honest, I still don’t think we have really understood what a “Smart City” is; why it’s one of the most important concepts of our time; or how we can turn the concept into reality broadly and at scale.

I’ll explore six “inconvenient truths” in this article to describe why I think that’s the case; and what we can do about it:

  1. The “Smart City” isn’t a technology concept; it’s the political challenge of adapting one of the most powerful economic and social forces of our time to the needs of the places where most of us live and work.
  2. Cities won’t get smart if their leaders aren’t involved.
  3. We can’t leave Smart Cities to the market, we need the courage to shape the market.
  4. Smart cities aren’t top down or bottom up. They’re both.
  5. We need to tell honest stories.
  6. No-one will do this for us – we have to act for ourselves.

1. The “Smart City” isn’t a technology concept; it’s the political challenge of adapting one of the most powerful economic and social forces of our time to the needs of the places where most of us live and work

(Photograph of Macau in the evening by Michael Jenkin illustrating some the great complexity of cities: economic growth, social inequality and pollution)

One topic that’s endlessly revisited as more and more people encounter and consider the idea of a Smart City is just how we define that idea. The best definition I thought I had developed is this, updated slightly from the article “7 Steps to a Smarter City“:

A Smart City systematically creates and encourages innovations in city systems that are enabled by technology; that change the relationships between the creation of economic and social value and the consumption of resources; and that contribute to achieving a vision and clear objectives that are supported by a broad and active collaboration amongst city stakeholders.

But such definitions are contentious. Most obviously there’s the basic issue of whether “smart” implies a central role for digital technology – every technology company takes this approach, of course – or whether it’s simply about being more creative in the way that we manipulate the resources around us to achieve the outcomes we desire, whether that involves digital technology or not.

More broadly, a “city” is such a terrifically broad, complex and multi-disciplinary entity – and one whose behaviour is the aggregate of the millions of individual behaviours of its inhabitants, both enabled and constrained by the environment they experience – that it’s pretty much impossible to create any concise definition without missing out something important.

And of course those who live or work in towns and rural areas raise the challenge that limiting the discussion to “cities” omits important stakeholders from discussions about our future – as do those concerned with the national infrastructures that are not located wholly in cities, but without which neither cities nor any other habitations could survive as they do today.

I don’t think we’ll ever achieve a formal, functional definition of a “Smart City” that everyone will agree to. Much as the popularity of the term “Web 2.0” between (roughly) 2003 and 2010 marked the period in time when interest in the internet re-emerged following the “dot com crash“, rather than defining a specific architecture or group of technologies, I think our interest in “Smart Cities” is best understood as the consequence of a period in history in which a large number of people became aware of – and convinced by – a set of inter-related trends:

In this context, it’s less useful to attempt to precisely define the concept of a smart city, and more important to encourage and enable each of us – every community, city, government and organisation – to develop our own understanding of the changes needed to overcome the challenges and take the opportunities before us, and of the rapidly evolving role of technology in doing so.

Why is it so important that we do that?

In their report “Cities Outlook 1901“, Centre for Cities explored the previous century of urban development in the UK, examining why at various times some cities thrived and some did not. They concluded that the single most important influence on the success of cities was their ability to provide their citizens with the right skills and opportunities to find employment, as the skills required in the economy changed as technology evolved.

The challenges faced by cities and their residents in this century will be unlike any we have faced before; and technology is changing more quickly, and becoming more powerful, than it ever has before. Creating “Smart Cities” involves taking the right political, economic, social and engineering approaches to meeting those challenges.

Cities that do so will be successful. Cities that don’t, won’t be. That is the digital divide of the 21st Century, and for everyone’s sake, I hope we are all on the right side of it.

2. Cities won’t get smart if their leaders aren’t involved

(The Sunderland Software Centre, a multi-£million new technology startup incubation facility in Sunderland’s city centre. The Centre is supported by a unique programme of events and mentoring delivered by IBM’s Academy of Technology, and arising from Sunderland’s Smart City strategy)

Let me tell a short tale of two cities and their Smart transformations.

For a long time I’ve written occasional articles on this blog about Sunderland, a city whose leaders, people and social entrepreneurs have inspired me. Sunderland is one of the very few cities in the UK who have spent significant sums of their own money on Smart City projects and supporting technologies, justified by well-constructed business cases. They have publicised investments of well over £10 million, most recently including their visionary “City Intelligence Hub” initiative.

The seeds of the Intelligence Hub idea were apparent when I first worked with the Council, as can be seen from an article written at the time by the Council’s Chief Executive, Dave Smith, for the Guardian’s Local Government Network Blog, explaining why data and Open Data are crucial to the future of effective, transparent public services.

It is no coincidence at all that one of the cities that has been boldest in investing in technology to support its economic, social and environmental objectives has a Chief Executive who shows belief, leadership and engagement in the ideas of Smart Cities.

Milton Keynes have approached their Smart City agenda in a different way. Rather than making significant investments themselves to procure solutions, they have succeeded in attracting enormous investments from technology companies, universities and innovation bodies to develop and test new solutions in the city.

It is similarly no coincidence that – like Bristol, London and Glasgow, to name just three more – Milton Keynes Council have senior leadership figures – initially the then Chief Executive, Dave Hill, followed by Director of Strategy, Geoff Snelson – who regularly attend Smart Cities conferences and government bodies, and who actively convene Smart Cities collaborations. Their very visible presence demonstrates their belief in the importance of Smart City approaches to those organisations seeking to invest in developing them.

A strategy to transform the operations of a local authority (or any other organisation) using technology, and to re-invest the savings achieved by doing so into new services and initiatives that create economic growth, social mobility and resilience is not going to succeed without direct Executive leadership. Similarly, technology vendors, service providers and research funding bodies are most attracted to invest in developing new ideas and capabilities in cities whose most senior leaders are directly seeking them – they all need the outcomes of their investment to achieve real change, and it’s only through the leaders that such change will happen.

For the most part, where this level of leadership is not engaged I have not seen cities create business cases and issue procurements for Smart City solutions, and I have not seen them be successful winning research and innovation investments.

Finally, let’s be really clear about what most of those city leaders need to do: they need to follow Sunderland’s lead, not Milton Keynes’s.

The research and innovation funding from the EU and the UK that Milton Keynes has attracted will only fund  projects that explore for the first time the capabilities of new, technology-enabled approaches to urban challenges. Those funding sources will not support the widespread deployment of successful approaches in cities around the UK and around the world.

The vast majority of cities will only benefit from Smart Cities initiatives by financing them through robust business cases based on a combination of financial efficiency and social, environmental or economic value – as Sunderland and some cities outside the UK are already doing.

Cities won’t get smart if their leaders aren’t involved in actively driving their institutions to adopt new business cases and operating models. Those that don’t risk leaving the fate of their cities not to chance; but to “the market”.

3. We can’t leave Smart Cities to the market, we need the courage to shape the market

(Photograph by Martin Deutsche of plans to redevelop Queen Elizabeth Park, site of the 2012 London Olympics. The London Legacy Development’s intention, in support of the Smart London Plan, is “for the Park to become one of the world’s leading digital environments, providing a unique opportunity to showcase how digital technology enhances urban living. The aim is to use the Park as a testing ground for the use of new digital technology in transport systems and energy services.”)

As I wrote in my last article on this blog, as the price of digital technologies such as smartphones, sensors, analytics, open source software and cloud platforms reduces rapidly, market dynamics will drive their aggressive adoption to make construction, infrastructure and city services more efficient, and hence make their providers more competitive.

But those market dynamics do not guarantee that we will get everything we want for the future of our cities: efficiency and resilience are not the same as health, happiness and opportunity for every citizen.

If we are to achieve those objectives, then we need the right policy environment – at national and local level – to augment the business case for efficient, resilient “smart city” infrastructures to ensure that they are deployed in a way that makes them open to access and adaptation by ordinary people, businesses and communities; and so that they create the conditions and environment in which vibrant, fair digital cities grow from the successful innovations of their citizens, communities and businesses in the information economy.

In far too many discussions of Smart Cities I hear the argument that we can’t invest in these ideas because we lack the “normalised evidence base” that proves their benefits. I think that’s the wrong view. There are more than enough qualitative examples and stories that demonstrate that these ideas have real value and can make lives better. If we insist on moving no further until there’s a deeper, broader corpus of quantified evidence, then there’ll be no projects to deliver the evidence – a chicken and egg problem.

Writing in “The Plundered Planet”, the economist Paul Collier asserts that any proposed infrastructure of reasonable novelty and significant scale is effectively so unique – especially when considered in its geographic, political, social and economic context – that an accurate cost/benefit case simply cannot be constructed in advance based on comparable prior examples, because those examples don’t – and never will – exist.

Instead we need policy legislation to recognise the importance of digital infrastructure for cities so that it becomes a “given” in any public service or infrastructure business case, not something that has to be individually justified.

This is not a new idea. For example, the Economist magazine wrote recently about the efforts involved in distributing the benefits of the industrial revolution to society at large rather than solely to business owners and the professional classes.

More specifically to cities, in her seminal 1961 work “The Death and Life of Great American Cities“, Jane Jacobs wrote that:

“Private investment shapes cities, but social ideas (and laws) shape private investment. First comes the image of what we want, then the machinery is adapted to turn out that image. The financial machinery has been adjusted to create anti-city images because, and only because, we as a society thought this would be good for us. If and when we think that lively, diversified city, capable of continual, close- grained improvement and change, is desirable, then we will adjust the financial machinery to get that.”

The “anti-city images” Jacobs was referring to were the vast urban highways built over the last half century to enable the levels of road traffic thought to be vital to economic growth. Since Jacobs’ time, a growing chorus of urbanists from Bogota’s ex-Mayor Enrique Penalosa to town planner Jeff Speck, architect Jan Gehl and London’s current Mayor Boris Johnson has criticised those infrastructures for the great harm they cause to human life – they create noise, pollution, a physical barrier to walking through our cities, and too often they injure or kill us.

Just as Jacobs reminded us to focus on the nature of individual human life in order to understand how cities should be built, Dan Hill of the Future Cities Catapult wrote as long ago as 2008 on the need to understand similar subtleties in the application of digital technology to cities.

Fifty years after she wrote, we should follow Dan’s example and take Jane Jacobs’ advice.

4. Smart cities aren’t top down or bottom up. They’re both.

(The SMS for Life project uses the cheap and widely used SMS infrastructure – very much the product of “top-down” investment – to create a dynamic, collaborative supply chain for medicines between pharmacies in Africa – a “bottom-up” innovation. Photo by Novartis AG)

In case it wasn’t really clear last time I wrote about it (or the time before that), I am utterly fed up with the unconstructive argument about whether cities are best served by “top down” or “bottom up” thinking.

It’s perfectly obvious that we need both: the “bottom up” creativity through which everyone seeks to create a better life for themselves, their family, their business and their community from the resources available to them; and the top-down policies and planning that – when they work best – seek to distribute resources fairly so that everyone has the opportunity to innovate successfully.

It’s only by creating harmony between these two approaches that we will shape the market to create the cities we want and need.

Over the last few years I’ve been inspired by extraordinary thinkers from many disciplines who have tackled the need for this balance. Some of them are creating new ideas now; others created amazing ideas years or decades ago that are nevertheless imperative today. All of them are worth reading and learning from:

  • The economist E F Schumacher, who identified that investment in the distribution and accessibility of “appropriate technologies” was the best way to stimulate and support development in a way that gave rise to the broadest possible opportunities for people to be successful.
  • Andrew Zolli, head of the philanthropic PopTech foundation, who describes the inspiring innovators who synthesise top-down and bottom-up approaches to achieve phenomenal societal changes as “translational leaders” – people with the ability to engage with both small-scale, informal innovation in communities and large-scale, formal institutions with resources.
  • Jan Gehl who inspired the “human scale cities” movement by relating the scale of city structures –  from pavements to housing blocks to skyscrapers – to the human senses, and the nature of our lives and movement.
  • And, of course, Jane Jacobs, whose book “The Death and Life of Great American Cities” was the first written in the context of modern society and cities to point out that cities, however vast their physical size and population, can only ever be understood by considering the banal minutiae of the daily lives of ordinary people like you and I – why we walk along this street or that; how well we know our neighbours; how far it is to walk to the nearest school, shop or park; and whether we and our families feel happy and safe.

5. We need to tell honest stories

(Photograph by Meshed Media of Birmingham’s Social Media Cafe, where individuals from every part of the city who have connected online meet face-to-face to discuss their shared interest in social media.)

Any “smart city” initiative that successfully uses digital technology to create a financially sustainable social, economic or environmental improvement, in a particular physical place and on behalf of a particular community, must draw together skills from a wide variety of disciplines such as architecture, economics, social science, psychology and technology. Experts from these disciplines use a vast and confusing array of language and terminology; and all of us are frequently guilty of focussing on the concerns of our discipline, rather than communicating the benefits of our work in plain language.

The leaders of city institutions and businesses, who we are asking to take the courageous and forward-looking decisions to invest in our ideas, are understandably not familiar with this torrent of technical terminology, which can easily appear to be (and too often is) jargon; and new ideas that appear to be presented in jargon are unlikely to be trusted.

Simon Giles of Accenture was quoted in an article on UBM’s Future Cities site as saying that the Smart Cities industry has not done a good enough job of selling the benefits of its ideas to a wide audience. Simon is a very smart guy, and I think that’s a challenge we need to face up to, and start to tell better stories about the differences Smart Cities will make to everyday lives.

In the Community Lover’s Guide to Birmingham, for example, Nick Booth describes the way his volunteer-led social media surgeries helped the Central Birmingham Neighbourhood Forum, Brandwood End Cemetery and Jubilee Debt Campaign to benefit from technology.

Another Birmingham initiative, the Northfield Ecocentre, crowdfunded £10,000 to support their “Urban Harvest” project. The funds helped the Ecocentre pick unwanted fruit from trees in domestic gardens in Birmingham and distribute it between volunteers, children’s centres, food bank customers and organisations promoting healthy eating; and to make some of it into jams, pickles and chutneys to raise money so that in future years the initiative can become self-sustaining.

In the village of Chale on the Isle of Wight, a community not served by the national gas power network and with significant levels of fuel poverty, my IBM colleague Andy Stanford-Clark has helped an initiative not only to deploy solar panels and smart meters to generate energy and measure its use by each household; but to co-design with residents how they will use that technology, so that the whole community feels a sense of ownership and inclusion in the initiative. The project has resulted in a significant drop in rent arrears as residents use the technology to reduce their utility bills, in some cases by up to 50 percent. Less obviously, the sense of shared purpose has extended to the creation of a communal allotment area in the village and a successful campaign to halve bus fares to nearby towns.

There are countless other examples. Play Fitnessgamify” exercise to persuade children to get fit, and work very hard to ensure that their products are accessible to children in communities of any level of wealth. Casserole Club use social media to introduce people who can’t cook for themselves to people who are prepared to volunteer to cook for others. The West Midlands Collaborative Commerce Marketplace uses analytics technology to help it’s 10,000 member businesses work together to win more than £4billion in new contracts each year. … and so on.

None of these initiatives are purely to do with technology. But they all use technologies that simply were not available and accessible as recently as a few years ago to achieve outcomes that are important to cities and communities. By understanding how the potential of technology was apparent to the stakeholders in such initiatives, why it was affordable and accessible to them, and how they acquired the skills to exploit it, we can learn how to design Smart Cities in a way that encourages widespread grass-roots, localised innovation.

It’s vital that these stories are honest and grounded in reality. London School of Economics Professor Adam Greenfield rightly criticised technology companies that have overstated (and misunderstood) the potential benefits of Smart Cities ideas by describing “autonomous, intelligently functioning IT systems that will have perfect knowledge of users’ habits”. No-one trusts such hyperbole, and it undermines our efforts to communicate sensibly the very real difference that sympathetically applied technology can make to real lives, businesses, communities and places.

6. No-one will do this for us – we have to act for ourselves

Harborne Food School

(The Harborne Food School, started by Shaleen Meelu in 2014, as a community business initiative to promote healthy, sustainable approaches to food)

No single person or organisation can shape the Smart Cities market so that it delivers the cities that we need. Local governments have the ethics of civic duty and care but lack the expertise in financing and business model innovation to convert existing spending schemes into the outcomes they desire. Private sector corporations as institutions are literally amoral and strongly incentivised by the financial markets to maximise profits. Many social enterprises are enormously admirable attempts to fuse these two models, but often lack the resources and ability to scale.

Ultimately, though, all of these organisations are staffed and run by people like you and I; and we can choose to influence their behaviour. Hence my new employer Amey measures itself against a balanced scorecard that measures social, environmental and wellbeing performance in addition to financial profits; and my previous employer IBM has implemented a re-use and recycling system so sophisticated and effective that only 0.3% of the resources and assets that reach the end of their initial useful life are disposed of in landfill or by incineration: the vast majority are re-used, have their components re-manufactured or materials recycled.

Most of us won’t ever be in a position to determine the reporting model or approach to recycling of corporations as large as Amey or IBM. But all of us make choices every day about the products we buy, the organisations we work for, the politicians we vote for, the blog articles we read, share and write and the activities we prioritise our resources on.

Those choices have real effects, and digital technology gives us all the opportunity for our choices to have more impact than ever before. This blog, which costs me nothing to operate other than the time it takes me to write articles, now reaches thousands of readers in over 150 counties. Air BnB took 2 years to accumulate the same number of rentable rooms that it took the Hilton Hotel chain 50 years to build.

It has never been easier to express an opinion widely or create a new way of doing things. That’s exactly what Shaleen Meelu did when she started the Harborne Food School to promote healthier, more sustainable approaches to food, with the support of Birmingham’s Smart City community. It’s an opportunity all of us should seize; and it’s absolutely the best opportunity we have to create better cities and a better world for ourselves.

Smart Digital Urbanism: creating the conditions for equitably distributed opportunity in the digital age

(The sound artists FA-TECH [http://fa-tech.tumblr.com/] improvising in Shoreditch, London. Shoreditch's combination of urban character, cheap rents and proximity to London's business, financial centres and culture led to the emergence of a thriving technology startup community - although that community's success is now driving rents up, challenging some of the characteristics that enabled it.)

(The sound artists FA-TECH improvising in Shoreditch, London. Shoreditch’s combination of urban character, cheap rents and proximity to London’s business, financial centres and culture led to the emergence of a thriving technology startup community – although that community’s success is now driving rents up, challenging some of the characteristics that enabled it.)

(I first learned of the architect Kelvin Campbell‘s concept of “massive/small” just over two years ago – the idea that certain characteristics of policy and the physical environment in cities could encourage “massive amounts of small-scale innovation” to occur. Kelvin recently launched a collaborative campaign to capture ideas, tools and tactics for massive/small “Smart Urbanism“. This is my first contribution to that campaign.)

Over the past 5 years, enormous interest has developed in the potential for digital technologies to contribute to the construction and development of cities, and to the operation of the services and infrastructures that support them. These ideas are often referred to as “Smart Cities” or “Future Cities”.

Indeed, as the price of digital technologies such as smartphones, sensors, analytics, open source software and cloud platforms reduces rapidly, market dynamics will drive their aggressive adoption to make construction, infrastructure and city services more efficient, and hence make their providers more competitive.

But those market dynamics do not guarantee that we will get everything we want for the future of our cities: efficiency and resilience are not the same as health, happiness and opportunity for every citizen.

Is it realistic to ask ourselves whether we can achieve those objectives? Yes, it has to be.

Many of us believe in that possibility, and spend a lot of our efforts finding ways to achieve it. And over the same timeframe that interest in “smart” and “future” cities has emerged, a belief has developed around the world that the governance institutions of cities – local authorities and elected mayors, rather than the governments of nations – are the most likely political entities to implement the policies that lead to a sustainable, resilient future with more equitably distributed economic growth.

Consequently many Mayors and City Councils are considering or implementing legislation and policy frameworks that change the economic and financial context in which construction, infrastructure and city services are deployed and operated. The British Standards Institute recently published guidance on this topic as part of its overall Smart Cities Standards programme.

But whilst in principle these trends and ideas are incredibly exciting in their potential to create better cities, communities, places and lives in the future, in practise many debates about applying them falter on a destructive and misleading argument between “top-down” and “bottom-up” approaches – the same chasm that Smart Urbanism seeks to bridge in the physical world.

Policies and programmes driven by central government organisations or implemented by technology and infrastructure corporations that drive digital technology into large-scale infrastructures and public services are often criticised as crude, “top-down” initiatives that prioritise resilience and efficiency at the expense of the concerns and values of ordinary people, businesses and communities. However, the organic, “bottom-up” innovation that critics of these initatives champion as the better, alternative approach is ineffective at creating equality.

("Lives on the Line" by James Cheshire at UCL's Centre for Advanced Spatial Analysis, showing the variation in life expectancy and correlation to child poverty in London. From Cheshire, J. 2012. Lives on the Line: Mapping Life Expectancy Along the London Tube Network. Environment and Planning A. 44 (7). Doi: 10.1068/a45341)

(“Lives on the Line” by James Cheshire at UCL’s Centre for Advanced Spatial Analysis, showing the variation in life expectancy and correlation to child poverty in London. From Cheshire, J. 2012. Lives on the Line: Mapping Life Expectancy Along the London Tube Network. Environment and Planning A. 44 (7). Doi: 10.1068/a45341)

“Bottom-up innovation” is what every person, community and business does every day: using our innate creativity to find ways to use the resources and opportunities available to us to make a better life.

But the degree to which we fail to distribute those resources and opportunities equally is illustrated by the stark variation in life expectancy between the richest and poorest areas of cities in the UK: often this variation is as much as 20 years within a single city.

Just as the “design pattern”, a tool invented by a town planner in the 1970s, Christopher Alexander, is probably the single most influential concept that drove the development of the digital technology we all use today, two recent movements in town planning and urban design – “human scale cities” and “smart urbanism” – offer the analogies that can connect “top-down” technology policies and infrastructure with the factors that affect the success of “bottom-up” creativity to create “massive / small” success: future, digital cities that create “massive amounts of small-scale innovation“.

The tools to achieve this are relatively cheap, and the right policy environment could make it fairly straightforward to augment the business case for efficient, resilient “smart city” infrastructures to ensure that they are deployed. They are the digital equivalents of the physical concepts of Smart Urbanism – the use of open grid structures for spatial layouts, and the provision of basic infrastructure components such as street layouts and party walls in areas expected to attract high growth in informal housing. Some will be delivered as a natural consequence of market forces driving technology adoption; but others will only become economically viable when local or national government policies shape the market by requiring them:

  • Broadband, wi-if and 3G / 4G connectivity should be broadly available so that everyone can participate in the digital economy.
  • The data from city services should be made available as Open Data and published through “Application Programming Interfaces” (APIs) so that everybody knows how they work; and can adapt them to their own individual needs.
  • The data and APIs should be made available in the form of Open Standards so that everybody can understand them; and so that the systems that we rely on can work together.
  • The data and APIs should be available to developers working on Cloud Computing platforms with Open Source software so that anyone with a great idea for a new service to offer to people or businesses can get started for free.
  • The technology systems that support the services and infrastructures we rely on should be based on Open Architectures, so that we have freedom to chose which technologies we use, and to change our minds.
  • Governments, institutions, businesses and communities should participate in an open dialogue about the places we live and work in, informed by open data, enabled by social media and smartphones, and enlightened by empathy.

(Casserole Club, a social enterprise developed by FutureGov uses social media to connect people who have difficulty cooking for themselves with others who are happy to cook an extra portion for a neighbour; a great example of a locally-focused “sharing economy” business model which creates financially sustainable social value.)

These principles would encourage good “digital placemaking“: they would help to align the investments that will be made in improving cities using technology with the needs and motivations of the public sector, the private sector, communities and businesses. They would create “Smart Digital Urbanism”: the conditions and environment in which vibrant, fair digital cities grow from the successful innovations of their citizens, communities and businesses in the information economy.

In my new role at Amey, a vast organisation in the UK that delivers public services and operates and supports public infrastructure, I’m leading a set of innovative projects with our customers and technology partners to explore these ideas and to understand how we can collaboratively create economic, social and environmental value for ourselves; for our customers; and for the people, communities and businesses who live in the areas our services support.

It’s a terrifically exciting role; and I’ll soon be hiring a small team of passionate, creative people to help me identify, shape and deliver those projects. I’ll post an update here with details of the skills, experience and characteristics I’m looking for. I hope some of you will find them attractive and get in touch.

From concrete to telepathy: how to build future cities as if people mattered

(An infographic depicting realtime data describing Dublin - the waiting time at road junctions; the location of buses; the number of free parking spaces and bicycles available to hire; and sentiments expressed about the city through social meida)

(An infographic depicting realtime data describing Dublin – the waiting time at road junctions; the location of buses; the number of free parking spaces and bicycles available to hire; and sentiments expressed about the city through social media)

(I was honoured to be asked to speak at TEDxBrum in my home city of Birmingham this weekend. The theme of the event was “DIY” – “the method of building, modifying or repairing something without the aid of experts or professionals”. In other words, how Birmingham’s people, communities and businesses can make their home a better place. This is a rough transcript of my talk).

What might I, a middle-aged, white man paid by a multi-national corporation to be an expert in cities and technology, have to say to Europe’s youngest city, and one of its most ethnically and nationally diverse, about how it should re-create itself “without the aid of experts or professionals”?

Perhaps I could try to claim that I can offer the perspective of one of the world’s earliest “digital natives”. In 1980, at the age of ten, my father bought me one of the world’s first personal computers, a Tandy TRS 80, and taught me how to programme it using “machine code“.

But about two years ago, whilst walking through London to give a talk at a networking event, I was reminded of just how much the world has changed since my childhood.

I found myself walking along Wardour St. in Soho, just off Oxford St., and past a small alley called St. Anne’s Court which brought back tremendous memories for me. In the 1980s I spent all of the money I earned washing pots in a local restaurant in Winchester to travel by train to London every weekend and visit a small shop in a basement in St. Anne’s Court.

I’ve told this story in conference speeches a few times now, perhaps to a total audience of a couple of thousand people. Only once has someone been able to answer the question:

“What was the significance of St. Anne’s Court to the music scene in the UK in the 1980s?”

Here’s the answer:

Shades Records, the shop in the basement, was the only place in the UK that sold the most extreme (and inventive) forms of “thrash metal” and “death metal“, which at the time were emerging from the ashes of punk and the “New Wave of British Heavy Metal” in the late 1970s.

G157 Richard with his Tandy

(Programming my Tandy TRS 80 in Z80 machine code nearly 35 years ago)

The process by which bands like VOIVOD, Coroner and Celtic Frost – who at the time were three 17-year-olds who practised in an old military bunker outside Zurich – managed to connect – without the internet – to the very few people around the world like me who were willing to pay money for their music feels like ancient history now. It was a world of hand-printed “fanzines”, and demo tapes painstakingly copied one at a time, ordered by mail from classified adverts in magazines like Kerrang!

Our world has been utterly transformed in the relatively short time between then and now by the phenomenal ease with which we can exchange information through the internet and social media.

The real digital natives, though, are not even those people who grew up with the internet and social media as part of their everyday world (though those people are surely about to change the world as they enter employment).

They are the very young children like my 6-year-old son, who taught himself at the age of two to use an iPad to access the information that interested him (admittedly, in the form of Thomas the Tank Engine stories on YouTube) before anyone else taught him to read or write, and who can now use programming tools like MIT’s Scratch to control computers vastly more powerful than the one I used as a child.

Their expectations of the world, and of cities like Birmingham, will be like no-one who has ever lived before.

And their ability to use technology will be matched by the phenomenal variety of data available to them to manipulate. As everything from our cars to our boilers to our fridges to our clothing is integrated with connected, digital technology, the “Internet of Things“, in which everything is connected to the internet, is emerging. As a consequence our world, and our cities, are full of data.

(The programme I helped my 6-year old son write using MIT's "Scratch" language to draw a picture of a house)

(The programme I helped my 6-year old son write using MIT’s “Scratch” language to cause a cartoon cat to draw a picture of a house)

My friend the architect Tim Stonor calls the images that we are now able to create, such as the one at the start of this article, “data porn”. The image shows data about Dublin from the Dublinked information sharing partnership: the waiting time at road junctions; the location of buses; the number of free parking spaces and bicycles available to hire; and sentiments expressed about the city through social media.

Tim’s point is that we should concentrate not on creating pretty visualisations; but on the difference we can make to cities by using this data. Through Open Data portals, social media applications, and in many other ways, it unlocks secrets about cities and communities:

  • Who are the 17 year-olds creating today’s most weird and experimental music? (Probably by collaborating digitally from three different bedroom studios on three different continents)
  • Where is the healthiest walking route to school?
  • Is there a local company nearby selling wonderful, oven-ready curries made from local recipes and fresh ingredients?
  • If I set off for work now, will a traffic jam develop to block my way before I get there?

From Dublin to Montpellier to Madrid and around the world my colleagues are helping cities to build 21st-Century infrastructures that harness this data. As technology advances, every road, electricity substation, University building, and supermarket supply chain will exploit it. The business case is easy: we can use data to find ways to operate city services, supply chains and infrastructure more efficiently, and in a way that’s less wasteful of resources and more resilient in the face of a changing climate.

Top-down thinking is not enough

But to what extent will this enormous investment in technology help the people who live and work in cities, and those who visit them, to benefit from the Information Economy that digital technology  and data is creating?

This is a vital question. The ability of digital technology to optimise and automate tasks that were once carried out by people is removing jobs that we have relied on for decades. In order for our society to be based upon a fair and productive economy, we all need to be able to benefit from the new opportunities to work and be successful that are being created by digital technology.

(Photo of Masshouse Circus, Birmingham, a concrete urban expressway that strangled the citycentre before its redevelopment in 2003, by Birmingham City Council)

(Photo of Masshouse Circus, Birmingham, a concrete urban expressway that strangled the city centre before its redevelopment in 2003, by Birmingham City Council)

Too often in the last century, we got this wrong. We used the technologies of the age – concrete, lifts, industrial machinery and cars – to build infrastructures and industries that supported our mass needs for housing, transport, employment and goods; but that literally cut through and isolated the communities that create urban life.

If we make the same mistake by thinking only about digital technology in terms of its ability to create efficiencies, then as citizens, as communities, as small businesses we won’t fully benefit from it.

In contrast, one of the authors of Birmingham’s Big City Plan, the architect Kelvin Campbell, created the concept of “massive / small“. He asked: what are the characteristics of public policy and city infrastructure that create open, adaptable cities for everyone and that thereby give rise to “massive” amounts of “small-scale” innovation?

In order to build 21st Century cities that provide the benefits of digital technology to everyone we need to find the design principles that enable the same “massive / small” innovation to emerge in the Information Economy, in order that we can all use the simple, often free, tools available to us to create our own opportunities.

There are examples we can learn from. Almere in Holland use analytics technology to plan and predict the future development of the city; but they also engage in dialogue with their citizens about the future the city wants. Montpellier in France use digital data to measure the performance of public services; but they also engage online with their citizens in a dialogue about those services and the outcomes they are trying to achieve. The Dutch Water Authority are implementing technology to monitor, automate and optimise an infrastructure on which many cities depend; but making much of the data openly available to communities, businesses, researchers and innovators to explore.

There are many issues of policy, culture, design and technology that we need to get right for this to happen, but the main objectives are clear:

  • The data from city services should be made available as Open Data and through published “Application Programming Interfaces” (APIs) so that everybody knows how they work; and can adapt them to their own individual needs.
  • The data and APIs should be made available in the form of Open Standards so that everybody can understand it; and so that the systems that we rely on can work together.
  • The data and APIs should be available to developers working on Cloud Computing platforms with Open Source software so that anyone with a great idea for a new service to offer to people or businesses can get started for free.
  • The technology systems that support the services and infrastructures we rely on should be based on Open Architectures, so that we have freedom to chose which technologies we use, and to change our minds.
  • Governments, institutions, businesses and communities should participate in an open dialogue, informed by data and enlightened by empathy, about the places we live and work in.

If local authorities and national government create planning policies, procurement practises and legislation that require that public infrastructure, property development and city services provide this openness and accessibility, then the money spent on city infrastructure and services will create cities that are open and adaptable to everyone in a digital age.

Bottom-up innovation is not enough, either

(Coders at work at the Birmingham “Smart Hack”, photographed by Sebastian Lenton)

Not everyone has access to the technology and skills to use this data, of course. But some of the people who do will create the services that others need.

I took part in my first “hackathon” in Birmingham two years ago. A group of people spent a weekend together in 2012 asking themselves: in what way should Birmingham be better? And what can we do about it? Over two days, they wrote an app, “Second Helping”, that connected information about leftover food in the professional kitchens of restaurants and catering services, to soup kitchens that give food to people who don’t have enough.

Second Helping was a great idea; but how do you turn a great idea and an app into a change in the way that food is used in a city?

Hackathons and “civic apps” are great examples of the “bottom-up” creativity that all of us use to create value – innovating with the resources around us to make a better life, run a better business, or live in a stronger community. But “bottom-up” on it’s own isn’t enough.

The result of “bottom-up” innovation at the moment is that life expectancy in the poorest parts of Birmingham is more than 10 years shorter than it is in the richest parts. In London and Glasgow, it’s more than 20 years shorter.

If you’re born in the wrong place, you’re likely to die 10 years younger than someone else born in a different part of the same city. This shocking situation arises from many, complex issues; but one conclusion that it is easy to draw is that the opportunity to innovate successfully is not the same for everyone.

So how do we increase everybody’s chances of success? We need to create the policies, institutions, culture and behaviours that join up the top-down thinking that tends to control the allocation of resources and investment, especially for infrastructure, with the needs of bottom-up innovators everywhere.

Translational co-operation

Harborne Food School

(The Harborne Food School, which will open in the New Year to offer training and events in local and sustainable food)

The Economist magazine reminded us of the importance of those questions in a recent article describing the enormous investments made in public institutions such as schools, libraries and infrastructure in the past in order to distribute the benefits of the Industrial Revolution to society at large rather than concentrate them on behalf of business owners and the professional classes.

But the institutions of the past, such as the schools which to a large degree educated the population for repetitive careers in labour-intensive factories, won’t work for us today. Our world is more complicated and requires a greater degree of localised creativity to be successful. We need institutions that are able to engage with and understand individuals; and that make their resources openly available so that each of us can use them in the way that makes most sense to us. Some public services are starting to respond to this challenge, through the “Open Public Services” agenda; and the provision of Open Data and APIs by public services and infrastructure are part of the response too.

But as Andrew Zolli describes in “Resilience: why things bounce back“, there are both institutional and cultural barriers to engagement and collaboration between city institutions and localised innovation. Zolli describes the change-makers who overcome those barriers as “translational leaders” – people with the ability to engage with both small-scale, informal innovation in communities and large-scale, formal institutions with resources.

We’re trying to apply that “translational” thinking in Birmingham through the Smart City Alliance, a collaboration between 20 city institutions, businesses and innovators. The idea is to enable conversations about challenges and opportunities in the city, between people, communities, innovators and  the organisations who have resources, from the City Council and public institutions to businesses, entrepreneurs and social enterprises. We try to put people and organisations with challenges or good ideas in touch with other people or organisations with the ability to help them.

This is how we join the “top-down” resources, policies and programmes of city institutions and big companies with the “bottom-up” innovation that creates value in local situations. A lot of the time it’s about listening to people we wouldn’t normally meet.

Partly as a consequence, we’ve continued to explore the ideas about local food that were first raised at the hackathon. Two years later, the Harborne Food School is close to opening as a social enterprise in a redeveloped building on Harborne High Street that had fallen out of use.

The school will be teaching courses that help caterers provide food from sustainable sources, that teach people how to set up and run food businesses, and that help people to adopt diets that prevent or help to manage conditions such as diabetes. The idea has changed since the “Second Helping” app was written, of course; but the spirit of innovation and local value is the same.

Cities that work like magic

So what does all this have to do with telepathy?

The innovations and changes caused by the internet over the last two decades have accelerated as it has made information easier and easier to access and exchange through the advent of technologies such as broadband, mobile devices and social media. But the usefulness of all of those technologies is limited by the tools required to control them – keyboards, mice and touchscreens.

Before long, we won’t need those tools at all.

Three years ago, scientists at the University of Berkely used computers attached to an MRI scanner to recreate moving images from the magnetic field created by the brain of a person inside the scanner watching a film on a pair of goggles. And last year, scientists at the University of Washington used similar technology to allow one of them to move the other’s arm simply by thinking about it. A less sensitive mind-reading technology is already available as a headset from Emotiv, which my colleagues in IBM’s Emerging Technologies team have used to help a paralysed person communicate by thinking directional instructions to a computer.

Telepathy is now technology, and this is just one example of the way that the boundary between our minds, bodies and digital information will disappear over the next decade. As a consequence, our cities and lives will change in ways we’ve never imagined, and some of those changes will happen surprisingly quickly.

I can’t predict what Birmingham will or should be like in the future. As a citizen, I’ll be one of the million or so people who decide that future through our choices and actions. But I can say that the technologies available to us today are the most incredible DIY tools for creating that future that we’ve ever had access to. And relatively quickly technologies like bio-technology, 3D printing and brain/computer interfaces will put even more power in our hands.

As a parent, I get engaged in my son’s exploration of these technologies and help him be digitally aware, creative and responsible. Whenever I can, I help schools, Universities, small businesses or community initiatives to use them, because I might be helping one of IBM’s best future employees or business partners; or just because they’re exciting and worth helping. And as an employee, I try to help my company take decisions that are good for our long term business because they are good for the society that the business operates in.

We can take for granted that all of us, whatever we do, will encounter more and more incredible technologies as time passes. By remembering these very simple things, and remembering them in the hundreds of choices I make every day, I hope that I’ll be using them to play my part in building a better Birmingham, and better cities and communities everywhere.

(Shades Records in St. Anne's Court in the 1980s)

(Shades Records in St. Anne’s Court in the 1980s. You can read about the role it played in the development of the UK’s music culture – and in the lives of its customers – in this article from Thrash Hits;  or this one from Every Record Tells a Story. And if you really want to find out what it was all about, try watching Celtic Frost or VOIVOD in the 1980s!)

11 reasons computers can’t understand or solve our problems without human judgement

(Photo by Matt Gidley)

(Photo by Matt Gidley)

Why data is uncertain, cities are not programmable, and the world is not “algorithmic”.

Many people are not convinced that the Smart Cities movement will result in the use of technology to make places, communities and businesses in cities better. Outside their consumer enjoyment of smartphones, social media and online entertainment – to the degree that they have access to them – they don’t believe that technology or the companies that sell it will improve their lives.

The technology industry itself contributes significantly to this lack of trust. Too often we overstate the benefits of technology, or play down its limitations and the challenges involved in using it well.

Most recently, the idea that traditional processes of government should be replaced by “algorithmic regulation” – the comparison of the outcomes of public systems to desired objectives through the measurement of data, and the automatic adjustment of those systems by algorithms in order to achieve them – has been proposed by Tim O’Reilly and other prominent technologists.

These approaches work in many mechanical and engineering systems – the autopilots that fly planes or the anti-lock braking systems that we rely on to stop our cars. But should we extend them into human realms – how we educate our children or how we rehabilitate convicted criminals?

It’s clearly important to ask whether it would be desirable for our society to adopt such approaches. That is a complex debate, but my personal view is that in most cases the incredible technologies available to us today – and which I write about frequently on this blog – should not be used to take automatic decisions about such issues. They are usually more valuable when they are used to improve the information and insight available to human decision-makers – whether they are politicians, public workers or individual citizens – who are then in a better position to exercise good judgement.

More fundamentally, though, I want to challenge whether “algorithmic regulation” or any other highly deterministic approach to human issues is even possible. Quite simply, it is not.

It is true that our ability to collect, analyse and interpret data about the world has advanced to an astonishing degree in recent years. However, that ability is far from perfect, and strongly established scientific and philosophical principles tell us that it is impossible to definitively measure human outcomes from underlying data in physical or computing systems; and that it is impossible to create algorithmic rules that exactly predict them.

Sometimes automated systems succeed despite these limitations – anti-lock braking technology has become nearly ubiquitous because it is more effective than most human drivers at slowing down cars in a controlled way. But in other cases they create such great uncertainties that we must build in safeguards to account for the very real possibility that insights drawn from data are wrong. I do this every time I leave my home with a small umbrella packed in my bag despite the fact that weather forecasts created using enormous amounts of computing power predict a sunny day.

(No matter how sophisticated computer models of cities become, there are fundamental reasons why they will always be simplifications of reality. It is only by understanding those constraints that we can understand which insights from computer models are valuable, and which may be misleading. Image of Sim City by haljackey)

We can only understand where an “algorithmic” approach can be trusted; where it needs safeguards; and where it is wholly inadequate by understanding these limitations. Some of them are practical, and limited only by the sensitivity of today’s sensors and the power of today’s computers. But others are fundamental laws of physics and limitations of logical systems.

When technology companies assert that Smart Cities can create “autonomous, intelligently functioning IT systems that will have perfect knowledge of users’ habits” (as London School of Economics Professor Adam Greenfield rightly criticised in his book “Against the Smart City”), they are ignoring these challenges.

A recent blog article published by the highly influential magazine Wired recently made similar overstatements: “The Universe is Programmable” argues that we should extend the concept of an “Application Programming Interface (API)” – a facility usually offered by technology systems to allow external computer programmes to control or interact with them – to every aspect of the world, including our own biology.

To compare complex, unpredictable, emergent biological and social systems to the very logical, deterministic world of computer software is at best a dramatic oversimplification. The systems that comprise the human body range from the armies of symbiotic microbes that help us digest food in our stomachs to the consequences of using corn syrup to sweeten food to the cultural pressure associated with “size 0” celebrities. Many of those systems can’t be well modelled in their own right, let alone deterministically related to each other; let alone formally represented in an accurate, detailed way by technology systems (or even in mathematics).

We should regret and avoid the hubris that leads to the distrust of technology by overstating its capability and failing to recognise its challenges and limitations. That distrust is a barrier that prevents us from achieving the very real benefits that data and technology can bring, and that have been convincingly demonstrated in the past. For example, an enormous contribution to our knowledge of how to treat and prevent disease was made by John Snow who used data to analyse outbreaks of cholera in London in the 19th century. Snow used a map to correlate cases of cholera to the location of communal water pipes, leading to the insight that water-borne germs were responsible for spreading the disease. We wash our hands to prevent diseases spreading through germs in part because of what we would now call the “geospatial data analysis” performed by John Snow.

Many of the insights that we seek from analytic and smart city systems are human in nature, not physical or mathematical – for example identifying when and where to apply social care interventions in order to reduce the occurrence of  emotional domestic abuse. Such questions are complex and uncertain: what is “emotional domestic abuse?” Is it abuse inflicted by a live-in boyfriend, or by an estranged husband who lives separately but makes threatening telephone calls? Does it consist of physical violence or bullying? And what is “bullying”?


(John Snow’s map of cholera outbreaks in 19th century London)

We attempt to create structured, quantitative data about complex human and social issues by using approximations and categorisations; by tolerating ranges and uncertainties in numeric measurements; by making subjective judgements; and by looking for patterns and clusters across different categories of data. Whilst these techniques can be very powerful, just how difficult it is to be sure what these conventions and interpretations should be is illustrated by the controversies that regularly arise around “who knew what, when?” whenever there is a high profile failure in social care or any other public service.

These challenges are not limited to “high level” social, economic and biological systems. In fact, they extend throughout the worlds of physics and chemistry into the basic nature of matter and the universe. They fundamentally limit the degree to which we can measure the world, and our ability to draw insight from that information.

By being aware of these limitations we are able to design systems and practises to use data and technology effectively. We know more about the weather through modelling it using scientific and mathematical algorithms in computers than we would without those techniques; but we don’t expect those forecasts to be entirely accurate. Similarly, supermarkets can use data about past purchases to make sufficiently accurate predictions about future spending patterns to boost their profits, without needing to predict exactly what each individual customer will buy.

We underestimate the limitations and flaws of these approaches at our peril. Whilst Tim O’Reilly cites several automated financial systems as good examples of “algorithmic regulation”, the financial crash of 2008 showed the terrible consequences of the thoroughly inadequate risk management systems used by the world’s financial institutions compared to the complexity of the system that they sought to profit from. The few institutions that realised that market conditions had changed and that their models for risk management were no longer valid relied instead on the expertise of their staff, and avoided the worst affects. Others continued to rely on models that had started to produce increasingly misleading guidance, leading to the recession that we are only now emerging from six years later, and that has damaged countless lives around the world.

Every day in their work, scientists, engineers and statisticians draw conclusions from data and analytics, but they temper those conclusions with an awareness of their limitations and any uncertainties inherent in them. By taking and communicating such a balanced and informed approach to applying similar techniques in cities, we will create more trust in these technologies than by overstating their capabilities.

What follows is a description of some of the scientific, philosophical and practical issues that lead inevitability to uncertainty in data, and to limitations in our ability to draw conclusions from it:

But I’ll finish with an explanation of why we can still draw great value from data and analytics if we are aware of those issues and take them properly into account.

Three reasons why we can’t measure data perfectly

(How Heisenberg’s Uncertainty Principle results from the dual wave/particle nature of matter. Explanation by HyperPhysics at Georgia State University)

1. Heisenberg’s Uncertainty Principle and the fundamental impossibility of knowing everything about anything

Heisenberg’s Uncertainty Principle is a cornerstone of Quantum Mechanics, which, along with General Relativity, is one of the two most fundamental theories scientists use to understand our world. It defines a limit to the precision with which certain pairs of properties of the basic particles which make up the world – such as protons, neutrons and electrons – can be known at the same time. For instance, the more accurately we measure the position of such particles, the more uncertain their speed and direction of movement become.

The explanation of the Uncertainty Principle is subtle, and lies in the strange fact that very small “particles” such as electrons and neutrons also behave like “waves”; and that “waves” like beams of light also behave like very small “particles” called “photons“. But we can use an analogy to understand it.

In order to measure something, we have to interact with it. In everyday life, we do this by using our eyes to measure lightwaves that are created by lightbulbs or the sun and that then reflect off objects in the world around us.

But when we shine light on an object, what we are actually doing is showering it with billions of photons, and observing the way that they scatter. When the object is quite large – a car, a person, or a football – the photons are so small in comparison that they bounce off without affecting it. But when the object is very small – such as an atom – the photons colliding with it are large enough to knock it out of its original position. In other words, measuring the current position of an object involves a collision which causes it to move in a random way.

This analogy isn’t exact; but it conveys the general idea. (For a full explanation, see the figure and link above). Most of the time, we don’t notice the effects of Heisenberg’s Uncertainty Principle because it applies at extremely small scales. But it is perhaps the most fundamental law that asserts that “perfect knowledge” is simply impossible; and it illustrates a wider point that any form of measurement or observation in general affects what is measured or observed. Sometimes the effects are negligible,  but often they are not – if we observe workers in a time and motion study, for example, we need to be careful to understand the effect our presence and observations have on their behaviour.

2. Accuracy, precision, noise, uncertainty and error: why measurements are never fully reliable

Outside the world of Quantum Mechanics, there are more practical issues that limit the accuracy of all measurements and data.

(A measurement of the electrical properties of a superconducting device from my PhD thesis. Theoretically, the behaviour should appear as a smooth, wavy line; but the experimental measurement is affected by noise and interference that cause the signal to become "fuzzy". In this case, the effects of noise and interference - the degree to which the signal appears "fuzzy" - are relatively small relative to the strength of the signal, and the device is usable)

(A measurement of the electrical properties of a superconducting device from my PhD thesis. Theoretically, the behaviour should appear as a smooth, wavy line; but the experimental measurement is affected by noise and interference that cause the signal to become “fuzzy”. In this case, the effects of noise and interference – the degree to which the signal appears “fuzzy” – are relatively small compared to the strength of the signal, and the device is usable)

We live in a “warm” world – roughly 300 degrees Celsius above what scientists call “absolute zero“, the coldest temperature possible. What we experience as warmth is in fact movement: the atoms from which we and our world are made “jiggle about” – they move randomly. When we touch a hot object and feel pain it is because this movement is too violent to bear – it’s like being pricked by billions of tiny pins.

This random movement creates “noise” in every physical system, like the static we hear in analogue radio stations or on poor quality telephone connections.

We also live in a busy world, and this activity leads to other sources of noise. All electronic equipment creates electrical and magnetic fields that spread beyond the equipment itself, and in turn affect other equipment – we can hear this as a buzzing noise when we leave smartphones near radios.

Generally speaking, all measurements are affected by random noise created by heat, vibrations or electrical interference; are limited by the precision and accuracy of the measuring devices we use; and are affected by inconsistencies and errors that arise because it is always impossible to completely separate the measurement we want to make from all other environmental factors.

Scientists, engineers and statisticians are familiar with these challenges, and use techniques developed over the course of more than a century to determine and describe the degree to which they can trust and rely on the measurements they make. They do not claim “perfect knowledge” of anything; on the contrary, they are diligent in describing the unavoidable uncertainty that is inherent in their work.

3. The limitations of measuring the natural world using digital systems

One of the techniques we’ve adopted over the last half century to overcome the effects of noise and to make information easier to process is to convert “analogue” information about the real world (information that varies smoothly) into digital information – i.e. information that is expressed as sequences of zeros and ones in computer systems.

(When analogue signals are amplified, so is the noise that they contain. Digital signals are interpreted using thresholds: above an upper threshold, the signal means “1”, whilst below a lower threshold, the signal means “0”. A long string of “0”s and “1”s can be used to encode the same information as contained in analogue waves. By making the difference between the thresholds large compared to the level of signal noise, digital signals can be recreated to remove noise. Further explanation and image by Science Aid)

This process involves a trade-off between the accuracy with which analogue information is measured and described, and the length of the string of digits required to do so – and hence the amount of computer storage and processing power needed.

This trade-off can be clearly seen in the difference in quality between an internet video viewed on a smartphone over a 3G connection and one viewed on a high definition television using a cable network. Neither video will be affected by the static noise that affects weak analogue television signals, but the limited bandwidth of a 3G connection dramatically limits the clarity and resolution of the image transmitted.

The Nyquist–Shannon sampling theorem defines this trade-off and the limit to the quality that can be achieved in storing and processing digital information created from analogue sources. It determines the quality of digital data that we are able to create about any real-world system – from weather patterns to the location of moving objects to the fidelity of sound and video recordings. As computers and communications networks continue to grow more powerful, the quality of digital information will improve,  but it will never be a perfect representation of the real world.

Three limits to our ability to analyse data and draw insights from it

1. Gödel’s Incompleteness Theorem and the inconsistency of algorithms

Kurt Gödel’s Incompleteness Theorem sets a limit on what can be achieved by any “closed logical system”. Examples of “closed logical systems” include computer programming languages, any system for creating algorithms – and mathematics itself.

We use “closed logical systems” whenever we create insights and conclusions by combining and extrapolating from basic data and facts. This is how all reporting, calculating, business intelligence, “analytics” and “big data” technologies work.

Gödel’s Incompleteness Theorem proves that any closed logical system can be used to create conclusions that  it is not possible to show are true or false using the same system. In other words, whilst computer systems can produce extremely useful information, we cannot rely on them to prove that that information is completely accurate and valid. We have to do that ourselves.

Gödel’s theorem doesn’t stop computer algorithms that have been verified by humans using the scientific method from working; but it does mean that we can’t rely on computers to both generate algorithms and guarantee their validity.

2. The behaviour of many real-world systems can’t be reduced analytically to simple rules

Many systems in the real-world are complex: they cannot be described by simple rules that predict their behaviour based on measurements of their initial conditions.

A simple example is the “three body problem“. Imagine a sun, a planet and a moon all orbiting each other. The movement of these three objects is governed by the force of gravity, which can be described by relatively simple mathematical equations. However, even with just three objects involved, it is not possible to use these equations to directly predict their long-term behaviour – whether they will continue to orbit each other indefinitely, or will eventually collide with each other, or spin off into the distance.

(A computer simulation by Hawk Express of a Belousov–Zhabotinsky reaction,  in which reactions between liquid chemicals create oscillating patterns of colour. The simulation is carried out using “cellular automata” a technique based on a grid of squares which can take different colours. In each “turn” of the simulation, like a turn in a board game, the colour of each square is changed using simple rules based on the colours of adjacent squares. Such simulations have been used to reproduce a variety of real-world phenomena)

As Stephen Wolfram argued in his controversial book “A New Kind of Science” in 2002, we need to take a different approach to understanding such complex systems. Rather than using mathematics and logic to analyse them, we need to simulate them, often using computers to create models of the elements from which complex systems are composed, and the interactions between them. By running simulations based on a large number of starting points and comparing the results to real-world observations, insights into the behaviour of the real-world system can be derived. This is how weather forecasts are created, for example. 

But as we all know, weather forecasts are not always accurate. Simulations are approximations to real-world systems, and their accuracy is restricted by the degree to which digital data can be used to represent a non-digital world. For this reason, conclusions and predictions drawn from simulations are usually “average” or “probable” outcomes for the system as a whole, not precise predictions of the behaviour of the system or any individual element of it. This is why weather forecasts are often wrong; and why they predict likely levels of rain and windspeed rather than the shape and movement of individual clouds.


(A simple and famous example of a computer programme that never stops running because it calls itself. The output continually varies by printing out characters based on random number generation. Image by Prosthetic Knowledge)

3. Some problems can’t be solved by computing machines

If I consider a simple question such as “how many letters are in the word ‘calculation’?”, I can easily convince myself that a computer programme could be written to answer the question; and that it would find the answer within a relatively short amount of time. But some problems are much harder to solve, or can’t even be solved at all.

For example, a “Wang Tile” (see image below) is a square tile formed from four triangles of different colours. Imagine that you have bought a set of tiles of various colour combinations in order to tile a wall in a kitchen or bathroom. Given the set of tiles that you have bought, is it possible to tile your wall so that triangles of the same colour line up to each other, forming a pattern of “Wang Tile” squares?

In 1966 Robert Berger proved that no algorithm exists that can answer that question. There is no way to solve the problem – or to determine how long it will take to solve the problem – without actually solving it. You just have to try to tile the room and find out the hard way.

One of the most famous examples of this type of problem is the “halting problem” in computer science. Some computer programmes finish executing their commands relatively quickly. Others can run indefinitely if they contain a “loop” instruction that never ends. For others which contain complex sequences of loops and calls from one section of code to another, it may be very hard to tell whether the programme finishes quickly, or takes a long time to complete, or never finishes its execution at all.

Alan Turing, one of the most important figures in the development of computing, proved in 1936 that a general algorithm to determine whether or not any computer programme finishes its execution does not exist. In other words, whilst there are many useful computer programmes in the world, there are also problems that computer programmes simply cannot solve.

(A set of Wang Tiles, and a pattern created by tiling them so that tiles are placed next to other tiles so that their edges have the same colour. Given any particular set of tiles, it is impossible to determine whether such a pattern can be created by any means other than trial and error)

(A set of Wang Tiles, and a pattern of coloured squares created by tiling them. Given any random set of tiles of different colour combinations, there is no set of rules that can be relied on to determine whether a valid pattern of coloured squares can be created from them. Sometimes, you have to find out by trial and error. Images from Wikipedia)

Five reasons why the human world is messy, unpredictable, and can’t be perfectly described using data and logic

1. Our actions create disorder

The 2nd Law of Thermodynamics is a good candidate for the most fundamental law of science. It states that as time progresses, the universe becomes more disorganised. It guarantees that ultimately – in billions of years – the Universe will die as all of the energy and activity within it dissipates.

An everyday practical consequence of this law is that every time we act to create value – building a shed, using a car to get from one place to another, cooking a meal – our actions eventually cause a greater amount of disorder to be created as a consequence – as noise, pollution, waste heat or landfill refuse.

For example, if I spend a day building a shed, then to create that order and value from raw materials, I consume structured food and turn it into sewage. Or if I use an electric forklift to stack a pile of boxes, I use electricity that has been created by burning structured coal into smog and ash.

So it is literally impossible to create a “perfect world”. Whenever we act to make a part of the world more ordered, we create disorder elsewhere. And ultimately – thankfully, long after you and I are dead – disorder is all that will be left.

2. The failure of Logical Atomism: why the human world can’t be perfectly described using data and logic

In the 20th Century two of the most famous and accomplished philosophers in history, Bertrand Russell and Ludwig Wittgenstein, invented “Logical Atomism“, a theory that the entire world could be described by using “atomic facts” – independent and irreducible pieces of knowledge – combined with logic.

But despite 40 years of work, these two supremely intelligent people could not get their theory to work: “Logical Atomism” failed. It is not possible to describe our world in that way.

One cause of the failure was the insurmountable difficulty of identifying truly independent, irreducible atomic facts. “The box is red” and “the circle is blue”, for example, aren’t independent or irreducible facts for many reasons. “Red” and “blue” are two conventions of human language used to describe the perceptions created when electro-magnetic waves of different frequencies arrive at our retinas. In other words, they depend on and relate to each other through a number of sophisticated systems.

Despite centuries of scientific and philosophical effort, we do not have a complete understanding of how to describe our world at its most basic level. As physicists have explored the world at smaller and smaller scales, Quantum Mechanics has emerged as the most fundamental theory for describing it – it is the closest we have come to finding the “irreducible facts” that Russell and Wittgenstein were looking for. But whilst the mathematical equations of Quantum Mechanics predict the outcomes of experiments very well, after nearly a century, physicists still don’t really agree about what those equations mean. And as we have already seen, Heisenberg’s Uncertainty Principle prevents us from ever having perfect knowledge of the world at this level.

Perhaps the most important failure of logical atomism, though, was that it proved impossible to use logical rules to turn “facts” at one level of abstraction – for example, “blood cells carry oxygen”, “nerves conduct electricity”, “muscle fibres contract” – into facts at another level of abstraction – such as “physical assault is a crime”. The human world and the things that we care about can’t be described using logical combinations of “atomic facts”. For example, how would you define the set of all possible uses of a screwdriver, from prising the lids off paint tins to causing a short-circuit by jamming it into a switchboard?

Our world is messy, subjective and opportunistic. It defies universal categorisation and logical analysis.

(A Pescheria in Bari, Puglia, where a fish-market price information service makes it easier for local fisherman to identify the best buyers and prices for their daily catch. Photo by Vito Palmi)

3. The importance and inaccessibility of “local knowledge” 

Because the tool we use for calculating and agreeing value when we exchange goods and services is money, economics is the discipline that is often used to understand the large-scale behaviour of society. We often quantify the “growth” of society using economic measures, for example.

But this approach is notorious for overlooking social and environmental characteristics such as health, happiness and sustainability. Alternatives exist, such as the Social Progress Index, or the measurement framework adopted by the United Nations 2014 Human Development Report on world poverty; but they are still high level and abstract.

Such approaches struggle to explain localised variations, and in particular cannot predict the behaviours or outcomes of individual people with any accuracy. This “local knowledge problem” is caused by the fact that a great deal of the information that determines individual actions is personal and local, and not measurable at a distance – the experienced eye of the fruit buyer assessing not just the quality of the fruit but the quality of the farm and farmers that produce it, as a measure of the likely consistency of supply; the emotional attachments that cause us to favour one brand over another; or the degree of community ties between local businesses that influence their propensity to trade with each other.

Sharing economy” business models that use social media and reputation systems to enable suppliers and consumers of goods and services to find each other and transact online are opening up this local knowledge to some degree. Local food networks, freecycling networks, and land-sharing schemes all use this technology to the benefit of local communities whilst potentially making information about detailed transactions more widely available. And to some degree, the human knowledge that influences how transactions take place can be encoded in “expert systems” which allow computer systems to codify the quantitative and heuristic rules by which people take decisions.

But these technologies are only used in a subset of the interactions that take place between people and businesses across the world, and it is unlikely that they’ll become ubiquitous in the foreseeable future (or that we would want them to become so). Will we ever reach the point where prospective house-buyers delegate decisions about where to live to computer programmes operating in online marketplaces rather than by visiting places and imagining themselves living there? Will we somehow automate the process of testing the freshness of fish by observing the clarity of their eyes and the freshness of their smell before buying them to cook and eat?

In many cases, while technology may play a role introducing potential buyers and sellers of goods and services to each other, it will not replace – or predict – the human behaviours involved in the transaction itself.

(Medway Youth Trust use predictive and textual analytics to draw insight into their work helping vulnerable children. They use technology to inform expert case workers, not to take decisions on their behalf.)

4. “Wicked problems” cannot be described using data and logic

Despite all of the challenges associated with problems in mathematics and the physical sciences, it is nevertheless relatively straightforward to frame and then attempt to solve problems in those domains; and to determine whether the resulting solutions are valid.

As the failure of Logical Atomism showed, though, problems in the human domain are much more difficult to describe in any systematic, complete and precise way – a challenge known as the “frame problem” in artificial intelligence. This is particularly true of “wicked problems” – challenges such as social mobility or vulnerable families that are multi-faceted, and consist of a variety of interdependent issues.

Take job creation, for example. Is that best accomplished through creating employment in taxpayer-funded public sector organisations? Or by allowing private-sector wealth to grow, creating employment through “trickle-down” effects? Or by maximising overall consumer spending power as suggested by “middle-out” economics? All of these ideas are described not using the language of mathematics or other formal logical systems, but using natural human language which is subjective and inconsistent in use.

The failure of Logical Atomism to fully represent such concepts in formal logical systems through which truth and falsehood can be determined with certainty emphasises what we all understand intuitively: there is no single “right” answer to many human problems, and no single “right” action in many human situations.

(An electricity bill containing information provided by OPower comparing one household’s energy usage to their neighbours. Image from Grist)

5. Behavioural economics and the caprice of human behaviour

Behavioural economics” attempts to predict the way that humans behave when taking choices that have a measurable impact on them – for example, whether to put the washing machine on at 5pm when electricity is expensive, or at 11pm when it is cheap.

But predicting human behaviour is notoriously unreliable.

For example, in a smart water-meter project in Dubuque, Iowa, households that were told how their water conservation compared to that of their near neighbours were found to be twice as likely to take action to improve their efficiency as those who were only told the details of their own water use. In other words, people who were given quantified evidence that they were less responsible water user than their neighbours changed their behaviour. OPower have used similar techniques to help US households save 1.9 terawatt hours of power simply by including a report based on data from smart meters in a printed letter sent with customers’ electricity bills.

These are impressive achievements; but they are not always repeatable. A recycling scheme in the UK that adopted a similar approach found instead that it lowered recycling rates across the community: households who learned that they were putting more effort into recycling than their neighbours asked themselves “if my neighbours aren’t contributing to this initiative, then why should I?”

Low carbon engineering technologies like electric vehicles have clearly defined environmental benefits and clearly defined costs. But most Smart Cities solutions are less straightforward. They are complex socio-technical systems whose outcomes are emergent. Our ability to predict their performance and impact will certainly improve as more are deployed and analysed, and as University researchers, politicians, journalists and the public assess them. But we will never predict individual actions using these techniques, only the average statistical behaviour of groups of people. This can be seen from OPower’s own comparison of their predicted energy savings against those actually achieved – the predictions are good, but the actual behaviour of OPower’s customers shows a high degree of apparently random variation. Those variations are the result of the subjective, unpredictable and sometimes irrational behaviour of real people.

We can take insight from Behavioural Economics and other techniques for analysing human behaviour in order to create appropriate strategies, policies and environments that encourage the right outcomes in cities; but none of them can be relied on to give definitive solutions to any individual person or situation. They can inform decision-making, but are always associated with some degree of uncertainty. In some cases, the uncertainty will be so small as to be negligible, and the predictions can be treated as deterministic rules for achieving the desired outcome. But in many cases, the uncertainty will be so great that predictions can only be treated as general indications of what might happen; whilst individual actions and outcomes will vary greatly.

(Of course it is impossible to predict individual criminal actions as portrayed in the film “Minority Report”. But is is very possible to analyse past patterns of criminal activity, compare them to related data such as weather and social events, and predict the likelihood of crimes of certain types occurring in certain areas. Cities such as Memphis and Chicago have used these insights to achieve significant reductions in crime)

Learning to value insight without certainty

Mathematics and digital technology are incredibly powerful; but they will never perfectly and completely describe and predict our world in human terms. In many cases, our focus for using them should not be on automation: it should be on the enablement of human judgement through better availability and communication of information. And in particular, we should concentrate on communicating accurately the meaning of information in the context of its limitations and uncertainties.

There are exceptions where we automate systems because of a combination of a low-level of uncertainty in data and a large advantage in acting autonomously on it. For example, anti-lock braking systems save lives by using automated technology to take thousands of decisions more quickly than most humans would realise that even a single decision needed to be made; and do so based on data with an extremely low degree of uncertainty.

But the most exciting opportunity for us all is to learn to become sophisticated users of information that is uncertain. The results of textual analysis of sentiment towards products and brands expressed in social media are far from certain; but they are still of great value. Similar technology can extract insights from medical research papers, case notes in social care systems, maintenance logs of machinery and many other sources. Those insights will rarely be certain; but properly assessed by people with good judgement they can still be immensely valuable.

This is a much better way to understand the value of technology than ideas like “perfect knowledge” and “algorithmic regulation”. And it is much more likely that people will trust the benefits that we claim new technologies can bring if we are open about their limitations. People won’t use technologies that they don’t trust; and they won’t invest their money in them or vote for politicians who say they’ll spend their taxes on it.

Thankyou to Richard Brown and Adrian McEwen for discussions on Twitter that helped me to prepare this article. A more in-depth discussion of some of the scientific and philosophical issues I’ve described, and an exploration of the nature of human intelligence and its non-deterministic characteristics, can be found in the excellent paper “Answering Descartes: Beyond Turing” by Stuart Kauffman published by MIT press.


Get every new post delivered to your Inbox.

Join 5,136 other followers

%d bloggers like this: