Trust, Enterprise Security, and Autonomous Technology

The technology writer Langdon Winner wrote an interesting book 30 years ago that has a lot of relevance to technologists today--especially when thinking about enterprise security. His core idea is one of technological autonomy. As the good folks at Cyborgology define it:

Technological autonomy is a shorthand way of expressing the idea that our technologies and technological systems have become so ubiquitous, so intertwined, and so powerful that they are no longer in our control. This autonomy is due to the accumulated force of the technologies themselves and also to our utter dependence on them.  ...Advanced technologies require vast networks of supportive technologies in order to properly function. Our cars wouldn’t go far without roads, gasoline, traffic control systems, and the like. Electricity needs power lines, generators, distributors, light bulbs, and lamps, together with production, distribution, and administrative systems to put all those elements (profitably) into place. A “chain of reciprocal dependency” is established, Winner says, that requires “not only the means but also the entire set of means to the means.”

Winner is not necessarily arguing that technology is autonomous in the sense of Skynet and Terminator. He is, however, pointing out that technology is not simply a tool animated by human will. Each successive layer of technology, in turn, creates a complex dependence through the supporting networks necessary to underpin it. Thus we cannot evaluate technology in isolation. Rather, we ought to think of techno-assemblages, mutually reinforcing systems of systems.

The experience of the modern user is by definition one of trust in incredibly complex systems that he or she cannot hope to completely master or have control over. Instead, we accept a limited understanding of expert systems and trust in the ability of the collected wisdom of experts (and when I say collective, I mean a combination since expertise is specialized in nature) that the systems we use will work as planned. The philosopher Anthony Giddens writes of this, for example, when talking about cars:

Everyone knows that driving a car is a dangerous activity, entailing the risk of accident. In choosing to go out in the car, I accept that risk, but rely upon the aforesaid expertise to guarantee that it is minimised as possible. […] When I park the car at the airport and board a plane, I enter other expert systems, of which my own technical knowledge is at best rudimentary.

I would argue that one of the major problems with enterprise security--and to some extent information security as a whole--lies precisely in the factors that both Giddens and Winner discuss. Information technology and the systems that underpin it are, in a sense, autonomous in the way Winner suggests. Cyber is ultimately an inescapable aspect of everyday life, making cybersecurity less of an exotic thing than it was when books like Black Ice were written. As more and more appliances become networked, we start entering into the world where the information user not only can't trust their toaster, but also becomes paranoid about people hacking into their cars. Moreover, the knowledge necessary to understand the sum of these techno-assemblages becomes not simply a problem for individual technologists, but a larger social issue that requires a diversity of expertise.

I think that as a company CrucialPoint itself is actually a very good response to this sort of new reality. My background is in political science and international politics, Dillon Behr is a former soldier, Matt Devost, and Bob Gourley have experience in the cyber security, national security and intelligence communities. Chris Barnes is a former federal CIO. I'm often amazed at the technical skills demonstrated on a consistent basis by Bryan Halfpap and Ryan Kamauff. Some of us have advanced degrees, others have many years of practical experience. Together, we have a mutually reinforcing basis of expertise for thinking about technology in a holistic fashion.

Winner and Giddens' ideas have great relevance for enterprise security. We aren't going to stop people from using various techno-assemblages or individual technologies. Mobile device security and the "death of the PC" are merely symptoms of this larger problem. And the implications associated with these technologies are policy matters for an manager with appropriate authority and perspective to set, not merely a technical domain for individual specialists. They are too complex and encompass way too many dimensions for a narrow perspective.

About AdamElkus

Adam Elkus is a PhD student in Computational Social Science at George Mason University. He writes on national security, technology, and strategy at and the new analysis focused Analyst One, War on the Rocks, and his own blog Rethinking Security. His work has been published in The Atlantic, Journal of Military Operations Foreign Policy, West Point Counterterrorism Center Sentinel, and other publications.



  1. CTOvision Monthly Summary for January says:

    […] predictions of his own, looking at the biggest hacking trends of 2011 to extrapolate future trends. Adam provided a look into the issue of trust as enterprise security grows increasingly autonomous and human beings can no longer monitor all […]