From carbon to silicon: Rohan Murty

Software creation is a creative endeavour that almost always requires imagination, patience and careful thought. Hence, until about the late ’70s, several computer scientists believed that to successfully build a large software system, the entire team of software writers must be geographically co-located, be constantly in touch with each other and coordinate on software development activities.

The early ’80s shattered this artisanal view of software creation. The development of the global delivery model, one of modern India’s great inventions in the technology world, changed the way software was built. It distributed the art of writing software across large teams of people around the globe, running relays to build complex software systems. Teams across the US, Europe, India and Japan transcended geographies and time zones to create a 24-hour working day. This required fundamental innovations in software engineering as software creation had to become less of an art form and more of a process. Its goals and steps needed to be clearly defined and broken down into smaller tasks, which could then be allocated to teams across the globe.
 
This, in essence, was an example of industrialising software creation. This has been India’s contribution to the software world and has been its technological story for the better part of the last 30 years. It also explains the birth and sustenance of the Indian software industry and has changed how global corporations operate.

Once built, all software systems must also constantly be maintained and upgraded as they must not only evolve with new paradigms in computing, but, being human creations, also suffer from flaws and bugs. This has afforded the Indian software industry a significant revenue generation opportunity: Our companies not only build bespoke software systems for global corporations, but also maintain, test, patch and upgrade them. With the advent of networks and the internet, software systems grew in complexity, spanning multiple machines across multiple geographies. Consequently, the Indian software industry found a gold mine in large contracts for managing these complex systems and networks.

Essentially, these revenue-earning streams were all by-products of building software systems for clients. Thus, the last 10 years have witnessed a large number of people hired to perform these tasks, which are often fairly deterministic—work that follows a set of pre-determined rules that can be written down on paper and followed faithfully to produce a desired outcome. But the Indian software industry has not yet done enough to move to the next step in the evolution of software systems. Hence, what started off as an innovation, and a lucrative business of software creation, has evolved into, arguably, a more lucrative business of managing human effort at scale. And this is where the Indian software industry has an opportunity to innovate further and change its future.

To put matters in perspective, let us consider the history of manufacturing. A hundred years ago, a typical assembly line for a car company consisted of scores of humans building cars. The early carmakers were artisans and each car was handmade, the proportions were slightly different from unit to unit, even in the same models.

As demand for cars increased, carmakers had to innovate to mass produce cars, reduce costs and introduce variety. Eventually, a series of innovations in manufacturing—which included interchangeable parts, enhancing the assembly line, division of labour, rigorous processes, decomposing work into tasks that are discrete and measurable, a focus on quality and cost control and mechanisation—gradually improved the productivity of these assembly lines. Today, the artisanal effort in car manufacturing remains in designing cars, layout of the plants, choosing the materials, etc. However, the act of producing a car has been ‘industrialised’: The same assembly line has been totally transformed with robots replacing repetitive human tasks.

Is there any limit to what human effort can be industrialised? Consider one of mankind’s oldest activities—stargazing. Professor Shrinivas Kulkarni, the John D MacArthur professor of astrophysics at the California Institute of Technology, leads the Palomar Transient Factory (PTF) project. Kulkarni’s team has built software to completely automate the act of discovering through a telescope. The large telescopes at Mt. Palomar in California constantly scan the sky, looking for patterns, and use an array of sophisticated machine learning and digital signal processing algorithms to crunch the observational data in real-time and ascertain whether a new discovery has been made. As a result, the cycle time for a single discovery in astronomy has now been reduced from two years in the early ’80s, to a few hours currently. This story should startle us. This transformation is a prime example of how sophisticated software has replaced complex human activities.

Or, consider a lesson in the history of computing itself. Over 400 years ago, the term computer was a job title, not a machine. Computers were often humans hired to perform calculations using pencil and paper. In the 19th century, Charles Babbage, often referred to as the father of computing, even applied for such a job.

In 1913, Harvard astronomer Charles Pickering hired a group of ‘computers’—mostly housewives—who were taught to execute fairly deterministic steps for processing observational data. These women, collectively called the ‘Harvard Computers’, played a crucial role in the development of modern astronomy. Pickering’s use of human effort to crunch astronomical data had huge ramifications on how science and engineering took place. Subsequently, during World War II, the US government hired large numbers of women computers to crunch navigational, ballistic and radar data. Back then, computers were entirely carbon-based life forms (read humans).

Now, compare that with what we see today in the Indian software industry. Despite several advances in computing, we have scores of humans working on tasks that are almost entirely deterministic and by-products of software construction. Clearly, the work done by human computers of the past is highly deterministic and hence it appears very primitive to us today. Will the work done by humans in the Indian software industry today look as primitive 20 years from now?

In many ways, the elimination of redundant mental work is the history of computing itself. In 1821, even Babbage grew frustrated with redundant mental labour (read human computers) and wondered instead if he could “compute with steam”. The software services industry of today does not appear to have learnt from the history of computing, manufacturing or science.

In Babbage’s days, steam was the motive power of commerce. For us, in the 21st century, it is silicon. Technology will inevitably obviate the need for the carbon-based labour of today and push us into the Silicon Age when software will replace large classes of human tasks. While history presents a mirror, the current reality presents an opportunity to change the future.

I do not prognosticate that all creative human endeavours will be dead or that software creation itself can be performed entirely by software. Well, at least not yet. But, there are several deterministic tasks for which the software industry still relies on volumes of human capital. People still believe these tasks require human effort, when, in reality, technology will do just fine. While we often confuse ‘complex’ human tasks with those that involve several steps, this is of no consequence to a machine. 

A first-order litmus test for whether software can eventually perform a human task is this: Does the task require a human to solve a problem? If the answer is no, then the task is nearly-deterministic and can be performed by software, replacing humans.  

A startling corollary is that if the first-order test is false and yet humans are very productive in performing some tasks then those tasks are even more amenable to being replaced with software. High human productivity by itself is no defence against automation. Perhaps, the very reason why humans are efficient at these tasks today is because they have broken them down into a series of finite and deterministic steps. Upon introspection, many tasks in our lives fall into the realm of the nearly-deterministic, unless, of course, one is an artist, judge or from a similar profession that requires creative ideation all the time.

This is important because fundamentally, most businesses consume and reshape themselves with technology as they are concerned about productivity. And that is why replacing human effort with software is the logical next step in the evolution of the Indian software industry.

The global delivery model has become synonymous with the age of the (human) carbon footprint, where a large number of well-trained humans performed tasks at lower costs. The next step is to take this model to its logical conclusion: Move these tasks from humans to machines and dramatically improve speed, correctness, transparency and cost of work. This evolution will require more than empty hype about ‘artificial intelligence’, ‘cognitive machines’, ‘neuroscience’ etc. (These are, after all, bleeding-edge areas of research about which we do not yet understand enough.) Instead, it will require an organisation whose raison d’être is the elimination of deterministic human effort in the enterprise.

How might such an organisation be different from what we see today?
•    It will require deeper use of new ideas from computer science that includes machine learning, distributed systems, programming languages, software engineering and statistics.
•    It will require a new industrialised process that defines and measures human productivity in the development and maintenance of software.
•    It will have to seamlessly integrate intelligent software into a human workforce since not all work maybe immediately replaceable with software.
•    It will require a re-examination of the business models and pricing strategies of today’s software services companies.

Furthermore, while the cost of humans is inflationary, the cost of intelligent software, by virtue of Moore’s Law, is deflationary. Hence, the cost to the client will have to come down constantly and significantly over time. For technology vendors, it entails more pressure on sales quotas. As software systems continue to cannibalise human tasks, we will need to find more meaningful tasks for the human workforce in the software industry. This has ramifications on hiring, salaries, promotion cycles and HR practices. Therefore our software services business model must be re-tooled.

This leads to perhaps the most difficult cultural change of all: To stay relevant, the remaining human labour force must constantly adapt and learn technologies that are not yet cannibalised by machines, putting more emphasis on the education of our professionals. Without re-education, we may end up with a labour force that is no longer in tune with the latest trends in technology.

Thus far, India has disrupted how software was built and delivered. But now, there is an opportunity to create a disruption in the very business model that made this industry successful. This is an opportunity for imaginative entrepreneurs who are not shackled by the vestiges of the past. It will result in innovations and changes in technology and traditional business models.

They will no longer rely on a large carbon (human) footprint to serve global customers. The Age of Carbon is fast fading. The Age of Silicon is here.