The stealthy rise of functional programming

Haskell isn't about to take over the world, but functional programming is slowly making its presence felt

Programmers can be a tribal bunch. Systems programmers see themselves as a breed apart from UI experts, agile developers sit at a different table from waterfall practitioners and everyone keeps their distance from the security guys.

But there's one tribal divide that goes back to the very beginning of modern computing: that between functional and imperative programming. Professor Simon-Peyton Jones, an advocate of functional programming since the 1980s and a major contributor to the Haskell language, is the first to admit that his tribe is a minority. But rather than any inherent superiority, the dominance of imperative is down to an accident of history, he says.

"After 1936 the Turing machine came to embody what computing is, but at the same time Alonzo Church, his research colleague, described the lambda calculus, which is a completely different way of thinking about the whole enterprise. Anything you can do with the Turing machine you can do with lambda calculus and vice versa, but nevertheless they have a completely different feel. From Turing machines we got imperative programming languages, which everybody is very familiar with, and from lambda calculus you get functional programming."

Very briefly, the main difference between the two models is that in functional programming data is immutable and functions are 'pure' by default, meaning that they will always return the same thing, no matter where they occur in the program. This is not true of imperative where data is generally changeable and function outputs unpredictable. From imperative programming came the languages C, Java and Python, while functional programming gave us Lisp, Haskell and F#. While the former are obviously hugely more prominent, functional programming has recently enjoyed an upsurge in interest.

John Gordon a trainer at QA consulting who formerly worked at consultancy Atos has noted this trend.

"Several projects I worked on in Atos were written in nearly purely functional Scala. They were those roles that needed high throughput on large amounts of data, especially in banking," Gordon says.

Peyton-Jones sees the increasing use of parallel processing, which is often deployed in such circumstances, as a factor favouring functional programming.

"Functional programming languages don't have side effects," he says. "The computational fabric is inherently parallel - if anything you have to do some work to sequentialise it. For the parallel world a language in which side effects like mutating state are exceptional rather than part of the very fabric is a much better starting point."

In functional programming if x is 42 it will always be 42. In imperative programming data is generally mutable, so you can assign a value to a variable and change it later in the program. At one point x might be 42 and a line of code later it might be 42 billion. This can be a problem in a distributed system.

"In a distributed world where different things happen in physically different places you may ask ‘is it safe for me to copy this thing over there?'," explains Peyton-Jones.

"If it's a fixed immutable value then of course you can because it's a value not a location containing a value. So anything where you've got immutable values is much more amenable to remotely caching those values and copying them freely, whereas if you're mutating values in place you've got to go to that one place, which tends to get very hot and involves a lot of communication."

The stealthy rise of functional programming

Haskell isn't about to take over the world, but functional programming is slowly making its presence felt

A related factor favouring functional is the falling price of storage. Rather than overwriting a file it makes sense to store all versions as separate files.

"Storage is now cheap and mutation is expensive, whereas before it was the other way around," he says.

"When you program with immutable values, which is what functional programming is all about, the whole world changes. Instead of overwriting a logfile, I write the new version of the file in another part of the disk and then swing a pointer to it. Then I just have to keep track of what the current state of the world is."

As well as cheap storage and multithreaded processing, functional programming has also been given a boost by the arrival of Spark, the popular in-memory big data analytics platform. Spark is written in Scala, a language with many functional features.

"There seems to be more interest within the big data sphere, with Spark being written in Scala in many cases over Python," says James Thompson, also a trainer at QA.

"Scala is built on Java and it is easier to start with object oriented programming (OOP) and then slowly introduce functional elements.

"True functional programming is highly abstract and can be seen as a bit academic, so the more gradual transition of Java to Scala is highly beneficial to those not from an academic background," he says.

However, Thompson notes that while there has been a "fair amount of experimentation" with functional languages, some people "seem to be drifting back to OOP" as they find it difficult to make the switch.

"It is relatively easy to learn the syntax but very difficult to use the language properly," he says.

Given the difficulty in crossing over from imperative, would it make sense for a beginner to learn a functional language first? In some cases, yes, says QA's Gordon, although it is not common practice.

"Those with a purely maths background would likely do very well learning a purely functional language first," Gordon says, adding: "However, I've never knew anyone to start out learning a functional language,"

The functional programming tribe is unlikely to overthrow the imperative majority, but it may achieve many of its goals by stealth, by becoming an essential tool in the developers' kit. Newer programming languages such as Scala, Go, Julia, Clojure and Rust have many functional attributes, and the latest versions of Java and C++ also offer support for functional constructs.

"It won't be a quick process but I think over the coming years we'll see gradually more and more functional programming expressed through existing languages adopting more functional features," says Peyton-Jones.

"Or maybe there's be a wholesale movement to functional programming. Either way I think it will be increasingly influential."