Functional Programming Hurdle: Uninteresting programs
Functional Programming, as a paradigm, does not have users, it has devotees. I am myself one of those, spending time in Haskell as often as I can.
Why then, if it is so great, has FP not taken over the software development world?
There are many answers to this question, ranging from the condescending (“is hard”, “has no practical use”) to the hopeful (“just haven’t reached critical mass”). This post is about another, often overlooked, fundamental reason:
Most programs are mostly about I/O
Corporate programmers, in flavours of start-up to enterprise, mainly spend their days designing software that takes user input, moves it over the network, stores it in a database and even has the ability to fetch it back again. The furthest you get into processing data on a typical day is perhaps report generation. Then, outside of your main project, your build, deployment and test scripts are all about I/O as well.
The interesting tasks, such as rendering, query optimization or data aggregation is all handled by platforms or tools. Which is efficient, since those things can be time-consuming to develop, and you are paid by time units.
While the notion that most programs are mostly about I/O seems obvious to the legions of corporate programmers using Java, PHP and C#, it is not to everyone. Runar Bjarnason gave a talk in New York last week on I/O in FP. The talk itself, while great, is not what caught my interest.
Runar Bjarnason – I want one of those stickers!
Inadvertently, 8 minutes in, he brilliantly sums up the chasm between FP and the real world (for mundane, but very common, variants of the real world) in 32 seconds:
You have some code that interfaces with the outside world, and then you have, like, what your code actually does.
And those things are kind of intermingled.
But in reality, your code is not about I/O. I/O is just the code that retrieves the input to your program, and then takes the output of your program and delivers it somewhere.
And those parts are not really that interesting. Your code is not about that. Your code is about what happens in between.
I have spent years in the corporate world on a multitude of different projects, and quite often not a lot happens in between. While there is a “pure” core of business logic in there somewhere, it is dwarfed by the amount of I/O-related code. Look at a typical mobile app, how many lines of code should have been separated into pure modules?
For this reason, if we want to convey the usefulness of FP to the imperatives amongst us, we need to focus on elegant solutions to real world inputs and outputs. Most software needs to integrate with other software. We need to show how our I/O-libraries, often overlooked or thought of as mere “helpers”, are superior in efficiency and leads to cleaner and more maintainable code. While a pure HashMap is a beautiful thing, let’s focus more of our time on Haskell packages such as postgresql-simple, amqp (of which I am a minor contributor), aeson and the fact that there is no native MSSQL library, just ODBC. Let’s focus on the plumbing, so to say. Great packages for integrating with the outside world is what will bring FP to the masses. When that prerequisite is there, only then are they ready for all the other stuff we have.
And please do not go on and on about the beauty of lens, to the casual passer-by it seems of no other use than to demonstrate that basic functionality they take for granted is somehow missing. If you want to wow someone, instead demonstrate how one line of code adds ekg to your long-running process.
We could be doing our day jobs in Haskell. But most of the FP literature (and a lot of the packages) is focused elsewhere. As it is, one has to excpect or anticipate doing som library work when starting a project in Haskell, I really hope we will get past that soon.
Yes, yes, and oh god, yes.
You’re exactly right, but I/O isn’t really the right term (it opens the door to FP “devotees” saying that Haskell’s ability to “abstract away” I/O is a huge win and then stop listening). It’s really about integration.
The great hoards of developers out the world (hoards which are too big, because they work inefficiently in weak languages, certainly) are mainly involved in writing code that does very simple transformations, but which has to make the right transformations at the right time in the right order consistently (lazyness? Oh dear) on user input from these seven heterogenous, unreliable input channels, validated against reference data from these thirteen heterogenous data sources, wait for that mainframe to update, send an email to this repair queue, roll the response from the user there into the data, send the transformed data out to this random collection of unreliable, heterogenous channels, etc. etc. etc. And the upstream and downstream systems are changing all the time, and the rules about what transforms to do in what order are changing all the time, and so on.
A very few (a lucky few) get to write a program which uses a complicated algorithm to make a sophisticated transformation to some rich data, and banging on about how brilliantly Haskell can solve that problem is not going to impress anyone.
This comment is better than my post. I hope everyone reads it.
Thanks.
Before I became a programmer I studied Physics and Maths, so my sympathies lie with functional programming, and I’ve even had some programming jobs where it would have helped a lot, but many more where elegant implementation of complicated functions is just so not the point. I believe that problems of that nature constitute the bulk of commercial programming.
I’d love for FP to be the next big thing, but as currently formulated it doesn’t seem to address enough of the day–to–day concerns of working programmers in ways which are compellingly better than what they have now (a depressingly low bar) to get any traction. We have at least now moved on from the world Monad tutorials which are much harder to understand than Monads are, but lengthy expositions on lenses aren’t much of an improvement.
But what rankles me most is the all–to–common assertion by FP advocates that the problems I see teams deal with day in and day out in industry either don’t exist, or if they do are irrelevant. Not the way to win friends and influence people.
Keith – yes, absolutely. And don’t forget the two or three specifications of 100 pages each that it all must comply with – supposedly, but not really. Still, a good type system does add a lot for this and all of the other issues you mentioned.
It is funny how history just keeps repeating itself. It took 35+ years for OO to become mainstream. And the funny thing is that what people call OO is not even OO (Google Alan Kay). What people use instead is “let’s have a number of global states instead of only one – Oriented Programming” or “Place Oriented Programming” as the Clojure guy expresses it. Erlang is probably the only truly OO language available today. And the twist is that it is a functional language. How funny that is. Functional programming will absolutely become main stream. But it will take a long time. And so what? It doesn’t stop you or your team or your biz from taking advantage of the technology today. You don’t need to wait for the rest of the herd to catch up. The leading edge guys started using C while the rest of the world were still bashing away in Cobol. I can imagine the Cobol guys back then shaking their heads going “Why use C? No software is written in it. Cobol is what everybody use.” All I can say is don’t wait for the herd. Move ahead and be part of the next “C guys” and don’t stay with the “Cobol guys”. Any thinking/practical tool that makes you more productive today is a good thing 🙂
Here’s a blog post about a company that’s using Haskell and Scala to handle those heterogenous, unreliable data sources, input and output channels: http://janrain.com/blog/functional-programming-social-web
I think you’ve summed up the problem really well to a non-functional programmer.
This is so spot on. People who do their day to day work in language X, but wants to learn/use language Y, first really starts doing so once their day-to-day tasks is simple to do in language Y. And that often starts with a Google search and typing in some code you found on StackOverflow to hook up that Postgres database, or to set up that router for the web server. It does not start with “Learn you language Y for a greater good”, good as that might be.
The stupid, mundane stuff must be easy to do, and there has to be reference code you can easily look to. The amount of easy to use libs and great documentation is what really has made it easy for people to start using Node, as a recent example.