ezyang's research log

What I did today. High volume, low context. Sometimes nonsense. (Archive)

Haskell: Haskell and GHC: Too Big to Fail? (panel)

Talk about the future of Haskell. “FP has won”

Agenda: Where Haskell is at; and talk about the environment

  • SPJ: I am one of the 36 people with commit bits
  • BOS: I have directly committed one change
  • ST: That’s one more than me, I’ve written books and taught in Haskell
  • BH: I’ve worked on compilers, Eden compiler; aimed at … Haskell
  • GS: Scheme, lambda calculus, tail calls
  • AM: I helped Don Stewart port it to VAX. I do work in OCaml. Next Haskell should be strict, next OCaml should be pure. Want to figure out if we want to go where you are, or other way.

Let’s talk about strengths/weaknesses of Haskell

  • SPJ: Haskell’s real strenth is it served as a laboratory for developing ideas. Somehow, that’s stayed true. Several years ago, we declared GHC was full; and since then we’ve doubled the number of features. Yet, it’s been combined with increase in real world usage. That’s very valuable. The weakness is I don’t know how fragile this combination is. This tension between people who want to use it and those who want to change everything; maybe this should be a threat: I percieve this as OK so far, but I don’t know how close to the breaking point we are.
  • BOS: The landscape has improved for bringing Haskell to the masses; libraries and tools have improved; and use of Haskell in production is better. Large community of people who don’t understand these things: to make a case, there are people who will listen, especially if you motivate things. Build interesting software.
  • BH: A strength and weakness is GHC. You could argue Haskell is becoming the same as GHC. The language is growing, lots of extensions, makes it hard to understand what is going on. The third dimension of the threat is whether or not Haskell will stop being a language for education.
  • GS: The rigid adherence to purity is a huge strength. Laziness forced purity, leading to discoveries that didn’t happen in Scheme or ML, e.g. monads. I spent nine years on Fortress, a scientific programming language (the project is finished now, we decided to stop working on it). Focus was on parallelism; offhand comment, “If I’d known then, I would have dragged Haskell to Fortran, not the other way around.” Please think about exploring a version of Haskell that abandons laziness or deemphasizes it; Haskell is exploring more needs of multicore, where most of the time you say “that is strict.”
  • AM: In five talks, it had production problems with laziness. But they all loved purity and compositionality. Haskell has so much work going on. But I love going home to OCaml, with focus on a few problem domains; the consortium has stayed stable of 14 members, but it’s billion dollars of systems. Mission critical: Amazon, countries in some cases. Mission critical! OCaml is not looking for broad success like Haskell. So think about where you want to succeed.
  • SPJ: Just to be clear; we don’t have random language extensions.
  • AM: I remember Don’s post about genetic testing of optimization flags. I’ll never use that.
  • Edward Kmett: Don did an hour long podcast just last night; he spoke opposite point on laziness. At Standard Chartered, they had algorithms which were too strict, computing large models, where getting a piece would force an enormous report. Newer programmers had made too many things strict.
  • GS: That’s why I would not get rid of laziness entirely. Performance versus space leaks. If we had a better cost model that would be better.
  • C: I don’t count myself as a Haskell programmer, but Bob observed ML lives as a language, rather than a compiler. Does Haskell live as a language? Haskell development seems to be GHC. We had a talk today about Intel, where they are using GHC as the frontend. UHC was sharing libraries with GHC… but what is Haskell as a language.
  • BOS: Not to a very great extent. This is a matter of happenstance. The different Haskell compilers have ended up drying into the desert sands. Haskell2010 is only slightly larger than Haskell98. So if someone says “I’ll spend 5 years building a Haskell compiler”, it can do it
  • Lennart: We have more than a million lines of Haskell that is not compiled with GHC. So Haskell lives.
  • ST: We built a tool on top of Haskell98 refactoring; people said it’s wonderful, but what about extension X/Y/… For tool-builders, you have to support GHC. This was in 2002; GHC is in a better place now, but building persistent tools, that is a problem. Lennart, you build tools, but you have your own infrastructure.
  • AM: IIRC, you have a Strict version of Haskell. How is that?
  • Lennart: Lazy Haskell composes much nicer than strict Haskell. For composability, you need to be lazy. But strictness composes better for resource consumption; semantics, lazy. I have no answer.
  • SPJ: I can comment. PLs have an enormous lock-in effect. One reason is because compilers that compile realistic languages/have an ecosystem are large engineering achievements which are hard to replicate. And there are all the users; that’s also difficult to build. In principle, this is a bad thing, but in practice this is impossible to counter. I’ve ceased to worry about this. Instead of that, just make the compiler and the ecosystem as open as possible. Hence GHC API, plugins, open source, encourage people. So don’t worry about a monolithic thing; make it as componentized as possible, to reduce barrier of entry. It’s not MS’s compiler, it’s your compiler. That’s easier than beating our breasts about it. The same would be true of a language description.
  • Doaitse Swiestra: I’ve been teaching Haskell. The slide says some math things are hard, but these are not the problem. But inside Haskell, there’s another language; a halfway Prolog interpreter with lots of extensions which say few people know how to get amazing things done; I’m convinced in the end, this is not the way we want to do them. So I think this is going the wrong way; take things away from the current state of depenetly typed languages; this is not teachable or transferable. Libraries depend on these extensions; it’s not clear if it’s coherent or not. That’s the real threat.
  • SPJ: The thing about laboratories, is you test things to destruction. So if we find that the phenomena happens, then that encourages people to come up with better ideas, because it will stand out in high relief.
  • C: That is definitely the laboratory, but that is incompatible with having production libraries. Libraries will depend on experimental things.
  • Didier Remy: You said something about Haskell, keep innovation in language, but keep users happy. I would be interested in knowing, in OCaml we stopped making new research in the language. What in your opinion helped keep the research; why do you see a weakness?
  • SPJ: That’s hard. One thing I’ve clung onto is Guy’s cling to a few core principles and change everything else. Laziness and a strong typed intermediate language has stayed stable. Why is it been compatible with people being able to use it? I don’t know! One component, a strength of Haskell, a really supportive Haskell community; helping people in. But maybe have different experiences.
  • C: We use Haskell commercially. We just don’t use the froth on top. We can still compile Haskell code from 10 years ago. So this core has been very stable.
  • ST: Is that Haskell98?
  • C: Maybe a few extensions.
  • ST: Haskell2010 was trying to identify those few extensions that were out there. But as people chased it, it was difficult to work out what they were.
  • C: I volunteer to scan our code and see what it is.
  • SPJ: A deliberate goal is to try hard not to gratuitously change things that already work.
  • Ryan Newton: As someone new, GHC 7.8, that we’re not very conservative. The burden of proof is not excessive. That lets it keep changing.
  • C: I’m a Racketeer, I’m interested in language evolution. Specification follows what works. People had success in all the extensions of GHC, so are there plans for Haskell spec to allow extensibility as part of the language, like macros?
  • SPJ: I’m very envious of Racket macros, because it’s very extensible. But I don’t know how to do it for Haskell. TH is the nearest, but it’s nowhere near.
  • C: I’m not talking about macros, but extensibility mechanisms. If you extend the type system, is there a general interface for extending type checker, standardizable across different implementations.
  • SPJ: No, but that would be great. In OutsideIn(X), we described parametrized checker X, but we never made that real.
  • BOS: Generic answer “is the right plan for X”, the answer is, if someone executes, we can say, “Yes”
  • Duncan Coutts: Are we going to decide between type families and functional dependencies?
  • SPJ: Probably not. Type families are really easy to do, really convenient for some things, but they are awkward in some cases. And fundeps have some defenders, and GHC says, if something has defenders, we’ll keep it. So you have something big, with slightly overlapping features, we have to leave with it. If you want a small finely crafted jewel, design something new.
  • ST: The thing that comes to mind, is we should have some subsets. With my Colin (indistinct) hat on, “We need some small identified subset”
  • BOS: The tension between TFs and FDs (SPJ: unnamed feature!) speaking with my industry hat on, if you use either one in your API, you lose 80% of the people who can keep up with them, people don’t use them, so people produce APIs that we are capable of using. (laughter)
  • Oleg Kiselyov: Iavor could disagree, but it seems fundeps are on the way out, but overlapping instances still help. Also when you have … it does not compose. So you just refactor with TFs and the use goes down. This is unfortunate because FDs is sometimes nicer, but I think the fight is over. About features of Haskell, I use Haskell for production, since 2006, runs every day and hour; the feature I use is Haskell98 + pattern guards + existential types, nothing more. I compiled 7.6, the pain is libraries change, some functions move, but with a few hours of work it compiles.
  • Gershom Bazerman: I want to ask, Haskell Prime has stalled for the third stall in the row. Is it OK it’s stalled? Is it fine? Is it on the backbruner?
  • SPJ: I think that’s a question for people on the room. You’re right it’s stalled, I think that’s an indication the community doesn’t care enough.
  • Edward Kmett: Is it stalled because of community, or just because Ian moved on.
  • SPJ: No, nothing to do with that
  • AM: We ran a fifteen year experiment ocamlp4; syntax transformer, completely insane. We tried to be data driven about extensions, trying to figure out what ocamlp4 packages are used. 80% proposed extensions were dropped. do-notation, threading, not used. 1-2 packages are used by everyone; like type-generated IO, dynamic types, we take this data, and integrate back to language. Data-driven approach. Not Ian, who sounds overworked. Use package manager as hypothesis tool. This seems more sustainable.
  • BOS: That’s what we converged on for Haskell Prime, but that well got tapped and no one did it
  • SPJ: Patch to GHC, send a UDP packet saying what language extensions are being used, saying FDs are big in Oregon, etc. We can get data that way. But it’s kind of snoopy.

OK, lets conclude the inward discussion. So let’s look outwards. Haskell is disruptive, it attracts brightest/motivated programmers, what about the rest? Maybe he was being unflattering to the rest, but is Haskell a language for the rest of them? How can I get all my students? Not just the top 15%. Is the terminology really forbidding? Monads? But they’re made more acute by FP looks like it’s winning.

  • GS: What would it take to have Haskell be language of choice for mobile applications
  • C: Small runtime
  • C: We have three Haskell-JS compilers, enumeral DSLs/EDSLs, so you don’t need to have your entire team understand Haskell. Just a small fragment to cope with basic FP, or some sort of smaller subset. It’s perfect. The purity is perfect for doing things high-level.
  • Carter Schonwald: Mobile devices: we didn’t have a GHC iOS until this summer, which people could use. For mobile app, people are working on it, people are hobby hacking. For industry, it’s similar to asking someone, “We have some people who do engineering, how do you make them become engineers”? It’s an acculturation problem. As long as we are aggressively friendly. “I’m confused right now, help” is helpful.
  • Getting back to attracting pointy headed people, I have a theory: it’s pessimistic, which is that peopel come to Haskell because it’s different/weird/hard. Pointy-headed people like puzzle things, writing pure programs. In an imperative language, throw enough state and it will solve itself. (laughter) In Haskell you have to think hard, it’s a lot of fun, but it’s something that is hard, and can be hard for many programmers out there.
  • BOS: The obvious way to deal with it being hard is you just lie. This is very successful! The rise of the node.js ecosystem, they’ve been telling themselves of . They’ve been introducing . We imported monads specifically to avoid writing continuations. They’ve been in this happy state of denial, and they’ve acquired accolades as a result. So, suppose you’re going to base your publicity around a giant lie, is that a world you want? I don’t want to be part of node.js because they are all bonkers. Be careful what you wish for. XXX
  • AM: Speaking of lies, look at the hardware community. It’s terrible. All of Haskell’s mistakes in software are great in hardware, e.g. Bluespec. So if everyone grabs hardware, it’s transformative. We teach Bluespec in Cambridge, not Verilog. It’s not as crowded, and you make a real difference. It’s untrodden, where Haskell has unique benefits. Just go in weird and wonderful. There are very few good hardware toolkits.
  • To respond to Joseph, in order to learn a complicated language, you have to layer complexity. GHC has gone the right way, it lets you run programs that don’t typecheck. So let them run the program, see the type error. So when they see the problems, and there is a system that prevents that, keep layering on. There just needs to be a curriculum. Sriram has this idea of progressive typing.
  • ST: Dr. Scheme did that as well. A small number of language features with very good error diagnostics; with larger language power, more incomprehensible.
  • SPJ I bet your pardon. (that’s one of the error messages?)
  • John Hughes: Is FP hard? Well, many people have IP background and not FP. How do we know if it’s hard in absolute terms? 30% of my class failed. Then of them took a Java course, and 30% failed too. So it’s not clear Haskell is failed. It’s hard if our education system does not prepare people. (applause)
  • Jonathan Fischoff: Datapoint about Haskell being hard. In my company, all 70 engineers are going to have to learn Haskell. I’ve been watching PHP developers learn Haskell. They’re doing a great job. It’s not been that huge of a barrier, because itw as for me when learning on my own, but having people who know Haskell immediately help them, they’re not struggling. But something I do think is a problem is how we sell Haskell. I have been trying to convince people to use Haskell by talking about correctness, it doesn’t work. So they just showed one demo of how much Haskell is faster and scales up, and that was it. So I think we have a simpler problem.
  • Aaron Contorer: Is Lebovitz in the room? Greg knows the only person whose job is to answer this question. What have we learned at FP Complete? I have a lot of data. Companies are not worried about learning a new language, syntax, type system, “it’s another language”. The issue is it integrates very poorly with everything else they have. That is the reason, my data tells me, they are not using Haskell. Haskell is a gem in itself, but it lives in its own universe. Companies say we’ve invested millions of dollars on Java runtime, .NET, dataflows with metadata and schema systems. “We love the performance, the scalability, what the correctness can do, fast prototyping”, they love Haskell when they love it, but then they say they can’t, because it has nothing to do with anything people. So we need to figure out how to got companies which are not Haskell shops. So how do we combine Haskell with gigantic preexisting environments. Schemas, code, interfaces, devops; if we can do that, then Haskell is already twice as good to impress.
  • SPJ: To what extent is that a library question?
  • AC: It’s the number one issue. Number two, it’s devops, like logging. Then the third thing is FFI with managed langauges. Someone says “there’s no way to access a database with a schema that I developed.”
  • Manuel Chakravarty: Similarly, I teach Haskell to students every semester. I get problem reports. I rarely get a question about type errors, I never get a question about laziness. Maybe something about type classes. I get questions all about Cabal gives me these strange errors, I can’t install this library, it doesn’t work on Windows. We had a conference on practitioners, and again we hear the same questions. The language is perfectly fine, it’s just the infrastructure around it.
  • ST: How does that get changed? One problem is we can write papers about types, but not devops. Academic development is driven by papers. What is the mechanism for devops?
  • BOS: A modest number of us don’t come from academia, and are here for the drinking. And those of us in that world have the capability, and have written code to help. Bread and butter stuff. It’s straightforward, roll up your sleeves, but you have to be motivated by a need.
  • C: Two comments, is Haskell inherently hard? And teaching. I don’t think Haskell is inherently hard. It’s acculturation and training. For example, diff eq is hard, but people think it’s essential. But we haven’t convinced our colleagues that lambda calculus is important. Layering helps for captivated/motivated, but I don’t have that. I don’t have the luxury of 75 people who are mandatory. Colleagues are skeptical, and so if you do a layered approach, it is in tension with “let me show you amazing things you can do with this language.” This is for Bluespec, but I bet it’s true for Haskell too.
  • Hughes: Another story about teaching, about parallel FP. That’s been great, 50 students, but we were using bleeding edge stuff. And we found that 25% of students used some form of Linux, and they were happy, 50% used a Mac, nightmare, 25% used Windows, and they had a nightmare. Please, test your libraries on those three platforms. This is a real problem when you can’t install them on the platforms people are using.

Extract one thing from the panel.

  • AM: It sounds like putting resources into Cabal is your biggest missed opportunity. So build it up to a fully automated system, tests, etc. In OCaml, I can spin up 200 machines in parallel. Automate everything, and you can focus on the interesting problems, and I’ll buy you drink. Also, packaging, I’ve been a BSD developer, packages we usually take releases and package them. Now it’s not practical. So automate that for metadata is important. Automate all of packaging/metadata, this is really boring. I can’t tell you how bad Makefiles are.
  • GS: Abstraction is powerful but hard, and making ita ccessible is worthy.
  • BH: Laziness is here to stay.
  • BOS: Subvert your curriculum committee
  • SPJ: An opportunity: libraries/packaging is the biggest practical impediment. Opportunity; this is a pain point, may have some bandwidth, but each individual thinks they can’t move the mountain. So get together and move it. 5500 libraries! HP/Stackage/opportunity. Cycles are there.
blog comments powered by Disqus