> I must not segfault. Uncertainty is the mind-killer. Exceptions are the little-death that brings total obliteration. I will fully express my cases. Execution will pass over me and through me. And when it has gone past, I will unwind the stack along its path. Where the cases are handled there will be nothing. Only I will remain.
I have to respect Jane Street for proving that HM-typed functional programming can be production-ready and extremely fast when given enough love.
I still occasionally hear things about how the more academic-styled functional languages can't work in production, but Ocaml shows that it absolutely can work, even with high performance requirements.
I was fortunate enough to work at Jet.com before Walmart completely destroyed it, which was an F# shop. I really liked it, and I never really felt "limited" by it.
The stuff I was working on didn't have nearly the same requirements as high-frequency trading like Jane Street does though. I never did any super low-latency stuff with F#, so it's tough for me to say how well it fair with that kind of environment.
They did an episode of Signals and Threads (Jane Street's technical podcast) about their build systems back in 2020¥. Highly recommended, as are the other episodes, especially The Future of Programming with Richard Eisenberg° and What is an Operating System? with Anil Madhavapeddy§.
The only thing worse than a complex codebase is a complex build system. I think CMake is an abomination. “Hey go learn this new language to compile this other language!”
I don't mind CMake. It's awful, but I'm already using C or C++, so the bar is set pretty low. It does do a few things right, and at least some of the things it does wrong are manageable.
I'll eat at least a bit of shit if it means I can get more than one platform's-worth of build process out of a single set of human-editable configuration files.
The ideas embodied in modern target-oriented CMake are worthwhile. It's all hobbled by a dreadful script language but that could be improved with sufficient motivation.
I wonder if anyone has used Jane Street's open source OCaml libraries and tools for somewhat serious projects.
I took a brief look at these things, and my impression is that their stuff isn't "ready" for anyone outside Jane Street, even though they put a lot of effort in building the ecosystem and open source their code.
kinda surprising that it still makes sense to develop your own new build system, and then do the migration to it, instead of adopting an off-the-shelf one like buck2 (or bazel/etc...). Not saying that these fancy build tools are easy to adopt, but compared to building your own? is there something remarkable that makes dune much different?
Without getting into any specifics of it - I'm sure there's people with much more experience with these tools who can comment - I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
JS does have a bit of a NIH culture, but I'm not sure if that was really at play here. There just...weren't very many good build tools available at the time, particularly for a company using an unorthodox tech stack.
> I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
But Dune started (according to this blog post) in 2016 and JS started seriously improving and adopting it last year. So to me Jenga sounds like a reasonable step in 2012, but pouring significant effort into migrating from Jenga to Dune (and improving Dune) in 2024 sounds more weird
Yes and no. This is all spelled out in the post, but it's a little thorny.
Dune is a rename of Jbuilder (2016). Jbuilder uses Jenga (2012) configuration files.
> By 2016 we had had enough of this, and decided to make a simple cross-platform tool, called Jbuilder, that would allow external users to build our code without having to adopt Jenga in full, and would release us from the obligation of rewriting our builds in OCamlbuild [...] Jbuilder understood the jbuild files that Jenga used for build configuration.
So in 2012 it made sense for them to build Jenga, because there weren't any good alternatives - Bzl etc. didn't exist, so they couldn't have solved their problems.
And in 2016 they had open-source code they wanted others to be able build; those people didn't want to use Jenga, and JS didn't want to rewrite their builds so that they could use something else. Thus, Jbuilder was a shim so that JS could still use their Jenga builds and others could build JS' code without using Jenga. Bzl etc., even though they existed, wouldn't have solved these problems either.
I don't know ocaml particularly well but my understanding is that only with buck2 (of those mentioned) do you have a build system with the semantics to actually compile ocaml properly.
In particular I was under the impression one needed to be able to run ocamldep before hand (or compile twice) - buck2 can do this, bazel needs hacks iirc.
I loath autotools with a passion, and it's awkward to set it all up, but it handles OCaml code just fine. We use it for multi-language projects (including OCaml) like nbdkit & libguestfs.
I respectfully have to disagree. As Fedora packager for OCaml packages, dune's configuration is yet another one-off, with a weird Lisp syntax, hard to understand, difficult to patch. I'd far rather people stuck to using more familiar Linux build systems. As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
(This rant more or less equally applies to other language-specific build systems.)
"Familiar" is not a property of any system. It's a relation between a system and a user.
Some Linux build systems maybe be more familiar to some users, but will be less familiar to others. When picking a build system, you can't just look at the system itself and declare it familiar or not.
It's not even enough to look at the total number of users familiar with some thing. Hindi is one of the most familiar languages in the world, but you're probably gonna have a bad time if you use it for the menu in a cafe in rural Texas.
You have to look at your actual cohort of users (and potential future users) and see what's familiar to them. This is one of the key reasons why usability is actually a deeply hard problem. So much of usability hinges on familiarity, but familiarity is a human-specific highly variable property.
As a user of Linux, I wish Linux distributions would abandon their build systems which really only work well for C and maybe C++, and also only work on one platform (not even the whole of linux!) meaning that packaging has to be duplicated for every OS, and stuck to using more familiar language-specific build systems that work everywhere.
(this rant more or less equally applies to all os-or-distro-specific build systems)
---
This rant is only semi-serious. I do see some value in the Linux distribution style packaging. In particular, I do sympathise with the need to do cross-language builds. But goodness are they a pain to work with, and probably the biggest barrier to me shipping software on Linux.
My hope is that eventually an evolution on build systems like bazel/buck2 will lead to a truly universal build system that is both cross-platform and cross-language. But unfortunately it doesn't look like it's coming soon.
I haven't been doing much OCaml in recent years, but I do sometimes check in to see what's going on in OCaml-land, and it seems like dune is what everyone is using now - what are the alternatives?
> This rant more or less equally applies to other language-specific build systems.
And yet those language-specific build systems are overwhelmingly winning, in pretty much every language.
> As Fedora packager for OCaml packages,...
I honestly think traditional Linux packaging is in the wrong here and the problems are essentially self-inflicted (not in the sense that individual maintainers are doing something wrong, but in the sense that the policy that traditional Linux distributions are following is inherently unsustainable. It's designed for a pre-CPAN world)
> As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
True up to a point, but frankly the worst case is falling back to a terrible C-style build, and "always do terrible C-style builds in case you need to integrate with C code" is not a proposition that has much appeal.
Much as I wish the whole world would standardise on Maven or Cargo, I can't see a realistic path to there without first eliminating C, because the C people are never going to agree to follow a standard for package repositories.
Dune's crash message:
> I must not segfault. Uncertainty is the mind-killer. Exceptions are the little-death that brings total obliteration. I will fully express my cases. Execution will pass over me and through me. And when it has gone past, I will unwind the stack along its path. Where the cases are handled there will be nothing. Only I will remain.
I have to respect Jane Street for proving that HM-typed functional programming can be production-ready and extremely fast when given enough love.
I still occasionally hear things about how the more academic-styled functional languages can't work in production, but Ocaml shows that it absolutely can work, even with high performance requirements.
> HM-typed functional programming
Mercury also uses Haskell for their backend https://mercury.com/
Also very cool!
I was fortunate enough to work at Jet.com before Walmart completely destroyed it, which was an F# shop. I really liked it, and I never really felt "limited" by it.
The stuff I was working on didn't have nearly the same requirements as high-frequency trading like Jane Street does though. I never did any super low-latency stuff with F#, so it's tough for me to say how well it fair with that kind of environment.
Too bad they didn't use their namesake:
https://mercurylang.org/
Would honestly be a lot more interesting than Haskell.
They did an episode of Signals and Threads (Jane Street's technical podcast) about their build systems back in 2020¥. Highly recommended, as are the other episodes, especially The Future of Programming with Richard Eisenberg° and What is an Operating System? with Anil Madhavapeddy§.
¥https://signalsandthreads.com/build-systems/
°https://signalsandthreads.com/future-of-programming/
§https://signalsandthreads.com/what-is-an-operating-system/
The only thing worse than a complex codebase is a complex build system. I think CMake is an abomination. “Hey go learn this new language to compile this other language!”
The only thing worse than a complex build system is a simple build system that can't build what you need to build. :(
This means the architecture is also bad, and that is what needs to be fixed.
> CMake is an abomination. “Hey go learn this new language to compile this other language!”
"go learn this awful new language"
I don't mind CMake. It's awful, but I'm already using C or C++, so the bar is set pretty low. It does do a few things right, and at least some of the things it does wrong are manageable.
I'll eat at least a bit of shit if it means I can get more than one platform's-worth of build process out of a single set of human-editable configuration files.
The ideas embodied in modern target-oriented CMake are worthwhile. It's all hobbled by a dreadful script language but that could be improved with sufficient motivation.
I wonder if anyone has used Jane Street's open source OCaml libraries and tools for somewhat serious projects.
I took a brief look at these things, and my impression is that their stuff isn't "ready" for anyone outside Jane Street, even though they put a lot of effort in building the ecosystem and open source their code.
I tried to use their magic-trace tool but I get at least 4 differents crash when using it on trivial dotnet programs.
I spent several years in Haskell land a while back, my first functional experience after Lisp.
Ocaml never clicked for me, I have a rare form of semicolon allergy and Haskell just looked a lot nicer to me.
But then I recently tried Reason and enjoyed it A LOT, so everything Ocaml is suddenly interesting.
i wish the reason syntax was the main one
kinda surprising that it still makes sense to develop your own new build system, and then do the migration to it, instead of adopting an off-the-shelf one like buck2 (or bazel/etc...). Not saying that these fancy build tools are easy to adopt, but compared to building your own? is there something remarkable that makes dune much different?
Without getting into any specifics of it - I'm sure there's people with much more experience with these tools who can comment - I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
JS does have a bit of a NIH culture, but I'm not sure if that was really at play here. There just...weren't very many good build tools available at the time, particularly for a company using an unorthodox tech stack.
> I'll point out that neither buck nor bzl existed when JS decided to start building their own tool in 2012. Bazel's first release was in 2015, Buck's was in 2013.
But Dune started (according to this blog post) in 2016 and JS started seriously improving and adopting it last year. So to me Jenga sounds like a reasonable step in 2012, but pouring significant effort into migrating from Jenga to Dune (and improving Dune) in 2024 sounds more weird
Jenga and dune are the same thing, it was just renamed.
The blog post clearly describes them as two different systems, and how Jane Street migrated from one to another.
Yes and no. This is all spelled out in the post, but it's a little thorny.
Dune is a rename of Jbuilder (2016). Jbuilder uses Jenga (2012) configuration files.
> By 2016 we had had enough of this, and decided to make a simple cross-platform tool, called Jbuilder, that would allow external users to build our code without having to adopt Jenga in full, and would release us from the obligation of rewriting our builds in OCamlbuild [...] Jbuilder understood the jbuild files that Jenga used for build configuration.
So in 2012 it made sense for them to build Jenga, because there weren't any good alternatives - Bzl etc. didn't exist, so they couldn't have solved their problems.
And in 2016 they had open-source code they wanted others to be able build; those people didn't want to use Jenga, and JS didn't want to rewrite their builds so that they could use something else. Thus, Jbuilder was a shim so that JS could still use their Jenga builds and others could build JS' code without using Jenga. Bzl etc., even though they existed, wouldn't have solved these problems either.
I don't know ocaml particularly well but my understanding is that only with buck2 (of those mentioned) do you have a build system with the semantics to actually compile ocaml properly.
In particular I was under the impression one needed to be able to run ocamldep before hand (or compile twice) - buck2 can do this, bazel needs hacks iirc.
I loath autotools with a passion, and it's awkward to set it all up, but it handles OCaml code just fine. We use it for multi-language projects (including OCaml) like nbdkit & libguestfs.
but why?
What percentage of JS programmers actually work on this stuff I wonder?
Dune is the single best name of a build system out there, nothing even comes close to.
Man I must be under a rock. Never heard of Dune!
Cmake mostly
I respectfully have to disagree. As Fedora packager for OCaml packages, dune's configuration is yet another one-off, with a weird Lisp syntax, hard to understand, difficult to patch. I'd far rather people stuck to using more familiar Linux build systems. As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
(This rant more or less equally applies to other language-specific build systems.)
> more familiar Linux build systems
"Familiar" is not a property of any system. It's a relation between a system and a user.
Some Linux build systems maybe be more familiar to some users, but will be less familiar to others. When picking a build system, you can't just look at the system itself and declare it familiar or not.
It's not even enough to look at the total number of users familiar with some thing. Hindi is one of the most familiar languages in the world, but you're probably gonna have a bad time if you use it for the menu in a cafe in rural Texas.
You have to look at your actual cohort of users (and potential future users) and see what's familiar to them. This is one of the key reasons why usability is actually a deeply hard problem. So much of usability hinges on familiarity, but familiarity is a human-specific highly variable property.
As a user of Linux, I wish Linux distributions would abandon their build systems which really only work well for C and maybe C++, and also only work on one platform (not even the whole of linux!) meaning that packaging has to be duplicated for every OS, and stuck to using more familiar language-specific build systems that work everywhere.
(this rant more or less equally applies to all os-or-distro-specific build systems)
---
This rant is only semi-serious. I do see some value in the Linux distribution style packaging. In particular, I do sympathise with the need to do cross-language builds. But goodness are they a pain to work with, and probably the biggest barrier to me shipping software on Linux.
My hope is that eventually an evolution on build systems like bazel/buck2 will lead to a truly universal build system that is both cross-platform and cross-language. But unfortunately it doesn't look like it's coming soon.
> more familiar Linux build systems
What's a 'familiar Linux build system'? make?
They likely mean 'generally used with other languages'. make is an example.
I haven't been doing much OCaml in recent years, but I do sometimes check in to see what's going on in OCaml-land, and it seems like dune is what everyone is using now - what are the alternatives?
> This rant more or less equally applies to other language-specific build systems.
And yet those language-specific build systems are overwhelmingly winning, in pretty much every language.
> As Fedora packager for OCaml packages,...
I honestly think traditional Linux packaging is in the wrong here and the problems are essentially self-inflicted (not in the sense that individual maintainers are doing something wrong, but in the sense that the policy that traditional Linux distributions are following is inherently unsustainable. It's designed for a pre-CPAN world)
> As an upstream OCaml developer, the whole thing falls down the minute you need to integrate other programming languages into your build (or OCaml code into a code base written in another language).
True up to a point, but frankly the worst case is falling back to a terrible C-style build, and "always do terrible C-style builds in case you need to integrate with C code" is not a proposition that has much appeal.
Much as I wish the whole world would standardise on Maven or Cargo, I can't see a realistic path to there without first eliminating C, because the C people are never going to agree to follow a standard for package repositories.