Andrew Oberstar

Why am I writing DERL?

December 27, 2020

I recently started a project called DERL (Data Eval Return Loop, pronounced like Daryll). This is intended to be a data-oriented/REPL-oriented IDE for Clojure, and I felt like I should explain my motivations longform.

My Clojure background

My experience is mostly in CI/CD/DevOps, primarily developing in the Java language. My open-source work has mostly been in Gradle plugins, since before Gradle 1.0 (almost 10 years ago now). As discussed in previous posts, I first used Clojure while reading the 7 Languages in 7 Weeks book which exposes you to languages in different paradigms, and I've been fascinated by it since.

No other programming language has changed how I think about programming more than Clojure. My Java code style has been fundamentally changed and I just write better, simpler code because of it in any language.

However, when working in Clojure, I find myself in a constant tension that feels ingrained in Clojure as a language and community. The "embrace the host" mentality is at odds with a more pure Lisp approach, but was done as a pragmatic compromise to extend the reach of Lisp to the JVM and JS ecosystems.

My instincts are to utilize what I know of the host, which is why I like trying to use some of the newer features of Java (some of which aren't so new now). I made ike.cljj to make it easier to use some Java 7 and 8 features. And I (sporadically) maintain clojurephant the Gradle plugin for Clojure. However, Clojure seems to have a different sense of "embrace the host" than I do. For build tooling, they've largely thrown away the concept and have moved towards tools.deps which is only classpath management and snubs its nose at traditional features of build tools. Similarly, Clojure seems to have little to no interest in smoothing Java interop around newer Java features.

My challenges with Clojure editors/tooling

Code is data and data is code, unless you're writing code, in which case it's still text. While I think Lisp syntax is superior, as a text format it just doesn't play well with typical editors that are more optimized for Algol-like syntax.

For all the benefits Parinfer and Paredit provide, the fact that you're still editing a text file in an editor made for text files is a leaky abstraction.

Of the existing editors:

While I've gotten used to the "write code in a file and send it to the REPL" workflow, I feel like there's a lot of potential that I haven't tapped yet, and I don't think these editors push me hard enough to go there.

Ultimately, I feel like I need to shake myself out of the host-centric thinking by going as far the other direction as I can and see what I learn from it. Which leads me to...

Data Eval Return Loop

While at a base level, we're still dealing with text, or at least visual representation, I want to remove te emphasis on text files that need to be read and results being printed.

The idea would be:

  1. Start DERL
  2. Load project and dependencies
  3. Compose code as data, manipulate it as data (keyboard shortcuts run Clojure functions that manipulate the data)
  4. Evaluate (initially this probably means printing/reading text on either end of a traditional REPL)
  5. Return data
  6. Loop

Goals for DERL

All DERL is right now (0.1.0) is a JavaFX prepl client. That's pretty far removed from an IDE. But ultimately I want a few key things:

I'm sure there is prior art for all of this, I see this as a learning experience that could result in something interesting to the Clojure community.

Perhaps this project will serve as nothing more than ingraining structural editing into my head and get me to embrace Paredit, allowing me to switch to Calva or Emacs more easily.

Perhaps I'll shelve this project in a year, a month, or a week.

Time will tell.