Writing my own REPL

I'm playing around with a new project, DERL (Data Eval Return Loop, pronounced like Daryll), which I intend to be a data-oriented/REPL-oriented IDE for Clojure.

My Clojure background

My experience is mostly in CI/CD/DevOps, primarily developing in the Java language. My open-source work has mostly been in Gradle plugins, since before Gradle 1.0 (almost 10 years ago now). As discussed in previous posts, I first used Clojure while reading the 7 Languages in 7 Weeks book which exposes you to languages in different paradigms, and I've been fascinated by it since.

No other programming language has changed how I think about programming more than Clojure. My Java code style has been fundamentally changed and I just write better, simpler code because of it in any language.

However, when working in Clojure, I find myself in a constant tension that feels ingrained in Clojure as a language and community. The "embrace the host" mentality is at odds with a more pure Lisp approach, but was done as a pragmatic compromise to extend the reach of Lisp to the JVM and JS ecosystems.

My instincts are to utilize what I know of the host, which is why I like trying to use some of the newer features of Java (some of which aren't so new now). I made ike.cljj to make it easier to use some Java 7 and 8 features. And I (sporadically) maintain clojurephant the Gradle plugin for Clojure. However, Clojure seems to have a different sense of "embrace the host" than I do. For build tooling, they've largely thrown away the concept and have moved towards tools.deps which is only classpath management and snubs its nose at traditional features of build tools. Similarly, Clojure seems to have little to no interest in smoothing Java interop around newer Java features.

My challenges with Clojure editors/tooling

Code is data and data is code, unless you're writing code, in which case it's still text. While I think Lisp syntax is superior, as a text format it just doesn't play well with typical editors that are more optimized for Algol-like syntax.

For all the benefits Parinfer and Paredit provide, the fact that you're still editing a text file in an editor made for text files is a leaky abstraction.

Of the existing editors:

• Outside of Clojure I primarily work in VS Code, but I've found the Clojure experience lacking:

  • Calva is impressive, however I like using VS Code because of muscle memory and familiarity. However, Calva's default keys stomp on a lot of my normal VS Code workflow.

  • Clover provides a simpler experience, but only works with Socket REPLs.

  • Parinfer support is lacking, which I've read is due to limitations of VS Code's core APIs.

• Atom has Chlorine and good Parinfer but, as an overall editor, languishes in the shadow of VS Code these days.

• Cursive is the most impressive Clojure environment I've used, with great Parinfer support, but the overall experience of Intellij is too Eclipse-y rather than VS Codey for me to want to stay there.

• Emacs has Lisp mindshare, but it's just too foreign everytime I've tried to learn it.

While I've gotten used to the "write code in a file and send it to the REPL" workflow, I feel like there's a lot of potential that I haven't tapped yet, and I don't think these editors push me hard enough to go there.

Ultimately, I feel like I need to shake myself out of the host-centric thinking by going as far the other direction as I can and see what I learn from it. Which leads me to...

Data Eval Return Loop

While at a base level, we're still dealing with text, or at least visual representation, I want to remove te emphasis on text files that need to be read and results being printed.

The idea would be:

1. Start DERL
2. Load project and dependencies
3. Compose code as data, manipulate it as data (keyboard shortcuts run Clojure functions that manipulate the data)
4. Evaluate (initially this probably means printing/reading text on either end of a traditional REPL)
5. Return data
6. Loop

Goals for DERL

All DERL is right now (0.1.0) is a JavaFX prepl client. That's pretty far removed from an IDE. But ultimately I want a few key things:

• Data-based editing (not structural editing). The equivalent of Paredit, but purely working on data, where the text on the screen is just a view to this. Maybe it doesn't even get saved to a traditional source file.

  I want it to feel more like Parinfer, (i.e. not as many keyboard shortcuts) but expect it will be more like Paredit.

• You edit entirely in the REPL, and you save it out as a way to run it later. Source of truth is data in the REPL, not a textual source file.

• Supporting socket REPL, prepl, and nREPL so you aren't bound to any of them. Maybe making a more DERL-y version you can run from the Clojure socket server, if it makes anything easier.

• Support multiple REPL sessions simulatneously in one project.

• Separate process from the JVM my project is running in. It may not always be an issue, but I don't want the classpath confusion of having the IDE and project together.

I'm sure there is prior art for all of this, I see this as a learning experience that could result in something interesting to the Clojure community.

Perhaps this project will serve as nothing more than ingraining structural editing into my head and get me to embrace Paredit, allowing me to switch to Calva or Emacs more easily.