Short version: I wrote a Prolog language server implementation – give it a try!
Since I started using Prolog last year, I’ve been wanting to improve the editor story. Most of my actual work is using Clojure and I’ve been using Lisps for years, so I’ve gotten used to having something like CIDER or SLIME. Prolog (SWI-Prolog in particular) has a lot of pretty powerful introspection capabilities, so the actual information gathering part of such a thing for Prolog would be straight-forward enough. I never got started on my efforts to make Cider for Prolog – the required architecture seemed non-trivial and the work I’d need to do Emacs-side to build a user interface seemed like a lot (while I’ve done a bunch of Emacs customization stuff, it’s mainly been modifying existing systems; I’m still not entirely clear how to build something on the level of Cider) – but it was always high on my list of “someday” projects I really wanted to do.
Eventually, I read about Microsoft’s “Language Server” project. It’s essentially a client-server model for editor extensions – instead of writing code that runs in the editor, in the classic manner of Emacs and Vim, the extension is a separate process that communicates with the editor via a JSON interface. This became relevant to me when I realized that there’s an Emacs Language Server client library, so I could write my Prolog introspection code as a stand-alone program and just implement the Language Server Protocol. Not only would this free me from having to write all of the Emacs side of things, it would also make whatever I built usable in other editors that implement the LSP as well!
Writing the server itself was fairly straightforward.
I made the decision to have the server not actually load the code that it inspected, mainly due to concerns with how challenging it would be to isolate the loaded code from the running process and from each other (SWI-Prolog’s module system seems slightly leaky). SWI-Prolog has lots of ability to get information on loaded code, but also good support (via library(prolog_xref)) for getting information from static analysis. I did have to make some enhancements to the library for some of the things the server needed, but it was simple enough (as I’ve mentioned previously, contributing to SWI-Prolog is always a very good experience).
prolog_xref library gives most of the information needed for the basic features – finding all defined predicates in a file, finding where a predicate was defined, finding the uses of a predicate – but the complex part for my implementation turned out to be figuring out what the input to those predicates should be:
The way, for example, the “find definition” LSP method works is that the server receives the file and the position in the file the request is coming from.
This means that the server needs to figure out what the predicate at that point in the file is; for full generality, I had to parse the Prolog code & walk over the terms to determine what the predicate at that position is.
Even more complicated is finding the references of a predicate –
xref_called/5 only gives the line the calling predicate’s definition begins on, so to actually jump to the exact location of the reference, the server needs to parse the clause starting at that line, then search through the term to find the referred predicate.
Luckily, this also turned out to not be too bad – although it’s definitely the most complex part of the server.
read_source_term_at_location/3 parses a clause starting at a given line and includes all the position information for the subclauses, which is exactly what I need for my use-cases.
Another neat aspect of Prolog is that the predicate I wrote to find the clauses at a given position (
clause_in_file_at_position/3) can actually be run either way – if the clause is a variable and the position is ground, it will find the clause at that position, while if the clause is ground and the position is a variable, it will find the position of the clause!
It was pretty neat writing the code initially to find the clause at position and realizing that I could also get the other way more-or-less for free.
Predicates that can run in different modes is one of my favourite features of Prolog; it’s one of the best forms of “DRY” (“Don’t Repeat Yourself”) I’ve come across in any programming language (I previously wrote about how I was able to leverage the same feature in my HTTP/2 client).
SWI-Prolog has a very nice built-in documentation system, so I knew that – in theory, at least – it should be a simple matter to show the documentation for the predicate under the user’s cursor. In pratice though, I’d been some what vexed before by the documentation not appearing in the way I wanted it – sometimes showing up in my terminal window as I wanted it, sometimes in a GUI window, which would annoy me.
Fortunately for me, SWI-Prolog’s source is very readable, so I just looked at the implementation of
library(help) and hacked together a predicate that gets the built in help when available & falls back to parsing the pldoc for user-defined predicates.
It uses some private predicates from the
help library, but they seem like ones that are pretty core to the library & unlikely to change…🤞
The most recent feature I added to the server is showing errors and warnings. Because I’m still trying to stick to static analysis, it doesn’t yet have all the diagnostics and warnings that SWI-Prolog is capable of, but it has some of the helpful ones – primarily syntax errors and singleton-variable warnings.
My first attempt to capture these warnings in a useful way was extremely hacky.
xref_source/1 would display warnings for the file it was attempting to parse, I tried redirecting
stderr to a string then parsing out the error messages from that.
This kind of worked (thanks to Prolog DCGs making it so easy to write parsers) but it was pretty obviously not the idea way of handling the situation.
Yesterday, while trying to fix something (that turned out to be an unrelated issue) I decided I should do this the “right” way & re-implemented it using
message_hook to intercept the warning & error messages before they actually got printed out, which let me operate on the Prolog terms that the system was sending, not just the printed-out representation.
One issue that came up with this approach was that, while the messages that actually got printed out would include the line number, the terms I could intercept with the hook didn’t have line numbers associated. Once again, I had to dig in to the source to figure out how it worked and once again, it turned out to be pretty simple.