Questions on go to features

[gsouf]

Hey there, started recently with OCaml and I noticed a few things with the LSP.

Here is one thing, I'm not sure whether it's a missing feature or a bug, but at least it didn't work as per my expectation (may it be right or wrong).

With the LSP we gain the ability, given a symbol, to jump to references, definition, declaration, implementations, type definition of the given symbol (note that these wordings are the ones coming from my neovim shortcuts, actual names might be different).
Below are somes cases where I couldn't get it to work as I expected.

To illustrate my findings here is a very simple code sample:

module type FOO = sig
  type t

    (* 1 *)
  val bar : string -> t
end

module Foo : FOO = struct
  type t = string

    (* 2 *)
  let bar s = s
end

          (* 3 *)
let f = Foo.bar "s"

Everything I will demonstrate here revolves around the bar function, I annotated the 3 locations where it appears to make explanations clearer.

So first of all here are few things that worked as expected:

• ✅ Go to definition for 3 -> go to 2
• ✅ Go to declaration for 3 -> go to 1

Here are things that worked partially as expected:

• ✅ Go to references for 2 -> go to 3. ❌ But does not go to 1
• ✅ Go to references for 3 -> go to 2. ❌ But does not go to 1

Now here are things that didn't work as expected

• ❌ Go to implementations for 1 -> nothing (expected to go to 2)
• ❌ Go to references for 1 -> nothing (expected to go to 2 and 3)
• ❌ Go to declaration for 2 -> nothing (expected to go to 1)
• ❌ Go to type definition for 2 -> nothing (expected to go to 1)
• ❌ Go to type definition for 3 -> nothing (expected to go to 1)

Any thoughts on this? Are there features missing, or is there something I'm not understanding correctly?

Thanks

[voodoos]

Hi @gsouf thanks for taking the time to list all this cases, I think it's a very good starting point to discuss these features !
First, I want to say that all the behaviors your describing are, currently, the expected ones and are not bugs per see. (Apart maybe 2 -> 1)

In particular, "go to type definition" from 2 or 3 should not jump to 1. Indeed, the type of bar is string -> t which is not explicitly defined in this module. You could give it a name: type foo = string -> t and then use that name and it would work.

I think we need a slightly bigger example to understand the current design choices and why choosing extensions is not obvious:

(* In the [example.mli] file *)

module type FOO = sig
  (* 0 *)
  val bar : string -> unit
end


(* In the [example.ml] file *)

module type FOO = sig
  (* 1 *)
  val bar : string -> unit
end

module Foo : FOO = struct
  (* 2 *)
  let bar _s = ()
end

module Bar : FOO = struct
  (* 2' *)
  let bar _s = ()
end

(* 3 *)
let f = Foo.bar "s"

(* 3' *)
let f = Bar.bar "s"

In this example the declaration chains are:

0 ---> 1 ----> 2 ---> 3
        ∟--> 2' ---> 3'

It's a tree! (and can even be a dag), which is the source of the complexity for these features that provide partial navigation in that tree, often in ways constrained by the LSP protocol (which was not designed with OCaml in mind).
(In addition to this example, one declaration can be related to multiple other declarations, not only definitions.)

Currently:

• "go to decl/def" functions only travels upstream in that tree. Going in the other direction requires making choices when there are multiple branches, or return multiple answers to the user, which is not part of the LSP protocol for these features.
• the "references" query lists all the usages of a definition. Not all the occurrences of that definition's identifier. There are multiple ways this could be extended but I suspect that there is no one-size fits all here, and the scope should probably be parametrizable which, again, does not fit well the LSP protocol.

The main advantage of these design choices is that the results are easily predictable and well scoped. Also they fits the protocol well.

I do think we should work on improving the current behavior in the future (and we are happy to receive contributions). Here a a few I could think of:

• Have "upstream" queries always work as expected like 2 -> 1
• A custom query (or set of queries) and client UI to navigate the tree in both directions. Choice will be given to the user if there is a branching.
• References could return the entire branch in which the target definition is, not only the usages but also the definition and upstream declarations.

[gsouf]

@voodoos thanks for your detailed answer. I'm not familiar with the internal of the LS protocol, and I only have minimal experience of OCaml so I might be missing something and I appreciate that advanced features of OCaml might make this difficult to fit the standard LSP function.

Just for reference I'm giving typescript snippet, using some class hierarchy and draw the parallel I might have wrongly expected with OCaml. I understand that typescript's nominal class hierarchy differs fundamentally from OCaml's structural modules, but I believe the user need for navigation remains similar in the module system.

Here is the snippet:

interface IFoo {
  foo(): void;
}

type FooMethod = IFoo["foo"];

class T implements IFoo {
  foo(): void {}
}

class U extends T {
  foo(): void {}
}

class V extends T {
  foo(): void {
    super.foo();
  }
}

class W extends V {
  foo(): void {}
}

const t = new T();
t.foo();

// For reference, another class with the same method
class Z {
  foo(): void {}
}

References

For references, it basically lists all references of a symbol, regardless of whether it is higher in the tree or down the tree, in a sibling branch, or when the method/function/property/class is defines/declared/override/referenced/called. With our example above; for any foo (other than the class Z), when I ask for references, it will lists all others foo (other than the class Z).

So basically "references" consists in finding the root, then listing all of its descendants. Note that depending on languages, the root might either be the actual top level item, or a "local root" such as the import statement in the current file and therefore list only the reference relative to this import. Then to find even more references, we will look for the references of import statement itself, or from the symbol defined in the imported file.

Similarly in OCaml, when a module satisfies a module type signature, all modules that share that signature could be considered related references, couldn't they?

Implementations

It will only go down the tree, but only lists an actual "implementation" of the method, not the usages.

In OCaml, it might not be different, once we have the references we can filter the implementations.

Declaration, definition, type definition

For those it's going up the tree In the example above, using any of these on the foo method will always resolve to the root class T, so it seems to target the first concrete class (it's not reaching the abstract class or interface), but honestly (nitpicking) here I would have expected that at least one definition or type definition goes to the interface (the distinction between these in the LSP has always been a source of confusion to me). So to navigate even higher in the tree and effectively being able to discover everything step by step, I'll just use what comes after extends or implement (the class name or interface name) and use "go to definition".

Here OCaml is a much more complex language, and it becomes more arbitrary to decide how to fit the protocol, I suppose. But from what I know of the language, I can't think of a case were we can't find what's up in the tree.

---

Overall, I think that "go to references", "go to implementation" and "go to definition" are the most important things here, because together they are a tool like no others to navigate the code base comprehensively. To simplify:

• the "references" should list unconditionally all descendants from the (local) root of the symbol
• definition/declaration help to jump to the (local) root, and to go up the tree
• implementation helps going down the tree without listing the usages

At the moment it's not fully possible with the OCaml LSP, notably the "implementations" that I couldn't get to work.

---

Alternatively, it seems that LSP allows for custom methods via the prefix $/ (https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#dollarRequests). If parts from the standard protocol don't map well for OCaml, it might be worth exploring the idea of implementing custom methods and supporting them through editor plugins (requires more work, but because it's optional it becomes a "nice to have" addition that does not force the implementation in all existing editors). But I'd personally explore that way as a last resort if nothing else is possible.

[gsouf]

@voodoos as a follow up of my previous message. I scoped my answer to the case of a single file. Actually declaration/definition should behave differently in a multi file setup (and in the case of OCaml not only with the mli file, but with other modules from other files too). I believe that definition should try to stick to the current file and stop at the import statement, in case of qualified imports, not sure in the case of, for example, open module . While declaration will go where it is declared, regardless of the file