Interesting article, but the approach still seems like the opposite of how language server sync usually works.
Typically, the LSP is notified on any file change and is responsible for keeping its own copy of the files. This way it doesn't need to request the "latest" files - on a keystroke, it just needs to update the actual file being edited.
It seems like the Deno LSP doesn't follow this push model for some reason, which is going to make it inherently unscalable in very large projects. I'm curious why.
I believe Deno's LSP does use this push model, the missing ingredient is that the LSP server itself is implemented in Rust, but bridges to TypeScript's Language Service API in JavaScript/TypeScript. The TypeScript side calls functions like `readFile()` that Deno then needs to provide, so they need to get the in-memory file contents from the Rust side to the JS side.
As implied here in other comments, the architecture they designed their LSP around immediately suggested where the issue might be.
Sure, in a project like this, observability becomes harder, but that's pretty much the biggest challenge. The fact that this was in any way passable for so long indicates how much engineers are willing to take suboptimal experiences, and how fast computers are today.
Similarly, 75kloc is not a huge project, and even 750k with deps is simply "big". Eg. Linux kernel is >25M, LibreOffice is >10M, Wordpress itself is around 1M, Gimp is 900K, etc. All that is without external dependencies.
Even final code completion that returns in "under 1s" is too slow (I can't imagine ever turning on a feature that would take the original 8s!).
Glad to see improvements, but hasn't this been an issue since the the introduction of `deno lsp`? My guess is that the Deno team have been a bit overwhelmed and only a paying (and likely enterprise) customer's complaint was able to raise its priority to be high enough for the team to dedicate someone to address it.
It seems likely, but it's probably also those same customers who are the ones with large enough codebases for performance to be "unusable" instead of merely "not great".
I have to wonder if maybe writing the whole thing in TypeScript from the start may have been a better choice. It's not clear what they get from Rust and it introduces the binding layer complexity described in the article.
Easiest way to make something faster, is to start with something very slow.
Working with deno using intellij's deno language server support[0] is extremely frustrating, slow and missing features, compared to it's normal TS/JS support.
Glad to hear that things are improving though, I'll have to give it another go.
You have to know a lot about Rust to write performant Deno.
Is there a list of language code smells? If the best parts of your language come from another language are you better off skipping the middle layer now instead of later when there's more code to rewrite?
That was not my experience with Node.js and C++. The options were to either accept mediocre performance and all JS/TS or start writing the important bits with the Node-API.
I think optimization in general is about understanding the layer of abstraction below you and arranging your workload to exploit it's efficiencies and avoid it's inefficiencies. (Cases where you can reimplement using a different algorithm and end up doing less work notwithstanding.)
Typically, the LSP is notified on any file change and is responsible for keeping its own copy of the files. This way it doesn't need to request the "latest" files - on a keystroke, it just needs to update the actual file being edited.
It seems like the Deno LSP doesn't follow this push model for some reason, which is going to make it inherently unscalable in very large projects. I'm curious why.