-
Notifications
You must be signed in to change notification settings - Fork 194
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discussion: Change packaging approach #115
Comments
Why not just distribute the sources? You need the compiler and whatnot available anyway and this way you could just on-the-fly compile sources for the specific OTP+Elixir version (and cache them, of course, making it a mostly-one-time thing) |
(w.r.t. package conflicts - I've known Java code that just rewrites packages on-the-fly so you'd end up with ElixirLS.deps.Jason, say, as a package name. I don't think it'd be super hard with Elixir) |
This is an interesting idea that I hadn't considered before! Especially because there are no nifs (which would make compilation more difficult)
Interesting, I didn't realize that was sometimes done in Java world. I think dynamically renaming the modules wouldn't be too difficult, but I'm more concerned about successfully renaming the application name (e.g. |
I've been thinking about this problem for a while and I think that all current options are not 100% satisfactory IMHO. I've been studying the source code and the core mechanisms that the server uses in order to provide its capabilities and it all boils down to having the project source and its dependencies compiled and loaded in a running BEAM instance. ElixirSense and ElixirLS depend on this core principle so that they can use (I've taken some general notes of the initialization/build process of the server as I think this could be included in the project to help people trying to understand/solve things. If you think so I'll gladly open a PR. I plan on writing a more in depth post about it) I think the definite solution would be something akin to what the Erlang language server has done: we would have on BEAM instance running with ONLY a minor dep that would just broadcast compiler events/state/diagnostics about the project to another BEAM instance that could have any dependencies it would want. On this scenario it would be something like: 1 - Elixir Language Server; The server itself could be an escript that would run on many versions of Erlang and the broadcaster could be an archive as it would automatically be in all projects' scopes. But before diving into possible implementation details, I think would solve the issue of having a "clean" BEAM instance with the project's code loaded and checked on the same Erlang version as the project runs. I might try this approach some time in the future but I think this could add to the discussion. |
Excellent writeup @victorolinasc. I added a small comment on code loading. Can you create a PR with this wrapped as Regarding splitting elixir-ls into several os processes - I think it's the cleanest. I don't know much about Java class loaders etc. but a few years ago I did something related in .net land using ApplicationDomains (you could load an .dll assembly into a sandboxed domain, do some introspection with Reflection and unload it when done). |
@lukaszsamson thanks for your comment! I'll add it to flow and open a PR later today! Do you think it would be a lot of work to have it as separated processes? |
I'm gonna see today how far I get (we have hackday at work) https://github.com/cdegroot/elixir-ls/tree/beam-splitter; I'm gonna start inverting the "who starts what" relationship somewhat (versus @victorolinasc's proposal) because I think that it's easier to get it correct. See the fork's new subapplication's README for details (that's all I have done so far, I like README-driven development :-) ) @lukaszsamson the JVM makes a lot of this simpler, because a class is uniquely identified by not only its name, but also its classloader. But let's not go there, this should work and smells more "OTP-y". |
I've updated the issue description with goals and requirements, are there any that seem incorrect or that I missed? |
@lukaszsamson I wonder if you have time to take a look at #136 and if you have any thoughts. I'm starting to think that we may need to take an incremental approach to get there because so much code in ElixirSense (and ElixirLS) as well is dependent on running in the same beam instance as the user's source code. Maybe we need to initially keep ElixirSense running in the same beam instance but run ElixirLS in a stand-alone instance, but then move more and more of the code to run in the stand-alone instance. |
I’ll take a look but I won’t have time to make any tests. I wonder if an incremental approach wouldn’t complicate things too much. |
(no promises about time commitments - my espresso machine's thermostat seems to be wonky which may mean I'll be sidetracked PIDing it using Nerves - but I'll plod on if only to get a feel for the code. Structurally, I think the PR is were it needs to be modulo one nagging bug I'm ignoring, so it's a good time for me to get acquainted with the actual LSP bits) |
erlang_ls is moving to at least 2 nodes for similar reasons as us: erlang-ls/erlang_ls#670 once that PR is merged maybe we can learn something from it. |
This comment suggests a workaround for the problem with dependencies JakeBecker/elixir-ls#97 (comment) |
I wonder if it would be possible to go one step further and make elixir_ls dev dependency of a project, like credo or dialyxir. It would guarantee, elixir_lsp uses the same version of Erlang/Elixir and JSON libraries as the project. It seems simpler to me than a global server with multiple instances running different Elixir versions communicating with it. It could produce different problems: e.g. someone needs to update their JSON lib version to use elixir_ls or maybe editors could have problems finding binary relative to the project path instead of a global one. Not sure if it is worth the tradeoff. Excuse me, if I am talking nonsense, I am only starting to learn about LSP. |
Just a minor input from bystander, adding to @tomekowal's suggestion. Its installation instruction states:
Opam is the counterpart for |
I'm not sure that I like the compromises required for a dependency-based approach. Namely:
I'm not saying that I couldn't be convinced, but currently I don't favor that approach. But it is also true that the current packaging approach is not working very well. |
It could work with if there's a sane fallback to default build. I fear that as @axelson said we'd have to deal with a slew of bugs from older elixir-ls versions. |
Maybe the new https://github.com/burrito-elixir/burrito can help us here? |
ElixirLS used to run as an escript but that was changed in v0.2.4: 25 Oct 2017: Package ElixirLS as .ez archives instead of escripts. This should make asdf installs work.
Maybe. There is also a proposal of distributing it as source code #121
Replicating the solution used in Livebook is something I have in plan |
I don't think that should be a problem anymore. I believe you just need to reshim elixir on installing an escript. |
@Benjamin-Philip for reference I looked up the PR and issue that removed escripts JakeBecker/elixir-ls#12 JakeBecker/elixir-ls#13 |
I see why we don't use escripts. One possible thing we could do is compile ElixirLS with all supported permutations of Elixir and OTP, and then depend on the package manager to install the most appropriate option. This starts to breakdown when a user has an unsupported permutation. The package manager also may not install the correct option. This would also be non trivial to implement on the CI. I am not sure if escripts installed using a git source are precompiled or compiled on a user's machine. |
I have another idea. Since we are going to require 1.12 we can think of distributing elixir-ls as a |
I am going to take a stab at implementing this. They are 2 years into this architecture and still leaning into it. I think it could solve quite a few headaches we experience in this project. I will report back. |
Currently ElixirLS is packaged with https://github.com/JakeBecker/mix_task_archive_deps as
.ez
archives. This approach means that the language server is run by a different version of elixir than it was compiled with which can cause issues such as #107ElixirLS used to run as an escript but that was changed in
v0.2.4: 25 Oct 2017
:Goals/Requirements:
.tool-versions
or installed globallya. This means that reported warnings/errors match the version of Elixir that they're using, not earlier or later versions
Possible approaches:
Potential issues to be aware of:
Jason
andDialyxir
(but if we added more dependencies it could become a larger issue)a. This is the issue that is currently causing the most trouble
I will come back and flesh out the issues and possible approaches later, along with more clear guidelines on what the requirements are.Edit: added above
The text was updated successfully, but these errors were encountered: