Leveling Up Developer Tooling for the Modern Rails & Hotwire Era

Summarized using AI

Leveling Up Developer Tooling for the Modern Rails & Hotwire Era

Marco Roth • November 20, 2024 • online • Talk

The video titled "Leveling Up Developer Tooling for the Modern Rails & Hotwire Era" features Marco Roth, who discusses the evolution of developer experience (DX) tooling in the Ruby on Rails ecosystem, particularly focusing on the implementation of Language Server Protocols (LSP) for tools related to Stimulus and Turbo. This presentation sheds light on the landscape of modern developer tooling and how Rails developers can enhance their productivity by adopting advanced tools.

Key Points Discussed:

  • Introduction to Developer Experience (DX): Roth emphasizes the significant changes in DX tooling and how they impact web application development, debugging, and enhancement.
  • Role of Language Server Protocols (LSP): LSP is introduced as a means of synchronizing language knowledge between the editor and the server, thus allowing broader support across editors without needing to build custom plugins.
  • Personal Journey in Open Source: Roth shares his experiences and contributions to the Stimulus and Turbo frameworks, which fueled his motivation to enhance the developer experience in the Rails community.
  • Development of Autocomplete Features: The discussion includes details on how to implement features that greatly assist developers in writing less error-prone Stimulus applications by providing real-time suggestions and autocompletion.
  • Community Impact and Feedback: Roth mentions the feedback from the community regarding the tools he's developed, indicating that collaboration and user input are crucial to the ongoing improvement and refinement of these tools.
  • Future Directions: The talk includes reflections on his recent work, such as web extensions for better usability in the Hotwire ecosystem and plans to improve diagnostics and code actions within the editor environment.

Significant Examples:

  • Roth explains the LSP in the context of Ruby and how it allows for dynamic suggestions while coding in Stimulus, preventing common mistakes such as typos or incorrect references to controller actions.
  • He highlights the practical challenges developers face when working in the Rails ecosystem compared to modern JavaScript tooling and how LSP can bridge this gap.

Conclusions and Takeaways:

  • The presentation concludes with a call to action for Ruby developers to embrace these advancements in tooling for enhanced productivity and to encourage continued innovation in the Rails ecosystem.
  • Roth emphasizes the need for the Ruby community to keep pace with tools available in other programming environments in order to attract and retain new developers, which he believes is essential for the future relevance of Ruby on Rails applications. Roth encourages ongoing dialogue and contributions within the community to foster a more robust and unified tooling ecosystem.

Overall, this talk provides valuable insights into modern developer tooling for Ruby on Rails, emphasizing the utility of LSP and community engagement in enhancing developer productivity.

Leveling Up Developer Tooling for the Modern Rails & Hotwire Era
Marco Roth • November 20, 2024 • online • Talk

The evolution of developer experience (DX) tooling has been a game-changer in how we build, debug, and enhance web applications.

This talk aims to illuminate the path toward enriching the Ruby on Rails ecosystem with advanced DX tools, focusing on the implementation of Language Server Protocols (LSP) for Stimulus and Turbo.

Drawing inspiration from the rapid advancements in JavaScript tooling, we explore the horizon of possibilities for Rails developers, including how advanced browser extensions and tools specifically designed for the Hotwire ecosystem can level up your developer experience.

https://kommunity.com/ruby-turkiye/events/leveling-up-developer-tooling-for-the-modern-rails-hotwire-era-671f5b80

Ruby Türkiye Meetup November 2024

00:00:01.440 okay I started now so let me do like a quick just you know welcome thing and
00:00:07.480 then I will let you have the stage sure uh I'm going to do that part of it
00:00:13.240 in Turkish so just to be like
00:00:24.000 sure leveling up developer tooling for the modern rails and Hardware era up
00:00:43.960 fore I think you can St awesome let me try to figure this
00:00:54.760 out can you see this yeah we can per yes okay let me
00:01:06.080 then all right then welcome thanks for having me here at the Ruby turkey Meetup
00:01:11.640 it's great to see that there are so many online meetups happening and also so many iners again and um yeah thanks for
00:01:18.040 having me today I want to talk about um the modern Rail and hot tooling I marker
00:01:25.040 R if you haven't met I would love to chat after or reach me on any of the socials online
00:01:31.920 um this talk is more about to kind of give you an Insight of what it takes to
00:01:37.200 build these tools and kind of give you some I guess transparency that it's not
00:01:42.280 that hard to build these tools and kind of help contributes to make these tools happen so what we going to do today is
00:01:48.680 kind of trying to understand the language server protocol the LSP for short building a simple LSP and then
00:01:54.680 kind of show you what's next for tooling I've been on the stimulus reflex
00:02:00.479 and C ready c teams for the past few years and I've been maintaining stimulus as well for the last two or so years so
00:02:07.119 this kind of gave me all these insights that I kind of had to kind of build these tools and really made it my um my
00:02:16.280 job easy to build these tools so I love open source I have been
00:02:21.879 doing this for a few years now and I really think I had an awesome Journey so far I was able to contribute to all
00:02:30.040 these tools are listed here and probably many more but most of you can see here are somewhat related to Turbo to hot
00:02:36.560 wire to stimulus or in some way to that I've also built a few other tools and
00:02:41.840 I've want to focus on one of them briefly because that's kind of what we are doing today as well so we have been
00:02:47.920 building this website called Ruby video the past few months or so we have been
00:02:53.000 really focusing on adding more and more um data to it and I'm happy to announce that as of today we also have the Ruby
00:02:59.040 turkey meetup on there too so I'm hoping that we can also add the future ones and
00:03:05.040 including this one ter too so this is just a way to show that all these events
00:03:10.080 are happening we are a really awesome community and we have all of these tools
00:03:15.280 and videos and talks available on this website now it gives you kind of a list
00:03:20.560 of all the talks and you can see what's happening and you get the recent ones there too so if you haven't seen this
00:03:28.040 check this out it's on Ruby video the def SL organizations as Ruby turkey and
00:03:33.120 there you can see this one and I guess the feature ones as well uh but to come go back to the topic
00:03:39.599 of today um I started to actively contribute in 2019 and at the time I was working at
00:03:46.400 the job where we we were using react but I kind of had enough of that
00:03:53.480 um the side mindset right I love Ruby I love rails and I want to keep building
00:03:59.280 these ra and Hotwire apps because that's just the way I think and the way my brain works and it feels like it gives
00:04:06.120 me this kind of superpower to just build these apps I the want I the way I want them to
00:04:11.239 build but also I think they are the features the future of mod ra apps and I
00:04:16.440 wanted to going back to do these full stack apps build these apps because it's
00:04:22.160 just so much more productive but I also started to miss these modern tooling that I was used to
00:04:29.160 from the react act and JavaScript um ecosystem that's kind of why I wanted to
00:04:34.199 go for a few years fulltime on open source to just build out some of these tools and I had a lot of fun doing so so
00:04:41.199 during the time between 2022 and 2023 I was doing pretty much full-time open source for the fun of
00:04:47.639 it mostly to work on these hot fire and related projects and yeah I noticed something
00:04:54.000 these react tools they had really awesome developer experience and they really made me more productive doing the
00:05:01.320 work I was doing but it didn't seem like we have something similar in the Ruby World it
00:05:06.759 seems like we were lacking that or if there were tools that weren't
00:05:12.000 really as easy to get started to set up in react World you just inst um use your
00:05:18.360 editor you open the project and it just works you didn't have to install and configure anything it pretty much just works out of the
00:05:24.319 box with Ruby we had this solor graph language show for the longest time but it's kind of felt that something is
00:05:30.440 wrong with it or it I never couldn't figure it out how to set it up or it was always kind of annoying to get it
00:05:35.840 started with it so I felt like we have to improve that experience to get more people to use these
00:05:42.280 tools and yeah I started to miss working with these tools I was used to in the react World I not the person who uses an
00:05:49.280 IDE in that sense they always kind of felt clunky and heavyweight for me to use and I was just wanted something
00:05:56.000 light R just works and is reliable but in the last few years stuff changed
00:06:02.000 a little bit we got kind of editor Evolution going on I think and that's thanks to the language Serv
00:06:09.120 protocol so what is that protocol and why do we want to use it it's kind of the way of sharing
00:06:16.440 knowledge between the server like language knowledge and the editor so we
00:06:21.919 can communicate between these two so it's like a language intelligence tools without having the need for a custom
00:06:27.840 buil IDE for that program language so you can think of this as this before we had LSP we would have to build a
00:06:35.639 plugin for each of the editors to add support for Ruby for JavaScript or whatever else you have but thanks to LSP
00:06:42.919 we can build one language server and because it supports the S the protocol
00:06:49.080 all dats that support the protocol too just support it after box if you want to think in terms of like rupan rails um
00:06:58.280 parallels you can think of LSP as like rack Rack kind of connects the web servers and like the um the um and the
00:07:07.000 Frameworks and it's just this common interface that you can use to build these um tools so how does it work
00:07:14.400 whenever you open a file that's associated with one language your editor starts a background process which is the
00:07:20.919 server itself and it keeps running and keeps serving these requests you can think of this as like a web server and
00:07:27.039 your client being the editor that just uh makes request to that server usually this happens over sent it
00:07:33.759 in or send it out we can also configure it to do it over TCP I but it's less common and it uses a protocol called
00:07:40.919 Jason RPC which if you aren't familiar with it it's kind of similar to http and it
00:07:46.639 looks a little bit like this so you have the headers up top as well you have some
00:07:52.240 um body in this case it's just a Json blob and here we can see it just has a Json RPC it has a version it has ID and
00:08:00.599 have some methods in param that you can pass along it in the LSP we have some messages that
00:08:07.000 we can send between the client server usually these aren't requests response and notification
00:08:13.280 messages but what we mostly care about are the text document synchronization so
00:08:18.720 whenever you open a file in your editor or you save a file or you close a file
00:08:24.240 your editor sends a notification request to the server to notify the server about the files that changed so we as the
00:08:31.560 server implementers can um react to those um events this brings us to the SSP today
00:08:39.880 and we kind of want to talk about what we're trying to solve here so when you are building out a stimulus application
00:08:47.240 you have this markup here this is the most basic example from the website but
00:08:52.279 what you really care about here are the attributes that are the controllers the targets the actions and so on all the
00:08:58.079 ones that are related to stimulus and what we want to do is when we kind of start to write our HTML we want to
00:09:04.560 have another complete to give us all the possible values or attributes that are valid in this context so this example
00:09:12.000 here we want to show all the possible controllers all the possible targets and everything that you might have in your
00:09:19.720 application so if you complete this controller you get to the value and then
00:09:26.000 you want to autocomplete the value here too so if you type this out you get us a profile control for example here
00:09:32.720 but if you noticed I I um made this on purpose so it's should actually be user Dash
00:09:39.440 profile and that's kind of the common problem that we saw with kind of people using stimulus that it's so easy to make
00:09:45.360 these mistakes it's just like so easy to make the T um to make these um typos and
00:09:52.240 you don't really get any feedback from the framework at all of what's actually wrong it just doesn't work it doesn't give you any output nothing so I thought
00:10:00.279 we can do better we want to support the people to write better simus controllers or code so back a few years I was looking
00:10:08.200 it up and just thought yeah let's see how we can build this so I found this documentation and it's about building
00:10:15.680 any extension for um vs code and it had this REO with all the examples and I
00:10:21.760 found this specific LSP example in here it just had this basic
00:10:27.640 interaction where you have have a text document you have some text and it will always give you two options to choose
00:10:33.240 from just like an autocomplete so let's see how this basic example works
00:10:38.440 here so when you open this file it sends this request to the server so what we
00:10:44.079 see here is a text document completion request it sends along the text document
00:10:49.760 reflect a path to where this file is the position of where your cursor currently is and some more um parameters we don't
00:10:56.880 really care about for right now and the server can do anything to actually generate a response but it has
00:11:03.079 to respond to this request with a Jason blob in this example we return this Json
00:11:09.519 blob where we have a result it's an array and it contains some of these um objects and the label is what you
00:11:16.279 actually see in your editor and when the client receives this it just displays it as like this um drop down
00:11:23.279 dialogue so if we go back to our simus example we have a similar situation here we have HTML here
00:11:29.800 we want to um autocomplete this attributes the only difference here is
00:11:35.880 that we are dealing with a embedded language so here we have HTML that
00:11:42.040 already is a standard it already is a completion um mechanism in most tivs for
00:11:49.120 HTML so we have to extend the existing HTML Behavior which is why we talk about
00:11:55.279 embedded languages the stimulus grammar or the stimulus attribute attributes are not part of this HTML spec it just uses
00:12:02.320 data attributes and you can pretty much choose arbitrary data attributes it doesn't really Define how that works so
00:12:09.079 we have to find a way to extend the HTML like documents to make this happen and
00:12:14.519 you can do this thanks to language Services um now we have to talk about
00:12:20.839 how we Implement one of those and most commonly you have to ask which language do you implement this in we are Ruby
00:12:27.160 developers so ideally we would want to use Ruby for this um but realistically speaking we
00:12:33.360 have a lot of Frameworks that are using Hotwire so it felt kind of weird to require Ruby to be installed for all of
00:12:40.480 these other Frameworks languages to use this um uh LSP so we decided to just use
00:12:47.079 typescript for now to just make it more portable so we can use the existing data providers and reuse existing language
00:12:55.760 services and speaking of that luckily the vs codes organization actually
00:13:01.519 exposes a language service for HTML already so we can just plug into this and Implement very easily with that all
00:13:08.880 we have to do here is to implement a data provider and data providers look something like this so you have a class
00:13:16.000 that you extend from or Implement from an interface that is package exports and
00:13:21.240 you have to implement these three methods to make it work so let's Implement them first up is the provide
00:13:28.639 taxs and this one is to extend new tags to the HTML syntax HTML tags but since we
00:13:35.959 are using data attributes for our markup we don't really need any new tags we
00:13:41.399 just need more attributes so all we have to do here is pretty much just return an Mt because you don't care about these
00:13:47.240 tags or now more interestingly we have the provide attributes that we want to
00:13:53.480 provides more to the editor so here we have this list of Comm
00:13:59.160 attributes you might use with stimulus some of them are Dynamic and some of them are static so that the static ones
00:14:04.800 up here are easy to implement so let's start with those what we do here is also return an
00:14:11.320 um an array and in here we just return a object or objects with a name attribute
00:14:18.120 and with that we pretty much just have this implemented already so this works for the static ones but now let's look
00:14:24.720 at the the dynamic ones the problem here is that these dynamic ones are dependent
00:14:30.519 on our application so we have to know which controls you have in your application and which classes targets
00:14:37.120 these um controls Implement in your application too or in your controllers
00:14:42.160 so we have to parse JavaScript or more specifically we have to par our stus controllers to see what these controller
00:14:49.120 hold in them and for that we Implement a control
00:14:54.160 definition class which just holds the reference ref to where this Val is and
00:15:00.360 what these controllers exp expose so the methods targets classes and the
00:15:06.320 values we use Acorn for this which is a jka parser and don't really have to know all
00:15:14.000 the details here but we just Implement a parsel class it has a par controller method we take in the codes which is a
00:15:20.440 source code and the file name and when you pass in or when you look at this file here for example our hello
00:15:26.920 controller um which looks like this it has two Targets the name and outputs it has some
00:15:33.600 methods on it and what we really care about here is just the targets and the method names you don't really care about
00:15:39.360 what's in the method themselves so what we can do is write some code in JavaScript to go over all
00:15:46.000 these files we have a glob to look for all these controllers we Loop over all of them read the files and pass the
00:15:54.560 source codes and the files into our parser parse controller method
00:16:00.519 in that parse controller method we can of can parse the code we get back an abstract syntax tree which just allows
00:16:06.839 us to look into this file and how it's defined and what it's actually exposing we inen a new control
00:16:13.240 definition class which we defined earlier and what we do now in here is we Traverse this ABS tree and look for
00:16:21.120 method definitions and whenever we find one of those method definitions we just take
00:16:27.399 the name of it and push it into our methods array so we know which methods
00:16:32.680 this control defines and with that we have the basic
00:16:38.199 um parser for parsing at all the method names our controler exposes we do something very similar for
00:16:45.000 the targets and for the other ipis we have and this turns this controller with
00:16:51.759 the Targets in the method names into an instance of our control definition um
00:16:57.519 class that looks like this we have have the path we have the methods as like strings of the methods it defines and we
00:17:04.000 have the two Targets as strings so we can actually work with that now and with this information we can go
00:17:09.720 back to our Dynamic attributes and now interpolate these values into our attributes
00:17:15.959 here so for our provide attributes um method we want to implement we can
00:17:22.880 extend this now to Loop over all our targets interpolate the identify power
00:17:29.320 controller and then return all of these targets with the remaining static ones and with this we get a auto
00:17:37.440 complete that looks like this now editor so we have the action the controller which are the static ones but we also
00:17:44.559 now get the data hello Target which is because we have a hell controller in our application so we kind of have already
00:17:51.160 good autocomplete and no understanding of our application so let's say you complete
00:17:57.320 this controller now you are this situation where you want to know which are the possible values for this
00:18:04.000 controller um so you can implement this third method the final one which is the
00:18:09.080 provide values um method we get in past pass in a tag and the attributes again
00:18:16.400 we don't really care about the tag it just has to be on the elements but we actually care about the attribute in
00:18:21.520 this case because you want to know which attribute we are trying to complete for so when we see attribute is our data
00:18:29.679 controller attributes we just return all the attributes we know or all the
00:18:34.880 identifiers we know from our controllers and with that we get the list of
00:18:40.480 possible values for the controllers if it doesn't match this we can see to see
00:18:45.960 if it matches the target attribute and if it does we can fight the right controller and if you have the right
00:18:52.440 controller we can return all the targets it defines and with that we get an
00:18:58.200 autocomplete like this where if you have the controller um attribute it gives us
00:19:03.840 all the possible identifier names and if we Define data hell Target attribute it
00:19:10.039 gives us all the targets that are valid on our controll room and with this you
00:19:15.400 shouldn't be making any typers anymore because you can just autocomplete this and it knows so okay this was the basic um idea
00:19:24.159 for I wanted to prove so like after a long night of like working on this I committed this up had
00:19:29.840 like initial commit and just let it sit for a few days so a few days later I was
00:19:35.960 trying to see if this is actually something I want to work on more and I just sent kind of a demo video of what I
00:19:42.840 showed you just now to my friend and he was pretty much saying yeah please work
00:19:48.559 on this more take my money I want to have this more in now my daily use so I
00:19:54.919 thought what else can we do we can extend this we can make this even better
00:20:00.760 so the other question here now is can AI all do that right now in today's day and
00:20:07.400 Ag and I would say maybe but I think there are different use cases here so I
00:20:15.440 feel like AI is great for boil plates you can generate a lot of code really quickly but if you want to actually have
00:20:22.720 accurate results and actually know that it is right you want to have LSPs
00:20:29.000 adding to that the context window of llms today are still very limited and what we have here are like
00:20:36.200 this location of the file so our simus control sits here our View files are over here and it doesn't really
00:20:42.640 understand the relation between those so it's really not that great for that use case but even if AI could do all of this
00:20:50.039 what we just chose you now which would be great so I don't have to do it anymore so that's good we still want to
00:20:55.760 know about Diagnostics and kind of inspiration for this was in Ruby
00:21:01.440 itself so whenever you use Ruby and kind of mistype a method it always gives you
00:21:08.159 this did you mean suggestion of what you could have meant what actually right so this kind of inspired this new um
00:21:14.960 approach I had to say if we see you misspelled something and we don't recognize this value we can give you a
00:21:21.960 suggestion of did you actually mean this controller and um give it to the
00:21:27.200 user and you we can do this for the controllers we can do this for the actions we can do this for targets we
00:21:34.520 can do it for pretty much everything even for if you're using values and you are mismatching the types between what
00:21:40.679 do you defined in your controller and what do you have defined in your HTML so what else can we do if we
00:21:48.679 already know something is not right but could be right we could kind of guide
00:21:55.760 the user to do the right thing and give it a them a code action to fix it for
00:22:01.200 them so when you see we have this controller it doesn't know about it but it could say did you mean Hello we could
00:22:08.400 just say replace this value with the right thing that we actually meant or if you're prototyping prototyping something
00:22:15.440 and you're defining a controller that you don't exist yet you can ask the LSP to create the file for you for the right
00:22:22.320 name at the right location so you don't have to think about all the naming and all the other things that go with it so
00:22:29.080 just gives you some ways to work with that and now fast forward a few years to
00:22:35.080 October last year where I felt comfortable enough releasing this whole thing this was all kind of um sitting in
00:22:42.240 my um work in progress but last year I was finally happy with what I had and I
00:22:48.720 was able to release this at my rails World talk last year and thanks to the LSP protocol we can just ship with all
00:22:57.360 these compatible editors so we don't have to do anything more it just works with all of them so if you aren't using
00:23:03.200 it today I would highly encourage you to give it a try and if it doesn't work for you we can see if we can improve
00:23:10.360 it so I released this thing I asked for feedback for the community and I got a
00:23:16.080 lot of awesome feedback and new ideas and I also started to see that some of these approaches we were taking
00:23:21.960 started to fall short especially the ones that were ready to parsing I'll be parsing
00:23:29.159 JavaScript because there's a lot of um syntax in JavaScript and with all the es ecmascript versions five six and now
00:23:37.360 with all the version numbers we didn't really support all of that so we had to
00:23:42.600 switch out the parser and make it work so that was one learning the other thing was if you remember from earlier this
00:23:50.080 glob we had we just hardcoded this string of where the controllers are but it turns out not all the projects have
00:23:57.440 the controll at these location so we have to make this more Dynamic so we could actually detect where these
00:24:02.760 controls are then we had some issues with the identifiers we kind of can always assume
00:24:09.760 that the file name is actually the same as the control identifier because you could rename them you could register
00:24:15.760 them twice you could do some weird things with it so we had to account for that and the other problem was the way
00:24:23.640 you are um initializing instantiating all of your applications so this example
00:24:29.559 here is from the most recent version how it is in default in rails today it uses the import Maps the simus loading um
00:24:36.240 version and it just e loads all of these controls but if you have been using rails for a while and there you know
00:24:43.120 there are some Legacy applications too you might have seen some of these bundlers some of these approaches of how
00:24:49.720 to build JavaScript and if you're building a tool like this you have to support pretty much all of these setups
00:24:56.200 because people are using them and people wouldn't be able to use this tool if it wouldn't support that so we have to make
00:25:02.679 sure that it kind of works with all of these setups and I think it does today
00:25:07.960 so if you can break it in any way in your application please let me know and I'll try to fix
00:25:13.200 it and the thing is technically the language server works it's actually
00:25:18.240 doing its job but if you if you can really parse some of the parts of it you
00:25:23.600 get a lot of inaccurate results and false positives which are Super annoying to work with because you just want this
00:25:29.559 tool to help you not to be in your way so as a use this is super annoying and that's why we focus on
00:25:37.279 addressing all of these issues and fixed most of them I think it's in a good shape today and all of that kind of lets
00:25:44.279 to rebuild this parser from scratch so it's not kind of what I showed um today
00:25:49.360 in the slides anymore it's a little bit more complicated to support all these setups but what we get today um for that
00:25:57.720 though is a more accurate and more um reliable partial that just works for all
00:26:02.760 these setups so if you fast forward to 20124 early this year I was able to release
00:26:09.360 this parser and with that I was also able to release the stimulus LSP 1.0
00:26:14.520 version where I felt like this was super um reliable this works for most cases so
00:26:20.200 it all support for typescript for inheritance for importing um controls
00:26:25.440 from MPN packages and so much more so all of that I felt comfortable shipping
00:26:31.000 it was the 1.0 release and that's kind of what you can have today so what's next for this um
00:26:39.360 recently Ruby mine added support for stimulus too so it's kind of cool to see
00:26:45.320 all these other editors catching up and adding support for that so it just means there's something to it there's
00:26:50.760 something right about doing that and putting in all this effort the other thing is all these
00:26:57.520 Diagnostics I show show you earlier are kind of locked into the editor now or into the LSP now but it doesn't have to
00:27:03.440 be we can kind of extract that out and make it not exclusive to the LSP so we can kind of run these
00:27:10.279 Diagnostics independently you can think of it kind of like a tool as ruop so you can just run this tool and it will go through all
00:27:17.080 your files as like a CLI and that's kind of what I want to do I want to build a stimulus lint package
00:27:22.600 that you can run independently from the LSP and it will give you all the errors and Diagnostics as you I showed you and
00:27:30.480 you can run it as part of your Ci or the local setup to just make sure that you catch these errors early so you don't
00:27:35.720 kind ship them to production the other thing is that we
00:27:40.760 don't really support HTML and Erb super well especially if you're using the
00:27:46.279 rails tag helpers for generating some of this in Ruby which is outputting EB so
00:27:51.960 oh HTML taxs so this doesn't quite work today but we are working on this as well
00:27:57.480 you want to for the same Diagnostics and code action in Erb so that's kind of coming as
00:28:03.200 well and related to that I was working with turbo LSP which has super basic support now
00:28:10.159 it's also available already but it's doesn't do much yet just because it has
00:28:15.200 the same problem um turbo makes heavy use of these Erb helpers too so we have to properly understand turbo uh um Erb
00:28:23.240 and rail tack help us to make this more um useful
00:28:28.679 so if you don't really understand Erb we can really provide any more intelligence as we what we have today so we have to
00:28:34.440 make that uh work first to make it happen so in the LSP for stimulus
00:28:41.159 understands HTML and Erb documents but it doesn't understand Ruby that outputs
00:28:47.679 HTML and I kind of showed this in my slide last year at the race World talk to I had this um thing up here that you
00:28:55.320 want to support this as soon as Ruby LSP support e and that kind of sparked this whole
00:29:01.279 conversation with the Ruby LP Creator and we kind of were talking about it back and forth and what that might mean
00:29:07.000 for that so last year at the Ruby day Ruby confa day we worked on it together to
00:29:14.080 make Erb support happen in a ruby LP so we put we kind of sat together and work
00:29:20.080 we working on this to just see what it takes to add a basic implementation of
00:29:25.200 that we quickly realized that it doesn't isn't that straightforward as you thought um as usual in software
00:29:32.159 engineering so there's that but the approach we were taking is something like this so if you have this um markup
00:29:40.039 here we were just ripping out all the HTML attributes around the Ruby so we
00:29:45.480 could kind of just putting this mask over it removing all the attacks around
00:29:51.120 it so it just left with the Ruby codes and if you just have Ruby codes you can
00:29:56.440 also use an existing Ruby parer to Parish this code and that's kind of what we did we just stripped out the HDM
00:30:03.159 around it just have the Ruby left and then we use that to um build the basic
00:30:08.640 integration for earb files this shipped thanks to Stan earlier this year and if you haven't
00:30:16.240 been using the ru LSP I high encourage to check this out too they have this re awesome really awesome aticle from a few
00:30:23.000 months back where they kind of summarized what they have been doing in the ru LSP but coming back to this
00:30:29.840 problem this worked for the basic support but in this process we kind of lost the HTML context and it's kind of
00:30:37.240 important to know where you are interpolating in Erb documents so if you look at this here if you just
00:30:42.799 interpolate a string at the top of the document it doesn't really matter but if you interpolate in between tags it might
00:30:49.720 have some security implications because you could insert some JavaScript some uh vulnerabilities
00:30:57.519 you just have to be careful where we are interpolating but if we just RP out all the HML context around it we don't
00:31:03.720 really have any sense of where we are interpolating so we have to know about where you are
00:31:09.320 interpolating and that's kind of f you want to support HTML with better Ruby more also because you can pretty much
00:31:17.000 use earb for whatever you want right it's just like a way of interpolating something in a text document so you can
00:31:23.080 use EB for generating emails or for generating other things which aren't
00:31:28.799 HTML but what we need for making this better is a HTML HTML earb parser and
00:31:36.960 I've been experimenting with this just so we can improve the Erb support in all
00:31:42.000 of these tools possibly more but what it kind of tries to do is to understand
00:31:48.240 syntax like this where you have tative where you have content tag where you're using Dom ID where you're using
00:31:55.360 conditionals where you're using tag to diff with the data syntax in the hash we are using
00:32:02.159 T um p in this example for and passing the block or where you have loops that
00:32:08.039 generates a lot of HTML around it that kind of leads me led me to think
00:32:13.399 that we need a better extraction for this so these two examples are seen
00:32:19.760 equivalent because in the end they are this will just output A P tag the same way as if you would write it on the left
00:32:26.519 side but for us to implement a tool on top of that we don't really care if it's
00:32:31.639 the left or the right we just care about if it's going to be a ptag or not and
00:32:36.960 that's something we just want to try to extract to make easier to work with with this parser similarly if you have this these
00:32:44.120 two examples here these output the same Ruby syntax in the end or the same HTML syntax it's just a way different way of
00:32:51.240 writing it but since these vales are all static we can figure this out um
00:32:56.480 statically in a um in the posture so all of this need a little bit more
00:33:01.880 exploration but I feel like there's something to it to make all of these tools way
00:33:07.039 better and I think if you do this the right way you might also be able to support Hammer slim or other temp
00:33:13.559 languages more easily because you just need to understand how these syntaxes
00:33:18.960 translate into HTML and if you can do that we can build support for all of
00:33:24.200 them and if you do this again the right way we might be able to to extract a language service again kind of what we
00:33:30.679 did earlier with the HTML one we just used it and were able to build something on top of it very easily but if you can
00:33:36.639 expose this for the wider community on how you might be templating in HTML we can provide these services and
00:33:43.799 then everybody can just build tools on top of it which is super awesome and one more thing which I want
00:33:50.679 to highlight here is the way we are rendering files partials views
00:33:56.559 components because I think we can also power more Advanced Diagnostics here especially for
00:34:02.919 stimulus so if you think about this example here where you have a controller at the outer level and some targets
00:34:09.919 within it you have to know that you have to put the target inside this element for it to work if it's outside of it it
00:34:17.320 doesn't work they give you any hints so what we want to do is we want to say if
00:34:22.639 you use this target outside of this element it's not valid we want to show this to you
00:34:29.159 so this we can do today already that's kind of doable but as I said the problem
00:34:34.280 is today modern rail apps have components have partials have multiple views and they are they might be split
00:34:41.440 apart so you have one file here the other file there the target is defined here the control is defined over there
00:34:47.679 and we have no connection between these files but if we know that this is Rend call is rendering this view file we can
00:34:55.200 put them together kind of stitch them together again and give you more Advanced Diagnostics even across files
00:35:01.839 and that's super powerful for the US stability aspect of um these
00:35:07.000 tools so we have to build understanding into the par to our
00:35:12.040 rail surrenders these View files so if you kind of do this here
00:35:17.640 where you have the controller where have render the file outside of this element we can actually highlight this
00:35:23.720 to you and say there's something wrong here you are using this target outside of the control element and that's
00:35:29.680 something I'm really excited about that's super helpful just because I spent I lost so much time debuging stuff
00:35:35.680 like that didn't realize it's not the way you think it is and that's if the
00:35:40.800 edit just tells you at um the way at the time you write it you're so much more
00:35:48.079 efficient so that's why I think a advanced as yearb par could prove super helpful it could power these tools and
00:35:57.040 while I was re in some of this I found some of these gems even could make use of that too to make their lives easier
00:36:02.760 on how they parse HTML and drb and possibly we can um build a new
00:36:09.400 formatter for formatting earb on Save really quickly there was a HTML format
00:36:15.720 gem for EB but I never got it to work for my setup as well so I think that
00:36:20.760 could something we can build something which also works quite nicely without any
00:36:26.079 setup and I don't know maybe possibly we can even build a more advanced earb engine if we actually understand earb so
00:36:32.119 much in depth that we can build something more um better for the modern
00:36:37.760 day as well similarly if you have been following the Ruby um part of it prism
00:36:44.680 has had a big effect on Tooling in that landscape and it just shipped with the
00:36:51.560 Ruby 3.4 pre-release a few months ago or not last month actually so that's had a
00:36:56.920 big effect and I think if we kind of do the same thing we could have a similar effect on
00:37:02.359 the Erb Tooling in general across rails and Ruby but maybe even in the v web
00:37:07.960 community of how you might be dealing with um statically rendering those um
00:37:13.839 View files okay that's a lot I know so I have
00:37:19.400 one more thing to show which is a little bit more visual so we have been working on a hot via browser extension as well
00:37:25.599 for you to kind of inspect stuff that happen happening in the browser this kind of all sparked from
00:37:30.800 this um conversation on Twitter where Matt Swanson was showing that he had a CSS hack for highlighting turbo frames
00:37:38.800 in um his application so you can see the blue outline and the frame with like the
00:37:43.839 name next to it and we just thought it's kind of annoying to copy this around and
00:37:49.040 that's why can bundle it up into a hot fire browser extension so you can download this today too it works for all
00:37:55.200 the major browsers if it doesn't please let does know um it gives you this panel
00:38:01.119 where you can disable some of these features um by yourself kind of what you care about and what it gives you is also
00:38:08.200 like outlines for stimulus controllers of where they are defined so the green outlines here are steamless controllers
00:38:14.560 you can see where they have um effect in the page and what what they um what they
00:38:21.400 do and if you kind of have this enabl and browse the web you might also realize that like overflow is using
00:38:27.520 stimulus for some reason and you can see here they are using um stimulus for
00:38:33.480 pretty much all the interactive pieces on the page for uploading for following for sharing for searching which is kind
00:38:40.319 of interesting and you can also see in the bottom right the control names of the these controls they are using for
00:38:46.800 powering this so you can just see and expect what's happening on a website that uses stimulus or
00:38:52.720 Turbo if you have been dealing with forms and turbo it's been you might have
00:38:58.319 spend some time there too so we kind of built in some event logging so you know what's going on in a sequence of use sub
00:39:04.599 meting of war in Turbo that's kind of useful and I have some sneak peeks here to share too which isn't released yet
00:39:10.960 but um we have been trying to visualize turbo frames and which parts of the page
00:39:17.480 influence how a frame is being updated or rendered so in this case here you can see all the TS on top um are controlling
00:39:25.400 this frame in the middle so even GitHub uses turbo frames for most of their interactions and all these buttons
00:39:30.839 control this frame in the middle to advance or navigate the page but as you can see here on this um
00:39:37.960 screen it gets a little messy if you have too much going on so we have to figure out a way to make this more I
00:39:43.720 guess uh easy to navigate if you have a bunch going on
00:39:49.200 and something else we've been working on in stamus reflex which isn't happening
00:39:54.560 in Turbo right now yet but we had this outline on the page where you had like
00:40:00.920 if you click on this button it would rerender this part of the page it would rerender these views and partials and it
00:40:08.079 would update that part of the page as a reaction to it and if you have all of
00:40:13.119 that kind of um put together and have it all in um the browser and the editor I
00:40:19.040 think you really have a good time building mod um V and rail
00:40:25.960 apps y one more thing we had here was the debug bar for Turbo streams they are
00:40:32.440 usually super quick to execute so sometimes it's super hard to catch if something actually happened or if it
00:40:37.680 actually executed or what it did so we want to build in a debug bar so you can pause the execution of Turbo streams
00:40:44.839 just so you can insect what's happening if it goes too quickly and the other thing we had was a
00:40:52.240 Reload button for Turbo frames if you are kind of iterating over or something you just click this button and refresh
00:40:58.599 the page I have now have lost some time doing that manually on the console so I just thought we should add this to the
00:41:05.680 frame directly there's so much more we have ideas about if you have any more ideas
00:41:11.800 to open issue on the repo and we might take a look and see if you can make it
00:41:17.400 happen as I said it's all work in progress so if you can break it please let us know and we'll try to fix
00:41:25.200 it so yeah to con clude here I think the whole developer experience and develop
00:41:31.720 argon matters I think we in the Ruby Community have been kind of putting this to the side a little bit we always
00:41:38.119 thought we have this Head Start of all the other um ecosystems but now I think they all
00:41:44.160 catched up or even overtook us and I think we have to kind of follow up now and have answers to these questions
00:41:50.800 too and I think these are the kinds of tools we have been missing to make it more productive and more fun to work
00:41:56.480 with and I think there's so much potential to level up even more than what I showed today so I personally
00:42:02.680 super excited to see this tool come to um life and I also think that these
00:42:08.599 tools are what make a language or framework stay relevant or if you kind of fun put the
00:42:14.560 other way if a new person comes to our ecosystem they expect these tools to be
00:42:20.880 here otherwise they feel weird if they are not they just feel like oh they don't have any tools something is
00:42:26.079 missing something is weird about this language and that's not the feeling I want to kind of give new people that are
00:42:31.599 join our community so I want to help improve this and make this happen as said we've been lacking behind
00:42:37.240 I think because I want to keep building rails and hotway
00:42:42.400 ups and for that to be a future um I am
00:42:47.480 trying to build these tools so if any of this is interesting to you and you want to follow up more of
00:42:54.200 what I showed today I've been running this Hotwire week newsletter for the past year or so and it's a newsletter focus on
00:43:01.800 Hotwire it kind of delivers every week on Sundays it just gives a highlight of what's happening in the world give
00:43:08.280 updates on um stuff youve building on hot.io and yeah if
00:43:13.760 you want to subscribe you can do this on hotfire weekly.com we recently had the
00:43:19.079 one year anniversary too so it's been going for a while so if you want to follow along online we have the socials
00:43:26.079 here or come and all I have to say now I want to stay in for a more productive hot ecosystem and for a more unified hot
00:43:34.000 ecosystem so that's all I have today so thank you so much for having
00:43:39.599 me thank you it was like a really really great talk to be honest thank
00:43:46.160 you like does anyone have any question I have a question about the
00:43:53.559 turbo stream deog bar sure yeah um so you said it is also like an in progress
00:44:00.960 uh project right is it like an extension like browser extension or how that works
00:44:07.880 yeah it's a browser extension so you can kind of install this um in your browser I mean it's not live today this one not
00:44:13.760 yet but it will be in the same package so you can enable the toggle for the browser H for debug bar and it will give
00:44:20.280 you this um uh floating toolbar you can kind of
00:44:25.400 push around on your page and then stop the execution for the turbo
00:44:32.880 streams I assume you had some issues in the past too so like where it goes too
00:44:38.079 quickly to see what's actually going on yeah yeah this would be very useful
00:44:44.800 when I was having some issues right um definitely yeah most of these tools and
00:44:52.079 and the features are inspired by the struggles I had in the past with dealing with um debugging these issues so it's
00:45:00.640 not um something I just thought about it's something that actually I spend some time debugging and thinking this
00:45:07.599 could be better okay and hi I am NS from E cloud
00:45:15.440 and I have another quick question sure and I think the hot fire generally used
00:45:21.480 by rails application what do you think about how how it is our hard to use the Hotwire
00:45:29.079 with the other Ruby we Frameworks other than the rails if it's hard
00:45:34.319 or is it hard or how is how is it hard I think it's so much straightforward so
00:45:40.000 all you pretty much have to do is you have to um set up the JavaScript part so
00:45:47.160 that's kind of I guess the most um challenging part but as soon as you can start the simus application and you can
00:45:53.119 um import the turbo uh um start or yeah s commands you should be good it doesn't
00:46:00.280 require any more than that and most of the um turbo interactions just expect that you return
00:46:06.240 a full HTML page and that's what most Frameworks already do so if you are visiting a new page with turbo uh too
00:46:14.040 drive it will just fetch the page update all the um the insights and that should
00:46:19.359 just work the only problem that you might have is um um with the action cable and
00:46:26.480 gu the we sockets with the updates live updates where you need some server side
00:46:32.800 um uh counterpart to make it work but as long as you just kind of care about
00:46:38.240 simulus Turbo Drive turbo frames and turbo streams without the
00:46:44.079 broadcasting it should be super straightforward to get started yeah makes sense thank you you're
00:46:51.440 welcome are you using it for any specific framework or yeah we are using Roda R yeah from the Jens yeah I was
00:46:59.920 just talking to him last week and I was asking him how he's managing JavaScript in um R
00:47:05.839 apps and yeah he didn't say he said like he doesn't have any um specific setup so
00:47:11.680 I guess as long as you can somehow just vendor the JavaScript part
00:47:16.720 and import it in your head HTML tag um that just just work out of the box yeah
00:47:23.200 yeah yeah actually the Jeremy evens just joined our company two weeks ago so I
00:47:28.640 think he probably asked for that um are you working for um UB Cloud right yeah
00:47:35.960 cool
00:47:41.200 awesome I have one uh like when you were basically started did you have to
00:47:47.480 do any like TW weeks here and there for different LSP
00:47:54.040 clients or was it all covered with the protocol there the only thing I had to do was um vs code so it requires to you
00:48:01.920 to wrap it in a small um yeah in a wrapper where you
00:48:07.839 just start the um language server from the clients and you have to publish the
00:48:12.880 marketplace so people can find it but that's all it took kind of just a small
00:48:18.720 wrapper to start um the server and that's all it is yeah and I guess you need some model set
00:48:25.160 up a little bit more for I guess V neovim where you just have to say here's the server executable and here are um
00:48:31.839 the initialized arguments but that's all you have to do that to just kind of set this up once and then you should be good
00:48:37.680 good to go I'm a nail user so I know the LSP
00:48:43.160 pains right yeah there's a um we added um config to LSP config for Nim so if
00:48:52.280 you use that you should just be able to enable stimulus LS in your array of
00:48:58.160 metal piece and that should just do it automatically
00:49:03.400 yep any more
00:49:13.680 questions okay I guess then um well thank you first of all like joining us
00:49:20.480 and having this great uh talk it was really good thank you thank you for
00:49:26.240 having me yeah yeah and maybe another topic on another week sure
00:49:32.799 yeah thank you for this great tools thank you for your thank you support for
00:49:39.160 the you have if you have any questions or any kind of um requests or are struggling with
00:49:46.160 something please please feel free to reach out on any of the channels and I'm happy to
00:49:51.599 help thank you
Explore all talks recorded at Ruby Türkiye Meetup
+3