Should we migrate the web-app component to Lua?

[GUI]

For background, API Umbrella's architecture can be separated into 3 main components:

• gatekeeper: Custom proxy and routing. The main API gateway portion of the code. Written in Lua inside nginx/OpenResty
• web-app: Admin APIs and admin login. Provides all the CRUD administrative APIs and handles admin logins via various authentication providers (e.g. Google oauth2, or LDAP). A Ruby on Rails application.
• admin-ui: The admin user interface. An EmberJS client-side app that utilizes the various admin APIs provided by web-app.

Since we migrated the gatekeeper to Lua, it's sort of been in the back of my mind to consolidate the web-app APIs to be written in Lua too. This type of task is more in the realm of "cleanup" or "maybe nice to have," so I don't think this is particularly pressing (and perhaps doesn't even make sense at all), but I wanted to create a ticket to track this and list out some current pros/cons.

Pros:

• Simplify the code base and reduce the polyglot nature of things a bit. If all of the APIs were in Lua, that would simplify the languages and architecture a bit, so that all the server-side code was written in Lua (with the admin interface still being JavaScript).
• Reduce number of server components. We could forego running the Rails server (Puma), and the Rails background processing (delayed_job). Less moving parts is always nice.
• Reduce server memory usage. Running the Rails app uses a fair amount of memory for a component that isn't heavily utilized. It would make our stack footprint a little lighter with these APIs living in Lua.
• Speed. While speed hasn't really been an issue for the web APIs to date, Lua-based APIs would probably be faster. But these APIs aren't heavily hit with the exception maybe of the API key signup API that might see benefits under heavier load.

Cons:

• Lack of OmniAuth: This is one of the bigger current pitfalls. We offer a variety of ways to login to the admin that hook into other authentication providers, like Google, GitHub, LDAP, CAS, etc. Being able to easily leverage OmniAuth and its multitude of login strategies is pretty nice and a significant efficiency gain.

  There's just not an equivalent mature and pluggable authentication system for Lua/OpenResty that I'm aware of. We could certainly implement the necessary authentication code (in a lot of cases, it's not actually that complex), but it's nice to not have have to deal with this. And just last month, I was needing to look at adding SAML integration for the admin login, and it's certainly easier to add an OmniAuth plugin to handle that versus rolling our own.

• Mailer libraries: Another bigger current pitfall. We primarily send e-mail for the API key info. Using Rails, we have a pretty easy and configurable interface with ActionMailer. Using the same ActionMailer API, we can easily send mail via sendmail and SMTP. Or there's theoretically hooks to send via other means (eg, I've worked on the sendgrid-actionmailer library to send via an HTTPS API). All that's required to switch implementations is some configuration tweaks.

  Again, this hits upon a relative weakness in the current Lua/OpenResty ecosystem. There is a lua-resty-smtp library, but it seems to still be missing some important functionality, like STARTTLS support. And if we wanted to support sendmail, we'd presumably have to do something custom, probably utilizing lua-resty-shell.

• Lack of ORM: We currently use Mongoid as an ORM for our data in Mongo. This provides some niceties around validation and organizing our models. There's not really an equivalent ORM for Mongo in Lua. That being said, our data models are pretty simple, and we have to fight to workaround Mongoid sometimes, so this doesn't seem like as big of a deal to me.

• It's more work: Keeping things the same is obviously the quicker and easier solution. :)

[GUI]

And I guess to summarize my current stance, given some of the current cons (notably, mail support and OmniAuth), I'm more inclined to keep things as-is with the web-app remaining a separate Rails app. While I think maybe someday it might be nice to consolidate the code-base so the server-side code isn't split between languages, I don't currently see it as particularly beneficial or worth the lift.

[cmc333333]

I agree with your analysis, but will highlight one factor. The major cons you listed stem from Ruby/Rails being a well-understood, well-used framework. They create a big framework, though, with frequent security vulnerabilities and releases. One pro argument is around reducing our attack surface. I don't mean to imply OpenResty is more stable/secure/what-have-you (I'd argue newer technologies are inherently insecure), but OpenResty is essential to the current stack while Ruby/Rails is not.

In any event, I agree that there are much bigger fish to fry. However, if there comes a point where maintaining the Ruby/Rails component becomes particularly burdensome, an OpenResty rewrite seems in scope.

[GUI]

Thanks for the feedback, @cmc333333! Those are excellent points about the surface-area and what's essential in our stack.

In the very beginning, the whole admin was much more of a typical server-side Rails app (no APIs, all server-side rendering), so we were probably getting more benefit out of Rails then. But as we've transitioned to driving everything via APIs with the Ember.js admin interface, our reliance on Rails has obviously diminished (and thus, the benefit we get from Rails). I think we're in agreement that it's not worth tackling now, but I like your thought process of deciding when Rails maintenance becomes burdensome to figure out when a rewrite might be worth revisiting.

Related, it's probably worth noting that master is on an unsupported version of Rails (so to some degree, this has already bit us), but I have already tackled the upgrade to a supported version in a branch (see #259--the merge into master is pending some of the Ember and testing upgrades it's unfortunately gotten tied up with). I was contemplating some of these same issues before I upgraded Rails, but I guess right now, it still seemed like a Rails upgrade was easier than the alternatives. But that might change the next time we need to weigh a Rails upgrade.