The Rise of JS and the New Server-side

I started writing this article in my excitement of Server-side Javascript (SSJS) and how I’ve become increasingly interested in Javascript (JS) over the past years. But at the end of the day, Javascript is just another programming language, its importance is defined by the frameworks and applications that are built on top of it. But while I was writing this, I noticed that together with the rise of Javascript, something much bigger is happening in the way services are and need to be structured.

Javascript in the Browser

A little more than a year ago, I wrote about the grim future of Adobe Flash. I think by now, there are even more signs that Flash is dying in it’s current form. Flash got a big stab in the back recently by Youtube when they started supporting the HTML5 video embed and also Apple is once again not supporting Flash on the Apple iPad.

Javascript is getting faster and stronger in the browser. A good example is the powerful, open-source V8 Engine developed for Google Chrome. Also, mobile devices are increasingly centered around Javascript, all Palm Pre apps are written in Javascript; the iPhone Google App is pretty much a JS interface in Safari; And there are cool new frameworks out there that allow you to write native iPhone/Android/Desktop apps completely in JS.

A simple client-server web application using the LAMP stack.
An oldschool client-server scenario with a web application running on the LAMP stack.

Javascript Beyond the Browser

At the end of the day JS is just a programming language, it’s success is determined by the amount of adoption and the problems it can solve. So far most of the problems solved by JS have been in the browser. But this is about to change.

Server-side Javascript is nothing new, but it’s not common yet. There are many open-source frameworks sprouting up now and I think the rise of cloud-services like the Joyent Smart Platform will definitely help SSJS taking off. Also, there are some serious economic benefits to having JS be the only programming language used in a web application. Do you know any programmers that don’t know how to write HTML? How about Javascript? JS and it’s relatives – ActionScript and Java – are very widely known by developers. This means that finding and training JS developers is easier.

These SSJS frameworks need to accommodate very different use-cases than their client-side counterparts (jQuery, Prototype, …). A SSJS framework shouldn’t care about the DOM, but it should care about things like IO / file access, network and database connectivity, template rendering, communication with existing dynamic libraries, etc. The server-side use-cases are completely different!

The Migration to Client-side

For the past years, I have been involved in scaling a huge service called Smart.fm. Over the course of 2009 we’ve made some big adjustments in our technology strategy. We’ve been offloading more and more controller logic into the browser using jQuery/Javascript. This also means that we’ve had to beef up our API and make sure all data can be accessed smoothly using JSON feeds. This has given us huge benefits in performance, modularity, manageability and user experience. But when looking at top-sites out there like Facebook and Google, this seems to be common practice nowadays. More and more logic is now running in the browser.

A typical Rails stack application.
A typical client-server scenario with a web application using the Rails stack.

So all of this has got me thinking: what does the server-side still mean for a web application?

The controlling of the UI flow and interactions is now done by Javascript in the client. This means that access control is purely done by the API, the client will not get the data it doesn’t have access too. This means that all access control and data routing will be controlled by the API.

What about server-side rendering of templates? Surely we need this for the search engine bots, but do we really need to treat them the same as users? Search engine bots like the Google Bot need to be able to scrape server-side rendered HTML information and will not execute any client-side logic. But as simple keyword search is becoming less relevant, we need to start building different interfaces for machines. Why not render some simple HTML for those dumb Google Bots and not worry about it in our UI? Oh and while we do that, render something more semantic for those smarter bots!

Architecturally, we can define these components in a modern web application:

  • An API that serves as an interface to the business models and provides access control
  • A user application that runs completely in the browser that uses the API for access to the domain’s data, but also interfaces with a multitude of other JSON API’s (Google Analytics can also be seen as a one-way JSONP call).
  • Simple server-side rendering capabilities for delivering the user application code and for serving semi-structured content for the search bots.

So when all rendering and UI flow logic moves to the client, what responsibilities remain for the server? It is very important to ask this question when you build a new server-side framework. What problems does it need to solve? Most server-side web frameworks are built around the increasingly irrelevant Model-View-Controller (MVC) pattern.

A Next-generation Architecture

Now that the responsibilities of both the client and the server have changed, we need to build new frameworks accordingly.

An example next-gen architecture

The browser, technically known as a ‘user agent’, needs a framework that takes care of all the UI flow. This means that we need the MVC pattern running on the client. This ‘user agent framework’ needs controllers that can render views and it needs a comprehensive routing system based on anchors.

As a business, now that more of your code (a.k.a. intellectual property) is running openly in your browser, where does your strategic advantage come from? Well, it comes from making your backend very intelligent.

The server, who I like to call ‘machine agent’, needs a very solid Application Programmer Interface. The sole purpose of this API is interfacing with different user interfaces (web, iphone, ipad, …). However, it should have a separate facility to deal with other ‘machine agents’ like Google Bots. Other servers that interface with your system have very different needs than third-party user interface applications. An example of this would be API calls that perform bulk operations (like Twitter’s mass-following call).

In order to deliver smart services that can compete, machine agents need to be able to analyze the state of their internal universe (business models) and act on them. This could be a simple iPhone push notification that gets sent after a certain event happens, or it could be after the completion of a statistical analysis. Web backends need to become more event-driven (like Javascript) because a simple cronjob will no longer suffice.

Machine-agents need to be able to integrate with other services. Sometimes this means that they need to be able to fetch many feeds from a single service. I consider this to be one of the hardest technical challenges of most projects I work on. For example, if I want to do something semi-intelligent with someone’s Twitter activity, I will need to first fetch all his data, import it and then analyze it. This is very expensive and I haven’t found any economically viable solutions yet. Of course no one wants to passively fetch feeds and then analyze them, it would be much better to do something like server-to-server long polling (e.g. listening for Tweets with a certain keyword), but sometimes you have no choice: you have to aggregate.

Conclusion

The use of Javascript on websites has been unstoppable and now it has spread to other platforms like the Palm Pre and iPhone. Javascript is a programming language that’s here to stay.

It is now common practice for web application developers to off-load a lot of work to the browser. This migration to the browser together with the changing technology landscape requires us to re-think web application architectures and frameworks.

This entry was posted in Uncategorized. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.
  • About this Blog

    Hi, I'm Dominiek. I'm the CTO & Co-founder of Bottlenose.com. This blog is about Technology, Innovation and the Web.