Landscape of Backend Development
Background
It happens when I was having the regular 1:1 with the all mighty frontend developer and he shared the challenges of being a frontend developer nowadays, and also the difference between now and in the early 2000.
Framework are releasing quickly; either as new version or as an all new framework. Developers are now expected not only capable to know the basics - HTML, JS and CSS but also specialized in using framework to develop application. And here’s the thing, every framework is different and once you get over the basics and turn your focus into performance, you would need to know the framework inside-out. All takes a lot of time and effort.
And the last statement in the conversation is, the principle for backend development has not been changed and they (thebackend developer) would not need to deal with all these issue rooted from the frontend framework.
I give it a little thought as I think while his statement is somewhat true, new framework for “backend development” (I am mainly referring to the common languages - SQL, Java, .Net, Ruby, Python, PHP) are not popping up as frequent as framework for frontend development. But the way backend application develop has indeed evolved quite a lot.
And let’s see what it is.
History
Before we start, let’s give the discussion some context. The landscape of development that I talk about will be spanning from the time when the first browser goes mainstream (i.e.Netscape around 1994) until now (i.e.2018). Besides, the application I talked about would be focusing on “online application”, i.e. application that serve users on the public internet. A rough timeline is bound to each of the stage and it is by no means a hard and definite time split for each stage. You should be seeing it more from the perspective of progression.
From static to dynamic (1995- 2000)
Since the original web server developed by Tim Berners-Lee, static pages started popping up. Soon after, people would like to add dynamic element to the page and a group of people worked out an approach to execute program on the server which generate web content dynamically. Perl was one of the popular language in developing such “program”, which a more precise definition should be “script”, or commonly known as “Common Gateway Interface” (CGI). The script is sitting on the serving host and is triggered remotely through web request and return the result of the operation as a response. It opens a lot of flexibility but also exposing risks since it allows remote execution and direct manipulation of the resources underneath. Access control is conducted on the operating system level by limiting access of the user that execute the script on behalf of, or through mechanism like “chroot jail” which isolate a process from the rest of the system.
In terms of web hosting, there are not much option - either owning the server and managed it by oneself (or through services from the hosting company) or procuring web hosting services which is usually sharing web servers with other subscriber.
Sandboxing and virtualization(2000- 2005)
The interest in the Internet growth in a rapid pace, there were people seeing the opportunity which generate needs for new technology to cater for the growth.
Since 1995, a new language targeted for internet programming was developed, with focus on security, performance and architecture. This language is also promoting the idea of “write once, run everywhere” since it aims to deploy the program on different device, ranged from computer server to set-top boxes. This language is JAVA and it has helped to define a newer approach to developing and operating internet application. At 2000, it starts to pickup the momentum after the first inception since 1995.
JAVA application would first compile to a platform independent byte code (hence the support of write once, run everywhere) and executed within the java virtual machine (JVMis indeed platform dependent). This approach allows portability but also has an added benefits of tightening the security.
The security model in JAVA consists of several layer, each layer is serving as a gateway and is responsible for safeguarding a particular area of focus. It can be summarised in the following order,
- Source code schema verification
- Class level security (from where could compiled class to load from)
- API level security, for which operation does it allow to be executed
- Security management, there includes authentication mechanics (against other system) and also the security protocol that would be using (algorithm, hash key format etc).
JAVA is not only for developing application running on the backend but also as a component executed in the browser for interacting with user, in the form of JAVA applet. The same security model apply.
Unlike the CGI approach, hosting the JAVA application involve the java virtual machine, a “middleware” application server that provides the JVM container environment for the JAVA application to execute upon. What’s more, the MVC architecture pattern is widely implemented. This architecture pattern promotes the separation of business logic, flow control and business domain model into separate layer / component.
Since the application would now involve more components and in turns, require more computing resources and thus, application is usually hosted in dedicated server. Either in the form of bare metal or embracing the budding idea of virtualisation - partitioning a physical server into many “virtualised” machines. Instead of putting everything into a single machine, each component will now sit within a dedicated environment. Managing the application is now more complicated.
Centralize to distributed(2005- 2010)
Internet keeps growing after the Dotcom burst - smart phone and mobile internet are gaining popularity, website are mashing up content from other websites through RESTFul API. The demand for a faster delivery and the change in the the channel / approach on internet application results in developers looking for solution that allow them to move faster and provide support to ease the development of the current trend.
There are benefits in adopting the MVC architecture but the JAVA approach is too “heavy” - in terms of the architecture requirement and the skill set involved (installing and configuring a Weblogic middleware is not a task for everyone). On the other hand, the application development using scripting language has the benefits of increased agility (changes can be introduced without compilation, deployment and restarting of server) and development framework for the different languages starts picking up the momentum as well. The development framework provides not only a “framework” but also a methodology on how application can (or should) be developed, following the “best-practise” suggested. The separation of concern, configuration management, data access layer, error handling - these are some of the functionality which the framework can usually provides.
Unlike JAVA, the execution of these languages does not involve the setting up of application servers (e.g. Jetty,JBoss, Tomcat), which means application is easier to setup and manage. On top of this, the landscape of “Cloud Computing” has changed tremendously; more affordable (bill by usage rather than upfront investment), speedy provisioning and consume the needed components as a service. This is the first time that a developer can easily get access to a production grade environment without a heavy upfront investment (someone would argue VPS can do the same but one would need to do all the hard-wiring to stitch to the level of resilience that a cloud provider can provide up-front). Besides, company like Heroku is pushing the envelope further, that shields the underlying infrastructure which allow developer to focus on developing their apps.
Code management has some new breath as well, GIT is the new player and decentralized version control is becoming the norm. Developer can start creating local branch, committing code locally and merge back later. This provide an even bigger freedom for the developer to experiment, make changes and rolling out releases.
Application scaffolding, GIT and Cloud platforms are the main drivers on increasing the development agility and promoting application architecture design that could leverage the elasticity that these services offer. This development methodology is being summarised as the “Twelve Factors App”.
Concurrency and Serverless (2010- NOW)
The proliferation of smart phone and mobile internet increases the population of internet user. This brings up the need of concurrent access than ever and a newer approach to deal with the increasing demand is also surfacing. Programming language that focus on concurrency is gaining popularity - Golang, Scala, etc. Backend developer who is used to stateful programming will need to design their application that works stateless. They would need to change the mindset of having the code to work on one machine which would now be running across multiple machines.
The cloud platform is becoming a commodity and infrastructure can now be handled like code through the use of API, the line between development and operation is getting blurry. Infrastructure can now be provisioned and versioned through API and configuration management tools. This advancement ease deployment and streamlined the development/deployment pipeline further. If 10 deploy a day is not enough, it can now be done continuously (CI/CD).
On top of this, a newer approach to allow partitioning of multiple application on a single host, running as if they are in their own entity starts showing up. This approach is named as the container approach and each container will have their own namespace. They works without the knowing of the other container and since it doesn’t involve a hypervisor, resources is better utilised than a virtual machine. Container can also be moved across environment and this enable developer to setup a production-like infrastucture on their machine easily.
As you can see, developer no longer just need to know the programming language, framework and design pattern but getting more and more involved in system level component too.
Conclusion
No matter if you are coding for the frontend or the backend, the magnitude of changes over the years are just as enormous. It doesn’t seems to be slowing down and on the contrary, changes are happening even faster. Developer would need to know even more than before in getting the jobs done. But at least it is an interesting journey.