What is server-side testing?
Over the past decade, A/B testing has evolved as one of the most popularly used strategies for growth by digital marketers, product managers, and UX designers alike. By enabling them to determine winning variations of critical web pages on their digital properties, it empowers them to optimize their experiences and in turn, their conversion funnel based on data-backed consumer insights to deliver supercharged growth.
Server-side testing is a method of A/B testing wherein the variations of a test are rendered directly from the webserver and then premeditatedly sent to the visitors’ device. Implementation directly on the server allows you to run more sophisticated tests that might otherwise hamper user experience if implemented on the client-side. In addition to that, server-side testing is also opted for in cases when it is simply unfeasible to experiment on the client-side. For example, testing two different product recommendation algorithms for an eCommerce website.
Put differently; server-side testing unlocks the door to a whole new world of experimentation possibilities by allowing you to test deep within your tech stack and run advanced tests that are beyond the scope of UI or cosmetic changes. Since all the heavy lifting occurs at the server level, the user experience becomes significantly smooth and fast.
More importantly, server-side testing allows you to look beyond your website and optimize even your product, mobile app, and IoT devices’ experience. Hence, empowering product and development teams to seamlessly build truly revolutionary, experience-optimized products.
Server-side testing vs. client-side testing
Consider the following situation. An eCommerce store owner, with an aim to optimize for better conversions, wants to test two different checkout flows on both their website and mobile app at the same time. To do so, they need an advanced testing methodology, which is more inherent to the process and integrates with the website to ensure a seamless user experience. Neither of which is possible by simple A/B testing on the client-side, and that is why server-side testing is required.
As opposed to client-side testing which involves delivering variations of a webpage that rest on the visitors’ browser (client), server-side testing allows tests to be run and modifications to be made directly on the webserver and then render the same to the visitor’s browser or mobile app. Both have varied scope and serve diverse business needs. Server-side testing is more robust and is built for fairly complex tests, while client-side testing deals with simpler, UI-related tests.
Here’s a look at the core differences between the two:
|Parameter||Client-Side A/B Testing||Server-Side A/B Testing|
|Getting Started||Simple implementation; requires a single code||Complex implementation; additional code to be deployed to run multi-channel tests|
|Implementation of Tests||WYSIWYG/DIY editor required to create test variations||Each variation needs to be coded separately, no drag/drop available|
|Scope||Limited to cosmetic changes; experiments revolve around the design, placement, messaging of the key elements on a web page||Ability to test deeply; experiments revolve around products’ UX, algorithms’ efficiency, sign-up flows, etc|
|Performance||Asynchronous code can cause Flicker Effect
Synchronous code can increase page load time
|No noticeable performance impact|
|Privacy||Privacy can be compromised as experiment attributes are visible on the end user’s client||More secure as experiment attributes are safe with the company servers|
|Channels||Restricted to desktop and mobile web. Doesn’t work for app.
Cannot seamlessly run cross-channel and omnichannel tests simultaneously
|Works for both web and app.
Can seamlessly run cross-channel and omnichannel tests simultaneously
While client-side testing can help you quickly figure out and make simpler improvements to your website that can positively impact your conversion funnel, server-side testing can help you go deeper and run advanced tests to optimize your products’ entire lifecycle. Each of the techniques complements the other and helps you overcome unique roadblocks in your optimization journey. So, deciding between the two depends completely on the scenario at hand, availability of resources, business goals, and teams’ optimization maturity.
Applications of server-side experimentation
A/B test deep within your stack
One of the most fundamental uses of server-side testing is to test two or more variations of any webpage or mobile app. While this works just like client-side testing, the primary difference is in the way variations are rendered. For server-side, rendering is done directly via the server while for client-side, changes are applied on the browser. A server-side testing tool is used instead of a client-side one in cases when tests are run on complex or dynamic elements that are beyond the scope of UI or when the modifications to be made are beyond the scope of a basic visual or code editor. An example of this would be running an A/B test on the shipping amount for an eCommerce store. Since the prices shown on the screen are rendered dynamically from the server, a sophisticated server-side testing tool is required to run this test.
Evaluate your features before rollout with feature testing
With Feature Testing, you can test multiple product features as well as multiple variations of a feature until you are sure you are rolling out the best one. Server-side testing allows you to create test variations with different product features or different variations of a product feature so you can experiment with them thoroughly, measure their impact based on critical metrics and then roll out the winning variation. Most importantly, you get real-time visibility into your products’ features’ performance. For instance, if you are a SaaS company and want to launch a free trial for your product, you can create multiple variations with different duration of free-trial periods, identify the winning variation and then make it available universally.
Manage features’ lifestyle with feature flag management
Also referred to as Feature Toggle, Feature Flags enable you to turn your features ON/OFF and thereby alter user experience, without having to deploy new code. Once rolled out, you can simply control your features – enable them for some users, disable them for others, hide them for a certain period, and so on. If necessary, you can also roll out an incomplete feature and simply hide it until you are confident about making it live. From studying user response to a certain feature to turning it ON/OFF, all of it can be done in real-time.
Deliver features confidently with staged feature rollouts
Testing on the server-side allows you to mitigate risk by rolling out features in stages. You can segment your audience, make a particular feature available only to a small percentage of them, measure its performance, and systematically make it available to all your customers in progressive stages (or even create your alpha and beta launch groups). You can also roll back a feature immediately in case a bug is found, modify it and re-launch it on the go. The valuable data and feedback accumulated from these iterations can help you drive constant innovation and rapidly evolve your product features.
Server-side tagging by GTM
GTM has recently launched server-side tagging that could be important for companies to explore experimentation on the server-side.
As per the recently released server-side tagging containers in Google Tag Manager (GTM), tags can now be loaded from the server via the Google Cloud instead of the client or browser. As opposed to a typical tagging configuration, where the page’s container is responsible for sharing visitor data with data collection tools, with server-side tagging, the container runs on the server through which data is shared with collection tools.
It offers the following advantages over client-side tags:
Better performance: Decreased number of tags on the client-side results in less code to be run over the website or app and hence considerably improves their performance.
Improved security: In server-side tagging, data is first sent to the server and then to 3rd party analytics tool(s) such as Google Analytics or Facebook. Therefore, visitor data cannot directly be accessed by these tools and hence is more secure.
More control: The server container does not run on users’ browser, but instead on the server that is completely in your control.
Why should you use server-side testing?
Build world-class products with agility
By helping product managers experiment across the entire product lifecycle and architecture, server-side testing provides a giant leap towards facilitating agile product management. As a product manager, you can leverage full-stack experimentation to run countless advanced tests, figure out what’s working, generate better ideas to refine your product user experience, and deliver improved features and develop market-ready products, faster. By running advanced tests, you can better understand your product features and deliver them continuously. Not only that, with time, better experiments can help you produce more powerful and definitive results that drive fruitful decision-making.
Reduce risk while launching features
Using server-side testing, you can test your features, measure their impact, make improvements and refinements to safely roll them out in stages, instead of making them live to all your users, universally, at once. This way, you can mitigate risk, learn from inconclusive tests, faster and iteratively, and as a result, launch features confidently.
Leverage omnichannel & cross-platform optimization
One of the key competencies of server-side testing is enabling omnichannel and cross-platform experimentation. Whether it be web applications, mobile apps, OTTs, beacons, or connected devices; you can experiment deeply and infinitely. You need not design tests specifically for each platform, instead, just run the same test across platforms, channels and devices, deliver consistent experiences and record the unique user behavior across all your digital properties under one roof. This way, you can gather insights on how your users respond differently to tests on different platforms, compare data for one channel to another, and use these insights to optimize your experiences across all your customer touchpoints.
Test at scale without impacting performance and usability
One of the inevitable side effects of running a simple client-side A/B test using an asynchronous code is the Flicker Effect (also known as the Flash Of Original Content). It refers to the phenomenon wherein the original content of a page appears for a very short while before the variation loads. While there are quick fixes to minimize the time, it is almost impossible to get rid of it completely. This is harmful because the Flicker Effect not only hampers the user experience, but can also lead to a substantial contrast in the derived test results from the actual ones. Imagine if a visitor notices a particular element changing post the page has loaded. They immediately realize that they are a part of an experiment, and hence have a tendency to change their behavior on the web page.
Server-side testing, on the other hand, is immune to the Flicker Effect or any other performance-related implications; hence, you can expect accurate results while running tests on the server.
Create a foundation for seamless product delivery, continuously
Server-side testing helps facilitate continuous integration and continuous deployment (CI/CD) in the software development lifecycle. Feature Flagging allows small pieces of code to be integrated to the main software branch and then gradually deployed for a segment of the audience, and slowly for the rest in subsequent stages. By allowing product managers and dev teams to test their features in production, make modifications iteratively, and deliver them gradually, server-side testing enables an enhanced development, delivery, and improvement speed.
When should you use server-side testing?
Server-side testing is most useful in the following scenarios:
Run experiments on single-page applications (SPAs)
As opposed to static web sites, Single Page Applications (SPAs) serve dynamic content, which refers to modifying certain elements of a particular page based on users’ interaction with it rather than loading entirely new ones from the server. Examples of such websites include Facebook, Instagram, and all other sites/apps that work behind a login. Testing dynamic content is more technically complex and hence, can only be tackled by server-side testing tools. And in a complex website built on multiple dynamic elements, client-side testing might slow down the page load time as developers have limited control over the load time of the page rendered on the user’s browser. Moreover, since all the dynamic elements on your website are constructed and controlled by the web server, you can only flexibly test them on the server-side.
Confidently launch new products/features
Before the launch of a product or a product feature, there are a plethora of ideas that product managers might want to validate to ensure that the launched product is exactly what they envisioned it to be. Server-side testing allows product and UX teams to test deeper into their stack and experiment thoroughly with their product features, measure their performance and impact, validate hypotheses, and roll products and features out confidently. The data-backed insights gathered enable them to take better decisions that help launch pioneering products, efficiently.
Run tests where data security is paramount
When it comes to web pages or apps that work with sensitive data, such as eCommerce payment pages and internet banking apps, etc., client-side testing can be challenging as data security is the single most uncompromisable aspect that such businesses can’t afford to be unwary of. This is the reason many marketers and product managers are hesitant towards deploying A/B tests on such pages. Server-side testing becomes the obvious and only choice for testing on such websites/apps as it ensures testing attributes are safe internally with the server. With server-side testing, you can experiment all you want with your critical pages without worrying about any security threat to your confidential data.
Perform a website revamp or UX overhaul
In today’s day and age, usability and user experience of your digital properties play a pivotal role in determining the fate of your business. As more and more marketers, product managers, and UX professionals are beginning to realize this, periodic website revamps or UX overhauls have become an integral part of growth roadmaps of the world’s leading businesses.
In an effort to ensure that the new version of your website or mobile app offers a significantly enhanced and seamless user experience to millions of users simultaneously, businesses need to experiment at scale. Server-side testing becomes the obvious choice as it allows you to test deeply and thoroughly to identify and fix any flaws related to site navigation, architecture, or user experience.
Enhance mobile in-app user experience
Mobile app technology is vastly different from web technology. So, the exact logic for client-side experimentation on websites cannot be replicated for mobile apps. Even if you do decide to implement client-side testing on your mobile app, the result will not only be poor user experience, but also tons of UI bugs. Moreover, fixing bugs on mobile apps involves a lot more steps than on websites. You can’t simply ship a new version of the app and make it live to your users instantly like in the case of a website, but will have to wait for approvals from Android and iOS stores and then for users to update their apps. Therefore, to successfully run experiments on mobile apps, you will require a sophisticated server-side testing tool that can handle the complexity of a mobile app.
With server-side testing, you can run mobile in-app UX tests seamlessly without having the users download multiple variations of their app. This allows you to continually enhance your in-app user experience, roll out features that optimize conversion rates, and improve your core metrics such as customer engagement and retention.
How does server-side testing work?
- Server-side testing requires a one-time SDK installation. The SDK is generally available in all the common backend languages such as NodeJs, PHP, Python, Java, .NET, Ruby, and platforms like iOS and Android. Once installed, SDK can be integrated into your server application by simply requiring it. This is, again, a one-time task.
- Whenever a visitor lands on the client browser, a call is sent to the backend server, where the SDK of the server-side testing tool is installed.
- The SDK acts as an interface between the website’s server and the testing tool’s server. It is responsible for fetching all campaign-related information from the tool’s server in JSON format, such as number of variations it has, the number of goals it contains, overall traffic distribution, and variation level traffic allocation.
- These settings are then used to create a client instance of SDK which offers various capabilities like A/B testing by providing a variation to a particular user – enabling a feature for a visitor, tracking goals, etc.
- Once the variation is assigned or the goal is tracked at server-side, the data is sent to the tool for tracking purposes. This data is fetched while viewing the campaign’s report or application dashboard.
Server-side testing in VWO
VWO FullStack allows you to run A/B tests on the server-side. To get started, you first need to create a Project and a Feature. A project is representative of the goal you wish to achieve and the element you are going to test. Testing your search algorithm could be one project, for example. Another one could be testing a pricing change, and so on. A feature, on the other hand, represents certain capabilities and functionalities of your product.
Once you have created a Project and a Feature, setting up a server-side A/B testing campaign is pretty straightforward. All you need to do is select A/B test under the FULLSTACK option in your VWO dashboard and follow some simple steps to successfully run your campaign. These will typically involve selecting the relevant Project, Feature, Environment, and setting up the variations, goals, and traffic allocation (like you would for any regular A/B test). You can also schedule tests and customize traffic allocation for each variation using advanced options.
Server-side testing use cases
• Product recommendation engine
To upsell/cross-sell, it’s almost customary for eCommerce stores to recommend a few curated and relevant products at the bottom of product pages. This product personalization for every user is based on parameters such as their browsing history, products’ popularity among other users, and so on. Now, in order to find out which of the multiple available algorithms are leading to a higher increase in sales and revenue, you need to test the efficacy of these product recommendation algorithms. Server-side testing allows you to test multiple algorithms, and determine the optimal selection of recommended products that could lead to an increase in sales and revenue.
• Cart value threshold for free shipping
The ideal cart value limit for free shipping is that sweet spot between happy and returning customers and an online store’s profitability. Since shipping fee is one of the crucial metrics in influencing customers’ purchase decisions, cautiously deciding the cart value threshold to qualify orders for free shipping becomes imperative for online retailers. From an infrastructural standpoint, testing the cart value for free shipping isn’t possible at the UI-level. You need to test on the server-side to figure out the ideal free shipping threshold that leads to increased average cart value and returning customers.
• Search algorithm
You have a stunning catalog of endless products. But, how do you ensure that your visitors get to easily browse through it and never end up disappointed? More importantly, how do you make sure that they can quickly find what they are looking for? Whether or not visitors land where they wanted to and even consider making a purchase depends on how you showcase your products on the search results pages. This, in turn, depends on the efficacy of your search engine algorithm. Whether the products to be shown on this page should be stored basis their pricing, popularity, brand, reviews, or latest views; all can be determined by experimenting with your search algorithm. Since doing any of this will require modification to your existing code, along with the robustness and flexibility to test deep, turning to server-side testing becomes a no-brainer.
• Length of free-trial and other website forms
• Discounts and deals on the homepage
Showcasing deals and discounts on your homepage in order to entice visitors into quickly making their bookings on your website is an age-old tactic. However, factors such as the value of the discount, its duration, or eligibility criteria for a certain offer can’t be randomized. You need to test to determine the optimal values for all these variables that lead to increased conversions. Moreover, these values need to be consistent across every channel for a particular user. A robust server-side testing tool is needed to dynamically alter these values, randomly render them to the users, and ensure the same variation is sent to the user across every device or channel that they interact with.
Media & Entertainment
• Subscription flow
Irrespective of the number of steps involved, a video streaming site’s subscription flow shouldn’t hamper the user experience. A complicated flow might lead to an increase in drop-offs. The challenge is to ensure that the flow is organic and seamless. To do so, you need to test multiple aspects of the subscription flow such as whether or not to provide social logins, the ideal number of steps involved in the checkout process, how the features of the various plans must be displayed, etc. Since each of these elements is required to interact with the entire IT system, even the slightest modification on the client-side can lead to a multitude of implications on the backend. Therefore, experimenting with the subscription flow of a video streaming platform can only be successfully handled by a server-side testing tool.
• Paywall access
A paywall is used to restrict access to certain content on a website or app with a payment system. It limits the quantum of free content a visitor can access without a subscription. For publishers, OTT platforms, media, and entertainment websites, it is essential to optimize their paywall access to amp up their content monetization efforts. In order to do so, they need to test the ideal placement of the paywall that leads to an increased revenue while not hampering the user experience.
Testing the paywall can be done on both client and server-side. However, the client-side testing, in this case, is not foolproof as visitors can get around it and access restricted content simply by deleting their browser history or opting out of cookies. To accurately measure the impact of different variations of the paywall placement on your subscription rate, the tests will have to be run on the server-side.
• Pop-ups in gaming applications
Pop-ups can make or break users’ gaming experience and greatly affect gaming apps’ conversions. Whether they are meant for in-app purchases, rewards, or hints; pop-ups should be appealing and displayed at the right stage. There are numerous server-side tests you can run to figure out which pop-up to send which player at which particular level to positively impact your core metrics. For instance, you can experiment with a certain hint by showing it to one player at level 4 and the other one at 9 to determine if this has an impact on drop-offs. The best part is that testing on the server-side allows you to do so without disrupting the users’ gaming experience.
• Mobile banking experience
With people exceedingly moving their financial lives online, banks have started building robust mobile banking applications that offer a multitude of services in a few clicks. However, ensuring that consumers enjoy a seamless digital banking experience can still be challenging. Whether it is the sign-up flow for a loan or the credit card payment page, there are multiple elements that can be optimized to improve user experience and make mobile banking less demanding. Since data security is utmost important and cannot be compromised in any banking application, it is recommended to experiment with them only on the server-side.
Server-side testing challenges
Dependency on developers
Right from creating and running a test to making modifications after a winning variation is declared, a dedicated development team is required to run an end-to-end server-side test. This poses two challenges. First, it makes marketers, product managers, and UXers less autonomous. They can’t single-handedly run tests and optimize user experiences, and improve experiences solely at their end. Second, there is an additional investment required for hiring resources dedicated for the same. However, if you use server-side testing continuously and strategically, and make it an inherent part of your business growth, you can rest assured of a return worth the investment.
Relatively slower implementation and execution
If compared to its counterpart, server-side testing is more complex in its operation as variations have to be coded before the test can begin. Hence, implementing and executing an entire server-side test campaign can be slower than running a client-side one. However, since server-side testing is used for testing more impactful website elements and crucial product features, dedicating that extra time is immensely rewarded with deep insights into how you can optimize your entire stack, as well as UX.
In today’s competitive digital landscape, customers are the ultimate source of validation your business needs (at least the only one that matters). Test deep within your entire stack and wide across your product features to let your customers decide what works and what needs to be fixed.
With server-side testing, you need not hold back with your ideas. Even the most complex of mobile app pages or most sophisticated of product features can be optimized to fuel conversions and improve your bottom line.
To ensure your website, product, and in fact, every digital touchpoint is optimized for the best, most seamless user experience, be sure to embed deep testing in your marketing and product endeavors.
With VWO’s server-side testing feature, you’re all set to get started on your deep experimentation roadmap. So, set the ball rolling, deliver delightful experiences, and build futuristic products- with server-side testing!
Frequently asked questions on server-side testing
Server-side testing is a method of A/B testing wherein the variations of a particular test are rendered directly from the webserver & then deliberately sent to the visitors’ device.
Client-side testing is used when an optimization test involves delivering variations of a webpage that rest on the visitors’ browser. Server-side testing allows tests to be run and modifications to be made directly on the webserver & then render the same to the visitor’s browser or mobile app.
An eCommerce store owner, with an aim to optimize for better conversions, wants to test two different checkout flows on both his/her website & mobile app at the same time. As this is not possible by running a simple A/B experiment on the client-side, Server side optimization is required. This type of optimization is more robust and is built for fairly complex tests.
Server-side A/B testing requires a one-time SDK installation, which is integrated into your server application. Whenever a visitor lands on the client browser, a call is sent to the backend server, where the SDK of the server-side testing tool is installed. These settings are then used to create a client instance of SDK, which offers various capabilities like A/B testing by providing a variation to a particular user. Know more in this guide.