Server-Side Testing

What is server-side testing?

Over the past decade, A/B testing has evolved as one of the most popularly used strategies for growth by digital marketers, product managers, and UX designers alike. By enabling them to determine winning variations of critical web pages on their digital properties, it empowers them to optimize their experiences and in turn, their conversion funnel based on data-backed consumer insights to deliver supercharged growth. 

Server-side testing is a method of A/B testing wherein the variations of a test are rendered directly from the webserver and then premeditatedly sent to the visitors’ device. Implementation directly on the server allows you to run more sophisticated tests that might otherwise hamper user experience if implemented on the client-side. In addition to that, server-side testing is also opted for in cases when it is simply unfeasible to experiment on the client-side. For example, testing two different product recommendation algorithms for an eCommerce website. 

Put differently; server-side testing unlocks the door to a whole new world of experimentation possibilities by allowing you to test deep within your tech stack and run advanced tests that are beyond the scope of UI or cosmetic changes. Since all the heavy lifting occurs at the server level, the user experience becomes significantly smooth and fast. 

More importantly, server-side testing allows you to look beyond your website and optimize even your product, mobile app, and IoT devices’ experience. Hence, empowering product and development teams to seamlessly build truly revolutionary, experience-optimized products.

Server-side testing vs. client-side testing

Consider the following situation. An eCommerce store owner, with an aim to optimize for better conversions, wants to test two different checkout flows on both their website and mobile app at the same time. To do so, they need an advanced testing methodology, which is more inherent to the process and integrates with the website to ensure a seamless user experience. Neither of which is possible by simple A/B testing on the client-side, and that is why server-side testing is required.

As opposed to client-side testing which involves delivering variations of a webpage that rest on the visitors’ browser (client), server-side testing allows tests to be run and modifications to be made directly on the webserver and then render the same to the visitor’s browser or mobile app. Both have varied scope and serve diverse business needs. Server-side testing is more robust and is built for fairly complex tests, while client-side testing deals with simpler, UI-related tests.

Here’s a look at the core differences between the two:

While client-side testing can help you quickly figure out and make simpler improvements to your website that can positively impact your conversion funnel, server-side testing can help you go deeper and run advanced tests to optimize your products’ entire lifecycle. Each of the techniques complements the other and helps you overcome unique roadblocks in your optimization journey. So, deciding between the two depends completely on the scenario at hand, availability of resources, business goals, and teams’ optimization maturity.

Applications of server-side experimentation 

A/B test deep within your stack 

One of the most fundamental uses of server-side testing is to test two or more variations of any webpage or mobile app. While this works just like client-side testing, the primary difference is in the way variations are rendered. For server-side, rendering is done directly via the server while for client-side, changes are applied on the browser. A server-side testing tool is used instead of a client-side one in cases when tests are run on complex or dynamic elements that are beyond the scope of UI or when the modifications to be made are beyond the scope of a basic visual or code editor. An example of this would be running an A/B test on the shipping amount for an eCommerce store. Since the prices shown on the screen are rendered dynamically from the server, a sophisticated server-side testing tool is required to run this test.

Evaluate your features before rollout with feature testing

With Feature Testing, you can test multiple product features as well as multiple variations of a feature until you are sure you are rolling out the best one. Server-side testing allows you to create test variations with different product features or different variations of a product feature so you can experiment with them thoroughly, measure their impact based on critical metrics and then roll out the winning variation. Most importantly, you get real-time visibility into your products’ features’ performance. For instance, if you are a SaaS company and want to launch a free trial for your product, you can create multiple variations with different duration of free-trial periods, identify the winning variation and then make it available universally. 

Manage features’ lifestyle with feature flag management

Also referred to as Feature Toggle, Feature Flags enable you to turn your features ON/OFF and thereby alter user experience, without having to deploy new code. Once rolled out, you can simply control your features – enable them for some users, disable them for others, hide them for a certain period, and so on. If necessary, you can also roll out an incomplete feature and simply hide it until you are confident about making it live. From studying user response to a certain feature to turning it ON/OFF, all of it can be done in real-time. 

Here are some detailed advantages of using feature flags:

• Feature functionality verification

Using feature flags, you can ensure the progressive release of a product feature and verify its functionality. The feature flag is usually switched ‘off’ by default and is switched ‘on’ for a specific user group when a new feature is ready to be released. After assessing user feedback if you see the feature has some issues, you can switch it off and prevent any further user exposure to this feature. This way you can go back and iterate on the feature to improve its functionality. 

• Risk reduction

Feature flags allow you to roll out the optimal version of a website feature that improves your core business metrics. With this feature, you can decide to accept or reject testing ideas. Since soft roll-outs form the key foundation for feature flags, you can continue server-side testing by eliminating any apparent intermittent issues. 

• Code simplification 

With feature flags, you can integrate code into the main trunk without creating persisting branches. Whenever a feature is ready to be released, you can simply switch it on, thereby activating the code. Anyone from your development team can change the behavior of the feature without needing to update any code. This is how feature flags simplify code integration and deployment circumstances. 

• Progressive Delivery 

The beauty of using feature flags lies in the fact that you can test small changes on a small user base, and then you can scale it based on that particular feature’s impact on your KPIs. This helps you understand how your systems perform under increasing load and how users react to your new feature release. These combinedly help limit blast radius in case any issues appear. 

Deliver features confidently with staged feature rollouts

Testing on the server-side allows you to mitigate risk by rolling out features in stages. You can segment your audience, make a particular feature available only to a small percentage of them, measure its performance, and systematically make it available to all your customers in progressive stages (or even create your alpha and beta launch groups). You can also roll back a feature immediately in case a bug is found, modify it and re-launch it on the go. The valuable data and feedback accumulated from these iterations can help you drive constant innovation and rapidly evolve your product features. 

Server-side tagging by GTM

GTM has recently launched server-side tagging that could be important for companies to explore experimentation on the server-side.

As per the recently released server-side tagging containers in Google Tag Manager (GTM), tags can now be loaded from the server via the Google Cloud instead of the client or browser. As opposed to a typical tagging configuration, where the page’s container is responsible for sharing visitor data with data collection tools, with server-side tagging, the container runs on the server through which data is shared with collection tools. 

It offers the following advantages over client-side tags: 

Better performance: Decreased number of tags on the client-side results in less code to be run over the website or app and hence considerably improves their performance. 

Improved security: In server-side tagging, data is first sent to the server and then to 3rd party analytics tool(s) such as Google Analytics or Facebook. Therefore, visitor data cannot directly be accessed by these tools and hence is more secure. 

More control: The server container does not run on users’ browser, but instead on the server that is completely in your control.

Why should you use server-side testing?

Build world-class products with agility   

By helping product managers experiment across the entire product lifecycle and architecture, server-side testing provides a giant leap towards facilitating agile product management. As a product manager, you can leverage full-stack experimentation to run countless advanced tests, figure out what’s working, generate better ideas to refine your product user experience, and deliver improved features and develop market-ready products, faster. By running advanced tests, you can better understand your product features and deliver them continuously. Not only that, with time, better experiments can help you produce more powerful and definitive results that drive fruitful decision-making.

Reduce risk while launching features

Using server-side testing, you can test your features, measure their impact, make improvements and refinements to safely roll them out in stages, instead of making them live to all your users, universally, at once. This way, you can mitigate risk, learn from inconclusive tests, faster and iteratively, and as a result, launch features confidently.

Leverage omnichannel & cross-platform optimization

One of the key competencies of server-side testing is enabling omnichannel and cross-platform experimentation. Whether it be web applications, mobile apps, OTTs, beacons, or connected devices; you can experiment deeply and infinitely. You need not design tests specifically for each platform, instead, just run the same test across platforms, channels and devices, deliver consistent experiences and record the unique user behavior across all your digital properties under one roof. This way, you can gather insights on how your users respond differently to tests on different platforms, compare data for one channel to another, and use these insights to optimize your experiences across all your customer touchpoints.

Test at scale without impacting performance and usability

One of the inevitable side effects of running a simple client-side A/B test using an asynchronous code is the Flicker Effect (also known as the Flash Of Original Content). It refers to the phenomenon wherein the original content of a page appears for a very short while before the variation loads. While there are quick fixes to minimize the time, it is almost impossible to get rid of it completely. This is harmful because the Flicker Effect not only hampers the user experience, but can also lead to a substantial contrast in the derived test results from the actual ones. Imagine if a visitor notices a particular element changing post the page has loaded. They immediately realize that they are a part of an experiment, and hence have a tendency to change their behavior on the web page.

Server-side testing, on the other hand, is immune to the Flicker Effect or any other performance-related implications; hence, you can expect accurate results while running tests on the server.

Create a foundation for seamless product delivery, continuously

Server-side testing helps facilitate continuous integration and continuous deployment (CI/CD) in the software development lifecycle. Feature Flagging allows small pieces of code to be integrated to the main software branch and then gradually deployed for a segment of the audience, and slowly for the rest in subsequent stages. By allowing product managers and dev teams to test their features in production, make modifications iteratively, and deliver them gradually, server-side testing enables an enhanced development, delivery, and improvement speed.

When should you use server-side testing? 

Server-side testing is most useful in the following scenarios: 

Run experiments on single-page applications (SPAs)

As opposed to static web sites, Single Page Applications (SPAs) serve dynamic content, which refers to modifying certain elements of a particular page based on users’ interaction with it rather than loading entirely new ones from the server. Examples of such websites include Facebook, Instagram, and all other sites/apps that work behind a login. Testing dynamic content is more technically complex and hence, can only be tackled by server-side testing tools. And in a complex website built on multiple dynamic elements, client-side testing might slow down the page load time as developers have limited control over the load time of the page rendered on the user’s browser. Moreover, since all the dynamic elements on your website are constructed and controlled by the web server, you can only flexibly test them on the server-side.

Confidently launch new products/features

Before the launch of a product or a product feature, there are a plethora of ideas that product managers might want to validate to ensure that the launched product is exactly what they envisioned it to be. Server-side testing allows product and UX teams to test deeper into their stack and experiment thoroughly with their product features, measure their performance and impact, validate hypotheses, and roll products and features out confidently. The data-backed insights gathered enable them to take better decisions that help launch pioneering products, efficiently.

Run tests where data security is paramount

When it comes to web pages or apps that work with sensitive data, such as eCommerce payment pages and internet banking apps, etc., client-side testing can be challenging as data security is the single most uncompromisable aspect that such businesses can’t afford to be unwary of. This is the reason many marketers and product managers are hesitant towards deploying A/B tests on such pages. Server-side testing becomes the obvious and only choice for testing on such websites/apps as it ensures testing attributes are safe internally with the server. With server-side testing, you can experiment all you want with your critical pages without worrying about any security threat to your confidential data.

Perform a website revamp or UX overhaul 

In today’s day and age, usability and user experience of your digital properties play a pivotal role in determining the fate of your business. As more and more marketers, product managers, and UX professionals are beginning to realize this, periodic website revamps or UX overhauls have become an integral part of growth roadmaps of the world’s leading businesses. 

In an effort to ensure that the new version of your website or mobile app offers a significantly enhanced and seamless user experience to millions of users simultaneously, businesses need to experiment at scale. Server-side testing becomes the obvious choice as it allows you to test deeply and thoroughly to identify and fix any flaws related to site navigation, architecture, or user experience.

Enhance mobile in-app user experience 

Mobile app technology is vastly different from web technology. So, the exact logic for client-side experimentation on websites cannot be replicated for mobile apps. Even if you do decide to implement client-side testing on your mobile app, the result will not only be poor user experience, but also tons of UI bugs. Moreover, fixing bugs on mobile apps involves a lot more steps than on websites. You can’t simply ship a new version of the app and make it live to your users instantly like in the case of a website, but will have to wait for approvals from Android and iOS stores and then for users to update their apps. Therefore, to successfully run experiments on mobile apps, you will require a sophisticated server-side testing tool that can handle the complexity of a mobile app. 

With server-side testing, you can run mobile in-app UX tests seamlessly without having the users download multiple variations of their app. This allows you to continually enhance your in-app user experience, roll out features that optimize conversion rates, and improve your core metrics such as customer engagement and retention.

How does server-side testing work?

1. Server-side testing requires a one-time SDK installation and integration with the backend systems. The SDK is generally available in all the common backend languages such as NodeJs, PHP, Python, Java, .NET, Ruby, and platforms like iOS and Android. 
2. Whenever the code, where the SDK is integrated, is executed, a call is sent to the VWO server. This happens generally only once as the data is cached for any consecutive executions.
3. The SDK acts as an interface between the customer’s server and the testing tool’s server. It is responsible for fetching all campaign-related information from the tool’s server in JSON formats, such as the number of variations it has, the number of goals it contains, overall traffic distribution, and variation level traffic allocation. These settings are then used by the SDK which offers various capabilities like A/B testing by providing a variation to a particular user – enabling a feature for a visitor, tracking goals, etc.
4. Once the variation is assigned or the goal is tracked at server-side, the data is sent to VWO for tracking purposes. This data is fetched while viewing the campaign’s report or application dashboard.

Server-side testing in VWO

VWO FullStack allows you to run A/B tests on the server-side. To get started, you first need to create a Project and a Feature. A project is representative of the goal you wish to achieve and the element you are going to test. Testing your search algorithm could be one project, for example. Another one could be testing a pricing change, and so on. A feature, on the other hand, represents certain capabilities and functionalities of your product. 

You can simply log in to your VWO account to configure Projects and create Features in a few simple steps. 

Once you have created a Project and a Feature, setting up a server-side A/B testing campaign is pretty straightforward. All you need to do is select A/B test under the FULLSTACK option in your VWO dashboard and follow some simple steps to successfully run your campaign. These will typically involve selecting the relevant Project, Feature, Environment, and setting up the variations, goals, and traffic allocation (like you would for any regular A/B test). You can also schedule tests and customize traffic allocation for each variation using advanced options.

Server-side testing use cases 

eCommerce

• Product recommendation engine

To upsell/cross-sell, it’s almost customary for eCommerce stores to recommend a few curated and relevant products at the bottom of product pages. This product personalization for every user is based on parameters such as their browsing history, products’ popularity among other users, and so on. Now, in order to find out which of the multiple available algorithms are leading to a higher increase in sales and revenue, you need to test the efficacy of these product recommendation algorithms. Server-side testing allows you to test multiple algorithms, and determine the optimal selection of recommended products that could lead to an increase in sales and revenue.

• Cart value threshold for free shipping 

The ideal cart value limit for free shipping is that sweet spot between happy and returning customers and an online store’s profitability. Since shipping fee is one of the crucial metrics in influencing customers’ purchase decisions, cautiously deciding the cart value threshold to qualify orders for free shipping becomes imperative for online retailers. From an infrastructural standpoint, testing the cart value for free shipping isn’t possible at the UI-level. You need to test on the server-side to figure out the ideal free shipping threshold that leads to increased average cart value and returning customers.

• Search algorithm 

You have a stunning catalog of endless products. But, how do you ensure that your visitors get to easily browse through it and never end up disappointed? More importantly, how do you make sure that they can quickly find what they are looking for? Whether or not visitors land where they wanted to and even consider making a purchase depends on how you showcase your products on the search results pages. This, in turn, depends on the efficacy of your search engine algorithm. Whether the products to be shown on this page should be stored basis their pricing, popularity, brand, reviews, or latest views; all can be determined by experimenting with your search algorithm. Since doing any of this will require modification to your existing code, along with the robustness and flexibility to test deep, turning to server-side testing becomes a no-brainer.

SaaS

• Length of free-trial and other website forms

For a SaaS business, free-trial and request demo forms, among others, are indispensable. And to ensure they are able to capture just the right information from a visitor while keeping the conversion rates high, their length needs to be optimized. A very long form can lead to high drop-offs, whereas a very short one might not be able to cover all the necessary information. Server-side testing can help you estimate the ideal form length that positively impacts your conversion rate. This is because your forms are directly linked to how your database is structured. If a field is obligatory, you can’t just hide it using JavaScript, because the form’s validation will fail on the server-side. Testing the length and complexity of a form can only be done on the server-side. 

Travel

• Discounts and deals on the homepage

Showcasing deals and discounts on your homepage in order to entice visitors into quickly making their bookings on your website is an age-old tactic. However, factors such as the value of the discount, its duration, or eligibility criteria for a certain offer can’t be randomized. You need to test to determine the optimal values for all these variables that lead to increased conversions. Moreover, these values need to be consistent across every channel for a particular user. A robust server-side testing tool is needed to dynamically alter these values, randomly render them to the users, and ensure the same variation is sent to the user across every device or channel that they interact with. 

Media & Entertainment

• Subscription flow 

Irrespective of the number of steps involved, a video streaming site’s subscription flow shouldn’t hamper the user experience. A complicated flow might lead to an increase in drop-offs. The challenge is to ensure that the flow is organic and seamless. To do so, you need to test multiple aspects of the subscription flow such as whether or not to provide social logins, the ideal number of steps involved in the checkout process, how the features of the various plans must be displayed, etc. Since each of these elements is required to interact with the entire IT system, even the slightest modification on the client-side can lead to a multitude of implications on the backend. Therefore, experimenting with the subscription flow of a video streaming platform can only be successfully handled by a server-side testing tool.

• Paywall access

A paywall is used to restrict access to certain content on a website or app with a payment system. It limits the quantum of free content a visitor can access without a subscription. For publishers, OTT platforms, media, and entertainment websites, it is essential to optimize their paywall access to amp up their content monetization efforts. In order to do so, they need to test the ideal placement of the paywall that leads to an increased revenue while not hampering the user experience.

Testing the paywall can be done on both client and server-side. However, the client-side testing, in this case, is not foolproof as visitors can get around it and access restricted content simply by deleting their browser history or opting out of cookies. To accurately measure the impact of different variations of the paywall placement on your subscription rate, the tests will have to be run on the server-side. 

• Pop-ups in gaming applications 

Pop-ups can make or break users’ gaming experience and greatly affect gaming apps’ conversions. Whether they are meant for in-app purchases, rewards, or hints; pop-ups should be appealing and displayed at the right stage. There are numerous server-side tests you can run to figure out which pop-up to send which player at which particular level to positively impact your core metrics. For instance, you can experiment with a certain hint by showing it to one player at level 4 and the other one at 9 to determine if this has an impact on drop-offs. The best part is that testing on the server-side allows you to do so without disrupting the users’ gaming experience.

BFSI

• Mobile banking experience 

With people exceedingly moving their financial lives online, banks have started building robust mobile banking applications that offer a multitude of services in a few clicks. However, ensuring that consumers enjoy a seamless digital banking experience can still be challenging. Whether it is the sign-up flow for a loan or the credit card payment page, there are multiple elements that can be optimized to improve user experience and make mobile banking less demanding. Since data security is utmost important and cannot be compromised in any banking application, it is recommended to experiment with them only on the server-side.

Server-side testing challenges  

Dependency on developers

Right from creating and running a test to making modifications after a winning variation is declared, a dedicated development team is required to run an end-to-end server-side test. This poses two challenges. First, it makes marketers, product managers, and UXers less autonomous. They can’t single-handedly run tests and optimize user experiences, and improve experiences solely at their end. Second, there is an additional investment required for hiring resources dedicated for the same. However, if you use server-side testing continuously and strategically, and make it an inherent part of your business growth, you can rest assured of a return worth the investment.

Relatively slower implementation and execution 

If compared to its counterpart, server-side testing is more complex in its operation as variations have to be coded before the test can begin. Hence, implementing and executing an entire server-side test campaign can be slower than running a client-side one. However, since server-side testing is used for testing more impactful website elements and crucial product features, dedicating that extra time is immensely rewarded with deep insights into how you can optimize your entire stack, as well as UX.

Top 5 server-side testing tools you can try

To launch sophisticated changes on your website, mobile, or even IoT devices, you must rely on a server-side testing tool that allows you to test deep within your tech stack. If you want to build an experimentation strategy aligning with changing customer behavior, choose a robust tool that takes your product experience to the next level. Find here the top 5 server-side testing tools that can meet your needs. 

VWO

VWO FullStack allows you to test anything, be it a subscription flow or a search algorithm, to uncover more conversion opportunities while delivering matchless user experiences. With minimal impact on loading time, VWO eliminates the risk of flickering altogether. Further, it supports 9 SDKs and is adding more so you can explore the one most relevant to your needs.

 

Projects

Organize multiple projects to manage testing of different application parts with VWO FullStack’s Project. Using this feature, you can choose an SDK language according to the code that best suits your application. Additionally, you can configure different code environments, such as  – staging, QA, and production, where you wish to run a particular campaign. 

Feature rollouts

Also known as Feature Toggles, this capability helps you make a feature available to a set of users or hide it from them by simply switching it on/off. Turning features on or off for a set of users can be done in real-time without having to deploy a new code. Moreover, with this feature, you can decide the allocation of traffic percentage you would like to include in a campaign. 

Feature tests

Can’t decide which variation to roll out that can help meet your conversion goals? VWO FullStack lets you test multiple feature variations with different feature variable values and understand their impact in real-time on user experience by measuring the performance of key metrics.

Traffic allocation and user whitelisting

Split traffic among multiple versions of your website or mobile app and decide what percentage of visitors should be included in your experiment. Want to ensure specific user segments are served only a particular variation? You can do that using VWO’s user whitelisting feature.

Targeting and mutually exclusive

Increase the accuracy of your A/B test by targeting a specific audience with custom segments, which you can save for future uses as well. Also, Mutually exclusive groups ensure that there is no audience overlap between tests intended for the same audience. 

Intuitive reporting and segmentation

Rich reports powered by VWO SmartStats will help you reach statistically significant results, thereby enabling you to roll out the winning variation to all users. Further, you can slice and dice your report to dive deeper into how each customer segment converts and gather learnings for future tests. 

Request a demo to explore the server-side flexibility of VWO FullStack.

ABTasty’s Flagship.io

Flagship empowers non-technical teams to design feature variations simply through a UI without facing any technical hassles. With the flip of a switch, you can expose your user segment to a particular feature. 

• Not only can you measure outcomes from the platform’s statistical engine, but you can also rest assured of flicker-free tests on all apps and websites.  
• Create and manage feature toggles of any type – String, Number Boolean, Array, and Object. 
• You can set deployment intervals to release features over time and monitor users’ reactions to them by tracking associated metrics. 

Kameleoon Full Stack

Cross-platform-ready SDKs and flexible release using feature flags make Kameleoon Full Stack a good choice for server-side testing.

• Test different versions of your features and measure their impacts on key product metrics across over 25 native filtering and breakdown criteria. 
• Kameleoon Hybrid saves developers’ resources because except for the initial SDK installation, they needn’t do any extra coding to get started with the platform. 
• By transferring third-party segments into the platform, you can maximize the effectiveness of your server-side experiment targeting, thereby offering relevant and personalized experiences to your users. 

Statsig

Statsig empowers an organization’s experimentation culture with its robust full-stack testing capabilities and by making data easily accessible. 

• This platform facilitates product experimentation and feature tooling, and its SDKs offer a whole gamut of logging tools that allows easy analysis of events and metrics. 
• To enable product managers and engineers to test and ship new features quickly, Statsig’s SDKs work seamlessly with any environment your application is in. 
• You can convert any feature roll-out into an A/B test and get automated analyses of its result so that you chart a well-thought-out product roadmap. 

A/B Smartly

A/B Smartly is the first private cloud experimentation platform that allows you to run multiple tests simultaneously across your entire infrastructure – starting from apps and websites to emails, ML models, and beyond. 

• Its real-time reporting helps you keep a tab on all experiments and measure their impacts on your business goals. 
• As A/B Smartly runs locally, you face no flickering or lag issues; instead, your team stays in full control of data which you can integrate with any other internal tools for greater development flexibility. 

Conclusion 

In today’s competitive digital landscape, customers are the ultimate source of validation for your business needs (at least the only one that matters). Test deep within your entire stack and wide across your product features to let your customers decide what works and what needs to be fixed. 

With server-side testing, you need not hold back with your ideas. Even the most complex of mobile app pages or most sophisticated of product features can be optimized to fuel conversions and improve your bottom line. 

To ensure your website, product, and in fact, every digital touchpoint is optimized for the best, most seamless user experience, be sure to embed deep testing in your marketing and product endeavors. 

With VWO’s server-side testing feature, you’re all set to get started on your deep experimentation roadmap. So, set the ball rolling, deliver delightful experiences, and build futuristic products- with server-side testing! 

Frequently asked questions on server-side testing