Immediate growth in traffic, 2x increase in time spent on page, low bounce rate, featured carousel stories, and increased CTR on SERP — it’s no wonder why more than 50 million websites have adopted AMP framework and as you are here, we assume you are one of them. In case you haven’t already, check out our post on why AMP pages are under-monetized and how we solve the problem with the AMP monetization suite.
If you want to run a direct campaign on AMP pages, the process is pretty simple. However, getting started with header bidding (using Real-Time Config, Cache servers, etc.) isn’t. In this post, we are going to focus on how we have built a scalable, efficient server-side infrastructure to help our publishers monetize their AMP traffic.
Let’s start from the beginning. When a user loads your AMP page, RTC calls are made from the browser to the Prebid Server Cluster. So, the first thing is to ensure the browser and Server Cluster get connected immediately. In other words, DNS lookup has to happen faster.
Faster DNS Lookups
As you know, DNS lookup helps the user’s browser to establish a connection with your server and the faster it happens, the better it is for the users.
In the case of AMP, the page loads almost instantly. So, it is important to leverage the fastest DNS lookup. We use Amazon Route 53, a highly available and scalable cloud Domain Name System (DNS) web service. Route53 uses a Global anycast network of DNS servers around the world, designed to automatically route requests to optimal locations depending on network conditions.
So, when a user loads a page from anywhere in the world, we establish the connection b/w the browser and our Server Cluster instantly.
Now to run header bidding on AMP pages, you need two essential components.
- Prebid Server.
- Prebid Cache Servers.
It’s important to optimally set up Prebid Server and Prebid Cache Server to run ultra low-latency auctions and render ads quickly. Here’s how we do it.
Prebid Server
Technologies: Prebid Server (Golang app), Postgres Server.
Prebid server is responsible for processing the incoming ad requests. To maximize bid rate from the demand partners (SSPs), the server has to first enrich the ad requests with some metadata and then forward them to sell-side platforms.
We primarily use Postgres to store the metadata, but to speed up the lookups the Prebid Server makes use of a local in memory cache. Though cache invalidation rules ensure that the metadata stays up to date, we have APIs to purge the cache on demand. With the help of Connection Pool, we have also optimized how the app server interacts with Postgres DB.
Prebid Cache
Technologies: Prebid Cache Server (Golang app), Redis.
Prebid Cache server is responsible for storing bid responses along with additional metadata about an ad request and respond back when queried. We use Redis, an in-memory data structure as the data store. Why?
Redis is exceptionally fast and an apt solution to build high performance, scalable web applications.
In addition, to ensure the high availability and linear scalability of the data store, we run Redis in a Clustered Mode. Essentially we create a Cluster of Redis servers consisting of several master nodes and replica nodes so that we can scale the system as needed.
Partitioning is quite useful in large-scale applications. Why? It improves performance and reduces contention by letting us divide data into partitions that can be managed and accessed independently. The problem was Prebid Cache Server didn’t have the capability to query the data back from the right servers instantly. We have extended the functionality of Prebid Cache Server by creating a redis driver that can store and query the data in multiple redis shards.
The next step is to ensure scaling.
Distributed Server Clusters
As a publisher, you probably have seen an uptick in traffic over time and especially, AMP traffic can shoot up instantly — based on a breaking event, news, etc. Whenever there’s an increase in pageviews, the number of requests will multiply.
How do you manage to process the bid requests when there’s a sudden spike in traffic?
If you aren’t able to scale your server capability, then you are likely to end up losing revenue. That’s why at Automatad, we scale our servers horizontally, i.e., we add more server nodes (more servers) based on the load.
Scaling horizontally helps us to handle more requests efficiently and provide more fault tolerance (no single point of failure and high uptime). No matter how many pageviews you have, we ensure that the requests are processed without any latency.
Note that scaling is automated with the help of load balancers which we’ll discuss next.
But that’s not all.
Having a scalable server setup isn’t enough. You need them to be distributed across several geographical regions to ensure close proximity to the Sell-Side Platforms (SSPs). This is to reduce the latency in the round trip — sending the bid requests and receiving the responses from the demand partners, capturing the bids from all the demand partners, and send the bids within 1000ms* to the ad server.
*AMP auctions have the timeout of 1000ms.
On top of it, your users tend to be distributed across the globe too. We maintain our Prebid Server Cluster in multiple locations keeping all these factors in mind.
Ideally, we recommend our publishers to use our Prebid Server Cluster hosted in a region where the majority of their traffic comes from. For instance, if the publisher is from the US, we use our Prebid Cluster hosted in the US – East / West region. For Asia traffic, we recommend them to use our Prebid Cluster hosted in the Singapore region.
In a line, we put you on par with the heavily-invested media companies.
Load Balancing
Load balancing is used to efficiently distribute the network traffic across a cluster of backend servers in order to ensure uninterrupted performance.
As we mentioned above, Automatad leverages load balancers to automatically distribute incoming ad requests across a cluster of prebid servers and prebid cache servers. In other words, even during the times of heavy traffic, you have nothing to worry about.
Load balancers constantly monitor the health of the individual Prebid Server and Cache Server and automatically adds or removes them from the cluster. That’s how we ensure high availability of our service at all times.
If you are wondering, how the provision happens, we trigger it based on response time.
Provisioning
Whenever the load increases, the time to respond increases as well. In order to ensure the response time is under the optimal level, we automatically provision more servers based on a trigger.
To put it simply, we monitor the response time (the time Prebid Server takes to respond to RTC Callouts) and trigger the provisioning, i.e., add more servers to the Cluster when it hits the certain value.
We use a combination of Terraform and Ansible to provision the infrastructure. Specifically, we make use of Terraform for VM provisioning and Ansible to install and configure the required software packages.
Monitoring & Alerting
Prometheus and Grafana help us with the monitoring of the services. We get email and slack alerts from Grafana based on the health/performance of the services.
What metrics do you constantly track in order to find anomalies?
Here are some of the metrics we track constantly:
- Uptime
- Disk Space used
- Open file Descriptors
- System Load
- CPU usage
- Network Traffic
- Memory Usage
- Time Spent doing I/O’s
- Disk IOps
What’s Next?
While it’s easy to replicate a server-side setup in theory, when you begin implementing it, you’ll come across new problems and hiccups. One of the major advantages of using Automatad is we have already solved the problems and performance issues that you may experience while running header bidding on AMP.
Besides, maintaining the integration with the SSPs and ensuring they are actively bidding as they should — is a different story altogether. If you don’t have the bandwidth and the team, give us a shout to help you with AMP monetization.
