SaaS Essentials: #1 Ability to Scale

In this series I am discussing 10 essential characteristics of good Software-as-a-Service applications. In this first post I cover Essential #1: Ability to Scale, along with some thoughts on how to achieve scale on Microsoft Azure or Amazon Web Services. I'll be discussing specific features and cloud services of both AWS and Azure in this series, as I've implemented SaaS solutions on both platforms.

#1 Ability to Scale

Since SaaS applications are multi-tenant, effortless scale is a must. Just because your solution is deployed to a cloud platform does not automatically make it scalable: you need to utilize scalable cloud services, and you need an architecture that doesn't inhibit scaling. If you have for example a legacy application that you are transitioning into a SaaS product and it can only work on a single web server, you're going to have scale limitations (and availability problems) until you change the architecture to support a web farm.

Once you know your SaaS handles one client well (giving you confirmation you've chosen adequate machine sizes), the next consideration is handling N clients: can you add more clients without affecting reliability or impacting the performance of existing clients? You can achieve this by ensuring you have a solid scalability story for each tier of your solution: web servers, data, services, worker processes, etc.

Scaling the Web Tier

The best way to scale your web servers is horizontally, meaning you have a load-balanced server farm where the number of instances changes as load changes. It's also possible to scale vertically, meaning you increase the size of your web server(s) to handle increased load, but this is ultimately limiting and not recommended.

A web server tier, as deployed to Azure CloudService or AWS Beanstalk / EC2, can be set up to auto-scale based on min/max instance settings and specific criteria for growing/shrinking the instance count. For example, an additional instance could be automatically deployed when VMs are at 60% load or more. You want a minimum of two instances for high-availability, and you want to set the maximum based on budget. With this configuration in place, your web farm will grow or shrink all on its own. This is horizontal scaling.

Scaling the Data Tier

Your data tier also needs to scale, and scaling databases is more complicated than scaling web servers. Data scaling is particularly important if your client data all sits in a single database. You have multiple options to consider. A relational database service like AWS RDS or Azure SQL Database can leverage vertical or horizontal scaling. Vertical scaling means allocating a larger database VM size, increasing capacity. 

Horizontal scaling uses multiple read replicas to gain increased capacity for database reads. Horizontal scaling with read replicas won't help you scaling database writes, however, so you might need to combine both horizontal and vertical scaling.

You can also consider sharding, in which data is partitioned across a collection of databases with the same schema. Azure offers Elastic Database Tools for working with sharding, and on AWS RDS read replicas can be used for sharding. This doesn't mean you should be leveraging all of these options; you want to arrive at a scalable story that is reliable and not over-complicated (I can't stress that enough: a complicated architecture is a sign that you need to further refine things or change your approach.)

Personally, I favor having a separate database per client which reduces the scalability question to database instance size or service level. Doing this avoids the need to do complicated things like sharding and keeps clients' data isolated from each other. It also gives you the freedom to individually control things like database size or region for each client. In fact, clients' databases don't even have to all be on the same cloud platform. Lastly, it simplifies operations like backing up or restoring a single client's data.

Cloud Storage

So far we've been discussing databases, but your data tier may well include basic cloud storage (AWS S3 or Azure Blob / Queue / Table storage). These services are highly scalable out-of-the-box, and you don't generally need to be concerned about putting scalability in place unles your solution spans multiple data centers around the world (which I won't go into here—check your cloud providers' options for replicating storage across regions). If your users are widely distributed geographically, consider leveraging a Content Delivery Network (CDN) for accessing your cloud storage.

NoSQL Databases

Lastly, there are non-relational cloud databases. Newer kinds of NoSQL databases such as Azure Cosmos DB or AWS DynamoDB are built for auto-scale and automatically replicate data across regions. If you develop (or update) your solution to leverage these databases, in full or in part, you can take advantage of this automatic scale.

Scaling the Services Tier

You can scale services in several ways, depending on how separate your services are. If your services are provided by your web servers, they're part of the web tier and you've already addressed matters. If you have a separate services tier, you can choose between hosting them similary to your web servers (Azure Cloud Service or AWS Beanstalk / EC2) or you can consider server-less computing which has become very popular.

Formally structuring your services used to be the domain of Service-Oriented Architecture (SOA), but today there is the newer Microservices Architecture to consider. With microservices your business services are small components that have the characteristics of being single-purpose, testable, separately deployable, and loosely-coupled (independent). You can read about the differences here. Serverless computing is particularly well-suited to microservices, because functions (services) can be easily deployed individually.

Serverless computing, provided by services like AWS Lambda or Azure Functions, allows you to deploy just your function code without worrying about scale: there are no instances to allocate, the scaling is automatic, and you are only charged when your functions are invoked (in other words, pay-for-use). A function you deploy can be hooked to an API Gateway (AWS) or HTTP trigger (Azure) and voila, it's a service—one that will auto-scale. You can also connect functions to other triggering events besides HTTP traffic. If you use the microservices approach, be sure to address security fully across your microservices tier.

Scaling a Worker Farm

If you have a "worker" compute instance whose job is to service tasks from a queue, that lends itself to a farm approach the same way used for web servers: instead of a load balancer distributing traffic to N web server instances, a common queue distributes traffic to N worker instances. If your cloud platform doesn't have a way to automatically scale a worker farm, you'll need to monitor appropriate and manually increase or decrease the number of instances.

Other Components

If there are other parts of your solution, you'll need to figure out their scalabity story as well. Some elements of your solution may not require scale. For example, you might have a single-instance compute instance such as a nightly background service that perhaps isn't affected by the number of clients.

If you use a distributed memory cache such as Redis, you'll want to monitor the amount of cache being used and adjust it larger or smaller to stay in line. I don't think there's currently an automated way to do this, so it's up to you to monitor the level of usage.

Finishing Touches

Test Scalability—Or Your Customers Will!

Once you have your software tiers scaling well, you'll want to test that it's all working by experimenting with different loads and verifying the cloud platform has increased or decreased instances as expected. Failure to do this means your customers will be doing the testing.

Put Monitoring In Place

You'll need to monitor the load on your SaaS and keep watch over it. For scaling activites that require you to manually make configuration changes in the cloud portal, you'll only know to do it if you're monitoring activity and load. Even for cloud services that provide automatic scaling, you'll want to be regularly verifying that the automatic behavors you're expecting are actually happening.

Get familiar with the metrics available in your cloud management portal. Also consider tracking your own metrics, such as number of business transactions per day / hour.

In Conclusion

Scale: it's a first-class consideration for any SaaS. Scaling up is necessary to accomodate growth, and scaling down is just as important to keep your costs down. Take advantage of the scaling features in your cloud platform. Cloud platforms provide many tools for scaling, but you have to be aware of them and put them to use—and they aren't all automatic. It's important to consider the scaling story for every tier of your solution. Thankfully, newer cloud services now often provide automatic scalability. The more of that, the better.

Next: 10 SaaS Essentials: #2 High Availability