Commentary: "Mistakes People Make Building SaaS Software"

I was amused to see a clickbait-y article pop in my RSS feeds from InfoQ: “QCon London: Mistakes People Make Building SaaS Software”. While it is hard to really judge the substance of the talk from this article, its “top ten mistakes” list is a pretty good conversation starter for people who actually are thinking about getting into a SaaS business.

The list is not perfect and probably isn’t exactly how I’d break things down, but it is nice to have a starting point.

Let’s go over some of these:

Building something users don’t want

Delivering unique features or different versions to tenants

Listening to consultants instead of understanding your customer base

I think these are actually facets of the most fundamental problem getting into this business:

• what kind of organization are you?
• who are your customers?

I’ve seen this firsthand: years spent in a large organization that was at its core product sales-focused trying to get into a service delivery business was an incredibly frustrating experience. No amount of introspection at all levels of management could ever change the leopard’s spots, so to speak. Large organizations have a culture and a way of doing things that won’t reinvent itself, regardless of how hard they try. This culture often dictates what kinds of customers they are willing to pursue. So, the first question is incredibly important.

Of course, the larger the organization, the more likely it is to engage consultants, with predictably mixed results at best. In addition to the poor customer fit the list alludes to, the other pitfall of engaging consultants without tightly working with them is that they can be blind to feasibility challenges. Unless they are deeply knowledgeable about the customer’s domain, they probably won’t know the functional and, especially, non-functional challenges that can make a seemingly brilliant product idea into a software development black hole. Yes, working closely with consultants and customers is a good way to avoid those surprises… But, at the risk of being cynical, that’s not something that happens very often.

At the other end of the scale, small organizations have a lot of flexibility, leading to a lot of the temptation to make mistakes John Topper alludes to in his talk. Some customers and domains have requirements that directly lead to these mistakes, so walking away must always be an option. Being lean is being able to recognize early that some ideas aren’t worth it.

To be fair, you have to understand the actual requirements – not just their superficial expression – to make some of these decisions. For example, people that have been burned by software that changes dramatically under their feet (the redesign in iOS 7 being a great example) will be tempted to make dramatic, sweeping, stability requirements that are in direct conflict with other fundamental properties of the software (e.g., security, scalability). These can be often negotiated into a more benign form. If not, walk away. That is business that is fundamentally incompatible with SaaS.

Not building tenancy concepts into application components

Agreed. Since the SaaS business model is predicated on scale, minimizing the marginal cost of adding tenants and users while meeting their isolation and performance requirements is fundamental to profitability. Having lived through the debacle of “let’s start with a single-tenant system, we can fix it later”, I can tell you that virtually no amount of money for that first sale will ever make up for the broken business model it entrenched.

Not calculating baseline per-tenant costs and testing this with the market

Agreed. But, to be fair, calculating costs can be difficult if you have no experience in the domain and don’t know what to expect. It’s relatively easy to calculate worst-case per-tenant costs (in the degenerate case where you only have a single customer), but the real trick (from a business perspective) is to know a realistic number of tenants to amortize base costs for. An additional (technical) challenge is to find the non-linearities in resource costs and how they map to business concepts – e.g., when do you need to go from N to N+1 database instances? How many tenants, users, data volume units does that map to?

The other baseline and marginal costs to keep in mind are the support and operations cost. It’s easy to fixate on the infrastructure costs, but the more specialized the domain, the higher the risk that customers will expect humans to support them.

Not automating tenant provisioning

Agreed. That was one of the things we worked very hard on in my last project for SLB. Even when transaction costs are overwhelmingly non-technical, being able to provision tenants quickly and cheaply is a game changer.

Deploying your software into your customers' AWS accounts

Agreed. I’ve seen this more often with Microsoft Azure, where the “Azure credits” included in other IT purchases from Microsoft incentivize this kind of behavior. It isn’t just a trap from the perspective mentioned in the article (it’s shrink-wrap software, not SaaS), it is also an IP protection and service level agreement perspective: the customer has full administrative control over your software, can see exactly how it is made and can prevent it from working properly any time they want.

In common with on-premise distribution, there is also licensing peril with doing this: arguably, this is “distribution” from the perspective of software licenses (e.g., the GPL), which triggers a number of obligations not present when just delivering a SaaS service using the software package. Also, commercial third-party dependencies may incur unexpected additional costs when the target infrastructure doesn’t belong to you.

Building your solution to be multi-cloud or cloud-agnostic

With the caveat that there are certain domains where being able to use multiple cloud providers is a key requirement (e.g., for data residency in regions with limited cloud provider coverage), agreed. The cost of building a product that is cloud-portable is huge. Most of this cost is opportunity cost: not being able to fully use the capabilities of your chosen provider because there is no equivalent elsewhere. The more direct cost of building compatibility layers, etc. is also significant but it is fairly easy to quantify. We are far less good at quantifying the features we could build if we’d just used the perfect platform service from our single cloud provider, instead of doing the extra infrastructure work to support other providers and make up for the lowest common denominator.

Now, there are businesses where cloud-agnosticism is a key feature. That is just an incredibly expensive feature.

Deploying your solution on-premise for customers

Agreed. Not that there aren’t valid business models that involve on-premise software delivery, but they are virtually always incompatible with a SaaS business.

Not thinking sufficiently about backups and disaster recovery

Operations, backups and disaster recovery are all fundamental aspects of running a SaaS business. You have to make disaster recovery your religion to be anywhere close to good at it.

Not classifying data assets

While this is probably only intended from the perspective of compliance (e.g., GDPR) and threat modeling (based on Jon Topper’s other presentations – e.g., this), data classification is an interesting area that I’ve found to be pretty fluid over the years. Without a doubt, you have to classify your data to meet compliance and threat modeling requirements. How finely you classify the data beyond indentifying PII and security-critical information is a choice that needs to be taken carefully. It can easily be a deep rabbit hole with very little tangible reward.

Ten years ago, we were trying to model various facets of data in our systems to cover a wide range of concerns:

• country of origin
• country of modification
• privacy
• security criticality
• allowed uses (model training, etc.)

And so forth…

The temptation was to tie data classification and data entitlements because this appeared to be the silver bullet for various legal requirements – which was hard to do and never worked well. In theory, you can construct your system out of classification-aware building blocks that apply an algebra of classification combinations to automatically classify outputs… In practice, it invariably falls apart and you realize that a less ambitious solution works just fine.

So, I think that I would rephrase the mistake thus: “not thinking about data classification and not classifying data where necessary”. The first step is to decide to what degree classification adds value (e.g., if it is a legal requirement, it is a no-brainer) before baking it into the system and development processes.