I get this question all the time. Sometimes, from my peers who are trying to make software purchase decisions and other times, from my engineering and product management friends, who are trying to understand a buyer’s mindset so they are well prepared for all possible customer questions.
So, here are a few things that I always look at before making a buy decision. You’ll notice that some of the criterion is not too different from on-premise services, but they become a lot more relevant when you move to a cloud-based service.
Functional Aspect
Data and Reports
One of the most common challenges with cloud-based services is the control over your data and how easy it is to access your data. It’s good if the cloud service provides its own reporting and data visualizations, but that doesn’t always serve all your needs. Most times, you’ll need to extract, enhance, cleanse, and merge the data with other data sources before it can be used. That means it should be easy to pull the data from your cloud service, in various possible ways through files and APIs.
Integration
This is also one of the non-negotiable asks. No matter how great the service is, it’ll need to integrate with other corporate systems and fit well in the grand scheme of things. No software or service can run in complete isolation. Even if you don’t see the need to integrate today, you’ll see the need in future.
Make sure you test the APIs for all possible inbound and outbound exchange of information.
Customization/Extensibility
One common limitation is the ability to customize your application. This can be a dual-edged sword for cloud-based applications. Multi-tenant cloud providers can only go to a certain extent while providing customizability. One of the popular advantages of cloud services is that all consumers are on the same version and need not worry about the upgrades. This makes it easy for the provider to focus on one version of its service that works the same way for all customers. However, this also means that the provider might not be able to handle your unique needs. There are some vendors that have taken a platform approach. You may not be able to use it exactly “out of the box,” but its highly customizable.
I have come across situations where we could only get 90% of what we wanted from a cloud-based service and the cost of building the rest 10% took away all the expected benefits.
User Experience
If you compare modern cloud-based applications with traditional ERPs, you’ll easily notice the difference in experience and convenience. Most of the cloud offerings provide a consistent, modern user experience that works seamlessly on a decent internet connection. The secret is in simplicity.
The design patterns have evolved over time, allowing the same core features that work equally well on desktops and mobile devices. Think about involving a UX expert while evaluating the application.
Non-Functional Aspect
Performance
The fact that cloud services always run on the web makes performance aspect really important. The user shouldn’t notice any difference in the performance when compared to an on-premise application. Its also worth noting that it’s not just the application performance that’s important. A lot of interaction with these applications happen over APIs . These APIs should be tested for performance as part of the evaluation exercise.
Security/Compliance
Although the world has changed and organizations are a lot more open to use cloud-based software, the security aspect is still very relevant.
Any information exchange or interaction with your online application needs to happen over a secure and well-authenticated channel. You should ask for security and compliance certifications from the provider. Some industry standard reports like SOC1 and SOC2 provide a lot of information about operations maturity of cloud-based, multi-tenant applications, including security. If you plan to capture credit card or other sensitive information over these applications, make sure that vendor is taking full responsibility of all compliance matters like PCI, HIPAA, etc.
Operations
The vendor should be asked to provide details about his hosting facility and related operational aspects. Inquire about his standard operating procedures—e.g. data backups, redundancy, capacity, disaster recover plan. Again, most of these aspects are covered in the certification process like SSAE 16, but you should review the audit reports and look into any open items identified in these reports. It’s also good to keep the right to audit the vendor’s hosting facility. It’s not a good sign if the vendor seems unprepared for such questions.
SLAs
Uptime, performance, and issue-resolution times are the most commonly used SLAs. Let’s take the example of uptime. I am sure all your vendors claim to provide 99.9% uptime SLA. So, whats the catch?
I suggest you spend some time on reading the exact algorithm for calculating the uptime. In many cases, you’ll find that the vendors very cleverly exclude some important features from the scope of calculating uptime. You should also check about how often the vendor provides the SLA information and what’s the penalty if he doesn’t meet it.
Bottom line is that you should read the fine print on how these SLA metrics are calculated, reported and handled.
Customer Support
So, why would customer support become more important for a cloud provider? Shouldn’t you face fewer issues compared to an on-premise software?
Remember that in the case of a locally deployed software, your IT folks get an opportunity to diagnose the problem and explore some workarounds. In critical times, the most commonly used solution is a “Restart.” But in case of a cloud service, you or your IT team don’t have any control over the application. In those critical times , you want to speak to a highly efficient, knowledgable, and super friendly customer support department. This is an area that gets generally overlooked while evaluating applications.
