There seems to be a group myopia around so-called ‘cloud computing’ and it’s definitions. What we’re really talking about are ‘cloud services’ of which, ‘computing’, is only a subset. It gets worse when you have people talking about Software as a Service (SaaS) as a ‘cloud’ service. Things continue to become murkier when the SaaS crowd, bloggers, and reporters start making up new definitions for cloud services using SaaS-like terms such as Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).
Soon we’ll have Hosting as a Service (HaaS), Services as a Service (SVCaaS), and Name Your Flavor as a Service (*aaS, better known as just plain ‘aaS’). Of course SaaS is just the re-hashing of the whole Application Service Provider (ASP) model from the late 90s anyway. You remember?
Let me explain what cloud services are, where they came from, and where they are going.
Cloud Services
Cloud services are not SaaS. They are far more akin to web services and this is part of why Amazon’s cloud services are simply called ‘Amazon Web Services‘. Cloud services encompass a number of different kinds of service, but they share some common attributes with each other and web services, including:
- simple point services meant to be integrated into a larger application
- machine-consumable via network-based APIs (commonly REST or SOAP)
- ‘infinitely scalable’ (meaning: ‘no practical limit’)
- in-the-cloud
- self-service
- on-demand
By contrast, most Software-as-a-Service products are fully-baked applications such as Salesforce.com’s CRM application. Another quick way to identify SaaS is that the user interface is usually the primary interface with the API being an afterthought. In comparison, most cloud services have an API as the first interface and a UI as a
secondary interface if one even exists.
Cloud vs. Web Services
Web services are historically a large business (i.e. ‘enterprise’) based technology trend. SOAP and the WS-* standards defined web services and their consumption. Web services, for this reason, existed mostly behind the firewall and were created and consumed within large organizations. Cloud services are similar to web services, but differ in terms of who uses them and where they are deployed (i.e. ‘in the cloud’).
Cloud services are fundamentally game changing, where web services are just ‘ho-hum’. They are the final democratization and commoditization of ‘web services’ out in the cloud, usually created by many different organizations, and consumed by anyone. This is a natural response to market forces seeking to have more control, scale, and usage from ‘web services’. Web services, like most recent Internet related technologies are more interesting when the power of ‘the net’ is applied. In other words:
web services (’ho-hum’) + network effect == cloud services (’holy cow!’)
If nothing else shows us how much cloud services look like web services, then ‘Amazon Web Services’ product offerings should. A short list of what they provide:
- Compute
- Storage
- Database
- Messaging / Queuing
- Payment processing
- Search
As you can see, these are all simple point applications, consumable via an API, for use as part of a larger application and of minimal use by themselves. They are not even marginally related to SaaS. As individual services they aren’t terribly interesting, but in terms of their ability to provide you leverage for your application, to quickly accelerate some need that you have when initially developing, or to allow you to get a large amount of ‘scale‘ once you grow is of tremendous value.
Clouds as Platforms and Infrastructure (aaS)
This recent RightScale blog entry inspired this post and what I think they are really talking about here is breaking ‘cloud services’ down into two main categories: platform-based models and infrastructure-based models. Both of these are defined fairly well in the RightScale blog posting (other than the association with SaaS terms). What is important to notice is that this isn’t an either/or situation. You don’t need to pick one or the other: platform or infrastructure. You’ll almost certainly wind up using both.
Build your prototype in Google’s AppEngine and have it use Amazon’s Flexible Payment System (FPS) to process payments. Or build it on Amazon’s Elastic Compute Cloud (EC2) and have it use Google Checkout. Do what is right for your application.
Cloud Service Trends
The continued trend is towards self-service, democratized, vote-with-your-dollars, cloud services that can be used to composite together or bolster your application development, decrease time to market, and provide new kinds of software development leverage. This means that cloud service providers, and the developers who build value added services on top of them, need to compete using strategies where their services have inherent synergies (a la the AWS ‘ecosystem’), on service or service level agreements, or by continuing to innovate in advance of competitors. Right now it looks like Amazon is doing this across the board while most others are playing catchup.
The continued driving forces will be ease-of-use, choice, and ability to use cloud services as part of larger applications for startups, small businesses, and soon larger enterprises.
Future of Cloud Services
The future of cloud services is one where you will select and composite the best (or cheapest) services for your application. Some combination of services will be right for your application and the tools that help you build and put together these services will be critical to your success. This means that tools that support multiple providers or abstract them in such a way that ease your ability to integrate and consume them will be clear winners.
May 30th, 2008
I’m quite pleased to see that you can now get OpenSolaris with Amazon’s EC2. OpenSolaris on EC2 means that you can now run your storage and/or databases in the cloud on top of the ZFS filesystem.
Among the number of truly outstanding capabilities of ZFS are, snapshots, cloning, compression and soon encryption. A little known feature is the ability to send a ZFS filesystem as a data stream. More importantly, you can send ‘incremental’ data streams.
What does this mean?
First and foremost it means that replicating data inside the EC2 cloud just got a *lot* easier.
It also means it is now feasible to have a replica of your data off-site at your office or another datacenter without paying exorbitant transfer prices. You’ll pay once to take make a ‘full’ replica and then afterwards you can keep that replica up-to-date by sending ‘incremental’ snapshot streams on an hourly (or more frequent) basis.
This will work quite well and be extremely cost-efficient even with a large dataset as long as you aren’t changing large amounts of data over short periods of time.
Given that a Yottabyte of capacity is on the horizon, this is welcome news for effectively managing your data.
May 7th, 2008
Storage requirements are exploding causing more and more small and medium businesses to employ creative solutions to stem the tide. In December, Hu Yoshida, CTO of Hitachi Data Systems (HDS), posted a blog entry about projected enterprise data growth. The entire posting is worth a read, but the included chart really paints the picture well:
Experience with our clients bears this out and we think most organizations are ‘feeling the pinch’. Perhaps most interesting about this trend is that, as Hu points out in the article, data falls into different kinds of buckets. He chooses to talk about structured vs. unstructured data, but there are other ways to slice this pie.
Case In Point
For example, we find that most clients are struggling with price/performance issues for large pieces of data. To take an extreme case, a friend of ours at LucasFilm is currently struggling with increasing the cost-effectiveness of their storage solutions.
LucasFilm has three types of storage solution:
- Tier-1, high speed, low capacity disk storage
- Tier-2, average speed, high capacity disk storage
- Tier-3, slow speed, medium capacity long term tape storage (AKA ‘archives’)
This is fairly typical for most businesses. The difference in costs can be quite dramatic between tiers. In LucasFilm's case their cost for tier-1 storage can be as much as 30x the cost for tier-2.
In order to be cost-effective, LucasFilm continues to expand tier-2 storage capacity and spends considerable time and money shuffling data between tier-1 and tier-2 storage.
Unfortunately most smaller businesses do not have LucasFilm's resources. Even for larger organizations, increased spending on storage and storage management is undesirable.
New Solutions
Fortunately, there is a very compelling new solution to this problem. Sun’s ZFS is now available and production worthy. ZFS is a revolutionary new open source filesystem that provides all of the capabilities of a NetApp storage appliance, in terms of redundancy, ease of use, and capabilities (e.g. NFS, iSCSI, Windows File Sharing). It requires Solaris, OpenSolaris, NexentaOS, or FreeBSD to run, but will work on most modern hardware.
ZFS provides a cost-effective option for tier-2 (and some tier-1) storage solutions. No special hardware or RAID controllers are necessary. It is designed to work on both inexpensive commodity and enterprise class hardware.
Using ZFS you can now build high-capacity, redundant storage systems for as little as $.25-.50/GB, which is pretty close to street price for the drives themselves. Alternatively, you can build tier-1 high-performance redundant storage systems, roughly equivalent in quality to enterprise solutions, for as low as $2/GB, which is practically unheard of.
For example, recently we priced a StoreVault S550 (NetApp’s SMB targeted appliances) vs. a ‘roll your own’ ZFS appliance using NexentaStor (see below). Total cost for a 10TB solution was $23K vs. $6K for a NexentaStor-based ZFS appliance. List price for a 1TB 7200 RPM SATA drive for StoreVault? $1500. List price for buying your own 1TB SATA drive? Less than $200.
When ‘New’ Works
Some might consider ZFS too ‘new’ to put in production; however, we have been running it successfully in production for over a year and are very happy with it. It’s reduced our storage and storage management costs dramatically and allowed a small consulting business to have enterprise class storage without enterprise class prices.
Still, if you need more or desire enterprise support, you can get it. We recommend one of the following two options:
- Solaris + Sun support
- NexentaStor from Nexenta
Right now we prefer Nexenta as they are 100% focused on delivering a ZFS-based storage solution. They provide a commercial solution and enterprise support using a specialized NexentaOS-based storage distribution called NexentaStor.
NexentaStor
NexentaStor turns commodity hardware into a sophisticated storage appliance, like NetApp, but at a quarter the cost. Best of all, the NexentaStor solution is based on NexentaOS, itself a re-packaging of OpenSolaris using Debian (Linux) userland and utilities. This means network-based upgrades, easier management, and a very small distribution.
Some items in the pipeline from Nexenta that are very exciting include compact-flash based versions of NexentaStor. With judicious selection of your hardware you can reduce most, if not all points of failure in your storage systems, dramatically increasing MTBF (mean time between failures) and putting your ‘commodity’ hardware into the same class as similar enterprise solutions such as EMC or NetApp.
Conclusion
Whichever way you go, ZFS is a rock-solid production capable filesystem with a compelling value proposition. We especially like it’s use as a tier-2 storage solution.
I’ll follow up shortly and talk about some specific exciting use cases for ZFS, including it’s capabilities for compression at the filesystem level and it’s use as a backing store for virtual machines.
Disclaimer: neoTactics is a Nexenta certified partner and a proud member of the Sun Startup Essentials program.
April 28th, 2008