2008-2009: Cloud Computing, the New Thing
In late 2008, working at Microsoft Partner Neudesic, our CTO Tim Marshall and I were invited to a Microsoft feedback session in Redmond about "Project Red Dog". Red Dog, it turns out, was about this new thing called Cloud Computing. Amazon had been doing this for a few years, and Microsoft was going to also enter the market. This "cloud computing" was a new idea and a new way of doing things—but it sounded exciting. A few months later, "Windows Azure" was released. As Neudesic is a consulting company, we started learning it and looking for early prospects.
When Microsoft introduces a new product or service, a lot of work goes into evangelism and education and finding early adopters. As a Microsoft partner, we did a lot of joint work with Microsoft: visits to prospects, proof-of-concept projects, training sessions, code camps.
Tim had his own ideas about developing the market, and one of those was starting Azure user groups in the ten or so locations we had across the United States. I and other colleagues (including Mickey Williams and Chris Rolon) started sponsoring monthly meetings, sometimes held at Microsoft field locations. Since this was all new, meeting attendance could just as easily be 5 or 20 or 50 people, depending. But we kept at it, and we got the word out there, and interest started growing. At meetings we would cover new cloud services that had just become available, or show off things we had built, or discussed useful patterns for applications. It was fun, and there was pizza.
We learned things about the cloud: the infrastructure was really advanced, but the individual hardware components could fail: you had to plan for redundancy and recovery. The economics of the cloud were different: you had to consider lifetime of the data and resources you allocated, else you would "leave the faucet running". Almost everyone who was an early adopter had an Unexpectedly Large Cloud Bill story. Developers giggled with pleasure at the ease of self-deployment; but sometimes you'd hear a horror tale where someone lost important data all because they weren't careful enough when clicking in the management portal. We started reinforcing the importance of separating Production accounts from Development accounts.
2010-2014 : Azure Evangelism and Early Adopters
As Windows Azure was evangelized, prospects started to line up. I participated in a great deal of proof-of-concept project work, sometimes arranged by and paid for by Microsoft. One that stands out was going to Coca Cola headquarters in Atlanta to show how readily existing web sites could be migrated to Windows Azure. The first web site we migrated was in ASP.NET/SQL Server, which was a slam-dunk and just took a handful of days. The second site used Java Server Pages and Oracle—definitely not in my wheelhouse—but in two weeks' time we had migrated it as well.
I wrote The Windows Azure Handbook in 2010, which I believe was the first book out for Azure. The book contained Microsoft messaging from the time: Platform-as-a-Service (PaaS) is better than Infrastructure-as-a-Service (IaaS) and so on. Today Azure is equally well-suited for PaaS and IaaS and the message has changed. We've learned that there are those who value the cloud for innovative new ways of doing things (the PaaS people); but also those who value the ability to leverage existing skills and don't want their world rocked (the IaaS people).
I also released through Neudesic an Azure ROI calculator, long before there was a comprehensive one available from Microsoft. You can see from this screenshot how few cloud services there were in those early years. The number of cloud services available today is vast and ever-expanding.
There were real cloud projects happening too by this time. At first, there had been a lot of interest but prospects seemed hesitant to actually take the plunge. There was for example a great fear of vendor lock-in. Eventually, and with increasing rapidity, adoption started happening. The vast majority of these projects were web site & database migration for established companies; but start-ups had a different mentality, they wanted to do everything in the cloud from Day L.
As head of the Custom App Dev practice at Neudesic, I made sure we had Azure-trained consultants in every region. As new cloud services appeared, this interested our other practices. SQL Azure database and (later on) Power BI interested the SQL / Business Intelligence practice. Service Bus interested the Connected Systems practice.
Badges Awarded to Consulants Who Completed Cloud Training
Microsoft started a Windows Azure category of their Most Valuable Professional program, and I was honored to be a Microsoft MVP from 2010-2014. I met some great MVPs on my visits to Microsoft (and hired one, Michael Collier), along with the Windows Azure product team.
Although activity was intense, Windows Azure wasn't perfect. For three years in a row, Azure went down during the annual MVP summit, usually for reasons like somone having forgotten to renew a security certificate. We MVPs were initially amused, but in later years it meant customers were affected. AWS also seemed to have a hiccup as well once or twice a year. We started educating customers about what dependency on a cloud platform meant for reliability, and fallback plans for when the a region or entire platform was unavailable. Both platforms have improved in reliability since then.
In 2011 Microsoft asked me to teach Azure training sessions in Amsterdam and Germany. This was a fun trip—except for the blistering winter snowstorm—and I met some MVPs including Kris van der Mast and Christian Weyer. This helped me realize that cloud computing was a worldwide phenomenon, and also that different regions had different problems to address: in Europe, for example, there were laws about where clients' data had to be stored, and that didn't always align well with existing data centers.
My Azure class in Munich, Germany
As the years went by, Azure added more and more services and would occasionally drop support for a service (never popular). New data centers were continually added around the world.
Azure Storage Explorer
I created a free storage tool named Azure Storage Explorer and placed it on CodePlex, which turned out to be a hit. Over the next few years, Azure Storage Explorer had over 280,000 downloads! I would do a handful of updates a year to ASE, usually because Microsoft had added a new feature or because the Storage API had changed.
Eventually, there was one breaking API change too many and I stopped maintaining it--but made the source available on CodePlex. A second reason for not working on it is simply how busy I was on cloud projects.
A few years later, Microsoft finally came out with their own tool, with nearly the same name: Microsoft Azure Storage Explorer. You can also now manage storage through the Azure Portal. It's about time!
Recently I've had some thoughts about creating some new, updated cloud tools. See the end of this post for more.
Cloud has exploded and is no longer something reserved for brazen early adopters or just a few specialists. At Neudesic, we consult widely on multiple cloud platforms: Microsoft Azure, Amazon Web Services, and now Google Cloud Platform.
New cloud services continue to arrive. There are services for Mobile and APIs and Non-Relational Databases and Distributed Memory Cache and Machine Learning. We now have Serverless Computing (AWS Lambda or Azure Functions), where you don't even have to allocate a server: just upload your function code and the platform takes it from there.
Names were changed. Windows Azure became Microsoft Azure, so the branding wouldn't be focused on one operating system. SQL Azure became SQL Database. Azure Web Sites became Azure App Services. Even Visual Studio Team Services / TFS Online was rebranded as Azure DevOps.
Software-as-a-Service (SaaS)
About 4 years ago I joined a product team to work on creating a Software-as-a-Service offering out of a legacy HR product named HRadvocate. It was a major amount of work to update the architecture and user interface, but eventually we had something deployed to Windows Azure with a reliable SaaS architecture that kept clients' data isolated from each other in separate databases.
SaaS Architecture on Azure
Authentication was initially through Azure Active Directory, with the idea that enterpises could use Microsoft's ADConnect to link their enterprise AD to AAD. It turned out that clients were demanding Active Directory Federation Services (ADFS) integration, so we added support for that. Later we added SAML support so products like PingFederate can be used to authenticate. Now our SaaS product could authenticate each client differently.
An Azure customer required a hybrid architecture, where Azure-hosted HRadvocate needed to integrate with multiple other systems--all of which were local to the enterprise. These systems connected to the former HR system via database integration, a structure that had to be maintained. To fit into this arrangement, I developed SQL Connector, a set of SQL Server functions written in C# that allow enterprise databases to query data in the cloud. This allowed the cloud data to be synced locally. Now, the local systems could continue to use their existing database integration, even though our SaaS was now part of the mix.
Amazon Web Services
I'd obviously been very focused on Microsoft Azure up until now, but that was about to change. Client requirements for HRadvocate led to a decision that we had to be able to run on Amazon Web Services as well as Azure. This led to several years of work on AWS and I am now proficient in it. Getting our solution to work on both Azure and AWS—while keeping a common source code base—was a lot of work but was also very educational. Azure's Cloud Service, SQL Database, Blob Storage, and Redis Cache mapped in a straightforward way to AWS's Elastic Beanstalk/EC2, RDS SQL Server, S3, and ElastiCache. About the only thing we couldn't transition was Azure Active Directory, but that's fine since we offer multiple ways of authentication.
SaaS Architecture on AWS
We also targeted Amazon's Commercial Cloud Services (C2S). To support this we added to the product the ability to run air-gapped (without Internet); this required locating and replacing any code (including from open source libraries) that was taking availability of the web for granted. Chart libraries like Google Charts had to replaced with Highcharts which could be local to the application. We added support for the FIPS 140-2 standard, using only algorithms and code for encryption that been certified to be compliant.
During this time, we continued supporting our product on Azure as well. Being able to run on two cloud platforms provided a lot of insight about what is the same and what is different between leading cloud platforms. There certainly seems to be a lot of copying going on between mainstream cloud platforms: when one provider comes out with a useful cloud service, it's not long before the competition has a very similar service. For example, Amazon has AWS Lambda for serverless-computing while Azure has Azure Functions. For those still worried about vendor lock-in, this keeping-up-with-the-Joneses activity should be comforting. The principles for building a good solution in the cloud transcend any one platform.
Ten years have gone by, and Cloud has certainly come into the mainstream. Just about all of us now use cloud computing every day, whether we realize it or not. Doing a web search? Streaming a movie? Using a social network? Making an online purchase? Cloud computing is an integral part of that.
Ten years ago, some big tech companies had cloud infrastructure but no one was providing cloud computing services to the public except Amazon. Now, there are clouds by Microsoft, Google, IBM, Oracle, SalesForce, SAP, VMWare, ...the list goes on and on. As for Microsoft, Azure is now also a leading cloud platform: it does PaaS and IaaS; half its VMs are reportedly running Linux; and there are a whopping 54 data centers worldwide. The growth has been phenomenal.
Cloud computing is no longer considered a speculative idea or a novelty for organizations: now, it's a common assumption that you'll be leveraging a cloud in anything new you develop. Ten years ago there was a lot of indecision about whether to go cloud or not; today, going to the cloud is a given, and the discussion is about which platform and which services to use. It's no longer a discussion of IaaS vs. PaaS; the debate now is about whether to leverage the newer cloud native architectures, NoSQL databases, serverless functions, and microservices vs. more traditional architectures. Serverless in particular is a major phenomenon that has opened cloud development to a broader number of people.
Some of my Neudesic colleagues from the early days have gone on to work at Microsoft or Amazon.
Cloud platforms seem to have improved uptime from 10 years ago, but there are still those moments when something goes wrong and a substantial number of clients are affected. You can still be in for a long wait when a cloud platform is recovering from an issue and each customer account has to be restored.
It's been a really interesting decade of cloud work, and there is plenty more to come. The do-it-yourself nature of the cloud is inherently satisfying, as is being able to change your mind and alter your deployment as will. Services that handle the details and let you focus on your application are a joy to use. You still need to know what you're doing architecturally and keep the cloud's different economic model in mind, but things like auto-scale and recovery are increasingly included in new cloud services. New services like Machine Learning are opening up new vistas for developers, and there's never been a more fun time to experiment—for just pennies.
1 comment:
Nice Post. I like your blog. Thanks for Sharing.
AWS Online Training
Post a Comment