Friday, October 31, 2008

First Time in the Cloud with SQL Data Services

I just completed putting data into the cloud for real and getting it back out using SQL Data Services for the first time, and I wanted to share the experience.

I've been chomping at the bit to do some real cloud computing ever since attending a Software Design Review about 7 weeks ago. I could develop locally with the pre-CTP SDK that was made available to me, of course, but I didn't have a cloud account. The whole idea is to use the cloud after all, so cloud computing on your local box isn't terribly exciting. At PDC2008 the platform was branded (Windows Azure), the CTP given out (updated software), and most important of all you can now sign up for cloud accounts.

I finally received an Azure Services account invitation this week which allows me to make use of SQL Services and .NET Services. Apparently a separate but similar process will give me the ability to put applications into the cloud. In the meantime, I'll have to be content creating local apps that can use the cloud for storage, access control, service bus communication, and workflow (those last 3 are what .NET Services gives you). This is what the portal looks like once you've set up your account:

I decided to start with SQL Data Services. There is an SDK for SDS, which you can download from the SQL Data Services Developer Center. Included in the SDK is SDS Explorer, which looks like this:

While the SDS Explorer is rather simple, simple is all you need to use SDS. HTTP GETs and POSTs are what you use to retrieve, query, or store data using REST. There are other ways to access SDS, such as using C# or Ruby, but using REST from the SDS Explorer seemed appropriate for first-time experimentation.

This is definitely not your father's SQL Server:
  • There's a learning curve. There's a new authority-container-entity hierarchy you need to understand. Authorties hold containers. Containers hold entities. Entities are like tables, but records are really just ids and data. An entity can be homogenous where all data is similarly structured, or jagged. Fortunately the documentation explains this nicely.
  • Access using REST is simple. It's way different from what you're used to in the enterprise (not necessarily in a bad way), but I found myself comfortable using GETs and POSTs to store and retrieve data in no time.
  • It's fun. Once I finally stored some records and was abel to retrieve them, I felt a real sense of accomplishment. And it was fun. Granted these are baby steps, but you've got to walk before you can run. Performance was excellent.

I have to admit, this experience is warming me to REST. Those restafarians might be on to something after all.

Wednesday, October 29, 2008

Cloud Computing User Group

I'm involved in helping form a national user group for Microsoft's cloud computing platform, Windows Azure. A community web site is now up at where you can post info, ask questions, and share everything from ideas to code. Check it out!

The user group will also have local chapters. These can be in any location where someone is willing to host the meeting each month and find a place to meet. Microsoft consulting partner Neudesic has offered to provide meeting space in its locations. It looks like the first meetings, which could be within the month, will take place in Seattle, L.A., Orange County (CA), San Diego, Phoenix, Denver, Dallas, and Philadelphia. Thank you Neudesic!

Cloud Computing is a big thing, and it's going to take a community to pioneer it.

How Cloud Computing Will Affect Every Developer

I've been working on a talk / presentation called How Cloud Computing Will Affect Every Developer, with Microsoft's Cloud Computing Platform (Windows Azure) specifically in mind. I used it for the first time today and it worked well.

Today at the PDC, Mickey Williams and I gave this talk in the form of a conversation using this presentation for Microsoft's Channel 9 (Internet video). It should be showing up on the Channel 9 site ( soon, I'll include a link once it appears.

A PDF of the presentation itself is available here:

Here's an outline of the deck:
  • Level Set - Clearing up Cloud Terminology
  • Cloud Computing in a Nutshell
  • Microsoft's Cloud Computing Platform
  • Impact all Developers will Feel
  • Impact on Enterprise Developers
  • New Patterns for Business Apps
  • Cloud Service Application Model
  • EAI Developers
  • SOA Developers
  • SQL Server Developers
  • Web .com Developers
  • Computer Science / Grid Computing Developers
  • Consultants
  • Product Developers
  • Start-ups
  • Leveling the Playing Field
  • One Last Category... "I Don't Have a Use for Cloud Computing"
  • Summary

PDC2008 Report, Day 2 (Tuesday)

Day 2 of the PDC began with another keynote. This time the topics covered were Windows 7, .NET, WPF, and Visual Studio. There was also a technical demo by Don Box and Chris Anderson.

Windows 7 highlights (Steve Sinofsky):
  • Addresses lessons learned from Windows Vista and Windows Server 2008. It has had important work in the areas of ecosystem readiness, standards (CSS in IE), compatibility (UAC "went too far"), and better support for certain scenarios.
  • Remote desktop can support multiple monitors.
  • Task bar - small icon support, customizable shut down button
  • Action Center - you can change (or hide) all messages for all Windows subsystems
  • UAC slider (you decide how much you want)
  • Developers get ribbon user interface, jump lists, libraries, multi-touch support, Ink/speech support, DirectX improvements
  • Fundamentals: work to decrease memory,m disk I/O and power; and increase speed, responsiveness, and scale.
  • Memory sticks can be encrypted, so if you lose one no one else can do anything with your files.
  • VHDs can now be created from hard drive images easily. You can also boot natively from a Win7 VHD.
  • Better DPI controls and multiple monitor controls.
  • Improved magnifier. Win+, Win- to zoom.
  • Amazing project set up via Win+P keystroke.

Building Great Apps - Scott Guthrie ("ScottGu"):

  • Win7 apps - win32/c++: New Win7 APIs. MFC for Win7 - ribbon UI and multi-touch support.
  • VS2010 - IDE support for very large code bases. Multi-core app dev.
  • Windows apps with .NET - .NET 3.5 SP1 improvements: streamlined setup, start-up performance improvements, graphics improvement, WPF DirectX interop, more controls.
  • New WPF toolkit (for all versions off Windows). New data grid, data picker, calendar, ribbon controls.
  • Windows apps with .NET 4.0 - WPF improvements (multi-touch, deep zone, visual state manager). Fundamentatls and interop (in-process side-by-side support for multiple CLRs, managed/native code interop, dynamic language support, extensible component model). Improved tooling in VS2010.
  • VS2010 - changing over to be WPF based. Very extensible. Multi-monitor support. Refactoring support. Better test/developer workflow.
  • IE8 - improved standards support, web slices, visual search, accerlators, built-in tools for Javascript debugging.
  • ASP.NET in 2008 - dynamic data, REST, MVC support, Ajax/JQuery. Intellisense + JQuery downloadable now at
  • ASP.NET 4 - web forms, MVC, Ajax, distributed caching ("Velocity") in CTP stage now.
  • VS22010 for web development - code focused improvements / editor improvements, JS/Ajax tooling, design view, SS2 support, publishing and deployment, possible to maintain a separate config file for each environment.
  • Sliverlight 2 - Netflix Instant Watch is live now using Silverlight. Silverlight toolkit (charting, tree view, dock panelo, wrap panel, view box, expander, ....). Silverlight designer in VS2010. Working on Silver in or outside the browser.

In the technical sessions, I concentrated today on Oslo, whicih is about modeling. I'm going to post on that separately.

Monday, October 27, 2008

PDC2008 Report, Day 1 (Monday)

The PDC formally began today, as usual with a morning keynote talk by a number of Microsoft executives. Due to excessive LA morning traffic it took my hotel shuttle over an hour to get to the convention center which cost me the first half hour of the keynote but I was there in time for most of it. Ray Ozzie and Bob Muglia are excellent presenters.

As expected, the cloud computing platform has been officially announced and is the star of the PDC. The platform is being called Windows Azure and sports a light blue Windows logo.

Here's the architecture:
  • Windows Azure is the cloud operating system This includes a utility computing infrastructure, a basic API for things like storage, a new app model, and a powerful portal for uploading and managing cloud services.
  • Azure Services are building block services for applications. This includes Live Services, .NET Services, SQL Services, SharePoint Services, and CRM Services.
  • Above that are Software as a Service (SAAS) applications from Microsoft. The consumer ones are branded with Live and the business ones are branded with Online.

There is a web site for Azure,

After the keynote, the sessions kicked off many of which explore the cloud computing platform in more detail. But that's not the only thing at the PDC. There's also Oslo (about modeling applications), Dublin (making Windows Server into a superior app server), Surface (awesome table-top interactive graphical surface), .NET 4.0, and more.

I've known about the cloud computing platform details for about 6 weeks but was under non-disclosure. Now that the platform is public I can discuss it openly.

A national cloud computing user group community has been started at (not a Microsoft web site).

Sunday, October 26, 2008

PDC2008 Report, Day 0 (Sunday)

This week I'm at the Microsoft Professional Developer Conference (PDC) and will be blogging daily to give a blow-by-blow account for those unable to attend personally. The PDC takes place every 2-3 years and is where Microsoft unveils the latest technology to software developers.

Although PDC2008 doesn't kick off officially until tomorrow (Monday 10/27), it was in full swing today for a pre-conference day of training sessions on existing technologies. In my case, I signed up for the all-day training on WPF which was given by the famous Charles Petzold. The training was very helpful and I think I know enough about WPF now to try to build something serious with it. Charles Petzold is a good and humorous speaker and I enjoyed listening to him. Now when I read through his WPF book I'll be able to hear his tone of voice speaking to me.

Although there were a lot of people here today, it'll be really packed tomorrow when the full conference is in attendance. I'm glad I got registration out of the way today. The "goody bag" this year is a black bag that says Windows 7 on the outside. Inside is a PDC2008 T-shirt, a schedule/guide, and various inserts from sponsors. The really good stuff is going to be given away on Tuesday it seems.

The buzz and expectation is that cloud computing is going to take center stage for this PDC. We'll find out tomorrow morning during the keynote address.

Thursday, October 23, 2008

A Good Night's Sleep

"My name is David, and I have sleep apnea. I've been sleeping well for seven days now." Does this sound like an AA meeting? There are more parallels than you might think. One week ago, I finally decided to do something about a sleep problem that had been tormenting me and my family for years. Now just one week later I feel like a new man, and wish I hadn't waited so long to take action.

My descent into sleep problems starts with my family. My Dad would always take a nap on the couch every Sunday afternoon, and the snoring was truly awful. There was no refuge from that snoring, you could hear it from anywhere in the house. Many sleep disorders are hereditary, and sure enough as an adult I found I had the same problem. Lots of people snore, so it never occurred to me that this was anything other than something to put up with.

The snoring grew worse over time, especially over the last few years. It was now more than an annoyance; it was making it impossible for me and my wife to get a good night's sleep. I would have trouble breathing, which would cause me to wake up several times throughout the night. Occasionally I would wake up in the middle of the night in a panic, gasping for air. I tried to compensate for my sleep problem by sleeping on my side instead of my back, but that led to chronic shoulder and neck pain during the daytime.

Finally things got to the point where my wife would have to leave the room in the middle of the night if she was going to get any sleep. I started sleeping downstairs in the guest room to spare her this nightly ritual. Anthony Burgess once said, "Laugh and the world laughs with you, snore and you sleep alone." The man knew what he was talking about. Still I did nothing about the problem.

The cumulative lack of sleep would make me very tired in the daytime. I was still able to get my work done each week through a supreme effort, but it was a sad way to exist, living life through a fog of drowsiness. About 10 times in the last year I even fell asleep on the freeway, which is incredibly scary. Even with this clear danger to myself and to others, I didn't act.

In the last year, I started thinking seriously about doing something about my sleep. My reluctance was partly rooted in having a busy schedule, and also the conviction that something as natural as sleep shouldn't require medical help. My colleague Mickey Williams had been telling me how his sleep apnea had been turned around by a CPAP machine and what a difference it had made to his quality of life. His testimony made a big impression on me.

Finally, I reached a turning point. Last month, my in-laws came to visit. While driving them back to the airport, they staged an intervention. "We heard you snoring last night and it sounded like a wild animal," said my mother-in-law. "You've really got to do something," said my father-in-law. I agreed at long last to finally take some action.

There happens be a sleep center not far from the Neudesic Irvine office, and I made an appointment to see a sleep specialist. I had to fill out some forms describing my symptoms, and log my sleep behavior for a week. When I saw the doctor, he took one look down my throat--apparently I have a narrow air passageway--and declared me an emergency case. That weekend, I was scheduled for a sleep study.

In a sleep study, you come into the sleep center and sleep overnight in a private room so you can be observed. The sleep technician connected about 25 sensors and electrodes to me--mostly on my head--which allowed them to monitor everything from brainwaves to leg motion to oxygen level. Feeling somewhat like a science experiment with all those wires, I did my best to fall asleep and eventually I did.

In the middle of the night, I was awakened. They had enough data on my problems, and wanted to see how I did with a CPAP machine. They strapped a facemask on me and told me go back to sleep. It was a little hard to fall asleep at first but before I knew it morning had arrived and it was time to go home. Even then, I felt more refreshed than usual just from the half-night on the machine.

The results from my sleep study were eye-opening: I had stopped breathing 159 times in the first few hours I had been observed and my oxygen level was down to 66%. Fortunately, when I was placed on the machine, my oxygen level went up to 96%.

I was sent home with a loaner CPAP machine, and a few days later I had a machine of my own. CPAP stands for Continuous Positive Airway Pressure, and all that means is that air is flowing into your nose and down into your mouth and throat. The idea is that this opens up your airway blockage and you get the oxygen you need. For some people this takes getting used to, but I had no problem even on the first night. About all you have to do is remember to keep your mouth closed; otherwise you're breathing normally. The machine is very quiet, the only noise is the whisper of air flowing and your own breathing.

The first morning after trying this at home, I felt like a new man! I had slept through the night, on my back, without waking up, for the first time in many years. I was alert. I felt great. The doctor told me this was common: 95% of his patients have that reaction. I now feel this way every morning.

If you're having sleep problems, don't be in denial as I was for many years. Modern medicine can really help. Get the problem looked at and make a change for the better. Your life will improve and those around you will be better off as well.

Tuesday, October 21, 2008

Science Fiction: Alive and Killing Me

Warning: this is a rant.

One of my great pleasures--both growing up as well as now--is science fiction. Whether we're talking books, television, or movies, science fiction has slowly but surely developed a pretty sizeable audience over the years--but oddly, the majority of that audience has very strange sensibilities (I present evidence for this below). It's as if the sport of football expanded its audience ten-fold, but most of the new people had no appreciation for what the game was about--and furthermore, were clueless that they were clueless!

Exhibit A: Science Fiction not equals Fantasy

Science fiction and fantasy have nothing to do with each other. There, I said it. If you disagree, you may consider yourself to be in that aforementioned clueless category.

I loved science fiction as a boy, devouring books by authors such as Arthur C. Clarke, Larry Niven, Isaac Asimov, Cifford D. Simak, Robert Heinlein, and Cordwainer Smith. By today's standards you would call this "hard" science fiction but back then it wasn't necessary to distinguish "hard" from the intruders that wanted to also be labeled sci fi.

There was also a completely unrelated genre called fantasy which I had no interest in. You could find both types of writing in a bookstore, but they would be nowhere near each other physically (since they have nothing to do with each other). No one would have thought to group them together; that would have made as much sense as combining gardening books with crossword puzzle magazines.

Somewhere in the 80s, someone got the idea that science fiction and fantasy were two sides of the same coin. Bookstores bought into this idea, resulting in the combined "Sci-fi/Fantasy" section you now see everywhere. This initial baffling development was bad enough--it made it harder to figure out which books were actually about sci fi--but that was just the beginning. Eventually sci fi authors themselves were persuaded that they should "try their hand" at fantasy, perhaps driven by arguments from publishers that there was more of a market for fantasy than sci fi. Whatever the reason, many writers off and went in this direction. This all but killed off "hard" sci-fi. All the books were now fantasy books.

How sci-fi and fantasy could ever be viewed as related remains a complete mystery to me. A sci fi story presents us with a semi-plausible premise where one or more interesting things are going on--such as being set in a future time, or in a society with different rules, or after a major new invention or discovery, or taking place on a different planet, or after being contacted by aliens--but the rest of the story has to follow the rules the reader expects (the rules of physics, the rules of human behavior, etc.) and be credible. That's what makes science fiction interesting, it makes us think about things that could actually happen (some more likely than others) and how they might change us or our world and how we could or should respond and what the consequences might be.

In fantasy, this pattern is inverted: the author is freely making up all of the rules of the world, and the world itself, and anything and everything is permissible in the name of magic. It's the most unsatisfying thing I can imagine reading.

"Hard" sci fi largely died for 20 years. Only a very few dedicated authors, such as Greg Bear and Charles Sheffield (sadly now deceased) were writing hard science fiction. Fortunately, there is good news to report here: hard science fiction is making a comeback, with a new generation learning what science fiction was originally all about. It's been a long hard wait, but it's very exciting to see.

Exhibit B: Science Fiction not equals Horror

If it's a stretch to confuse science fiction with fantasy, surely no one would confuse it with horror... or would they. Apparently Hollywood enjoys marketing films to science fiction fans and getting them into the theatre, only to rudely change the movie's genre midway through and abandon the plot. Here are some of the worst offenders:

Event Horizon". Wikipedia has the gall to call this a science-fiction movie. Go read the initial plot description and you'll be intrigued. If you started to watch it you would be convinced it was a promising science-fiction movie. That is, until you get past the first half hour of the movie. Space movie turns into vampire movie. All science goes out the window, to be replaced by the supernatural, blood, and gore.

Life Force." Wikipedia is a little more honest this time, calling this a "science-fiction/horror film." Out in space, we discover an alien vessel--and some of the crew is found alive in stasis. After bringing them back to earth, voila! you're transported to a vampire movie as everyone on earth turns into blood-sucking zombies.

Exhibit C: Science Fiction not equals Martial Arts

Not satisfied to taint science fiction by blending it with first fantasy and then horror, martial arts has to get into the act. And this brings us to the ultimate example of science fiction that is not science fiction.

The Matrix." A science fiction movie with a big budget, well-known actors, and special effects. And a great plot premise. Except, mid-way through the science and plot are abandoned. Completely. The rest of the movie is about martial arts and mysticism.

Amazing, people lap up The Matrix and happily pay to see its two sequels. I can only conclude people like watching special effects and action sequences so much that it doesn't matter if there is continuity, a plot, or suspension of disbelief.

Exhibit D: Inability to Tell the Difference Between Good and Bad Sci-Fi

In evaluating what's good and bad in sci fi--and here I'm primarily thinking of television series and movies--people are making some really odd judgments these days. Perhaps it's partly due to what the book publishers and movie makers have done to us as described earlier.

Let's be clear: if someone claims to be a "sci fi" fan but offers "The Matrix" as a shining example of it, something is very wrong. "2001: A Space Odyssey" and "Planet of the Apes' (the original) and "The Day The Earth Stood Still" are great sci fi movies. "The Matrix" is worse than bad sci fi, it's really not sci fi at all. By the same token, "Star Trek" is good TV science fiction and "Lost in Space" is horrible.

There were 2 television science fiction series I really liked in the 70s. One was "
UFO", the other "The Starlost", both short-lived because they couldn't find a large enough audience. I recently had the pleasure of catching up on "UFO" thanks to a colleague's DVD collection, and now "Starlost" has just come out on DVD.

Starlost had a great premise. The earth is destroyed, but before that happens the final generation launches space ship Ark to send a remnant of humanity to a new world, which will take many centuries. The ark is a huge ship that contains bio domes of different societies. An awesome setting for all sorts of interesting episodes, since each bio dome can have its own interesting culture and stories. Moreover, an accident of some sort occurred and the ark is on a collision course with a Class G star, a problem that really needs to be solved one of these centuries. This has all the makings of a great sci fi show.

Is this appreciated, by anyone? If you did a quick search online, you would conclude "No" in a hurry. There are plenty of articles on "Starlost" on the web, such as
this one and this one and this one, that claim Starlost is "the worst science fiction series ever." Frankly, I'm getting tired of it.

Worst sci-fi TV series ever? That's quite a statement. Did you not see Lost in Space? Or Cleopatra 2525?? Granted, Starlost had more than its share of production problems and was a far cry from what Harlan Ellison, Ben Bova, and Douglas Trumbull were after (the major talent left the show, the budget shrank and shrank, and it was shot on video in Canada during a writer's strike). Nevertheless, its brilliance shines through and what remained was still highly imaginative and quite watchable to avid sci-fi fans in 1973. To call it "the worst sci fi TV series ever" makes we wonder what these people regard as good sci fi, and what their criteria is. I happily just ordered my Starlost collection of DVDs, and as soon as I mentioned it to a colleague he did the same without hesitation. I predict it will sell well despite the mockery by the clueless.

Saturday, October 18, 2008

Speak Your Client's Language

Effectively Communicating with Business And Technical People

To be a good consultant, you have to learn how to effectively communicate. At most client engagements you will likely need to communicate with both business and technical people.

Business people and technical people are pretty much the same on the outside: 2 arms, 2 legs, etc. But they think, speak, and act differently. The only way you can discover whether someone is business-oriented or technically-oriented is to talk to them, listen to them, and find out what they do.

Business people and technical people live in very different worlds. Business people are focused on the business. They easily relate to profit/loss, sales, marketing, customers, risk, productivity / efficiency, agility / time to market, competitive advantage. Business people have a hard time understanding technical people--they seem to be of a different culture and speak a different language.

Technical people are passionate about technology. They easily relate to design, engineering, hardware / software, creating things, and find solutions to interesting technical problems. Technical people have a hard time understanding business people--they seem to speak a very ambiguous and imprecise language.

It might seem like the "right" way to address this problem is to get both sides to reach out to each other: business people should learn more about technology, and technologists should learn more about business. For a consultant, this is not the correct answer, for two reasons.

  1. First off, it's not the customer's job to bridge the gap, it's the consultant's job. Even if most of your work is technical, interacting with business and technical people is something a consultant should always be prepared to do. The consultant has to bridge the gap, not the customer. Always remember, a consultant is more than a developer.
  2. Secondly, the purpose of I.T. is to further the business's objectives. No matter how technical your work is, you should be able to relate your project to how it furthers the business strategy. Your work is part of a cycle where a business strategy is formed, departments (both technical and non-technical) execute on the strategy, and results are produced and measured. Every consultant should learn on Day 1 of a project how their work relates to the business objectives.
So just how do you bridge the gap between business and technical people?

First off, identify your audience. If you're at a meeting, who are all those people at the conference table? They're just question marks until you find out who they are and what they do. Once you know you're addressing a VP of Marketing or a salesperson, switch to business speak. When you're talking to a development manager or I.T. administrator, switch to techno-speak.

When speaking to business people, use terms they can appreciate. For example:

  • Strategy. Every business runs on a plan.
  • Agility. An agile, more nimble business is not mired down and can turn on a dime in response to executive direction.
  • Productivity. A more productive business creates products or delivers services faster and cheaper.
  • Customers. Customers are the engine that keeps a business running.
  • Industry terms. Learn something about the industry your client is in and get acquainted with the industry-specific terms.
Let's talk about some skills to develop:

  1. Listening. This is the most important skill of all.
  2. Speaking plainly. A great example of this was during the congressional investigation of the Challenger Space Shuttle disaster. After mounds of incomprehensible techno-babble from engineers, Feynman lifted some o-ring material out of his icewater and snapped it like a twig. Something even an 8-year old could understand.
  3. Relevance. Is what you are saying relevant to the other party? Is how you are saying it understandable to the other party?
  4. Make it Clear. It takes work to say things clearly and simply. "If I'd had more time, I would have written you a shorter letter."
All right, but if we're addressing a mixed audience? What good is it to know how to speak your client's language if you're in a room full of business and technical people?

There are ways. You can find things that are of interest to everyone (when somone yells "fire" in a crowded room, every person is interested regardless of their background). For example:

"These proposed changes will improve security?"

Everyone views security as important.

You can also sprinkle in something of interest to the technical people and something of interest to the business people in what you say. For example:

"This more efficient arrangement will improve the bottom line by processing orders faster."

Something for everybody. Technical people will like the efficient arrangement part, while business people will respond to improving the bottom line.

Non-ideal for a mixed audience:
"In the next spring we want to enable SSL and implement authentication/authorization using a custom membership provider."

"The next round of changes we are proposing will increase security of the web site, including tracking who does what."

Visual explanations are often helpful in getting things across to a mixed audience. A simple chart can link technical projects to business impact without being overwhelming.

You should never be afraid to ask "what do you mean?" (in a respectful way, of course). It's better to get clarity on what someone is telling you, right up front. By the same token, if you're presenting and some asks you "what do you mean?", realize you probably need to adjust what you're saying to better reach that person.

The better you know your audience, the more effectively you can communicate with them.

Software + Services (S + S)

Microsoft's vision for the future of computing is currently called Software + Services (or S+S). While S+S is a powerful concept, you won't necessarily "get it" from the name alone, since "software" is a very general term and "services" is an overloaded term which suggests different meanings to different people.

The basic idea is this: whether you're a consumer or a business, you're used to running software locally--that's the "Software" part of S+S, local software you run on your home computer or on your business premesis. In the near future, your options are going to open up. Some of the sofware you run might be "out there in the cloud". That's the "Services" part of S+S--software hosted remotely.

That might not sound like anything terribly new--we've all been "running software hosted remotely" for 10 years whenever we access the World Wide Web--but the capabilities we're going to get as we enter this new era of cloud computing will be new and compelling.

You can read about S+S on MSDN. We'll have lot more detailed information to chew on after the upcoming MS Professional Developer Conference (PDC) where Microsoft will unveil its cloud computing platform.

Are You Over-Engineering?

“Over-engineered.” Most software developers would cringe to hear their work described in this way. Yet over-engineering is common. One reason for that is, it’s easier to spot in someone else than in ourselves. To gain insights into what over-engineering is, we’re going to study what’s happened to the potato peeler over time. Below you see the traditional potato peeler that has served American households well for nearly a century. If anyone in your household cooks, there’s a good chance you have one of these in your kitchen.

There have been some incremental improvements along the way (nicer handles, angled blades) but the basic design hasn’t changed much.

The traditional potato peeler has a lot going for it. You can get the one above for about $2. It has only two moving parts, is compact, stores easily, usage is intuitive, and they last for decades.

Those design characteristics would seem hard to improve upon, but that hasn’t stopped companies from trying.

Enter the Rotato, shown below. To peel a potato (or some other fruit or vegetable), you mount it on the Rotato’s spiked base and lower the top arm. You then turn the crank and Presto! the peel comes off in one continuous strip. There is also the Rotato Express, an electric model.

Reviewers agree that when this works according to plan, the experience is fast and fun, and the potato is well-peeled. Reviewers also agree that the device does not always work well.

1. Unnecessary Complexity
At first glance, we can see the Rotato has many moving parts. While complexity is not an automatic indication of over-engineer-ing, it is one of the warning signs to pay attention to. When you see complexity that is unnecessary to do the job at hand, you have over-engineering.

Over-engineered designs are often unnecessarily complex.

2. Comparison Aids Design Assessment
To prove something is over-engineered you have to show a simpler and better design exists. Having another design to compare against reveals a great deal. We learn a lot about the Rotato when we compare it to the traditional potato peeler. The Rotato is a lot bigger, has many moving parts, costs ten times more, takes up a fair amount of space, and often lasts less than a year.

Over-engineering is best proven by comparison to a superior design.

3. Fatal Flaws Really Are Fatal
Design flaws come in two flavors, fatal and questionable. Fatal design flaws are not acceptable under any circumstances.

Does the Rotato have any fatal design flaws? Here’s one: the Rotato doesn’t peel the whole potato; in fact, it comes with a paring knife that you need for the top and bottom of the potato, and for “problem areas” of irregularly shaped potatoes.

Also hard to overlook is that the Rotato stopped working for many reviewers after they had peeled only 3 potatoes, requiring them to replace the blades.

Fatal design flaws cannot be compensated for by other aspects of the design.

4. Weighing Negatives against Positives
What about design negatives that aren’t fatal? A negative might turn out to be acceptable if there is a related benefit that is compelling, such as unparalleled safety, longevity, or an appealing feature. Or it might not. The negatives have to be weighed against the positives.

We identified some negatives about the Rotato earlier, but what are its positives? The Rotato peels potatoes much faster than a traditional peeler and doesn’t get your hands dirty. In the case of Rotato Express, there’s no manual labor involved in the peeling since an electric motor does the work. The experience is also said to be fun; this is a new benefit (you never hear traditional potato peeling called fun).

How do the positives weigh against the negatives? As you might guess, you’ll get different opinions from different people. Some reviewers love the Rotato while some hate it. Many of the favorable reviewers openly admit the Rotato’s problems yet love it nevertheless and recommend it to others. Many are on their second unit. The positives seem to outweigh the negatives for many users.

Perceived design negatives need to be weighed against positives.

5. Bells and Whistles Nobody Ordered
Another type of over-engineering is to festoon what you’re building with all sorts of interesting features that nobody asked for and nobody is likely to need. In soft-ware, this is often justified as anticipating future events or future needs.

The Rotato doesn’t have this particular problem, but a $2,320 peeler from Hobart might qualify. It comes with a stainless steel cabinet, a wheeled floor stand, leveling legs, a timer, and a peel basket.

Even if you know there will a future need for something, that doesn’t necessarily justify building it now. It’s much more important to focus on present requirements.

A design with unnecessary, unrequested features is over-engineered. Don’t design too far into the future.

6. Suitability is in the Eye of the Beholder
In reading reviews of the Rotato, it was interesting to notice that some of the most positive ones came from people with arthritis, or had limited use of their arms, or who had lost an arm. For them, the Rotato was the only way they could get back into the kitchen. We might rate the Rotato generally over-engineered, but for these people it is a godsend.

An over-engineered solution may be justifiable if it offers something crucial other designs do not.

7. Lack of Refinement and Refactoring
Over-engineering is a normal by-product of the software development process, which is why we now recognize the importance of refactoring. You often can’t see the ideal design until you’ve first achieved some prototype that works. The important thing is to not stop there. Now it’s time for iterative refactoring and design review.

Despite its problems, the Rotato has many loyal fans. It may be a diamond in the rough, a design not yet sufficiently refined.

Failure to refine and refactor often leaves an over-engineered result.

8. Trade-Offs
Design always includes trade-offs. In soft-ware, many of these trade-offs are well known. Security is at odds with performance. Database normalization is at odds with simplicity of queries. And so on.

Sometimes, people fool themselves into believing they don’t have to make any trade-offs. They reason, if we make this the best we can in every area, if we spare no expense, we don’t have to make any trade-offs. These people are wrong. The resulting complex design will bring consequences with it such as high cost, high maintenance requirements, fragility, or other risks. Luxury cars are a good example.

A design without trade-offs is almost certainly over-engineered.

9. Elegance
A sure sign of good design is elegance: over-engineered solutions are completely lacking in it. This is the place where technology meets art. Michelangelo said, “to sculpt, take a block of marble, and take away what doesn’t need to be there.”

All human beings recognize elegance when they see it. In music, not everyone is a composer but we all know good music when we hear it. The craze over iPods and iPhones is driven by design elegance.

Which is more elegant, the space station envisioned in 2001: A Space Odyssey (left) or the actual International Space Station (right)? Kind of like comparing the Sistine Chapel to an oil refinery, isn’t it?

Good designs are elegant, poor designs aren’t.

Final thoughts
An over-engineered solution is a job half done. As master craftsmen of our trade, we owe it to our customers and ourselves to make it all the way to the finish line.

While researching for this article, I explored what others had to say online and came across a blog posting using the potato peeler and the Rotato as an example of over-engineering. I thought this was a brilliant illustration--but I also felt the short blog posting did not really do the subject justice. That's my justification for borrowing the same example and giving it a fullter treatment. Unfortunately, I can't find the original posting to give proper credit for the idea.

Saturday, October 4, 2008

Clearing up Cloud Confusion

Cloud computing. Utility computing. SaaS. Software + Services.
What does it all mean?

Here's a quick primer on what these terms mean and how they are related.

Web 2.0
The second-generation uses of the web, which emphasize social networking, collaboration, and sharing.
Examples: Facebook, MySpace, Flickr, blogs, communities

Software as a Service (Saas)
Software you access over the Internet to use.
Examples:, NetSuite, Taleo, RightNow.

Utility Computing
Charge for use of hosted computing resources.
Pay-as-you-go based on what you use, just like electricity or water from a utility.

Software + Services
Microsoft's term for some software local ("Software") and some software out in the cloud ("Services").

Example: iTunes.

Cloud Computing
Internet-based development - platform is a data center.
Run your own software in the cloud with fantastic ability to scale.

Wednesday, October 1, 2008

Cloud Computing: The Next Big Thing

Unless you've been living under a rock or aren't involved with technology (kind of the same thing), you've likely noticed a growing level of buzz about something called Cloud Computing. You should pay attention, because this is the Next Big Thing.

The general idea is pretty simple: Right now, you probably do your computing using some hardware and software which is located locally on your premeses. Instead, you could move some of that to a data center "out there in the cloud"--in other words, accessed over the Internet. Both consumers and businesses will be getting into the act as cloud computing becomes real.

This probably doesn't sound like all that new an idea; we've had outsourced hosting for quite some time. And then there was all that excitement in the 90s about "Application Service Providers" (ASPs) that didn't seem to take hold.

Ah, but that was then. Things are different now. Cloud computing is hosting on steroids, using the mammoth data centers that have been built to support communities such as Amazon, Google, Yahoo, and MSN. Cloud computing can provide super-scalability and automated provisioning on-demand, in response to changes in usage levels. Cloud computing will offer interesting business models, such as pay-as-you-go. Virtualization technology, already very popular in the enterprise, has paved the way for a lot of this data center technology.

As a simple example, imagine your business decides to start using an Internet-based expense tracking application instead of handling it locally. If your business grows rapidly and you take on many more employees, you pay a bit more per month for the application but you don' t have to do anything to handle the extra load. Likewise, if times are tough and you reduce head count, your monthly charges go down. This is very different than running an application locally, where you have to anticipate peak load and ensure you have enough computing power to handle it. With cloud computing, it's automatic and resembles how a utility provides you with electricity or water.

That's just the tip of the iceberg. Depending on which of the cloud computing platforms we're talking about, you could conceivably be talking about
  • Running consumer or business software in the cloud
  • Applications that are fully or partly located in the cloud
  • Ability to run canned packages or your own applications in the cloud.

Cloud computing isn't just about relocating your applications from one place to another, it will give us new ways to design applications. An example of an application that has both a local component and a cloud component is iTunes. You download the software to your local PC or Mac, but it accesses a music store out there on the Internet.

It will be interesting to see how the big players stack up in terms of their offerings, business models, and competitive advantages. Some vendors are already out there with their cloud offerings, such as Amazon's Elastic Compute Cloud (EC2) initiative. Others will be announced shortly. My particular area of interest is in what Microsoft's cloud computing platform will look like, as I work for a national Microsoft consulting partner. All should be revealed at the end of October at the MS Professional Developer's Conference in LA. I can't wait.