Monday, April 9, 2012

DocShare: Illustrating the CQRS Pattern with Windows Azure and MVC4 Web API

The Command-Query Responsibility Separation pattern (CQRS) is a recent pattern that is gaining popularity. In this post I’ll briefly explain the CQRS pattern and show an example of it in a Windows Azure solution that also uses MVC4 & ASP.NET Web API.

The demo application, DocShare, allows documents to be stored and read from blob storage. There are two separate UIs and web services for queries (reading) and commands (writing). My online sample is at http://mydocshare.cloudapp.net (query on port 80, command on port 8080). The command site is password-secured so I can control who posts new documents. The source code is on CodePlex at http://docshare.codeplex.com.

DocShare Query (left) and Command (right) user interfaces

The CQRS Pattern

The concept behind CQRS is a simple one: instead of a single CRUD-style interface for querying and updating application data, queries and commands are implemented separately. In short, reading and writing are independent. The pattern is simple enough to understand and implement, but the profound benefits that come from using it may not be obvious at first glance.

CQRS Separates Queries and Commands

Martin Fowler has a good post on CQRS. In a nutshell, there are 4 key benefits:

• More Granular Scale. You can scale read and write implementations separately. You probably don’t have the same amount of reads as you do writes.
• Best-of-Breed Approaches. Once you come to grips with implementing reading and writing separately, you realize you can use different data models and approaches for each. For example, you might take advantage of caching, CDNs, or data snapshots in your read implementation.
• Simpler Code. A combined interface for reads and writes can often be complex. Separating them affords simplicity.
• Safety. The query implementation is by definition free of side effects. You can’t accidentally update or modify data when calling it. In a single CRUD-interface, accidental updates are a real possibility if something is wrong in the calling code.


DocShare

DocShare is a Windows Azure solution for simple document management that consists of two web roles, DocShare.Query and DocShare.Command. The DocShare.Query UI and underlying web service can be used to browse and open documents (files in blob storage); DocShare.Command adds the ability to create/delete containers and upload/delete documents.


DocShare Dual Web Role Architecture

DocShare stores documents (files) in blob storage. Since the reading and writing are in different web roles, this gives us flexibility in implementation approach and scale.

To scale the query or command implementation, simply change the number instances assigned to each web role. This can be done initially in the service management configuration for each role, and can be changed over time in the Windows Azure Management Portal. In the configuration shown below, there are 4 query (read) servers and 2 command (write) servers.

DocShare Query and Command Web Roles in Windows Azure Management Portal

Query

The Query implementation runs in a dedicated web role, DocShare.Query.

In the Query implementation, the UI allows users to select a container and see the documents in the container. A document can be clicked to open it in another window. The reads are routed through the Windows Azure CDN, which aids performance with locale-based edge caching.

DocShare.Query User Interface

To improve read performance, the query implementation makes use of the Windows Azure Content Delivery Network (CDN). For example, in my deployment of DocShare a Blob whose write-time URL is

http://mydocshare.blob.core.windows.net/pictures/wedding_02.jpg

becomes

http://az178546.vo.msecnd.net/pictures/wedding_02.jpg

at read time once adjusted to use the assigned prefix for the CDN.

The Web API web service for the query implementation is shown below, QueryController. It provides two methods, GetContainers and GetDocuments. Note the use of Get in the method name, which in Web API means an HTTP GET operation will be used to access them.

QueryController.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net.Http;
using System.Web.Http;
using DocShare.Models;
using DocShare.Models;

namespace DocShare.Query.Controllers
{
    public class QueryController : ApiController
    {
        private static DocumentDirectory DocumentDirectory = new DocumentDirectory();

        // /Query/GetContainers - Return a list of document containers.

        public string[] GetContainers()
        {
            try
            {
                return DocumentDirectory.ListContainers();
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }


        // /Query/GetDocuments - Return a list of documents in a container.

        public DocumentDescription[] GetDocuments(string id)
        {
            try
            {
                string container = id;

                return DocumentDirectory.ListDocuments(id);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }

    }
}

The controller uses a DocumentDirectory class to do its work. The IDocumentQuery interface, DocumentDescription object, and DocumentDirectory class are shown below.

IDocumentQuery.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using DocShare.Models;

namespace DocShare.Models
{
    // Document Query Interface. There are no side-effects to using this interface; no data is changed.
   
    public interface IDocumentQuery
    {
        // Return a list of document containers.
        string[] ListContainers();

        // Return a list of document descriptions enumerating the documents in a container.
        DocumentDescription[] ListDocuments(string container);
    }
}

DocumentDescription.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using DocShare.Models;

namespace DocShare.Models
{
    // A DocumentDescription contains all the metadata about a document and a link to its content.

    public class DocumentDescription
    {
        public string Name { get; set; }
        public string Owner { get; set; }
        public string Created { get; set; }
        public string Size { get; set; }
        public string Link { get; set; }
    }
}

DocumentDirectory.cs

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Web;
using DocShare;
using DocShare.Query.Models;
using System.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;
using Microsoft.WindowsAzure.StorageClient;
using Microsoft.WindowsAzure.StorageClient.Protocol;
using Neudesic.Helpers;
using DocShare.Models;

namespace DocShare.Models
{
    // Document Command Interface. This interface modifies content.

    public class DocumentDirectory : IDocumentQuery
    {

        private string CdnPrefix = System.Configuration.ConfigurationManager.AppSettings["CdnPrefix"];
        private BlobHelper BlobHelper;

        public DocumentDirectory()
        {
            string storageConnectionString = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
            BlobHelper = new BlobHelper(storageConnectionString);
        }

        // Create a document container. Return true if successful.
        public string[] ListContainers()
        {
            List<string> containers = new List<string>();

            try
            {
                List<CloudBlobContainer> containerList = new List<CloudBlobContainer>();
                if (BlobHelper.ListContainers(out containerList))
                {
                    if (containerList != null)
                    {
                        foreach (CloudBlobContainer cbc in containerList)
                        {
                            containers.Add(cbc.Name);
                        }
                    }
                    return containers.ToArray();
                }
                else
                {
                    Trace.TraceWarning("BlobHelper.ListContainers returned false");
                    return null;
                }
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }

        // Return an array of document descriptions enumerating the documents in a container.
        public DocumentDescription[] ListDocuments(string container)
        {
            List<DocumentDescription> descriptions = new List<DocumentDescription>();

            try
            {
                List<CloudBlob> blobList = new List<CloudBlob>();
                DocumentDescription desc;

                if (BlobHelper.ListBlobs(container, out blobList))
                {
                    if (blobList != null)
                    {
                        foreach (CloudBlob blob in blobList)
                        {
                            desc = new DocumentDescription()
                            {
                                Name = blob.Name,
                                Created = DateTime.Now.ToString() /* TODO */,
                                Link = MakeCDNLink(blob.Uri.AbsoluteUri),
                                Owner = "john.smith" /* TODO */,
                                Size = SizeString(blob.Properties.Length)
                            };
                            descriptions.Add(desc);
                        }
                    }
                    return descriptions.ToArray();
                }
                else
                {
                    Trace.TraceWarning("BlobHelper.GetDocuments returned false");
                    return null;
                }
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }

       
        // Convert a Blob URI into a CDN URI.

        private string MakeCDNLink(string uri)
        {
            int pos = uri.LastIndexOf("/");
            if (pos != -1)
            {
                int pos2 = uri.Substring(0, pos).LastIndexOf("/");
                if (pos2 != -1)
                {
                    uri = "http://" + CdnPrefix + uri.Substring(pos2);
                }
            }
            return uri;
        }


        private string SizeString(long length)
        {
            string result = length.ToString();

            if (length > 1024 * 1024 * 1024)
            {
                result = Convert.ToInt32(length / (1024 * 1024 * 1024)).ToString() + "GB";
            }
            else if (length > 1024 * 1024)
            {
                result = Convert.ToInt32(length / (1024 * 1024)).ToString() + "MB";
            }
            else if (length > 1024)
            {
                result = Convert.ToInt32(length / 1024).ToString() + "KB";
            }

            return result;
        }
    }
}


Command

The Command implementation also runs in a dedicated web role, DocShare.Command. Although a purist interpretataion of CQRS might forbid any use of the query interface on the command side, I often find it useful to provide read/write capability on the command side.

In the Command implementation, the UI allows users to do what the query interface does as well as create/delete a container or upload/delete a document. It is password-protected to limit who can perform updates. Whereas reads have been going through the CDN, writes go directly to blob storage.

DocShare.Command User Interface

The Web API web service for the command implementation is shown below, CommandController. It provides three update methods: PostContainer, DeleteContainer and DeleteDocument. By all rights there should also be a PostDocument method, but in my sample I ended up implementing that in the view because I wasn’t quite sure how to make an HTTP file upload work against the Web API; but that’s something I expect to correct in good time. Again note the use of HTTP VERBS in Web API method names: Post actions are for creating content, Delete methods are for deleting content.

CommandController.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net.Http;
using System.Web.Http;
using DocShare.Models;

namespace DocShare.Command.Controllers
{
    public class CommandController : ApiController
    {
        private string AdminPassword = System.Configuration.ConfigurationManager.AppSettings["AdminPassword"];

        private static DocumentDirectory DocumentDirectory = new DocumentDirectory();
        private static DocumentRepository DocumentRepository = new DocumentRepository();

        // /Command/GetContainers - Return a list of document containers.

        public string[] GetContainers()
        {
            try
            {
                return DocumentDirectory.ListContainers();
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }


        // /Command/GetDocuments - Return a list of documents in a container.

        public DocumentDescription[] GetDocuments(string id)
        {
            try
            {
                string container = id;

                return DocumentDirectory.ListDocuments(id);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return null;
            }
        }


        // /Command/PostContainer - Create a container.

        public bool PostContainer(ContainerActionModel request)
        {
            try
            {
                return DocumentRepository.CreateContainer(request.name);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }


        // /Command/DeleteContainer/ - Delete a container.

        public bool DeleteContainer(string id)
        {
            try
            {
                return DocumentRepository.DeleteContainer(id);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }


        // /Command/DeleteDocument/| - Delete a document.

        public bool DeleteDocument(string id)
        {
            try
            {
                string[] items = id.Split('|');
                if (items.Length > 1)
                {
                    return DocumentRepository.DeleteDocument(items[0], items[1]);
                }
                else
                {
                    return false;
                }
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }


        // /Command/GetPermissions/password - coarse security - validate an admin password. Return true if accepted.

        public bool GetAdminPermissions(string id)
        {
            try
            {
                if (id == AdminPassword)
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }
    }

    public class ContainerActionModel
    {
        public string name { get; set; }
    }
}

The controller uses a DocumentRepository class to do its work. The IDocumentCommand interface, Document object, and DocumentRepository class are shown below.

IDocumentCommand.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using DocShare.Models;

namespace DocShare.Models
{
    // Document Command Interface. This interface modifies content.
   
    public interface IDocumentCommand
    {
        // Create a document container. Return true if successful.
        bool CreateContainer(string name);

        // Delete a document container. Return true if successful.
        bool DeleteContainer(string name);

        // Store (create or update) a document. Return true if successful.
        bool StoreDocument(string containerName, string blobName, byte[] data);

        // Delete a document. Return true if successful.
        bool DeleteDocument(string container, string documentName);
    }
}

Document.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using DocShare.Models;

namespace DocShare.Models
{
    // A DocumentDescription contains a document's metadata and content.

    public class Document
    {
        public string Name { get; set; }
        public string Owner { get; set; }
        public string Created { get; set; }
        public byte[] Content { get; set; }
    }
}

DocumentRepository.cs

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Linq;
using System.Web;
using System.Diagnostics;
using Microsoft.WindowsAzure.ServiceRuntime;
using Microsoft.WindowsAzure.StorageClient;
using Microsoft.WindowsAzure.StorageClient.Protocol;
using Neudesic.Helpers;
using DocShare.Models;

namespace DocShare.Models
{
    // Document Command Interface. This interface modifies content.
   
    public class DocumentRepository : DocumentDirectory, IDocumentCommand
    {
        BlobHelper BlobHelper;

        public DocumentRepository()
        {
            string storageConnectionString = ConfigurationManager.ConnectionStrings["Storage"].ConnectionString;
            BlobHelper = new BlobHelper(storageConnectionString);
        }


        // Create a document container. Return true if successful.
       
        public bool CreateContainer(string name)
        {
            try
            {
                return BlobHelper.CreateContainer(name);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }


        // Delete a document container. Return true if successful.

        public bool DeleteContainer(string name)
        {
            try
            {
                return BlobHelper.DeleteContainer(name);
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }


        // Store (create or update) a document. Return true if successful.
        public bool StoreDocument(string containerName, string blobName, byte[] data)
        {
            try
            {
                BlobHelper.CreatePublicContainer(containerName);
                BlobHelper.PutBlob(containerName, blobName, data);

                return true;
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }

        // Delete a document. Return true if successful.
        public bool DeleteDocument(string container, string documentName)
        {
            try
            {
                BlobHelper.DeleteBlob(container, documentName);

                return true;
            }
            catch (Exception ex)
            {
                Trace.TraceError(ex.ToString());
                return false;
            }
        }
    }
}


Summary

The  CQRS pattern provides us with a better factoring of reading and writing than traditional CRUD approaches. It heightens scale, simplicity, and safety.

DocShare is one example of this pattern that makes use of Windows Azure and MVC4 Web API. Web roles on Windows Azure provide the separation and independent scale CQRS specifies. MVC4 and Web API also fit the pattern very naturally.

Download the code at http://docshare.codeplex.com or visit the online sample at http://mydocshare.cloudapp.net/.


Posted by at
Labels: , , , ,

3 comments:

Unknown said...

can you leave cqrs in a private
cloud?

May 1, 2012 at 1:56 PM
David Pallmann said...

William, you can apply CQRS anywhere - there's nothing cloud-specific about it.

May 4, 2012 at 4:40 AM
Azure DevOps said...

Informative blog.
Azure Development Online Training

April 6, 2020 at 1:16 AM

Post a Comment

Subscribe to: Post Comments (Atom)