The ExpandoObject Stringified

I had a requirement to output some data as part of a number of pages in an e-commerce site. This data would change from page to page and typically get richer as a user would go through an order path in the application. The data needed to be in JSON format and was going to be used by the marketing guys to help track information about user behaviour across the order process. The data was also subject to change regularly depending on what the marketing team wanted to track. In short, I had to construct a JSON string on the server side in C#.

I had seen something like this done before where a StringBuilder was used to construct a large JSON string which was then pushed out in the response with the rest of the page markup. This approach seemed messy and difficult to maintain. I did a bit of research on some approaches to constructing a potentially complex and changing JSON object using C# and came across the ExpandoObject class. MSDN has a nice succinct explanation of ExpandoObject:

"Represents an object whose members can be dynamically added and removed at run time."

You can find this class in .NET 4 in the System.Dynamic namespace. ExpandoObject lets you effectively "model" an object on the fly with properties that don't really exist (just like using the ViewBag in ASP.NET MVC). In my case, it saved me from writing a number of classes or complex strings just to hold data that was to be output in page responses. Instead, I could use ExpandoObject and then use a JSON serializer to stringify the object.

The example below shows how easy and flexible it is to use ExpandoObject with the Newtonsoft .NET JSON library (available as a NuGet package):

dynamic orderInfo = new ExpandoObject();

orderInfo.OrderDate = DateTime.Today;

orderInfo.ProductTypesAdded = new List<dynamic>();
orderInfo.ProductTypesAdded.Add("Radio");
orderInfo.ProductTypesAdded.Add("Television");
orderInfo.ProductTypesAdded.Add("Recorder");

orderInfo.ReachedCheckout = true;
orderInfo.PaymentMethod = "Card";
            
orderInfo.DeliveryInfo = new ExpandoObject();
orderInfo.DeliveryInfo.Method = "First";
orderInfo.DeliveryInfo.Country = "UK";

var json = JsonConvert.SerializeObject(orderInfo);

The resulting JSON is:

{
    "OrderDate":"2013-10-11T00:00:00+01:00",
    "ProductTypesAdded":[
        "Radio",
        "Television",
        "Recorder"
    ],
    "ReachedCheckout":true,
    "PaymentMethod":"Card",
    "DeliveryInfo":{
        "Method":"First",
        "Country":"UK"
    }
}

CRUD against MongoDB via C#

In my previous post, I gave some of my first thoughts about NoSQL in relation to MongoDB. In this post we will look at how you can perform basic CRUD operations against a MongoDB collection using the official C# driver. The purpose of this post is for me to use it as a handy reference when working with MongoDB in the future - hopefully you'll also find it useful. I will assume that the reader has some familiarity with using MongoDB on the shell.

Throughout the post I'll be using a simple collection of documents where each document stores some basic information about an aircraft. If you load the MongoDB shell and execute the following statements then we'll have a common ground to start with:

db.planes.insert({
    'Manufacturer':'Lockheed Skunk Works',
    'Model':'Blackbird SR-71',
    'FirstFlight': new Date('12/22/1964')
})
db.planes.insert({
    'Manufacturer':'Lockheed Skunk Works',
    'Model':'X-55 ACCA',
    'FirstFlight': new Date('06/02/2009')
})

At this point, you should have a database called mongotest with one collection containing two JSON documents. You can verify this by executing:

db.planes.find().pretty()

We'll now access and modify the planes collection using the C# driver. You can download the driver from here or add it to your solution using the NuGet package (it's easy to do using the NuGet package manager in Visual Studio). For this post, I'm using version 1.6.1 of the official MongoDB C# driver. Assuming you now have added references to MongoDB.Bson.dll and MongoDB.Driver.dll, add the following three using statements at the top of your class (FYI, I practiced with a standard console application directly within the Main method):

using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Builders;

Now we can define a class that will map to a single plane document. We will use this class along with the MongoDB generic C# driver methods to insert, retrieve, update and remove documents from our planes collection.

public class Plane
{
    public ObjectId Id { get; set; }
    public string Manufacturer { get; set; }
    public string Model { get; set; }
    public DateTime FirstFlight { get; set; }
}

Note how we're using the MongoDB.Bson.ObjectId type for our identifier property (we're relying on MongoDB to add unique id's to newly inserted plane documents). Also note that the Plane class must support a parameterless constructor for the driver to be able to work with it.

Let's now take a look at how you can insert a new plane document into the planes collection using the driver. We'll first get a handle on the planes collection, I have the following four lines at the top of my Main method that gives me the required reference (planesCollection):

var connectionString = "mongodb://localhost/?safe=true";
var server = MongoServer.Create(connectionString);
var mongotestdb = server.GetDatabase("mongotest");
var planesCollection = mongotestdb.GetCollection<Plane>("planes");

To insert a new document into the planes collection (note that the Id property will be filled-in by the insert method):

planesCollection.Insert(new Plane {
    Manufacturer = "Unknown",
    Model = "Tornado IDS/ECR",
    FirstFlight = new DateTime(1974, 8, 14)
});

To retrieve a list of all the documents in the planes collection:

var planes = planesCollection.FindAll().ToList();

To retrieve a list of filtered documents - let's say we want only the planes that are manufactured by Lockheed:

var lockheedPlanes = planesCollection.Find(
    Query.EQ("Manufacturer", "Lockheed Skunk Works"))
        .ToList();

To update a document (as a whole block) in the planes collection:

var planeToUpdate = planesCollection.FindOne(
    Query.EQ("Manufacturer", "Unknown")
);
planeToUpdate.Manufacturer = "Panavia Aircraft GmbH";
planesCollection.Save(planeToUpdate);

Note that you can also perform a partial update to a document (not as a block), refer to the Update method on the MongoCollection class.

Finally, to remove a set of documents based on a query from the planes collection - in this case we'll remove all planes manufactured by Lockheed from the collection:

planesCollection.Remove(
    Query.EQ("Manufacturer", "Lockheed Skunk Works")
);

Initial thoughts on NoSQL with MongoDB

I have heard about the NoSQL movement for a while now but only recently started to properly look into a NoSQL technology. A few weeks ago I found out that 10gen (the team behind the popular NoSQL database called MongoDB) are starting a free "MongoDB for developers" course. The course runs weekly, with each week containing a number of videos that you watch, take a quiz on and then submit homework for. So far it has been very useful despite some initial hiccups in administrating the course. We're now starting Week 3 and have already covered quite a lot about NoSQL in general and how MongoDB can be used in a CRUD sense.

 

Week 3 of the course dives deeper into schema design in MongoDB and compares this with schema design in the more familiar relational data model. The course also has a project that runs throughout - which is to create a simple blogging engine using MongoDB and Python. It seems like the future weekly homeworks are going to be geared towards extending the blogging engines' functionality. Although the course material is using Python to communicate with MongoDB, there are a number of driver libraries that you can download from the MongoDB website for other programming languages - such as C#. I haven't had a chance to play with the C# driver yet but plan on writing a blog post about my experience with it in the future.

 

My training at university and all of the work experience I have gained so far has been around working with relational database management systems. So it is quite exciting moving away from the relational mindset and thinking about a different way to persist application data. It is weirdly refreshing to work with JSON documents in collections than working with the tables and rows abstraction in relational databases.

 

One thing that the course instructors emphasise about MongoDB is that your schema design should be geared towards how your application will access your data. So unlike in a relational data model, where you typically normalise your model to the third normal form - in MongoDB your data model will reflect the access pattern your application will use to get or update the data. As MongoDB doesn't provide support for joining data, you typically model your data (or JSON documents) so that each document contains all of the necessary information your application will regularly need for a particular use case. This is interesting as I think there is an implicit assumption here that you'd only ever have one particular access pattern for your MongoDB database... for example what if I wanted to build another application that accesses the data for a purpose that is different from when the original MongoDB database was designed? I'm hoping the answer to this question is not along the lines of migrating/copying the data to a schema that's more appropriate for the new application!

 

At this stage of my exploration, another concern with MongoDB is that there doesn't seem to be a quick out-of-the-box method to generate a pretty looking report for the data stored in the collections. In the relational world, SQL with the table and rows abstraction lends itself nicely to generating reports that non-technical users can easily make sense of. With MondoDB, you can query your collections of JSON documents easily enough in the shell using mongo.exe and the find() method on a collection - but the resulting output is in JSON format which probably won't go down too well with a non-technical user. Having said this, there may still be solutions/explanations to these concerns that I haven't come across - so I'm definitely not letting them stop me from diving deeper into the NoSQL MongoDB world.

C# How To: JSON Serialisation and Deserialisation

I've started to make an active effort in using JSON over XML, the main reason being that I find JSON formatted data to be easier on the eye than XML formatted data. I found a project in our source control repository at work that used JSON formatted data as input. The project used a neat little JSON library that I'll now be using whenever required. The library is called Json.NET. The following code shows how easy it is to serialise a .NET POCO into JSON, and then reconstruct the object back from JSON using the aforementioned library.

Imagine we have a simple class that models a person:

internal class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int Age { get; set; }
}

We can create an instance of this class and then serialize the object into a JSON string using the JsonConvert class as follows:

var inputPerson = new Person() 
    { FirstName = "Alan", LastName = "Turing", Age = 41 };

string json = JsonConvert.SerializeObject(inputPerson);

The json string variable now contains the value:

{"FirstName":"Alan","LastName":"Turing","Age":41}

Reconstructing the object back from a JSON string is as simple as:

var outputPerson = JsonConvert.DeserializeObject<Person>(json);