Codestock 2012

This past weekend I was given the opportunity to speak about CouchDB at Codestock in Knoxville, TN. This is a talk I've been able to give a few times, but this is the first time I've attempted to record it. I've pulled out a 10 minute clip where we walk through storing a fast food order in a relational database and then storing the same order in a document database. The video is rough because all I had was my pocket camcorder.

CouchDB Bucket Demo, Codestock 2012 from digitalbush on Vimeo.

Also, here are the slides for the whole talk.

The sample code for the note taking app and map/reduce are in this repository. The wikipedia demo can be found in this repository. I'm still trying to get my legs with this whole speaking thing, so your feedback is much appreciated. Codestock was a blast and I hope to go back next year!

Manage Your Dependencies with Rake and NuGet

Earlier I blogged about how to perform some basic build tasks in your .NET project with Rake and Albacore. There was one bit about managing dependencies I left off though because I thought it warranted its own post. For the projects I've been working on lately, we've managed to keep our source repository light and nimble by not checking in binaries for all of the dependencies.

NuGet 1.6 came out this week and this functionality is baked in. You can check out the NuGet way in the documentation. The bummer of this is that you have to enable "Package Restore" for each project in your solution. You also now have multiple packages.config to maintain per project. Yes, you can manage it all though the GUI or the package manager console for your projects, but I want it all in one place. I also like not having to do anything on a per project basis other than standard references.

After several iterations on what Derek Greer started, I've ended up with the solution below. Dependencies are declared in the same packages.config format that nuget uses, so you can take something you've already created and centralize it. We have one build step to refresh our dependencies and it looks like this:

require 'rexml/document'
TOOLS_PATH = File.expand_path("tools")
LIB_PATH = File.expand_path("lib")

FEEDS = [
	#Your internal repo can go here
	"http://go.microsoft.com/fwlink/?LinkID=206669"
]

task :dependencies do
	file = File.new("packages.config")
	doc = REXML::Document.new(file)
	doc.elements.each("packages/package") do |elm|
		package=elm.attributes["id"]
		version=elm.attributes["version"]

		packagePath="#{LIB_PATH}/#{package}"
		versionInfo="#{packagePath}/version.info"
		currentVersion=IO.read(versionInfo) if File.exists?(versionInfo)
		packageExists = File.directory?(packagePath)
		
		if(!(version or packageExists) or currentVersion!= version) then
			feedsArg = FEEDS.map{ |x| "-Source " + x }.join (' ')
			versionArg = "-Version #{version}" if version
			sh "\"#{TOOLS_PATH}/nuget/nuget.exe\" Install #{package} #{versionArg} -o \"#{LIB_PATH}\" #{feedsArg} -ExcludeVersion" do |ok,results|
				File.open(versionInfo,'w'){|f| f.write(version)} if ok
			end
		end
	end
end

There's a little bit of code there, but we're getting some good benefits from this one task.

Control over where our dependencies go.
I'm not a big fan of the packages/ folder that nuget uses by default. You may be able to change this in the GUI somewhere, but I haven't seen it yet. Yes, I'm aware that this is trivial, but I got used to storing my dependencies in lib/ and I'm okay with keeping that. :) Every team has their own conventions they like to follow and it's nice to not have to change those just because you want to adopt a new tool.

No weird version number suffixes on our folders.
The default convention nuget uses is to store packages under a folder named {name}.{version}. That's cool until you need to update your dependency to a new version. When you do, you (or your tooling) will have to update the reference paths in all of your *.csproj files to accomodate the new path. I would prefer to store it in a folder with just the name of the package. Keep in mind, this removes the ability to run multiple versions of the same library for different projects within a solution. This hasn't come up on my projects yet though.

No need to keep tabs on what dependencies our dependency has.
I'm hoping this issue will change one day. As it stands right now (NuGet 1.6), if I have a single entry in my packages.config like so: <package id="NHibernate" version="3.2.0.4000"/> then calling $> nuget.exe install packages.config will not get NHibernate's dependency 'Iesi.Collections'. It turns out though, calling nuget like this: $> nuget.exe install NHibernate -Version 3.2.0.4000 will get that dependency for us, so that's exactly how our rake script does it.

I feel like the ruby syntax reads fairly easy even if you aren't familiar with the language. Still though, I think it would be beneficial to add a little commentary.

Line 5 is where we define our source(s) for nuget packages. At work we're using a file share to cache packages and then falling back to the default source when needed.

Lines 11 and 12 are where we load up the packages.config xml file using the XML parser that ships with a default Ruby install. From my reading, there are better gems to accomplish this faster, but this is a really tiny XML file we're dealing with.

Line 13 selects each package node and iterates over it. The next two lines just pick out the id and version attributes into variables. On lines 19 and 20 we read in the version file if it exists and also check if the package directory exists. We use all of that on line 22 to see if we need to restore this package.

If we're all systems go for NuGet launch, then line 23 turns the array of feeds from line 5 into '-Source' arguments for nuget.exe. Line 24 creates a version argument for nuget.exe if we have one. Finally, line 25 shells out to nuget.exe and assembles all of the command line arguments it needs to do the job. When we get our package, we poke(line 26) a version.info file to track the version we've downloaded for future runs.

Wrapping Up
That's it. I almost didn't write this post since NuGet 1.6 supports this scenario out of the box. I still feel like it's worthwhile to have this as part of our rakefile if for no other reason than to manage my packages from a single place. What do you think? Please let me know if you see anywhere I could improve the process.

Cross posted from Fresh Brewed Code. If you haven't taken a look over there, please take a moment to see what we've been up to.

Take Control of Your .NET Builds with Rake and Albacore

If Rake is a gateway drug to Ruby, then Derick Bailey is your dealer. He's created a project named Albacore which makes building your .NET projects stupid easy with Rake. Doing anything in angle brackets for msbuild was painful for me. I write code for a living, so it just makes sense to write code to build my stuff.

Lately I've been doing some work with our builds and TeamCity. A coworker pointed me to Rake and next I discovered Albacore. I just wanted to take a moment to show you how simple it is to set up a build that compiles your code, runs your tests and assembles the output.

require 'albacore'

PRODUCT_NAME = "Autofac.Settings"
BUILD_PATH = File.expand_path("build")
TOOLS_PATH = File.expand_path("tools")
LIB_PATH = File.expand_path("lib")

configuration = ENV['Configuration'] || "Debug"

task :default => :all

task :all => [:clean,:dependencies,:build,:specs,:copy]

task :clean do
	rmtree BUILD_PATH
end

task :dependencies do
	#future post. 😉
end

msbuild :build=>[:dependencies] do |msb|
	msb.properties :configuration => configuration
	msb.targets :Clean, :Build
	msb.verbosity = "minimal"
	msb.solution = "#{PRODUCT_NAME}.sln"
end

mspec :specs => [:build] do |mspec|
	mspec.command = "lib/Machine.Specifications/tools/mspec-clr4.exe"
	mspec.assemblies Dir.glob('specs/**/*Specs.dll')
end

task :copy => [:specs] do
	Dir.glob("src/**/*.csproj") do |proj|
		name=File.basename(proj,".csproj")
		puts "Copying output for #{name}"
		src=File.dirname(proj)
		dest = "#{BUILD_PATH}/#{name}/"
		mkdir_p(dest)
		cp_r("#{src}/bin/#{configuration}/.",dest)
	end
end

:default
So, let's start from the top. Line 10 defines a default task. This is what will get called when you just call rake without any arguments from the command line.

:clean
Line 14 defines a task which just nukes the build output directory. This makes sure we don't accidentally leave artifacts around from a previous build.

:build
Line 22 is my first albacore task. This is the task where I'm compiling my code. Line 23 would be 'Debug' or 'Release' if you're using the default build configurations. The line after is where I tell it to clean the build output and then Build. Point it at a solution file and you're good to go. Easy enough.

:specs
Line 29 is another albacore task to run my Machine.Specifications based tests. Tell it where mspec lives and what assemblies contain your tests. Done.

:copy
Line 34 is a simple file copy task to assemble the build output from src/ and copy them to the build folder. Find all of the project files and go to bin/{config} and get the output files. Move them to a folder with the name of the project.

That's about it. Thanks to Derek Greer for getting me started with Rake. I was able to look at his sample Rakefile and start hacking away. Within a few minutes I had my own rakefile running with albacore tasks. Ruby is pretty straightforward and fun. Playing with Ruby via Rake just makes me want to write more Ruby.

Since I'm a Ruby n00b, I'm sure my Ruby is less than perfect. If you have some suggestions for me to make my code suck less, please leave a comment.

Cross posted from Fresh Brewed Code. If you haven't taken a look over there, please take a moment to see what we've been up to.

devLink 2011

I was lucky enough to get subbed in last minute to give my CouchDB talk at devLink this past week. This was my first time speaking at a technical conference and it turned out to be a great experience.  I did the same bucket demo that worked so well at the user group and it was just as messy and just as awesome as before.

I'm guessing there was 40-50 people there and I had 30 people fill out response forms. The audience was really engaged and asked a lot of good questions.  The feedback was positive, so that's a bit of a relief. I really hope everyone enjoyed the talk as much as I enjoyed giving it.

The slides from my talk can be downloaded here. The sample app I demoed can be found on github (https://github.com/digitalBush/couch-samples) and the CouchDB/Wikipedia demo is also up on github (https://github.com/digitalBush/Couchipedia).

Extension Methods meet the Database

A co-worker and I had a conversation about extension methods today. Somehow this triggered a thought that I needed to see what could be done about the database monotony. Here's the result.

/*
 * DbExtensions.cs
 * Author: Josh Bush (digitalbush.com)
 */

using System;
using System.Data;
using System.Collections.Generic;

namespace Bush.Data{    
    public static class DbExtensions{        

        private delegate T DbAction(IDbCommand cmd);

        /// 
        /// Helper method that does the connection and paramter setup.
        /// 
        private static T ExecuteDbAction(IDbConnection conn, string commandText, IDbDataParameter[] parameters,DbAction dba){
            if (conn.State != ConnectionState.Open)
                conn.Open();
            using (IDbCommand cmd = conn.CreateCommand()){
                cmd.CommandText = commandText;
                cmd.Connection = conn;
                if (parameters != null){
                    foreach (IDbDataParameter p in parameters)
                        cmd.Parameters.Add(p);
                }
                return dba(cmd);
            } 
        }
        
        /// 
        /// Extension method to execute a query and return the resulting records
        /// 
        public static IEnumerable Query(this IDbConnection conn, string commandText, params IDbDataParameter[] parameters){
            return ExecuteDbAction>(
                conn,
                commandText,
                parameters,
                queryHelper  //ugh, can't use yield inside of an anonymous method
            );            
        }
        public static IEnumerable Query(this IDbConnection conn, string commandText){
            return conn.Query(commandText, null);
        }
        private static IEnumerable queryHelper(IDbCommand cmd){
            using (IDataReader r = cmd.ExecuteReader()){
                while (r.Read())
                    yield return r;                
            }
        }

        /// 
        /// Extension method to execute scalar query.
        /// 
        public static object QueryValue(this IDbConnection conn, string commandText, params IDbDataParameter[] parameters){
            return ExecuteDbAction(
             conn,
             commandText,
             parameters,
             cmd=>cmd.ExecuteScalar()
         );  
        }
        public static object QueryValue(this IDbConnection conn, string commandText){
            return conn.QueryValue(commandText, null);
        }

        /// 
        /// Extension method to execute a non-query.
        ///         
        public static int Execute(this IDbConnection conn, string commandText, params IDbDataParameter[] parameters){
            return ExecuteDbAction(
                conn,
                commandText,
                parameters,
                cmd => cmd.ExecuteNonQuery()
            );
        }
        public static int Execute(this IDbConnection conn, string commandText){
            return conn.Execute(commandText, null);
        }
    }
}


I haven't had a chance to really run this code through it's paces, but I did the following test. I used the MySql Connector, but the methods above should work for any database provider as long as it implements the IDbConnection interface.

using (MySqlConnection mConn = new MySqlConnection(ConnectionString)){
    foreach(IDataRecord r in mConn.Query("select post_title from posts"))
        Console.WriteLine(r["post_title"].ToString());               
}

Compare that to the craptacular way of doing it without the extension methods.

using (MySqlConnection mConn = new MySqlConnection(ConnectionString)){
    mConn.Open();
    using (MySqlCommand mCmd = new MySqlCommand("select post_title from posts", mConn)){
        using (MySqlDataReader r = mCmd.ExecuteReader()){
            while (r.Read())                        
                Console.WriteLine(r["post_title"].ToString());            
        }
    }                    
}

I like the improved look. I haven't had a chance to test this with transactions or much of anything else for that matter, so just consider this a proof of concept at the moment.

You can download the extension methods to see for yourself: DbExtensions.cs

Next Page »