Archive for July, 2010

The Importance of Liquidity

Not too long ago I blogged about how I found a job in an unconventional manner. The dev team I ended up working with was top notch. I was able to work with them for two months and I learned so much in such a short amount of time. Unfortunately the company that was supporting said dev team was a little less than stable. I'm happy to say that I've moved on and have found employment elsewhere, but that's not what I want to talk about tonight.

Getting booted on my ass revealed how well our family had prepared for tough times.  My wife and I have made it a point to live within our means and save money for a rainy day.  While people around us have been buying fancy new cars and big homes filled with a lot of toys, we lived in our small house and drove older vehicles to save money. When our son, Hudson, came into the world, we decided to upgrade homes, but still managed to be reasonable with our home purchase.

We bought a new home and decision time came.  Do I put a big fat down payment on the new home to keep the monthly payments low or just take advantage of the super low interest rates our shitty economy has bestowed upon us?  The problem with the big fat down payment is that I would have to give up a large chunk of our savings.  In the end I compromised and met somewhere in the middle.  I felt confident that I had enough remaining for my family to survive a while should bad things happen.

Bad things happened.
About a month prior to my previous employer's implosion, we decided that my wife should stay home with our child.  This was huge for both of us.  My wife has worked since she was 16 and staying at home was a huge lifestyle change for her.  For me, it meant some added pressure to provide for my family. We were up for it and took the plunge. Naturally now that we're solely
dependent on my income is when I would get laid off.  That's just the way my world works.

Is my World Ending?
As it turns out, losing my job wasn't the end of the world.  We had at least 6 months of funds to float us until I could find more work.  Granted, that's not 6 months of eating steaks every night, but it's still 6 months of mortgage, utilities,  health insurance, groceries, etc.  With that under my belt, I was able to go out an look for jobs confident that I didn't have to take the first thing that came my way.  I was able to pick where I wanted to go so that I could do what's best for my family.

This whole experience has just validated my theories about how our family's finances should be managed. I feel good about how things have turned out. Let my situation serve as an example for you. I would say that you need at least 3 months of money available "just in case." You just never know when the FBI and IRS will pay your company a visit.


CouchDB Bulk Load Performance

Yesterday I wrote a very long (sorry!) post about my technique for bulk loading data into CouchDB. I didn't want to make that post any longer, so today I'm going to talk about how well the bulk loading performs. All of these numbers come from my machine which is a Core2 Quad Q9000 @ 2Ghz with 8GB RAM and a single 7200 RPM drive.

As you may recall, the main problem I had was getting too much stuff in memory and having everything blow out with an out of memory exception. The solution I posted yesterday keeps memory under control and seems to keep all four processors busy. Here's a screenshot of task manager while it is running.

Armed with a stopwatch and the CouchDB web interface, I did some crude timings.  I may go back at some point and wrap my loader with some timing code so that I can generate some minute-by-minute graphs, but this will do for now.

Elapsed Time (minutes)Database Size (gigabytes)Document Count
1.3319,000
2.749,000
31.089,000
41.4134,000
51.8181,000
103.6503,000
155.1843,000
206.71,201,000
257.91,473,000
309.11,759,000

I was really impressed with the numbers! After 30 minutes, I was averaging 977 documents/second and and 5.2 megabytes/second. Keep in mind this is all running local on my machine, but the numbers sure are encouraging.