digitalBush

This post should be titled, "Dude, Where the Hell have You Been?" I'm sorry I have ignored this project for so long.

New Home
First, let me get started by saying that I moved the source to github a while back.  It now lives here. If you look at the commit history, I moved it there months ago, fixed a few outstanding bugs, added a feature that someone needed for a specific project and then just left it. I'm picking it back up somewhat, but we'll talk about that in a minute.

Email Bankruptcy
There have been a TON of emails from you guys; so many that I haven't been able to keep up.  Most of the emails have been about two bugs: an off by one goof I made for the completed function and forgetting to use charAt() to access a char in a string. These bugs are fixed in the github repo right now (I think). I'm calling my email situation a total loss.  Sorry to those that have emailed me and not gotten a response.

Birth of a New Project
Part of the reason I stalled on this project was a lack of tests.  For a while I wasn't sure how to even test this thing given that it is purely driven from user input.  Up to this point I had just been opening up a test page in every browser I could think of and running through a few things manually.  A couple of weeks ago I sat down one night and spiked out a rough version of a keystroke simulator which I'm now calling KeyMasher. Please be kind, it's still very rough. Once I get it more polished, I'll put up an official project page on my blog here.  I had found a few other projects which do this and jquery.autotype seemed to be the closest fit. Unfortunately I couldn't get it to work with my specific needs, so I've now written my own with a syntax I feel more comfortable with.  I've already worked out a few tests using this against my masked input plugin.

I'm Just One Guy
After I get a half way acceptable set of tests around it, then I can feel a bit more confident about what I change.  I would like to be able to implement some of the features I've seen come across my email. It will take some time to get everything to a place where it should have already been.  Please be patient, I'll get there.

The other day I called up a former co-worker to share an epiphany I was having. "Hey dude, I just wanted to call and tell you that you should be proud of what we managed to accomplish where we worked."

Courtesy of NeoGaboX

At the time I was pretty down on myself. I helped implement a system from the ground up which grew in a very organic manner. The thing it ended up being was not what we intended to write in the beginning. Web services talked to other web services in a chained together fashion. We only had one database instance where data loading, exporting and reporting all competed for resources. The applications were all coupled to an awkward schema and changes to it required changes to everything. I basically considered it a complete disaster and felt really crappy for having been a large part of its creation.

What a difference half a year can make. I've been exposed to some gnarly stuff both good and bad. The good stuff? Well it's better than most of the stuff I wrote. The bad stuff? Oh man, it stinks. It stinks in ways way I can't even describe, but I'll try my best:

Awkward schema?
The horribleness I created pales in comparison to what I've seen. I got to see a schema which was produced by a vendor which decided it would be awesome to obfuscate the schema by naming tables and columns with 3 letters and 5 digits. It was then picked up by a vendor which decided to add it's own tables and columns filled with inconsistencies. These were easily identified with vanity prefixes. Foreign Keys? Those are for suckers.

Applications coupled to one another via the database?
Oh yeah, I've seen worse there too. Except this time the schemas are bigger. Keys are repeated in places that seem to not need to be repeated. Columns are foreign keys to other tables with names that don't match up. Rogue applications are writing data in ways which break other applications because they forget to update denormalized fields or just omit data entirely. Sometimes they forget to update fields which hold similar data but in different ways. How many fields indicate status for a record in your app? Less than 5? Pffft, you must be doing something wrong.

Chained together applications?
I thought that chaining together services was bad enough. At least those had some form of a contract to communicate between one another. How about applications that are chained via batch job AND are coupled to the database? Run one process and it inserts into a table, run the next process and it processes those records. The 3rd one cleans up after the first two. If something breaks in the middle, may God be with you if you attempt to just re-run the process without cleaning up.

Looking back...
I would have loved to have had the resources to have multiple database instances. I mean, we were developing directly against the production environment. That's like running on a tight rope with no net while holding really sharp knives with your ass on fire. How the heck I never screwed something up in a catastrophic manner, I'll never know. I must have digital guardian angels.

This is when I started to get a warm mushy feeling about what we accomplished. We didn't have reporting databases (that would have been awesome though). We didn't have readonly copies for ad-hoc queries (that would have been awesome too). Nope, we had ONE database and we had a lot of junk coming in and we managed to produce a relatively clean stream of data going out. Here's the amazing part to me: we were handling a lot higher volume of data than these other systems I've been exposed to since. That gives me hope that I'm doing SOMETHING right.

Does it mean we are awesome software developers? Nope. We just did whatever we had to do to get the job done. We had limited resources and we made it happen by using whatever tricks we could find. It was all trial and error. Ultimately I'd say it was more expensive for our employer than if we had what we needed. Looking back, we could have added so much more technical value to the company instead of figuring out how to optimize every single process to run on one machine.

I think I figured out all of the wrong ways to write code. I screwed up a lot of stuff, but it was all fixable. Because of the amount of ridiculous optimization we had to do, it looks like we got quite a bit right the 2nd (sometimes 3rd) time around. Sure some of it was messy, but it isn't near as bad as some of the stuff I've seen since. If I had it to do all over again now, I would have done things quite differently. That's par for the course being a software developer though. We're constantly learning new ways to do what we've already done in a better/faster way. Going forward, I will not let current advancements affect how I perceive past accomplishments. I shipped a lot of software that works. I'm proud of that.

At work I've picked up an existing project and have been adding features to it. This has been a whole new thing for me since most of the stuff I've worked on in the past has been projects of my own creation. I recently got caught with my pants down when I let a bug slip over to the QA people. I got bitten because I trusted that a property name did what it said it did and because I didn't thoroughly test the possible scenarios around this particular feature. It's my fault for letting this by, but I just wanted to highlight how important naming really is.

It all started with a simple user story: User would like to track the expiration date of bacon.

Simple enough, I opened up the view model and added my ExpirationDate property.

    public class BaconViewModel {

        [Required]
        public virtual string Name { get; set; }

        [Required]
        public virtual DateTime? ExpirationDate { get; set; }

        // A lot more code here....

        public bool IsNew{
            get { return String.IsNullOrEmpty(Name); }
        }
    }

Up to this point, creating bacon really only had one property to fill out, the Name. What I failed to notice was that little devil of code in the IsNew property. Does a view model not having anything in the name really make it new? I'm getting a bit ahead of myself though, so ponder on that last question while I continue to tell my story.

The IsNew property itself isn't the only ingredient to this bug. The other part stems from the fact that in the project I inherited, the view does double duty for create and edit. Here's the basic view code before I added my feature:

    <h2>
        <% if(Model.IsNew) %>
            Creating Bacon is the most noble of all tasks.
        <% else %>
            Be careful editing bacon.
    </h2>

    <% using (Html.BeginForm()) {%>
        <%: Html.ValidationSummary(true) %>

        <fieldset>
            <legend>Fields</legend>

            <div class="editor-label">
                <%: Html.LabelFor(model => model.Name) %>
            </div>
            <div class="editor-field">
                <%: Html.TextBoxFor(model => model.Name) %>
                <%: Html.ValidationMessageFor(model => model.Name) %>
            </div>

            <p>
                <input type="submit" value="Sizzle" />
            </p>
        </fieldset>

    <% } %>

Of course this is all just a contrived example, but do you see that bit of code in the h2 tag? If we're creating bacon we want to show one thing and if we are editing then we want to show another. I went about my business and added the UI code to capture that bacon expiration date. When I did all of this I introduced a nice little bug.

What happens when I'm creating bacon and give it a name, but fail to give it a good expiration date? The post happens like normal and a partial view model gets filled out and the ModelState has errors. We throw that partially completed view model back to the view along with the errors so that the user can weep at their awfulness. Except this time that IsNew property isn't feeling so new anymore and now the view goes into edit mode and thoroughly confuses the user. Witness the carnage:

Like I said above, this is a contrived example. What really happened in my case was an exception when I tried to grab the value of the nullable property. Hopefully I've made my point though. The two lessons I'm taking from this experience is:

1. Don't have views that do double duty. If you have parts that are the same, you can use partials to keep things DRY.
2. Make sure that your property names do what they say they do all of the time and not only just out of coincidence.

No small animals got killed because of this bug obviously, but I did take a nice punch to the ego when QA found a fat bug due to an enhancement I made. That will just be one more thing I know to check when I'm spelunking in other people's code. Happy coding!

I've recently joined a team which is in the process of adopting agile/lean methodologies.  This has been somewhat of a challenge since the rest of the organization seems to follow a waterfall approach.  To the rest of the company, we're just a bunch of outlaws slinging software and so it's been difficult to get the buy in from what I understand. We're working towards it, and that's all that matters to me.

I came from a shop which was doing this process quite well.  Prior to that I was in a place where it was pants on fire all the time.  Given the choice, I much prefer the agile process done well. Software got churned out faster and there was less bugs.  Everyone seemed to have a firm grip on a wide variety of the product - there was less of the development silos.

"I want a thing that does a thing like this other thing but different, ya know?"
First and foremost: requirements specifications are important.  The place that had no process was pure insanity.  No one knew what was going on because they didn't really know what was being built.  That lack of knowledge was due to very broad and vague requirements. Without the requirements being specified to any granular level resulted in poor communication and a lot of waste in the form of time re-explaining things.

It sounded so simple when you explained it forever ago...
Big stories are almost as bad as not having any stories.  It's hard to estimate a big story because there is a lot of unknown and a lot of room for change.  The surface area is too large.  By the time you've iterated a big story, the scope is likely to have changed quite significantly from the original simple explanation presented as a afterthought at the end of that meeting you had 3 months ago.  Not only that, but the outside perspective of your progress will seem very slow and your early time estimates will be way off.

What is the smallest deliverable I can provide and still add functionality?
When you've narrowed down your efforts to a very small scope, you have removed surface area for change in the time that it takes you to iterate. That's not to say that feature won't change, because it will.  Once the sprint is done, it's out of our minds and we're moving on.  Change building on existing functionality is good because it builds a better feature.  Change in the middle of implementing functionality is just frustrating as hell.  You just wrote something that no one will ever see.  If they didn't see it, then did it even happen?

That feature demo is closure for you and for the people who sign your paycheck.  They know what you've been up to because they saw it with their own eyes.  They then consciously made a decision to alter direction on something they witnessed work.  Changing something that exists is a lot harder to swallow for a business than changing something that doesn't. I find myself not arguing for a features to be implemented in a certain way anymore because I have less emotional attachment to them once they've been seen.

The other good thing about demoing the smaller deliverables is that your perceived velocity is a lot higher. This is a good side effect because the last thing I want my boss questioning is what I've been doing. For me it also helps keep morale higher because I can get something done fast, get a small pat on the back and then move on.  I stop worrying about where things are going and start worrying about getting the next piece done.

For me, I think the smaller stories provide real velocity gains and not just the perception of such.  I spend less time trying to figure out how the pieces fit together in advance.  Instead worrying more about how this feature fits into what exists already.  If I hit a sticking point, then I know that I need to do some refactoring.  Refactoring code that exists and has tests is much easier than pre-planning all of the what-ifs.

This is all very new to me still.  It's taken me a while to detox from having daily fires to put out. I'm actually able to focus on building better software instead of running around the office with my arms flailing above my head.  As I'm getting to spend more time in agile/lean processes, I'm starting to see big gains in my productivity and mental health.

Wow.  I can't believe this, but it has been a full year since my son Hudson was born.  A lot has changed since then.  I've changed jobs twice, we've moved to a new house, and my wife is now staying home to take care of Hudson. From a tech standpoint, I've given up quite a bit of my non-work time to spend time with my family.  That means less frequent blog posts and less work on my open source projects, but I think it's worth it.  He's only going to be this age once.

The other day I was looking back at his early pictures and I can't believe how much he's grown.  It's just an incredible thing.  Those of you with kids know the feeling I'm experiencing right now. For the rest of you, saying that I'm awestruck would be an understatement. The time has flown by so fast, and I'm enjoying my opportunity to be a father.