Be Proud

The other day I called up a former co-worker to share an epiphany I was having. "Hey dude, I just wanted to call and tell you that you should be proud of what we managed to accomplish where we worked."

Courtesy of NeoGaboX

At the time I was pretty down on myself. I helped implement a system from the ground up which grew in a very organic manner. The thing it ended up being was not what we intended to write in the beginning. Web services talked to other web services in a chained together fashion. We only had one database instance where data loading, exporting and reporting all competed for resources. The applications were all coupled to an awkward schema and changes to it required changes to everything. I basically considered it a complete disaster and felt really crappy for having been a large part of its creation.

What a difference half a year can make. I've been exposed to some gnarly stuff both good and bad. The good stuff? Well it's better than most of the stuff I wrote. The bad stuff? Oh man, it stinks. It stinks in ways way I can't even describe, but I'll try my best:

Awkward schema?
The horribleness I created pales in comparison to what I've seen. I got to see a schema which was produced by a vendor which decided it would be awesome to obfuscate the schema by naming tables and columns with 3 letters and 5 digits. It was then picked up by a vendor which decided to add it's own tables and columns filled with inconsistencies. These were easily identified with vanity prefixes. Foreign Keys? Those are for suckers.

Applications coupled to one another via the database?
Oh yeah, I've seen worse there too. Except this time the schemas are bigger. Keys are repeated in places that seem to not need to be repeated. Columns are foreign keys to other tables with names that don't match up. Rogue applications are writing data in ways which break other applications because they forget to update denormalized fields or just omit data entirely. Sometimes they forget to update fields which hold similar data but in different ways. How many fields indicate status for a record in your app? Less than 5? Pffft, you must be doing something wrong.

Chained together applications?
I thought that chaining together services was bad enough. At least those had some form of a contract to communicate between one another. How about applications that are chained via batch job AND are coupled to the database? Run one process and it inserts into a table, run the next process and it processes those records. The 3rd one cleans up after the first two. If something breaks in the middle, may God be with you if you attempt to just re-run the process without cleaning up.

Looking back...
I would have loved to have had the resources to have multiple database instances. I mean, we were developing directly against the production environment. That's like running on a tight rope with no net while holding really sharp knives with your ass on fire. How the heck I never screwed something up in a catastrophic manner, I'll never know. I must have digital guardian angels.

This is when I started to get a warm mushy feeling about what we accomplished. We didn't have reporting databases (that would have been awesome though). We didn't have readonly copies for ad-hoc queries (that would have been awesome too). Nope, we had ONE database and we had a lot of junk coming in and we managed to produce a relatively clean stream of data going out. Here's the amazing part to me: we were handling a lot higher volume of data than these other systems I've been exposed to since. That gives me hope that I'm doing SOMETHING right.

Does it mean we are awesome software developers? Nope. We just did whatever we had to do to get the job done. We had limited resources and we made it happen by using whatever tricks we could find. It was all trial and error. Ultimately I'd say it was more expensive for our employer than if we had what we needed. Looking back, we could have added so much more technical value to the company instead of figuring out how to optimize every single process to run on one machine.

I think I figured out all of the wrong ways to write code. I screwed up a lot of stuff, but it was all fixable. Because of the amount of ridiculous optimization we had to do, it looks like we got quite a bit right the 2nd (sometimes 3rd) time around. Sure some of it was messy, but it isn't near as bad as some of the stuff I've seen since. If I had it to do all over again now, I would have done things quite differently. That's par for the course being a software developer though. We're constantly learning new ways to do what we've already done in a better/faster way. Going forward, I will not let current advancements affect how I perceive past accomplishments. I shipped a lot of software that works. I'm proud of that.

Size Matters: Small User Stories

I've recently joined a team which is in the process of adopting agile/lean methodologies.  This has been somewhat of a challenge since the rest of the organization seems to follow a waterfall approach.  To the rest of the company, we're just a bunch of outlaws slinging software and so it's been difficult to get the buy in from what I understand. We're working towards it, and that's all that matters to me.

I came from a shop which was doing this process quite well.  Prior to that I was in a place where it was pants on fire all the time.  Given the choice, I much prefer the agile process done well. Software got churned out faster and there was less bugs.  Everyone seemed to have a firm grip on a wide variety of the product - there was less of the development silos.

"I want a thing that does a thing like this other thing but different, ya know?"
First and foremost: requirements specifications are important.  The place that had no process was pure insanity.  No one knew what was going on because they didn't really know what was being built.  That lack of knowledge was due to very broad and vague requirements. Without the requirements being specified to any granular level resulted in poor communication and a lot of waste in the form of time re-explaining things.

It sounded so simple when you explained it forever ago...
Big stories are almost as bad as not having any stories.  It's hard to estimate a big story because there is a lot of unknown and a lot of room for change.  The surface area is too large.  By the time you've iterated a big story, the scope is likely to have changed quite significantly from the original simple explanation presented as a afterthought at the end of that meeting you had 3 months ago.  Not only that, but the outside perspective of your progress will seem very slow and your early time estimates will be way off.

What is the smallest deliverable I can provide and still add functionality?
When you've narrowed down your efforts to a very small scope, you have removed surface area for change in the time that it takes you to iterate. That's not to say that feature won't change, because it will.  Once the sprint is done, it's out of our minds and we're moving on.  Change building on existing functionality is good because it builds a better feature.  Change in the middle of implementing functionality is just frustrating as hell.  You just wrote something that no one will ever see.  If they didn't see it, then did it even happen?

That feature demo is closure for you and for the people who sign your paycheck.  They know what you've been up to because they saw it with their own eyes.  They then consciously made a decision to alter direction on something they witnessed work.  Changing something that exists is a lot harder to swallow for a business than changing something that doesn't. I find myself not arguing for a features to be implemented in a certain way anymore because I have less emotional attachment to them once they've been seen.

The other good thing about demoing the smaller deliverables is that your perceived velocity is a lot higher. This is a good side effect because the last thing I want my boss questioning is what I've been doing. For me it also helps keep morale higher because I can get something done fast, get a small pat on the back and then move on.  I stop worrying about where things are going and start worrying about getting the next piece done.

For me, I think the smaller stories provide real velocity gains and not just the perception of such.  I spend less time trying to figure out how the pieces fit together in advance.  Instead worrying more about how this feature fits into what exists already.  If I hit a sticking point, then I know that I need to do some refactoring.  Refactoring code that exists and has tests is much easier than pre-planning all of the what-ifs.

This is all very new to me still.  It's taken me a while to detox from having daily fires to put out. I'm actually able to focus on building better software instead of running around the office with my arms flailing above my head.  As I'm getting to spend more time in agile/lean processes, I'm starting to see big gains in my productivity and mental health. :)

I’m Going to Disneyland!

I had the most awkward conversation the other day and it was so awesome that I just had to share. I wish I could give you some kind of funny back-story which ultimately leads to this conversation, but there isn't one. Ladies and gentlemen, this conversation just happened randomly and without warning. I can't make this stuff up.

Person - So, did you go to school to learn about computers?
Me - Actually I went to school for Computer Science.
Person - So you learned how to, like build computers and stuff?
Me - No, I learned how to write software. This whole building computers thing is just kind of a bonus my employer gets from me. I kind of just picked it up along the way.
Person - (Light bulb dimly lights up over her head) Oh! So you could work for Microsoft!
Me - Yeah, I guess I could.
Person - So, I bet that Bill Gates is your hero, isn't he?!
Me - No, not really.
Person - So, then you must be a Mac guy then, right?
Me - No, I kind of use both. I'm not particularly tied to Macs or PCs.
Person - oh! Oh! OH! You could work for Disney!
Me - I suppose I could. I mean, I could work just about anywhere considering that businesses these days run off computers and all kinds of companies need software developers.
Person - (Blank stare)
(Awkward Silence)
Person - Oh, I guess you're right.
Me - (Swift Exit)

Image courtesy of Bruce Tuten

The funny thing for me if just realizing how little people understand about computing.  Since I have a "degree in computers", I'm qualified to write software, repair broken hardware, fix your recipe spreadsheet, and design microprocessors.  There is no difference really.  After all, it's just a computer, right?  In the same spirit, I fully expect a marketing person to be able to maintain the books of a fortune 500 company.  After all, it's just business.

Similarly, I received a phone call at work because an electrical outlet wasn't working.  Since computers run off electricity, then naturally I must be an electrician also. How nice. I'll be sure to email my past professors and let them know how much my degree helped with flipping a breaker. 😉

Increasing Difficulty of Software Development

One thing I'm starting to notice is that it is much harder to write software now than it was 4 years ago.  Am I becoming incompetent?  Hardly, I think I'm a much better developer now than I was then.  The "problem" is that my company has grown from a single table database with maybe 100,000 records to an unknown number of tables with tens of millions of records.  Back in the day, it was easy to churn out a piece of software because things were much simpler.

  1. The impact of my software was minimal.  I was writing things that weren't directly tied to revenue.  Most of my development was data analysis and simple data importing and exporting into that single table.  A lot of the data was crap already and there wasn't much I could do worse than what was already there.
  2. My software had no other systems to integrate with.  After all, I was paving new ground with what I was doing for my company.  There was nothing else to maintain other than this crappy software I was writing at the time. 

Now there are many things that have to be considered when writing software within my company's environment.  There are several different systems which share data.  I have to be very cautious about what my software touches to keep from disturbing other processes.  Suddenly we now have so much software that maintenance becomes a concern.  Copy and paste works fine (It's still a bad idea!) if you have 2 programs. Copy and paste becomes a nightmare when you have 100's of things.  The same thing goes for embedded logic.  You can get away with a crappy check for a specific login to do something special with small environment.  In a big environment, that kind of stuff won't cut it.  No one will remember it's there and it will undoubtedly rear it's ugly head later on.

So, the most frustrating thing in all of this is having to justify the increased time and resources required to complete tasks.  When someone says something like "Let's make this button for this one login do this special thing", it makes me want to cringe.  Great, now I need to create a mechanism to be able to do "special things" and stub this into perfectly normal code.  And since we can't embed client specific logic and data into the code, we must make a place to store that "special thing".  And since "special things" are forgotten by developers, then we must create an interface for someone to manage these things of special nature.  All of this for one client who probably isn't that big and definitely not worth the effort required to do something stupid that doesn't make sense anyway.  4 years ago I wouldn't have thought twice about completing said request, and I would have just opened up the code and typed up "if(stupidClient) specialThing(); else normalThing();", compiled that and deployed it.  I still find those beauties every once in a while and want to punch myself in the face for having done that.

Now instead of getting praise for quickly doing things, I'm greeted with the phrase "All you need to do is have it do this when this person is logged in and clicks this."  Yes, I'm aware that is what the result should be.  That is evident, but what isn't always evident is the implementation. It's hard to convey to those making these decisions why it isn't so simple.  If it sounds simple, then it must be simple, right?  I wish that were the case.  Every decision about changes or updates to a piece of software have to be considered and weighed carefully.  It's very easy to take a piece of good software that's easy to maintain and turn it into and nightmare over the course of a few feature requests. 

Show Me the Code

Who's governing the standards of the code behind the things you use on a daily basis? Tennessee's lottery has had an issue with their switch to computer generated numbers.  Big deal, the lottery screwed up, we all do it from time to time, right? The problem is there should have been some checks and balances to prevent a bug like this from reaching production.  Are they not using test driven development?  Did anyone actually review the results of the generated test sets?

How can we know that we can trust the software running the things that we interact with?  A friend of mine brought up electronic voting in a conversation we had today.  There is a large issue here.  How do I know that when I press the button that it really is counting towards the candidate that I select? More importantly still, how do I know that the code and hardware are hardened enough to prevent malicious activity?

Obviously, all of this just raises questions in my head.  I'm not sure I have the answers to this, but I know for sure I'd love to inspect the code of some of this stuff.  I'd at least like to know that real life code had some kind of audit.  Think of it as a Good Housekeeping seal of approval or FDA stamp for publicly consumed software.  For lack of a better name, we'll call it the Josh Bush Software Guarantee.  I like that, don't you?  It kind of has a nice ring to it.

Jokes aside, it's important to realize that computers control many aspects of our lives.  Even though the lottery is a tax on the mathematically challenged, I want to make sure it's fair for everyone.