digitalBush

Introduction

Photo courtesy of Leo Reynolds

A while back I ranted about one of the amazingly crappy projects I inherited when a developer left the company. I'm not going to beat a dead horse and tell you how sucky the old app was. I already did that once. Bitching about it wasn't going to do anything, so I fixed it.  It's been running along merrily for a month now, so my confidence is starting to build with it (Yes, I absolutely realized I just jinxed myself).  Instead I want to focus on what I did this time around that I think makes it better.  I'm quite proud of the simplicity of the app and I'd like to share.

Separation of Concerns

The old app was concerned with way too many things.  I've been working hard to componentize various pieces of our business into a service oriented architecture.  The basic workflow of a file from a client goes something like this:

1. File gets dropped onto our ftp site
2. File gets unzipped/decrypted/un-whatever-is-keeping-it-from-being-plain-text
3. Claims are extracted and network discounts are applied
4. File are zipped/encrypted/re-whatever-was-keeping-it-from-being-plain-text
5. File gets delivered to client

Simple, eh?  Well, the old app was concerned with all of this and had client specific logic hard coded into the app.  In place of the one app, we now have 3 services, 2 of which already existed before I even rewrote this thing. 

Service #1 is a windows service that reads client information from a database table.  This service is responsible for unwrapping and re-wrapping the files to and from plain text.  It could care less what the data actually is.  It only has a limited set of duties.  The only thing it knows is: "Where do I get the files?", "How do I make it plain text?", "Which internal web service gets the data when I'm done?", "How do I redo what I just undid?", "Where does the final file go?", and "Who do I notify when something bad happens?".   I didn't personally write this app, but I did have a hand in its direction.  It does what it's supposed to, and it does it well.

Service #3 is the other pre-existing service.  This is where our actual discount logic sits.  This is something I created a few years back and just recently revised.  I can't tell you how this works in detail because that's how we make our money.  If I told you, then I'd have to kill you.  If I did that, then I'd have no one to read my blog.

Service #2 is where the new x12 parser and translator sits.  This service can now be a lot more naive than the original.  There's no client specific logic in the app.  It has one focus, and that is to take the x12 formatted file and turn it into a web service call then take the web service response and but it back into the x12 format.

Focus on Only the Things that Matter

One of the things that plagued the old app was how stingy it was with it's input.  It was trying to validate data for segments that never really mattered for our purposes.  All of the parsing logic was the same for the segments, so I pulled that logic into a base class.  This class is responsible for taking a segment and breaking it's elements down into buckets and the reassembling these buckets via .ToString(); nothing less and nothing more.  For those segments we really cared about, I wrote more specified classes that inherited from the base class.  These classes exposed properties that did type conversion from those buckets I mentioned earlier.

Now, if I'm passed a segment that I've never seen, then I'll just treat it like all of the other segments that I don't care about.  I don't have to know what every segment does in order to do my job, so why try to validate it?

Make it Easy to Setup

I mentioned that client logic was embedded in the app.  This really made it hard to add new clients.  With the client information tucked away in the database, the code became a lot simpler to setup and maintain.  Setup is now a breeze because it's just a row insert.  Maintainability is improved because there is none of the repeated code that once existed for each of the clients.  Testing is easier too because I now have a limited set of inputs and outputs that I need to test. 

Conclusion

I'll admit, none of this is ground breaking stuff.  This is just good practices.  I really took pride in the final product though because I felt like the code expressed my original intentions in a very concise manner.  The other pre-existing services made this process a lot less painful and that's how it should be.  It shouldn't take a small act of God to implement something you are already able to do with client A for client B.  Only time will tell if the app is flexible enough to accommodate future needs.  

There is this one project that I maintain at work that really punches me in the gut.  It was developed by someone who is long gone who will remain anonymous.  Every time I open this project in visual studio, I die a little bit inside.

First let me give you a little background on the project.  It parses the wonderfully stupid 837 document standard mandated by HIPAA(X096 and X098 if you care) .  So, to defend the person who wrote this, it sucks to begin with.  The EDI format is comprised of loops and segments.  Each loop/segment piece is separated by a delimiter which is standardized to be a tilde ('~').  Each loop/segment piece starts with an identifier and then continues on with the data.  Data pieces within a loop/segment piece are separated by a delimiter which is standardized to be an asterisk ('*').  Furthermore, some of those data pieces, depending on the identifier, may have multiple elements which are delimited by a colon (':').  Congratulations, you have been briefed on the basics of an EDI document.  The official documentation for the standards mentioned above are 600+ pages.  Please shoot me now.

So how do you make it worse?  Let me count the ways.

The code is designed to specifically parse the intended documents.  Each segment has it's own class which is custom tailored to parse and produce the intended string.  Great, so now we have a metric crap ton of classes dedicated to parsing segments.  The class count for these segments is somewhere around 70 which are neatly organized into one file (Yipee!).  Each segment object takes a string as a constructor and breaks those data fields into named buckets.  It also contains a method to re-assemble those named buckets back into the appropriate string representation.  Basically each object performs these same basic functions with an extra bit of logic for validation. Is all of that parsing and assembling factored into a base class?  Nope!  There's no code reuse here. Why make smaller code when you can copy and paste!  Furthermore, the assembling method doesn't even override ToString() like a good little object.  Nope, it has it's own method "WriteString()".  This behemoth weighs in at around 15,000 lines of code.

With all of that said, it does its job.  It's not fast and it's very picky about the format, but it has been up and running for a year and a half now with occasional persuading by yours truly.  I have plans to rewrite it, but bigger projects are on my plate at the moment.  I do, however, have a working prototype that uses configuration files from the Perl Module X12::Parser.  The parser happened in 150ish lines of code, and it'll take at least that again to make use of the structure to do what I need to do.  Brevity can be beautiful.