Notes Are Not Developer Friendly

Photo courtesy of err_r

Attention all users of systems across the world: Note fields in applications are not a good place to put things that need to be indexed and parsed.  For whatever reason, the users at my company seem to love their note fields.  They love them so much that they begin to believe that just by the very nature of them sticking something in there that the server (a magic natural language processor, of course)  will make things happen just as they typed it. Damn, and to think: I spent 4 years in college and years later I'm still learning programming languages when all I had to do was just tell the freaking computer what I wanted to do in broken English.  Here, let me give this a go:

  1. Open notepad
  2. Type "Make a data export of our stuff and send it to these people via ftp with encryption.  Do this every month and call me before you send it out."
  3. Save
  4. ???
  5. WTF?  Why isn't this freaking computer working!  POS!

Some of them also seem to think that I actually read the crap they type in there too.  It seems that my brain is somehow physically connected to all things technology at my company.  By that very nature, it seems that anything that is typed into any application anywhere is immediately known to me.  Sweet.

I recently had a nice situation come up where some data was loaded with a note that said those records were part of a larger group.  Forget that we have tags for this type of thing or heaven forbid the group name field.  I get the call "Josh, when we're giving this data out to XYZ company, they need to know about these records that I marked with a specially crafted note field.  They need to know that they are members of ABC group."  It seems no one understands that notes are for evaluation by humans after one has already pulled that particular record up.  Crap, here I go creating a massive temporary index on this text field so that I can make this happen in under a year.

Once I identify these records and create data in the places they were supposed to be based upon these magical identifiers in the note field then I'm feeling pretty good about myself. I think that I'm done.  Wrong again.  I get the next call: "Josh, some of the records are not being identified correctly.  I'm emailing them to you so you can see why."  Oh great, how did I mess this up?  I used the magic identifier you told me: "Strategic Hospitals Incorporated of Tulsa" (FYI: This is made up for the sole purpose of comedic value) .  Once I get the email and start checking the notes of the records in question, it becomes obvious what happened.  I forgot to check all possible combinations of that magic phrase.  Apparently, it would have been too hard to use the same identifier, instead  they chose to shorten it to: "Strategic Hosp. Inc. of Tulsa", "Str. Hosp. Inc. of Tulsa", and even "S.H.I.T."!  "S.H.I.T.", can you believe that?  How in the hell am I supposed to know about this "S.H.I.T."?  Seriously.

What's the moral of the story?  Note fields are a necessary evil.  People want to store things that we (the developers) don't need to know about.  The only way to do that is to provide them a free form field where they can type anything in.  Some people (usually the same ones every time) will continue to put important pieces of data here that belong elsewhere.  I think it's just a fact of life.

  • Philip

    It was extremely hard for me not to laugh when reading this. I just had the wonderful experience of reading through a clients years worth of help desk calls sent to me in an access database exported by someone who knew NOTHING about table relations. My job was to “index” based on user’s initial problem comments and then on the analysts comments…necessary, yes…evil, definatly

    Feeling your pain.

  • Norman Harebottle

    I feel your pain and raise you some technology. :-)

    Next time you run into this you might check out a “little” project called ElasticSearch which is designed for indexing of textual data and rapid searching of said data.

    The feature that would likely be most compelling to you is a thing called Faceted Search which is functionality that is used by most large shopping sites to help users sift through mountains of products to find just the product that they are looking for.

    Great article though and best of luck in fighting the “notes” field wars!