Data Access IS The Problem

I have harped on this here before, but once again my blood-pressure is rising because I have seen a simple, efficient use of the powers of Excel marginalized by a client’s inability to get support and cooperation from the “Gate-Keepers” (“Crypt-Keepers” ?)  of corporate data stores.

The Gate-Keepers are often corporate DBA’s who are deathly afraid that users might hurt themselves if they actually get the data they need to help them analyze performance and plan their business futures (even though they are already doing it mostly in “crappy”, dangerous, silos of user-designed and maintained spreadsheets already anyway).  The rest are the makers of third-party accounting or BI software who’s sole motivation is simply to get total control of the data and the process so the business is forced to keep paying them forever and ever …

By forcing BDMs to flail along with stand-alone spreadsheets without direct or reliable integration with REAL corporate FACTS they are exposing the business to risks that may or may not be large.  Regardless, these processes are inefficient at best and frustrating to users and their bosses no end.  The efforts to eliminate spreadsheets altogether (which is the unspoken goal of most IT “Professionals” anyway)  simply has not and will not work.  That is unless they can succeed in moving everyone to the browser-based “calculators” being offered as Excel alternatives by Google and even Microsoft and eliminate the powerful Office Client Excel that has so much power and capability.  That would be unconscionably stupid.

In the case that has me worked up today it is data being collected by a 3rd party software provider who takes simple, straightforward collected data and presents it to the user in an awkward, complex and generally useless GUI in a browser that takes otherwise logical and straight-forward data and makes the export of the data to Excel a painful, inefficient, and in the end unusable, process – thus making use of their data outside of their interface (which they will sell the client and reporting module BTW 🙂 ) impossible.

I have run into this in the BI “Universe” as well where at one client we have been trying to get the data we need for a critical business analysis tool in Excel for TWO YEARS to no avail.  I have found more and more cases where DBA’s will simply not allow departmental users or developers access to the very data they need to run their businesses while it’s being collected and collected but is being used for absolutely nothing.

To me the efforts made by DBA’s and 3rd Party software developers to prevent the effective import of corporate data into Excel is a major impediment to Corporations.  Tragically these Corporations don’t even realize it’s true or how much inefficiency and extra cost they are accepting as normal.  BDM’s as a rule defer to the “Professionals” in their IT brain’s trust or worse they are afraid to cross IT (like you might not to complain to the Police for fear of repercusiions).   It is truly a tragedy.

We should all remember that it is THE BUSINESS that matters, not the careers of the IT “professionals”.  And we are supposed to be working at using technology to make business processes better and more flexible not less.  We are servants to the Business not the other way around and whenever that gets forgotten we all lose.


About Biggus Dickus

Dick is a consultant in London, ON Canada who specializes in Microsoft Excel and Microsoft Office Development.
This entry was posted in Uncategorized and tagged , , , , . Bookmark the permalink.

15 Responses to Data Access IS The Problem

  1. Ken Puls says:

    Great post!

    I’m very fortunate in my company to be both the accoutant and the guy in control of the data. The security/accessibility is always considered, but it’s a big thing to me to make sure that the people who have legitimate needs get access to that data. For us that is often me anyway, as I’m the guy who has to build the reporting structure since nothing we seem to do fits into any canned report format from any vendor.

    One of the first questions I ask when evaluating any new software is whether the data is accessible via SQL. If the answer is no, then the buy decision is most likely a no as well. Nothing drives me more crazy than capturing data into a locked storage vault!

    • DickM says:

      Good stuff Ken. Sounds like you have a good situation but that certainly is not the standard is it 😉 ?

      I’d keep that job if I were you :-).


  2. Omar says:

    Our main supplier has been developing browser based apps and pushing us to purchase and use them. But they won’t open the data to us. My arguments that the data belongs to us fall on deaf ears. Instead, they keep talking about their ability to create reports for us. In the end, when the dealers don’t toe the line, they start using their muscle to force us to use the apps.

    It’s unfortunate. All they would need to do to get me on board is to give me sql access to the data. Oh, and a report that shows all accesses to the data from their internal people so we could monitor the security of our data.

    I don’t have the chequebook, but the people who do will consider my input. It would be so much easier if our supplier wanted to listen to all our needs.

    • Biggus Dickus says:

      Actually all you need is one or more read-only Views or SPs on their beloved server that you can reach out to.

      What scares me sometimes is how these DBAs and 3rd Parties really don’t always understand their own data or how someone could get value out of it. It makes me question their motives frankly. And the fact that Businesses accept it is just plain ridiculous. Really bizarre IMHO.

    • Ken Puls says:

      The real issue you’ve got here is that the business model for your vendor is not about making your life easy, it’s about generating an income stream. Sure they can create custom reports for you, but at what cost?

      We do have one applicatin outsourced to the cloud (our payroll). They give us exactly the same line. As it turns out, they charged us $500 to develop the report, then $50 every two weeks to actually run it for us. Great model… for them! The real irritation is that those costs were not disclosed up front, and now we’re semi locked in to their product.

      We’re actually looking at going though the pain of dumping them to bring it back in house. It will be costly from a human and money perspective in the short term, but we can’t get access to the data the way we need it, and we can’t afford to have them develop all the reports we need either!

      • Biggus Dickus says:

        I have always had a strange business model. I believe that my value to my clients is based on my ability to create solutions for the least cost and that require as little on-going effort on my part as possible for them to get the value they want out of my “solution” – emphasis on the word “solution”.

        Then I hope that providing solutions that they appreciate will lead them to use me again for something else that I will build the same way with the same approach.

        This has worked for me over the years in some companies but has frankly been a failure in most. The reason I am not more successful or richer is because that approach is simply anathema to IT departments and the clients see me simply as a “cheap” solution rather than one that provides “value”. I am a serious threat to other 3rd parties and especially to IT departments and they do whatever it takes to get rid of me. That means that those clients who use my services usually have to do so under the radar and have little money to pay me.

        I have just completed a 2 month project that has made the IT department at the client go ape-shit crazy – because they said it would take 3 years and cost 25 million Euros to accomplish what I did in 2 months on an hourly rate. Rather than giving us credit for what we have accomplished the project is likely to die now. The user will end up with nothing rather than make some people look bad … Not an untypical situation I’m afraid.

        It’s been totally frustrating but I just cannot bring myself to run my business that other way where I try to screw my clients for the most I can squeeze out of them.

        This helps me sleep at night except when I think about how I am heading toward the end of my working career and I have left huge amounts of money on the table over the years.

        Maybe a bit too much honesty ?


      • Ken Puls says:

        Can’t seem to comment on your reply, Dick, so I’ll comment on mine.

        For reference, my philosophical stance is the same as yours. I’m trying hard to build a stable system here that can be supported after I leave as well. It’s not about the money, it’s about professionalism, ethics, standards and the greater good over personal gain. (We joke around here that we subscribe to the Freedom 95 plan!)

      • Biggus Dickus says:

        To the non-Canadians among us, “Freedom 55” is an ad campaign for a Canadian insurance company promising the ridiculous prospect of retiring at 55 through their efforts (yeah sure).

        I have another developer friend same as us who has long said he is on “Freedom 105” – how funny is that 🙂 ?

        p.s. We gotta meet some time Ken, eh ?

      • Ken Puls says:

        Man, that is too funny. I guess I’m just a bit more optimistic then? 🙂

        Definitely do need to meet some time!

  3. Omar says:

    Freedom 55. It never dawned on me that was a Canadian only ad campaign.

    The supplier I was mentioning does do some recovery of development costs, but there are other issues going on. They actually believe their products are improving us dealers. If we don’t play ball, they think we aren’t representing their products properly. Any market share problems then are blamed on the decisions we make including which of their products we use. They will tie other initiatives to our use of their software products. It can be tough to hold a principled stand about what is good for our business.

    Every once in a while, I get to talk to some of their program managers who love the ideas I come up with as to why we need direct access to the data. Of course, nothing ever comes of that. BTW, yes, all I need is read only access.

  4. Biggus Dickus says:

    Wow … “Any market share problems then are blamed on the decisions we make including which of their products we use” That’s bizarre !!! Aren’t you their customer?

    Funny … just tonight I got push back from the DBA at this client saying that we should be using “MS reporting as it was a more appropriate fit to how things are usually done with this type of operation” … aaaggghhh !!

    Every day it seems to be getting harder and harder to break through.

    • Omar says:

      Yes, we’re their customer in the sense we buy their products to resell to end users, but in reality, we’re their agent. The model we work under is similar to the car dealerships, but with even closer control. The local territory manager is really just a puppet. We can explain our needs and plans, but if someone at the manufacturer says jump, they come at us again until we cave.

      That sounds worse than it is. We have a great product line, and great customers. We are constantly being challenged to improve which does make us more competitive in a tough, mature business. Some of their tools they develop really do meet our needs.

      I just wish I had access to data so I could take it to the next level. My job is all about making our sales department more effective, which I really like. We’ve doubled in size recently through a merger and that has really stretched our abilities to communicate and control our business. All my development skills are being stretched.

      • Biggus Dickus says:

        Sounds like a good opportunity to use your skills with Excel – that is assuming you can get at the data ;-).

  5. m-b says:

    I guess I’m lucky to have access to an ERP system which actually uses Excel as the main reporting tool. You can first select the data you want and then an Excel sheet is created which is linked to that data. Then you can save that file back into the ERP system and every time you open it, it gets fed with the latest data.

    • Biggus Dickus says:

      You’re a lucky man 🙂 BUT it would be eve better if you could reach out to that report programatically or as the source of a query table – yes ? That is the ultimate.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s