How To Change Data Sources Dynamically In Excel With Power Query

My Crow Eating Exercise

This past week I asked on this blog whether anyone out there was using the technique I wrote about years ago for dynamically changing the Source Connections for Power Pivot Tables in Excel.  I have noticed that people keep coming to it but no one ever tells me whether they use it or not.

The only response I received was from 2 Power Query “gurus” named Frank Tonsen and a guy who’s name looks like Sam Hide.  I really appreciate their prodding me to look at finding a way to accomplish the same using Power Query.

They detailed to me that it is apparently possible to accomplish the same in Power Query – something I never looked into as I have been satisfied with what I’ve been using and frankly because i don’t use Power Query aggressively to this point.  I admit frankly everyone doesn’t have Power Query installed with Excel 2013 and IT can be a block … and I don’t deal with anyone who uses Office 2016 to this point.

So I decided to look into this and found as always an endlessly complex and frustrating set of Web hits, none of which really seem to resolve this issue specifically. But in all this were snippets of what I appear to need.

My Demands

There were two issues that have to be satisfied for me to make the switch (which I REALLY WANT TO DO)

1. It had to be straightforward and as simple as possible so that many can implement it and

2. It had to accommodate my requirement to change values in an Excel file and have the Data Sources change based on the UI … In other words the user should be able to pick a particular year or Division and retrieve just the data for that … This is to limit the impact of large data sets creating HUGE Excel files and of course eliminate those looooong refreshes.  To accomplish this I create SQL Server Functions and pass parameters to them like this:

select * from fnDataSummary(’01/01/2015′,’12/31/2015′)

where the dates in this formula:

=”select * from fnDataSummary( ’01/01/”&CurYear&”‘,’12/31/”&CurYear&”‘)”

are driven from a Slicer or a pick list inside the spreadsheet … very cool

My Solution

So here we go … This is a way to connect to a SQL Server or SQL Azure database.  I hope you find this simple .

First I suggest you save as often as you can in this process. I find that when developing in Power Query it does crash from time to time   . This doesn’t bother me as this is not a new thing in Excel and most of the time if you close Excel and reopen the file things go back to normal.  Once the file is complete and functional it appears to stay stable (just like Power Pivot.

Select a SQL Server that you have Windows rights to (for convenience), a database in that Server and a simple Select Statement against a Table in that database.

Create a blank Excel Workbook

Create a simple table like this, make it a table and name it “tblConnections” for our purposes (putting your SQL statement in the SQL Field in row 2):

Launch Power Query and ‘Launch Editor’ …

Create a ‘New Source’ … ‘Other Source’ …’Blank Query’, Open its Properties and rename it qryMasterTable …

Open qryMasterTable in the ‘Advanced Editor’ and Type or copy THIS into that query:


txtSQL= Excel.CurrentWorkbook(){[Name=”tblConnections”]}[Content],

#”Changed Type” = Table.TransformColumnTypes(txtSQL,{{“SQL”, type text}, {“Server”, type text}, {“Database”, type text}})


#”Changed Type”

Create a New blank query, rename it ‘qryData’, open the ‘Advanced Editor’ and copy this text into it:


Source = Sql.Database(qryMasterTable[Server]{0},qryMasterTable[Database]{0}, [Query=qryMasterTable[SQL]{0}])



Go ‘Close and Load’ …

Back in Excel while still on the Power Query Tab choose ‘Options’ … ‘Privacy’ … select ‘Ignore the Privacy ….’ … Click ‘OK’ … (see notes)

In Excel go to the Data Tab and go ‘Refresh All’ …

This should import the data from the SQL that you entered from the Server and Database you put in the tblConnections table.

And that’s it … two very small queries that drive the querying of data.

Important Notes ..

If you want another data source simply add other rows to tblConnections as necessary and duplicate ‘qryData’ and change the references to ‘{0}’ to [1] and then ‘{2}’ accordingly.  The name of each query in Power Query will become the name of the new table.  (You can even rename the tables by renaming the queries btw …).

And remember that any DAX that you place in any of these tables in the Power Pivot Data Libraries are preserved despite all these changes.

There is to me an odd feature (?) of Power Query called “Fast Combine” which is a mechanism designed to basically prevent you from mixing data from multiple sources in one spreadsheet … ?  I am really unsure why this is so (maybe some auditor or lawyer got to the developers and insisted on this).  While this pretty much would make it almost impossible without great effort to integrate multiple data sources into one Excel file (which I often do btw) there is also a way to override this “feature” by choosing to ignore it… Unfortunately this becomes a problem for us here because that little table in Excel is considered an “alternative data source” and so this rule kicks in unless you choose to “ignore” it…  There are all kinds of warnings of the risks inherent in doing this but frankly it strikes me as the exact same kind of overkill as Macro security settings (only MUCH worse).  This is because it is related to each individual user and the settings “Private”, “None” etc.  of every data source and every user has to personally set this to “Ignore” for every new file they get.  It is nothing short of a pain in the ass to me.

Interestingly though, in Excel 2016 they decided to provide a VBA workaround which to me indicates that I am not the only one who thinks this is a bit much in the REAL WORLD…  That is the capability to add this line in the Workbook_Open() Sub:

ThisWorkbook.Queries.FastCombine = True

This allows the Power Query queries to run just like you clicked “Ignore … “

If it’s a big thing you can make the setting FALSE after you do a refresh or even better make this setting only accessible with a password so that the refresh can only be done by an Admin of some kind.

To me this is a BIG reason to get your clients to move to Office 2016 asap.


So as far as I can tell Sam and Frank are absolutely correct that if you’re using Excel 2016 then Power Query and this relatively simple technique will make your solutions far more responsive to the real needs of your solution. I hope this means that you can teach an old dog new tricks and I hope this simple example makes sense to the most possible people so they won’t have to do all the searching I had to do to get this working..

Enjoy …


P.s. Ken Puls did a recent post on this exact issue that somehow was missd in all my searching … It’s a typically thoughtful Ken posting and offers an alternative way to accomplish this :

Pass Parameters to SQL Queries – The Ken Puls (Excelguru) BlogThe Ken Puls (Excelguru) Blog

Posted in Business Intelligence, Excel, Excel 2013, Office 2013, OLAP, Power BI, Power BI Desktop, Power Query, PowerPivot, Process Automation, Spreadsheets, VBA | 11 Comments

How To Change Connection Type In Power Pivot

Every day, day in and day out, the most popular posting I have here is this one:

and this one:

I personally use these techniques all the time, for years now and it is my standard modus-operandi for Power Pivot development in Excel.

Yet despite the fact that everyone comes here every day and looks at this page, I HAVE NEVER heard anything back from anyone whether they actually use this, if it works for them or not .. Not once…

That’s pretty frustrating people …:-)


p.s. go HERE …:-)

Posted in Uncategorized, PowerPivot, Excel 2013, Excel, Spreadsheets, OLAP, Office Automation, Process Automation, Power BI, Power BI Desktop | 19 Comments

Is Power BI For Real ?

Thought of the day …

I have been messing around with Power BI for a while now and while it “looks” great and does “good demo” I’m finding it almost impossible to build anything that I can rely on  .. it has a lot of supposed features that really don’t exist (vaporware) and frankly it’s VERY complicated to the point where I’d be reluctant to put a price on a Power BI implementation and I’d be worried that in the end it would let me down to the point where I would look bad for suggesting it.

I am TRYING to come up with a set of best practices that work reliably and consistently but to this point haven’t found that sweet-spot.

Also remember that Microsoft is selling this as an end user tool for analysts … and they have no idea that this will mostly NOT work because these “Analysts” will run into so many walls (data access, data quality, networking issues, security, internal data politics, etc., etc., etc.) that it should be done by professionals … But by Microsoft pricing it and selling it as an end-user product, companies will just never allocate the money needed to make  Power BI Solutions thing really work (if that is actually possible) ….

Oh yeah, sure Microsoft’s line is “We’re working on it and shipping new incremental improvements every day” … but what that means is that Power BI will NEVER be stable enough to host anything but demos .. How can I rely on a technology that will change regularly and is very likely to make things that work for me now stop working at some time in the future for no apparent reason and without warning…  ?

And if IT gets involved, none of these Power BI installations will ever see the light of day .. it’ll be just too fragile, too needy and too prone to embarrassing IT in the eyes of their users .. So IT will shut it down.

I am thinking that Power BI better level off technologically and soon and that Microsoft needs to work on making it SOLID, explaining what it REALLY can do rather than writing endless crap on the web about stuff that they’re “working on” as if those features really exist and begin spinning the fact that Power BI is NOT EVER going to be a tool for the amateur.

But that in the end Power BI COULD be a great product and that Microsoft is working to make it so.

No I’m NOT just in a bad mood today btw:-)


Posted in PowerPivot, Excel 2013, Microsoft Access, Spreadsheets, Business Intelligence, OLAP, Analysis Services, Office Automation, Cloud, Process Automation, the Cloud, Power BI, Power BI Desktop | Tagged , | 7 Comments

This post is about the Windows 10 Start Menu Corruption BS ….

I am going to talk about something that most users of Windows 10 have not run into yet, so most will probably be inclined to just write this post off as irrelevant to them .. Just wait !!

This post is about the Windows 10 Start Menu Corruption PROBLEM.

It WILL come to you some day – unless Microsoft somehow finds a way to fix this most pernicious of bugs in their Flagship operating system !  Which they have failed to do in over a year (?).  So please read on….

I just recently decided that it was time to move my world to Windows 10 finally – for real. For a week I just loved the new feel and the general environment.  Then suddenly my Start Menu stopped working and a dialog came up to the effect that my Start menu was corrupted and when you reboot it will “probably fix it” – or words to that effect.  So I restarted and it was the exact same… I could no longer use my Start menu .. it was pooched !!

So I searched the Internet and I found thousands and thousands of inquiries about this same exact issue and the usual thousands of mostly useless “fixes” posted by either well-meaning idiots or self-promoting a-holes.  Frankly while the apparently well-documented solutions do seem to work INITIALLY they do not tell you that within hours or days at most the problem will happen yet again and despite all your efforts, while you might get it back … it will happen again once it starts happening.

It appears to me that this issue cropped up after I customized the “Tiles” on my Start Menu to move my favourite apps front and centre.  It also in the end appears that the most likely ultimate solution is to reinstall Windows 10 from scratch and then DO NOT EDIT YOUR START MENU !!  Although this is only my guess because I just couldn’t bring myself to do that.

Instead I found a free download called “Classic Shell” ( ), installed it from my desktop and up came a very slick looking solution that not only gives me a Start menu that looks and works exactly like the one in Windows 7 (which I can live with) , but one that seems to work and work and work !

I have found zero complaints about Classic Shell out there, nothing but praise.  I have no idea why this is a free app … but I guess that’s not my problem.  But it seems to work for me.  We’ll see.

So what possible way can Microsoft explain why some freeware solution seems to be the only way to reliably resolve an bug in their operation system that has been known for over a year?  I just ran Windows update and am running Version 1511 OS Build 10586.218 which was installed from Windows Update this very morning … And the Native Start Menu does not work .. So I installed Classic Shell.

What bothers me is that I have seen zero response from Microsoft on this … it makes me think that Microsoft is unable to solve this issue themselves and have so little shame that they can’t even admit it… And yet how come some third-party startup has a solution and not the maker of the product itself?

This makes me nervous about everything Microsoft is up to with their never-ending incessant upgrades of all their software all the time !!  How can anyone make any decisions on Microsoft products if they never stand still long enough to be assessed? Further, if they don’t stay still long enough for us to actual implement solutions and for the clients to actually get some value out of them and what happens when inevitably Microsoft breaks something in one of these iterations (and they are going to) ?  How can these new incremental changes possibly be adequately tested before they ship… ?  Oh I get it .. we’re ALL BETA Testers now …  But sadly it looks more like we’re all ALPHA Testers instead.   

How can anyone make any decisions to implement serious solutions with this tech in this environment of a constantly moving floor?   What does this Start Menu bug say about how Microsoft is going to respond to this inevitability ?  I guess as long as only a few people are left out to swing in the wind then they’ll just look the other way.  Where is that going to leave Microsoft when people just simply stay away?

Where does that leave us all?



Posted in Access, Access Solutions, Business Intelligence, Excel 2013, Microsoft Access, Microsoft Access 2013, Microsoft Excel 2013, Office 2013, Office Automation, PowerPivot, Process Automation, Spreadsheets, the Cloud, Windows 10, Windows 10 Start Menu | 3 Comments

My Rules For Solution Engagements …

I’m baaaack … again …

I have decided that I just cannot not blog …

I have too many thoughts and I sure would hate to see all the knowledge I have accumulated disappear with me into retirement someday (or that other thing that shall not be discussed).  30+ years has taught me a lot and I hope that it is useful to somebody out there.

I would be glad to hear directly from you if you really read this btw .. But I do not consider this a discussion forum so if you disagree with what I say here please just go somewhere else and think what you want.  I admit I will not always be right but this is always my blog:-) ….

So take my thoughts or leave them please ..  I would however like to hear from you if you see what I say is valuable.  If not then ok contact me too – but be nice – please.  I do not really feel like a cat vs dog people argument anymore … this is what I think and that’s the way it is … unless I specifically have an error that you see.


My Rules For Solution Engagements …

Somebody asked this question recently on Linkedin:

“Hello, for those of you who develop Access db’s and apps for small and med businesses – How hard is to establish a common ground on which you and your customers establish a set of requirements?”

Interesting question that I simply felt I had some ideas about so I posted the following up there (slightly edited I’m afraid).  I thought maybe this deserved some kind of Blog posting.

Bottom line for me is that the main pain-point of development in Medium to Small (and also usually large) clients is not REQUIREMENTS “per se” but rather managing the engagement both at the beginning and throughout.  I have found that without appreciating the facts that I am detailing below nearly every project WILL FAIL to satisfy probably everybody involved.  Yeah sure you’ll likely get paid but they’ll likely never implement the solution or worse you will end up supporting a “wheezing dog” solution for a client that will get more and more pissed off as time goes on.  You MUST produce the right solution for the client period .. You cannot let the client force you to create a piece of carp that no one likes and doesn’t do the job for them…

Here we go – not totally random thoughts on this question …

Thirty years of doing solutions for Large, Medium and Small business has taught me that in many ways Medium and especially Small businesses are the toughest to satisfy. These businesses are most often owned and micro-managed by a person who is used to making all the decisions (as opposed to Corporate clients where there people are more used to compromise and collaboration).

To satisfy a Medium/Small business’s requests for a solution I find there is one key component I look for and that is a specific “Champion” in the business (sometimes the owner but not always) who “gets” what you are offering. By “get” I mean that they appreciate your skills as a business analyst and as a developer. He/she MUST respect you and especially respect that you are there to bring your skills in application development to their business need and they must have risked their careers on the success of this solution (really)..

The next consideration whether this is a NEW implementation or a spreadsheet replacement or a “Reno” of an existing Access database.

There are few opportunities anymore to create a database where there isn’t already some kind of automation in place. The first thing you have to do is determine who will fight that change and whether they are going to be an insurmountable problem. An effort has to be made to get ALL of them on-side. The more there are the faster you get to a failed opportunity. Regardless of the enthusiasm from the Boss or the “Champion” these folks will kill you if they are part of the decision-making process.Either they have to be won over or they have to be told that this is happening and they have to “get used to it”. Sad but true.

You have to insist on one rule without exception Saying “That’s the way we’ve always done it ” is NOT acceptable. Contd …

As developer you will, be doing your client and yourself no good if you let your database design be bastardize by their definition of some business process or some strangely laid out form or report that goes against good design practices and yet would deliver exactly the same info and result but in the right way. But at the same time you have to realize that you are just the developer and the business process has to be satisfied. So you often have to suggest alternative ways to accomplish something and fight for it. Again a reason why you need a “Champion” on your side.

After all these considerations then every solution goes it’s own way as every business requirement is different .. like snowflakes.

But without the relationships mentioned above and without respect for you and you skills as a solution developer (justified or not😉 ) no project will succeed.

Always keep in mind the old saying that you have to NEVER say “I gave you what you asked for but it’s not what you need …” Understanding the roadblocks in place and managing those relationships is ultimately way more important than any technical or best practices in the building of the actual solution.Sorry to say this (and I know it will sound self-serving) but it is only through EXPERIENCE with people and organizations that you can fine-tune these skills. But when I was younger I learned these rules early and have just fine-tuned them over time.

But one of the problems ends up that once you get to a certain age the temptation to tell clients what you really think about the way they are approaching the project can be hard to suppress:-) … If you know what I mean😉.

A very important part of any successful solution is NOT showing your work as you develop it to any more than a small number of people and most importantly 1. The people who will actually USE the solution and 2. NOT anyone Director-level or higher (if you can avoid it). If you ever show a solution partially finished to a Director or a VP I guarantee that they will suggest something be added .. EVERY TIME. ….

AND that something will likely have absolutely nothing to do with the original purpose of the solution and is very likely to mean going back the beginning in your design and development .

AND this one thing (or worse multiple things) will be the ONLY part of the solution they will care about and if you do not deliver it they will become negative on you and your solution and will very possibly kill the project.

I know you don’t always control this but if you can avoid this scenario TRY

If forced into it I suggest you be open and honest about the implications of these suggestions to the project right then and there and/or get your Champion to push back or manage this issue away.

I know that this is hard and I KNOW that the customer is always right …

– these extra people aren’t your customers …

– you have to keep the feature suggestions down to the least people who are most genuinely involved in the business process you are trying to automate ..

– AND you have to focus on delivering something that works and is not a dangerous piece of bad design and compromised spaghetti code to make it look good but that will tip over soon and will be impossible to revise or even maintain going forward..

One last thing … These “special requests” invariably are added without any revision to the budget or the timing. That is a BIG risk you have to manage immediately … The longer you delay that chat the more likely that YOU will end up eating the extra cost …

Talk soon ….


Posted in PowerPivot, Excel 2013, Microsoft Excel 2013, Microsoft Access, VBA, Office 2013, Microsoft Access 2013, Access, Spreadsheets, Business Intelligence, Office Automation, Access Solutions, Process Automation | 3 Comments

Here We Go … REAL JavaScript Macros For Excel ?

I have been completely open and honest (as usual) about my contempt for the useless offerings that Microsoft has tried to pass for “Automation” in Excel using JavaScript.  Their concept of “Add-Ins” which do little more than allow you to add Web content inside your spreadsheet that interacts with the values in some cells in the file and exposes some inane answer in the Pane (pain?) on the right taking up even more of the screen already grabbed too much by the Effluent UI “Ribbon” at the top…

I have commented many times here, there and everywhere that what is REALLy needed is some kind of true Macro capability for the “Automation” of actual spreadsheets.  What a strange idea.  Well finally it appears as if someone at Microsoft has come to the same conclusion after years of apparently aimless brain-storming.  Excellent !!

Here is a link to my buddy Maarten Van Stam (a genuine Excel MVP btw) that details the recent “coming out” of this technology …

and also here is more:

and especially here:

(this last one you have to get into it until the interview with Rob Howard …)

Rob says some really interesting things … This MIGHT be a big thing but I am seen so many BIG THINGS over the years that have turned to nothing but smoke in the end.  But this is different I think … Is this a breakthrough in Microsoft’s approach to Excel?  ACTUALLY caring about people who ACTUALLY use the product ?

Hope so..

Posted in Analysis Services, Cloud, Excel, Office Automation, OLAP, PowerPivot, Process Automation, Spreadsheets, the Cloud, Uncategorized, VBA | 7 Comments

My Thoughts On Excel TV

Earlier this week I appeared on the Excel TV broadcast with Rick Grantham, Jordan Goldmeier and Oz du Soleil …  It was great fun for sure and I hope it was somewhat valuable to the broad Excel community (and even the Access community).  I would really like to thank these guys for what they’re doing with Excel TV and I sure hope that they keep it up !!

As the guest I took it upon myself to cover as many things as I could and in the end I’m pleased with the discussion.   It seems like I talked a lot but as the guest I thought that was expected …  Hopefully what I said provided value regardless..

These guys, like me, want to get Excel the best, broadest, use in businesses around the world.  Anyone who comes here knows that I don’t pull my punches on what I think about the current status of Excel.  I sure wish I could just talk about the great things we can do with the product rather than the problems that are in front of us in trying to advance the use of the product, and maybe at the same time make us all some money either as a better employee or as a Consultant.  Some day maybe …

Anyway .. it’s pretty long (an hour and a half) but I understand that Rick chops the sessions up into subsections soon after so you can enjoy it in bite-size pieces.

Hope you enjoy it !!




Posted in Access, Access Solutions, Business Intelligence, Cloud, Excel, Excel 2013, Microsoft Access, Microsoft Access 2013, Microsoft Excel 2013, Office 2013, Office Automation, OLAP, PowerPivot, Process Automation, Spreadsheets, the Cloud, Uncategorized, VBA | 8 Comments