How To Really Publish Excel Data To Power BI … Part III

OK .. Let’s go … Part III … !!

In this post I will try to explain how (and especially how not) to get your Excel data into Power BI Desktop in a way that will allow you to schedule refreshes like a normal person … so that all you have to do is run a little Excel VBA code to export your data to Sharepoint (note I said Sharepoint not OneDrive … I’ll explain) and then at an acceptable scheduled time of day it will automagically refresh your data up in PowerBI.com and your consumers of your Dashboards will see the latest and greatest data …

The fact is that if you use the obvious available menus to connect to your Excel data IT WILL NOT WORK … and for reasons like I will explain (but not justify).

How NOT To Connect Excel Data Into Power BI

I considered only showing the positive here and only showing what DOES work … but in the end I think there is enough confusion in this topic that I have to show the bad and the good.

Using OneDrive

In all the literature on Power BI OneDrive for Business is constantly referred to as THE place to store Excel data for use in Power BI.  There is no doubt that OneDrive for Business does work for PBI and PBID BUT the fact is that OneDrive for business is actually a subset of SharePoint in the first place.  If you want to use OneDrive for Business as the “staging” area for your PBID Excel files you have to make sure the connection is to the ACTUAL URL in Sharepoint that gets you to the file… i.e.

https://<<Company name>>-my.sharepoint.com/personal/YourLogin_onmicrosoft_com/Documents/Folder1/USElection2016 Direct OneDrive.xlsx

Notice that this is in fact a Sharepoint site that exists under YourLogin name …  This means that if you quit, get promoted, fired, retire or die this site will go away with you … oops.

On the other hand, if your company uses Office 365 they automatically receive the ability to easily create and manage (including security of all types) the ability to create “Company” departmental SP sites at will that are maintained by an Admin(s) who can change and be managed easily any time.  Yes OneDrive can do all this but why not just sue SharePoint?  I don’t get this and would really like someone to tell me why the obsession with OneDrive and not Sharepoint (must be political inside MS I guess).

So in my future ALL my solutions are going to migrated to SharePoint from OneDrive for Business … unless someone convinces me of the advantages of using OneDrive .. and switching back would be fairly easy too.

Using the Get Data … Excel Menu-pick in PBID

This is a really weird one … This menu item simply does not work, never has and I assume never will.

Oh sure you can use Get Data – Excel to link to internal files inside you firewall.  But if you do so, you will be forced to ALWAYS open the PBID file on a network PC with access to the sources in their original network location every time the Excel data changes and then you have to republish the PBID file to PowerBI.com.

That is unless you just distribute the PBIX file (which I know many do) which is frankly ludicrous and pretty much defeats the whole advantage of Power BI to deliver to any device anywhere and to refresh data automatically simply by changes to the data source itself.  Oh sure there will be those who argue for this for security reasons, but that is bogus IMHO, and probably reflects the fact that they can’t figure out how to do what I’m gonna show you.

So why not use this menu item to connect to an Excel file in SP or OneDrive for Business? NOT

OK … say you path your Get Data – Excel to the actual URL out in :”the Cloud”?  Like

https://<<Your Company Name>>.sharepoint.com/Shared Documents/PowerBI

and then you pick the file from there?  Well …. Hmmm …  If you do that PBID picks up the path to your file in this format:

You will be forced to edit out any extraneous references in there so you end up with a Source reference like this:

= Excel.Workbook(File.Contents(“\\Your Company Name.sharepoint.com\Shared Documents\PowerBI\ImportSales.xlsx”), null, true)

If you publish THIS link to PowerBI.COM it fails because it will be looking for a Windows account for access and there is none for SharePoint or OneDrive for Business.  It simply will not refresh unless you do it manually.

Do not ask me why it does this, but the fact is that the path here is useless except to import your data ONE TIME ONLY!  It will not refresh under any circumstances.  Oh sure you could let it fail and then edit the path by copying in the full URL at that time and it MIGHT work (after you edit the security in PBI).  But that is a lot of work and it actually imports very slowly to boot.

So do NOT use the Get Data … Excel menu pick …

Use the Get Data … Web dialog instead !!

Choose Get Data … Web and paste the fully qualified URL path to your SP file in THIS dialog

then you will be able to schedule auto-refresh when this PBID file gets published to PowerBI.com … This is the reason why in the code in my previous post(s) on this topice I have added a cell where you can put the fully-qualified reference to your sP site and also a formula which shows (therefore you can copy it into the clipboard) the actual target Excel file that will be created when you run the code.  You simply have to leave this file open and copy and paste the reference to the source Excel file in PBID when this dialog appears … simple eh?

(Interesting weird fact … recently this process failed for me as PBID insisted on adding extra characters to the end of my URL after the file name (?)  I have no idea what that was about but it appeared to have something to do with having the actual site that the file was in open in the browser in SharePoint.  This was new but I resolved it by closing the browser completely while I was working in PBID…. This was disconcerting but I believe that it is not an ongoing problem .. Today while I am writing this I have everything open and it all works fine … Hmmmm.  Problem is that I have no idea who I would reach out to about this if it becomes an ongoing issue… It does not please me even though I can work around it …)

And these refreshes will happen automatically to bring in any changes made to the source EXCEL file … NOT to the PBIX file that is saved in PBID … the Excel file.  This means you users do not need to know anything about the plumbing of Power BI .. they just need to open and refresh or manually change the data in the file and either run the VBA to Publish the data to SP OR you could even include that code into the OnSave property of the Workbook.  Automatic Refreshing all the way through ….

I know this is hard to imagine and grasp.  It is REALLY hard to explain as well which is a great source of frustration for me…  But as I will show you next, this works and the other simply does not.

This all makes my brain hurt.

 

 

Set Up The Auto-Refresh In PowerBI.com

If you use the techniques I mention just above, once you have published the PBID’s PBIX file to PowerBI.com you still have a couple more things to do. Remember that PBI.com lives in “the Cloud” and for it to “talk” to your data in SharePoint you have to provide credentials for PBI to use tp reach the data.  This is done only in PBI (not PBID) and is actually quite simple 9if quite hard to find unless you know what and why you’re looking for it).

1. Open Power BI in the browser (app.powerbi.com) and move down to the DataSets.  Find the Dataset you JUST created and right-click the ellipsis on its right and choose “Schedule Refresh”…  There are 2 things you have to do there…
2. Open “Data Source Credentials” and choose “Edit Credentials”, click in the middle of the drop-down box of “Authentication Methods” (the pulldown menu simply does not work so you have to click in the middle of the text box (literally) to get the list of protocols and select “oAuth2”.  Why “oAuth2” ?  I assume that this is the protocol to connect to Cloud-based data sources like SharePoint … but to say this is NOT INTUITIVE is quite the understatemtn ..
3. When you “Sign In” with this it will ask you to confirm your login to SharePoint.  Click your account.  Once you have done that you will be connected to the source file in SharePoint in “the Cloud” !!
4. Then click on “Schedule Refresh” and click to “Yes” the “Keep your data up to date”.
5. If you leave it this way it will refresh daily … but you can now add up to 8 times during the day to refresh the data … (of course you could also use “Refresh Now” to force a refresh).  I hope some day we can sinply say “Every hour or ever half hour but this is more tha adequate for 99% of solutions, I believe.  This is IN THE BROWSER NOT IN THE PBIX FILE btw !!

Conclusion

So there ya go …

Now you can go back to PBID, open the PBIX file and add whatever Charts, tables, images or any of the cool stuff that you can do in Power BI.  Remember to DO IT ALL IN PBIX file in PBID, save that file locally and then re-publish it to PBI.COM.  Yes you can change things up in PBI.COM but why bother ?  Have a single point of development.

Why this is so insanely complex I have no idea …

Why this is effectively NOT documented anywhere I also don’t know either 😦 …

Why the Excel team isn’t all over this is absolutely beyond my comprehension but frankly nothing surprises me anymore from them… sorry guys and gals… Prove me wrong … please !!

But the fact is that by using these techniques you will be able to publish Excel data (including but not limited to data expressed in a Pivot Table) in Power BI though Power BI Desktop and with auto-refresh happily updating your Dashboards and Reports on a schedule of up to 8 times every day or manually through your browser.

I will next show how to customize your Power BI solution for each individual or group of consumers of your data … again using Excel as the natural platform.

I will also discuss details of how to build a combined Excel/PBID solution using relationships, DAX and KPIs using the best technique in the best environment.

I will also discuss how you can design your PBI implementation bring this data BACK INTO Excel using the Power BI Add-in for Excel in an effective way.

Yes I think that this is exciting stuff and yes I am willing to talk about it to anyone anywhere so feel free to reach out to me for guidance or help.  This is the most exciting thing in Excel since Power Query and before that Power Pivot.

I believe that it’s about time everyone got on board all the effort that the Excel Team has put into their product (including the Excel Team themselves btw) and we must all get started to leverage all this great tech for the benefit of our Companies and/or Clients and also benefit ourselves (frankly). 

Dick