Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

PowerBI

Hi - I am wanting to have Matillion fire an API Data refresh to the PowerBI service on successful completion of running our ETL orchestration (Final step being posting the data to snowflake).

I have found the PowerBI documentation (https://powerbi.microsoft.com/en-us/blog/announcing-data-refresh-apis-in-the-power-bi-service/), but would really appreciate someone pointing me in the right direction of how to implement this in Matillion; or even better, an example, or tutorial......

Thanks in advance!!

7 Community Answers

Matillion Agent  

Paul Johnson —

Hi Robbie,

I don’t have access to any power bi services. So I can only guide you on what I would attempt to do to implement this.

  1. First you would need to install powershell on the matillion instance, the linux distribution is not officially supported by Microsoft but I would follow the instructions for Centos7 here and see if that works
  2. Using the Bash Scripting compnents obtain an AAD access token, sample code here to help you
  3. Fire the API call to do the data refresh attaching your token (https://api.powerbi.com/v1.0/myorg/groups/{group_id}/datasets/{dataset_id}/refreshes)

If you are happy to share some credentials I can try and help you with it. Let me know how you get on.

Thanks,
Paul


Robbie Francis —

Hi Paul - Thanks for the info - It's a good start. How would you fire the API call from within the matillion flow though (Or is that the easy bit?)


Robbie Francis —

Can you e-mail me, and I'll share the login credentials?


Matillion Agent  

Laura Malins —

Hi Robbie

You can fire the API call from within Matillion by running a curl command in the Bash Script component

Thanks
Laura


Kevin Havice —

Hey, were you all successful with this? I am also interested in doing this, and I don't really know what I'm doing when it comes to API calls and scripting components.

I read the references linked above...plus the following Microsoft resources...
https://docs.microsoft.com/en-us/power-bi/developer/get-azuread-access-token
https://docs.microsoft.com/en-us/rest/api/power-bi/datasets/refreshdatasetingroup
...plus I did a little more research.

I've just never done this stuff, in practice.

I gather that an API call to refresh the Power BI dataset needs an auth token in the header. For this, you suggested installing PowerShell...and then, you're saying that you can run a PowerShell script from a Bash script? Is it possible to use a Python script for this, instead? It appears there is a Microsoft Azure Active Directory Authentication Library (ADAL) for Python. Is it possible to get the "adal" library, and make it available for use in a Python Script component, in Matillion? Do you think this would be simpler than installing and using PowerShell?


Kevin Havice —

...and, similarly, it seems like it would be possible to make the Power BI API calls from a Python script, right? Like using the "requests" library?

And to answer one of my own questions from above, I read this...
https://redshiftsupport.matillion.com/customer/portal/articles/2234735-python-script
...to discover how I can install additional Python libraries.

Any reason I shouldn't try doing all the same steps with one or more Python Script component(s)?


Matillion Agent  

David Lipowitz —

Hi Kevin,

I tend to favor “curl” because it doesn’t run into any of the package dependency issues you sometimes see with Python. That said, I built a proof-of-concept and the Python “requests” library does work for this as well.

Please let us know if you have any more questions.

Best Regards,
Dave

Post Your Community Answer

To add an answer please login