Bottom line: Learn how this FREE Excel add-in from Microsoft called Power Query will help transform your data into a format that is ready for pivot tables, reports, and so much more.
Skill level: Beginner
In this tutorial I provide an introductory explanation of Power Query. You will learn why this is my new favorite Excel tool for working with data, and how it can help automate processes and save you time!
The Power Query Data Machine
I was watching a TV show on how things are made, and they were explaining how a depositor machine worked in a pastry factory.
The basics of a depositor machine are:
- You add ingredients to it.
- Change some settings.
- And it magically creates pastries (cookies, donuts, biscuits) that are ready for baking.
Once the dials are set, the process can be repeated over-and-over again to make perfect pastries every time. Getting hungry…? 🙂
Power Query works in a very similar way!
- You add your data sources (Excel tables, CSV files, database tables, webpages, etc.)
- Press buttons in the Power Query window to transform your data.
- Output that data to your worksheet or data model (PowerPivot) that is ready for pivot tables or reporting.
Power Query is like a machine because once you have your query setup, the process can be repeated with the click of a button every time your data changes.
If you have used macros to transform your data, you can think of this as a much easier alternative to VBA that does NOT require coding.
Common Data Tasks Made Easy
Do you work with data that has been exported from a system of record? This could be a general ledger, accounting, ERP, CRM, Salesforce.com, or any reporting system that contains data.
If so, you probably spend a lot of time transforming or re-shaping your data to create additional reports, pivot tables, or charts.
These data transformations could include tasks like:
- Remove columns, rows, blanks
- Convert data types – text, numbers, dates
- Split or merge columns
- Sort & filter columns
- Add calculated columns
- Aggregate or summarize data
- Find & replace text
- Unpivot data to use for pivot tables
Do any of these tasks sound familiar? If so, then they probably also sound boring, repetitive, and time consuming. 🙂 Believe me, I’ve spent the better part of my career doing these tasks and trying to figure out faster ways to get them done.
Fortunately, Power Query has buttons that automate all these tasks!
Overview of the Power Query Add-in Menus
Once installed, the Power Query tab will be visible in the Excel Ribbon.
You use the buttons in the Power Query tab to get your source data. Again, your data could be stored in Excel files, csv files, Access, SQL server database, SharePoint, Salesforce.com, Dynamics CRM, Facebook, Wikipedia, websites, and more.
Once you have specified where your data is coming from, you then use the Power Query Editor window to make transformations to the data.
The buttons in the Power Query Editor Window allow you to transform your data.
Think about some of those tasks you do repeatedly as you browse the buttons in the image above. Each time you press a button your actions (steps) are recorded, and you can quickly re-apply the steps when you receive new data.
After completing your steps, you can output the data to a Table in your Excel workbook by clicking the Close & Load button.
You can also modify existing queries and refresh your output tables with the changes or updated data.
Here are a few examples of what Power Query can do with your data.
Unpivot Data for Pivot Tables
My favorite feature of Power Query is it’s ability to Unpivot data. This is a technique used to get your data ready for the source of a pivot table. This is also referred to as normalizing your data to get it in a tabular format.
The data might start out looking something like the following.
And you want the end result to look like this.
Power Query can do this with the click of a few buttons, and prepare your data for use in a pivot table.
Here is an article and video on exactly How to Unpivot Your Data with Power Query.
Checkout my article on how to structure your source data for a pivot table if you are unfamiliar with why your data needs to look like this for a pivot table.
Append (Combine) Tables with Power Query
The Append feature of Power Query allows you to combine multiple tables (stack them vertically) to create one large table. It can do this with multiple tables in one file, or it can pull in data from a bunch of different files/sources.
Let’s say you have a folder that contains CSV or Excel files with report data for each month. Throw all those files in the Power Query machine, and it will spit out one nice table that you can then use to create pivot tables and charts.
If the data in those reports also needs to be transformed (remove rows, split columns, unpivot, etc.), then Power Query can handle that in the same process.
Once it is setup, all you have to do is hit the refresh button every month when a new file is added to the folder and the rows will be added to your output table.
How awesome is that! 🙂
Merge Tables – A VLOOKUP Alternative
Power Query has the ability to merge or join tables. This can be used as an alternative to VLOOKUP or INDEX/MATCH formulas.
Let’s say you have this data table of sales records, and you are using a VLOOKUP to bring in information about the product based on the name of the product sold. Your product group information is located in another table on a different sheet or workbook.
Using VLOOKUP formulas is great, but it can often mean adding thousands of formulas to your workbook. Which increases the file size and calculation time.
Power Query makes it very fast and easy to merge two tables together with just a few clicks. It basically uses SQL joins, so you can even do more advanced merges like inner, outer, left, right, full, and anti joins.
Create Custom Functions
Power Query was designed so that you do NOT have to know how to code to use it. It is very easy to use because you can just click buttons and apply filters like you normally would in Excel.
However, Power Query can be programmed to create custom functions. This gives you seemingly unlimited potential to transform your data in just about any way possible.
It is based on the M language, and most of the functions are very similar to writing a formula in Excel. This also makes it more user friendly and easier to learn the code.
This new language and set of functions means there is a lot to learn, but I consider that the fun and challenging part. Plus, employers of the future will definitely be looking for employees with Power Query skills.
Power Query Records Your Steps & Automates Processes
Power Query not only makes all these tasks easier, but it also records your steps so you don’t have to do them over-and-over again. It will save you a lot of time if you are preparing the same data every day, week, or month.
It also does a pretty good job of handling errors. If the structure of your source data changes, Power Query will tell you what step it broke at and allow you to fix it. This makes maintenance easy and you don’t have to completely redo your process when something changes.
You can use Power Query to get your data ready for use in pivot tables, charts, and dashboard reports. This is a critical step in the process of summarizing and analyzing data.
The Power Query Machine & Power BI
Well, it can’t exactly make cookies, but Power Query is a pretty awesome tool! It will save you a ton of time when transforming your data.
Power Query is just one piece of the suite of Power BI (Business Intelligence) products from Microsoft.
If we go back to the analogy of baking cookies in a factory, you can think of Power Query as the first step in the assembly line. Once the cookies are formed, we then need to bake them (Pivot Tables, PowerPivot) and then package them for presentation (Power View, Power Map, Charts, Dashboards, etc.)
You can think of Excel as the factory building that houses all these tools. The exciting part is that there is a very bright future for Excel! All these advances in technology will help us make sense of our data in new ways, save us time, and impress our bosses. 🙂
How Do I Get Power Query?
The other nice part is that it is a FREE add-in from Microsoft for Excel 2010 and 2013.
I have a dedicated page that will help you determine if you have the right version of Excel to get Power Query. It also provides complete installation instructions and the download link.
To give you an idea of the importance of this tool, Power Query will no longer be an add-in in Excel 2016. Instead, it will just be part of the Data tab on the Excel Ribbon.
This article has provided an overview of the basics of Power Query that should help you understand some of the major features. Power Query has a ton of features and there is definitely a lot to learn.
I will be sharing more how-to articles and videos in the coming weeks. Here are a few resources that will help you get started.
Online Training Program
Our new program, Elevate Excel, is a comprehensive (beginner to advanced) training program that covers all areas of Excel, including Power Query.
What Do You Think?
Are you using Power Query yet? If so, please leave a comment below letting me know how you use it.
If not, do you think it would be useful for you? Are you doing any of the tasks I mentioned manually right now?
I will be writing more how-to articles and videos on Power Query in the future, so I’m interested to know what you want to learn.