Using Google Refine in competitive intelligence analysis

A category of tool that competitive intelligence analysts rarely use is that of software that requires some coding, or coding-style treatment. Some of these are hugely powerful, like Tableau. A more basic tool is Google Refine; which lets users tidy up large data sets, and meld these data sets with public databases like movie listings. With Google Refine, virtually any type of data file can be cleaned and prepared for use for analysis and review using this convenient application.

More broadly, Google Regine is an ETL tool. Extract, transform, and load - or ETL - is a term used by technology professionals for a complicated process of retrieving data from a particular source, editing the data so that it is usable, and then loading the data into a new source. Analysts of all skill levels often have to review data that can be messy and disorganized and clean it up for company use. When gathering competitive intelligence, this type of issue happens quite often. Google has come up with a very nice tool to help all skill levels to clean and organize their data, which makes the process of competitive analysis much easier.

When performing competitive analysis, information can come from a variety of sources. For example, if you competitive intelligence information provides a list of products and prices from a variety of cities and states, the files can vary greatly depending on the data sources and authors. The product number may or may not include dashes or spaces. The product description could vary from "assortment of hand tools" to "asst tools" and anywhere in between. Also the cities and states fields may be combined as in "San Diego, California" or they may be separated and abbreviated as in "Minn." or "MI".

Google refine allows the user to take disparate data and refine it so that the information is easier to consume for analysis. By starting a project you can have the same file modified in several ways if needed. The analysis tools in Google Refine then intelligently compare the data and find "like" patterns. The user can then decide to change the data to a consistent pattern (change every variation of "assortment" to "asst") or choose their own pattern. Then save your work as needed.

Competitive intelligence and analysis can be much simpler and faster by using the Google Refine tool. At Aqute, we have recently used this tool to standardize records downloaded from LinkedIn, and (an example Google themselves use) to automatically detect the languages of thousands of survey responses.