Web Scraping Tutorial: Tracking Your Competitors with Price Monitoring
Greetings, data scrapers — in today’s tutorial I’m going to show you how to monitor retail prices to give you a competitive edge. We’re using a tool by Mozenda — for more info on what they’re all about, check this link out).
And feel free to watch the video as you follow along with this article:
Let’s say you have a business: tracking your competitor’s online activities has numerous advantages. Mainly, it gives you a way to compare your own pricing to the marketplace. And by tracking this information, it gives you a chance to respond to your competitor’s fluctuating prices so you can come out ahead.
Let me give you an example. Let’s say you sell Beats by Dre headphones. If you data scrape your competitors, it allows you to monitor the price fluctuations. Maybe one day they’re charging $119.99 for a set, and the next week they’re charging $109.99 for the set. If you want to keep selling a consistent volume of headphones, you’ll need to re-price your own supply of these.
With a data scraping tool we no longer have to log on every day to all these various competitor websites and see what they’re charging, write that down, and so on. With data scraping we can fire up an instance of the scraper and start creating our own price database without the inconvenience of manually writing everything down.
While there are numerous scrapers out there — and you can even create your own if you’re a developer, but it’ll take a while — today I’m using Mozenda.
This option makes things go a lot quicker. We’re going to use this scraper to gather price info and we’ll be able to monitor any changes it. The first thing we’ll do is create an Agent, which is essentially an instance of the Mozenda scraper.
Then, we’ll navigate to our competitor’s website — I’m just using one today (Best Buy) for the sake of this demo, but you can use as many competitor sites as you want. Now I’m going to type in the product I want to monitor — in this case it’s Beats Studio 3 Wireless.
Next, I click the name of the product along with the price of the product. This is a nice feature of the Mozenda scraper — you just click, do a quick select here and there, and it’s gathering your info. I’m all done, so I’ll just save the Agent so it can run on Mozenda’s web interface and create a little database for me.
But here’s the thing — this agent isn’t going to continuously monitor prices for me; that’s going to eat up a lot of my credits, burn up a lot of energy and other resources. So what I’m going to do is set up this agent to run every day at a certain time — we’ll say noon.
And then I’m going to tell the program that I want any changes to my data to be tracked. So, if the price of these headphones change, I get to see it.
I’ll click the gear icon, select Schedule Agent, select my how often I want the scrape to fire, then hit save. Now I’m all set up to track important pricing information on my competitor’s website!
So again, whether you’re a business owner needing to do research, or you’re in the research department of a larger company and your boss told you to start monitoring competitors, this style of data scraping can really help.