Automating A Price Comparison Tool Using Python
Automating a price comparison tool using Python
In the retail world, a competitor price analysis is an important activity that Marketing departments regularly do as an effort to get the insgihts needed for setting their own product prices. Though knowking the competitor's product prices are not the only information required for setting a product price in retail, it is certainly important to be aware of it for reference.
There are multiple approaches for finding competitor's prices that varies in a wide range of time, effort, price, and accuracy of the results.
Some retailers ask their employees to research the competitor's prices by their own means which could include visiting stores, calling to Customer Service and/or copying manually data from the competitor's website. The list of prices that each employee gets is sent to another person who is accountable for compiling multiple files and providing an overall report.
Copying data manually from a website is a boring, tiring, time consuming, and prone to human error task.
There is a better approach for gathering massive information from a website that is called Web Scraping which is an automated technique for extracting required data from websites through software.
In this notebook, I'll walk you through the process of using Python libraries such as Requests, Selenium and Beautiful Soup for grabbing product prices by iterating a user-defined list of product categories from Farmacia San Pablo website and then fetching the price from a competitor website named Farmacias del Ahorro
The overall steps I'll follow are:
- Understanding the structure of Farmacia San Pablo and Farmacias del Ahorro websites
- Install Python libraries and modules
- Download and parse Farmacia San Pablo webpage using
- Extract product details from Farmacia San Pablo
- Lookup and extract regular and discount price from Farmacias del Ahorro
- Combine extracted data from Farmacia San Pablo and Farmacias del Ahorro into a dictionary
- Compiling all product details into a
Pandasdataframe and creating a CSV file
- Integrating all in a single main function
- Execution of main function
By the end of the project, is expected to create a csv file with the following information:
product, description, url, price, upc, url website2, regular price website2, discount price website2 Biretix Micropeel Tratamiento Exfoliante,50 ML Tubo,https://www.farmaciasanpablo.com.mx/dermocosmeticos/dermocosmeticos/geles-o-lociones-de-limpieza/biretix-micropeel-tratamiento-exfoliante/p/000000000005130301,385.0,8470001729057,377.0,358.15
How to run the code
Press Run button above.
Also refer to Execution of main function step at the end of the notebook