Web Scraping
Scraping of most Important Consumer Affairs
Use the "Run" button to execute the code.
Web Scraping:-
Web scraping is the process of collecting structured web data in an automated fashion. It’s also called web data extraction. Some of the main use cases of web scraping include price monitoring, price intelligence, news monitoring, lead generation, and market research among many others.
In general, web data extraction is used by people and businesses who want to make use of the vast amount of publicly available web data to make smarter decisions.
Introduction about Consumer Affairs:-
ConsumerAffairs is not a government agency. Companies displayed may pay us to be Authorized or when you click a link, call a number or fill a form on our site. Our content is intended to be used for general information purposes only. It is very important to do your own analysis before making any investment based on your own personal circumstances and consult with your own investment, financial, tax and legal advisers.
In this notebook we are going to scrap https://www.consumeraffairs.com/
Tools used during scraping are:- Python, BeautifulSoup, Pandas, Requests etc.
Project Outline
- We are going to scrape https://www.consumeraffairs.com/
- We will get a list of imporatant purchages required for life.
- For each item we have many companies.
- For each item we will grab company name, company star rating and their URL.
- For each item we will create a CSV file in following format.
Company Name, Star Rating, URL
Endurance Auto Warranty, 4.4, https://www.consumeraffairs.com/auto_warranty/endurance-warranty.html
