Project1 Webscraping

Introduction

The aim of this webscraping project is to scrape the steam's video game website to extract information like title, developer name, user review etc about video games. Below is the snapshot of the website.

As we can see, the games are divided into four categories:

1. New and Trending
2. Top Sellers
3. What's Being Played
4. Upcoming

We will let the user choose the category of game.
After that, we will scrape the webpages of games listed in the first 10 pages of the chosen category.

Brief Outline

The following numbered list describes the step by step procedure followed to scrape the website:

1. Import the relevant python libraries.
2. Store the base url of the steam's website.
3. Get the preference from user.
4. Create BeautifulSoup documents from each webpage.
5. Get URLs of each video game.
6. Extract information about video game.
7. Write the information in a csv file.
8. Wrapping up.
9. Summary.
10. Future directions.
11. References.

The details of each of the above step can be found in the respective sections.

Installing and Importing Libraries

We will need following libraries:

1. jovian ---for uploading and sharing the jupyter notebook
2. requests ---for performing http requests and obtaining webpages
3. beautifulsoup ---for creating beautifulsoup documents out of webpages obtained
4. pandas ---for reading the generated csv files.