Project 1 Web Scraping With Python
Scraping the top Data Science jobs from leading e-recruiting websites.
Outline of my Project :
We will use web scraping technique in this project.
"Web scraping is a technique used to extract structured data from websites containing unstructured data through an automated process."
---> Make a list of Data science job around my region from known website www.Linkedin.com , www.Stepstone.de, etc..
For each Job on page : get list of job titel , Company name, location of job, when job is posted, number of applicants, Experience Level , Job page URL.
By the end of the Project, we'will create a CSV file in following format :-
Create CSV file : Job titel, Company name, Location , Posting date, Total applicants already applied, Seniority Level , Employment level, Page URL
Example : Data Analyst (M/f/x), AVA & MAY, Berlin, Berlin, Germany, 2 weeks ago, 34 Applicants, Associate, Full-time, Click Url
# Execute this to save new versions of the notebook !pip install jovian --upgrade --quiet # Install the library (library can be installed using pip) !pip install requests --upgrade --quiet # Install the library !pip install beautifulsoup4 --upgrade --quiet import jovian # Import the library import requests # Import time library import time # Import the library from bs4 import BeautifulSoup jovian.commit(project="python")
[jovian] Attempting to save notebook.. [jovian] Updating notebook "darpandahake/project-1-web-scraping-with-python" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Committed successfully! https://jovian.ai/darpandahake/project-1-web-scraping-with-python
Pick a websites URL's and describe your objective
- Browse through different sites and pick on to scrape. Check the "Project Ideas" section for inspiration.
- Identify the information you'd like to scrape from the site. Decide the format of the output CSV file.
- Summarize your project idea and outline your strategy in a Juptyer notebook.Use the "New" button above.
# Berlin url_1 = 'https://www.linkedin.com/jobs/search?keywords=Data%20Analyst&location=Berlin&geoId=&trk=public_jobs_jobs-search-bar_search-submit&position=1&pageNum=0'