Jovian
Sign In
Learn practical skills, build real-world projects, and advance your career

Scraping the top Data Science jobs from leading e-recruiting websites.

Outline of my Project :

We will use web scraping technique in this project.

"Web scraping is a technique used to extract structured data from websites containing unstructured data through an automated process."

Imgur

My Objective:

---> Make a list of Data science job around my region from known website www.Linkedin.com , www.Stepstone.de, etc..
For each Job on page : get list of job titel , Company name, location of job, when job is posted, number of applicants, Experience Level , Job page URL.
By the end of the Project, we'will create a CSV file in following format :-
Create CSV file : Job titel, Company name, Location , Posting date, Total applicants already applied, Seniority Level , Employment level, Page URL
Example : Data Analyst (M/f/x), AVA & MAY, Berlin, Berlin, Germany, 2 weeks ago, 34 Applicants, Associate, Full-time, Click Url
# Execute this to save new versions of the notebook
!pip install jovian --upgrade --quiet
# Install the library (library can be installed using pip)
!pip install requests --upgrade --quiet
# Install the library
!pip install beautifulsoup4 --upgrade --quiet

import jovian
# Import the library
import requests
# Import time library
import time
# Import the library
from bs4 import BeautifulSoup

jovian.commit(project="python")
[jovian] Attempting to save notebook.. [jovian] Updating notebook "darpandahake/project-1-web-scraping-with-python" on https://jovian.ai [jovian] Uploading notebook.. [jovian] Committed successfully! https://jovian.ai/darpandahake/project-1-web-scraping-with-python 
'https://jovian.ai/darpandahake/project-1-web-scraping-with-python'

Pick a websites URL's and describe your objective

  • Browse through different sites and pick on to scrape. Check the "Project Ideas" section for inspiration.
  • Identify the information you'd like to scrape from the site. Decide the format of the output CSV file.
  • Summarize your project idea and outline your strategy in a Juptyer notebook.Use the "New" button above.
# Berlin
url_1 = 'https://www.linkedin.com/jobs/search?keywords=Data%20Analyst&location=Berlin&geoId=&trk=public_jobs_jobs-search-bar_search-submit&position=1&pageNum=0'