Exploratory Data Analysis - Women in Tech

Analyzing HackerRank Developer Survey 2018

This report is an analysis of HackerRank's annual Developer Survey with the aim to get an insight into women's representation in the tech field. In the past, women have known to face barriers in terms of progress in the IT industry, owing to the prevalent gender biases. The key idea is to understand trends, how things are changing, and find correlations based on educational background, age, etc. We also aim to study how the numbers vary across different regions of the world.

About the dataset

The dataset used in this study contains responses to the survey conducted by HackerRank wherein they inspect the demographics, education, job level, industry, gender, tech stacks etc. to get a better idea of their users. The original dataset can be found on this link: https://www.kaggle.com/hackerrank/developer-survey-2018?select=HackerRank-Developer-Survey-2018-Values.csv

Methodology

The following steps have been undertaken to perform this EDA:

1. Data cleaning: missing values, inconsistent data, outlier treatment
2. Non-graphical analysis: variables, data types, basic metrics
3. Exploratory Analysis and Visualizations
   • Univariate analysis
   • Bivariate analysis
4. Asking and Answering questions
5. Inferences and Conclusions

How to run the code

This is an executable Jupyter notebook hosted on Jovian.ml, a platform for sharing data science projects. You can run and experiment with the code in a couple of ways: using free online resources (recommended) or on your own computer.

Option 1: Running using free online resources (1-click, recommended)

The easiest way to start executing this notebook is to click the "Run" button at the top of this page, and select "Run on Binder". This will run the notebook on mybinder.org, a free online service for running Jupyter notebooks. You can also select "Run on Colab" or "Run on Kaggle".

Option 2: Running on your computer locally

1. Install Conda by following these instructions. Add Conda binaries to your system PATH, so you can use the conda command on your terminal.

2. Create a Conda environment and install the required libraries by running these commands on the terminal:

conda create -n zerotopandas -y python=3.8 
conda activate zerotopandas
pip install jovian jupyter numpy pandas matplotlib seaborn opendatasets --upgrade

3. Press the "Clone" button above to copy the command for downloading the notebook, and run it on the terminal. This will create a new directory and download the notebook. The command will look something like this:

jovian clone notebook-owner/notebook-id

4. Enter the newly created directory using cd directory-name and start the Jupyter notebook.

jupyter notebook

You can now access Jupyter's web interface by clicking the link that shows up on the terminal or by visiting http://localhost:8888 on your browser. Click on the notebook file (it has a .ipynb extension) to open it.

Downloading the Dataset

Now, there are three ways of downloading the dataset:

• importing urllib.request module and using it's urlretrieve function
• downloading the dataset directly from link and then uploading on Jupyter files section manually
• importing helper library opendatasets and using it's download function

Downloading using opendatasets:

Let's begin by downloading the data, and listing the files within the dataset.