Scraping Stock Price data from Zerodha Kite web software.

Objective:

• In stock trading while designing trading system we need stock price data of various timeframe. In "Intraday Strategy Backtest", we typically need data of 1min to 15min timeframe. Such low frequency time data is not available for student or learner. Main objective of this project is to scrape stock data for educational purpose only.
• Getting such data is difficult task so in this project we will build a small utility code that will allow us to download stock price data in csv format.

Introduction:

• Web scraping is a technique to obtain large amount of data automatically from websites using software or script. Mostly it is unstructured html data which is then converted into structured data and stored in spreadsheet or in database format.
• Various process likes data cleaning, data formating, are done on unstructured data.
• Different libraries such as pandas, numpy, datatime, requests, json are used to perform data cleaning and structing.

Steps to follow:

• Website to scrape data from: "https://kite.zerodha.com/"
• Read login credentials from text file save on local computer.
• Set up selenium driver for auto login to website.
• Read the stock dictionary from text file and store it in a variable.
• Build a data url and extract authentication code from cookies.
• Pass data url and authentication code to requests method.
• Convert the response of request method to json and create pandas dataframe.
• Perform data cleaning with help of pandas library,time module.
• Store dataframe into csv format at desired directory.
• Perform all steps to entire stock list.

Install and import required libraries.