Overview / Usage

This is a web application built using python django framework. This app uses django version 3.0.4 in the backend. A collection of small programs that extract data from multiple websites and packages it to be useful with the use of BeautifulSoup, a Python package for parsing HTML and XML documents. Once you retrive the raw HTML of a site, you can start to select and extract with BeautifulSoup, which parses raw HTML strings and produces an object that mirrors HTML documents' structure.

Methodology / Approach

1. This project is only for study purpose.
2. Check a website's Term and Conditions before scraping it and read the statements about legal use of the data.
3. Do not request data from the website too aggressively and ensure that your program behaves in a reasonable manner.
4. Revisit the website and rewrite code as needed as the layout of the site may change.

Technologies Used

Django, BeautifulSoup , PostgreSql

Repository

https://github.com/sagarbanik11/covidnews