Medium taps into the brains of the world’s most insightful writers, thinkers, and storytellers to bring you the smartest takes on topics that matter.The idea to scrap the data started off with the aim to understand what kind of articles are more well read. But, then I realised that much more can be done with the data available. Can i understand even before publishing my article the number of claps or number of users who will like my post? Can i make a more informed decision on where to publish my article?

Content

The data contains all posts tagged AI, Machine Learning, Datascience or  Aritificial Intelligence on Medium, along with user information like number of Followers and number of Following (as on the date the data was scrapped). It also contains information about the publication where the post was published. In addition to this, if a post is tagged to Multiple tags, there is one row in the data for each tag associated with the post.

As an additional level of scraping, for each post the entire content of the post is provided, with code Blocks being separated from the content. The number of code blocks is also provided as a metadata.

The data is collected for a period of September 2017 to September 2018.