Context Newspaper articles are popular and diverse data sources for data scientists in various fields. To collect this kind of information, I built newspipe a scalable pipeline to crawl and extract data from news websites. Showing the power and flexibility of the pipeline, I collected articles from German-speaking publishers by providing the pipeline only with the URL of the website. This dataset is the result of the demonstration. I mainly crawled articles from 15 news sources over an extended period of time. The remaining 33 news sources were only test runs. Content Main news sources: - berliner-zeitung - bild - faz - focus - handelsblatt - heise - mainpost - spiegel - stuttgarter-zeitung - sueddeutsche - tagesspiegel - taz - welt - zeit Every JSON has following keys: - `id`: unique id - `author` : author name as string (may contain unrelevant information) - `published`: publish date as unix timestamp (int value) - `title`: article title as string - `summary`: summary of the article (not cleaned) - `text`: article text as string - `link`: link to news article - `tags`: tags of the article (optional) Inspiration If you found this dataset useful, cite: @misc{steven mi_2020, title={German News Dataset}, url={https://www.kaggle.com/dsv/1679751}, DOI={10.34740/KAGGLE/DSV/1679751}, publisher={Kaggle}, author={Steven Mi}, year={2020} } I hope you can make some cool analyses with this dataset. I used this dataset to train GPT-2 and got some cool results, like: "Wenn das zu deutlich werden sollte, ist es eigentlich ein katastrophales Signal für die Patienten, sich anzustrengen