This corpus was originally created for performance testing (server infrastructure CorpusExplorer - see: diskurslinguistik.net / diskursmonitor.de). It includes the filtered database (German texts only) of CommonCrawl (as of March 2018). First, the URLs were filtered according to their top-level domain (de, at, ch). Then the texts were classified using NTextCat and only uniquely German texts were included in the corpus. The texts were then annotated using TreeTagger (token, lemma, part-of-speech). 2.58 million documents - 232.87 million sentences - 3.021 billion tokens. You can use CorpusExplorer (http://hdl.handle.net/11234/1-2634) to convert this data into various other corpus formats (XML, JSON, Weblicht, TXM and many more).
The database offers access to over 6 million dialectal linguistic evidences of the project "Dictionary of Bavarian Dialects" (German: Das Bayerische Wörterbuch) as image snippets, partly and forthgoing lemmatized.
The area covered by the Dictionary of Bavarian Dialects (Bayerisches Wörterbuch) comprises Upper Bavaria, Lower Bavaria, the Upper Palatinate and neighbouring regions of Bavarian Swabia, Middle Franconia and Upper Franconia. Over and above the vernaculars spoken today, Bavaria’s literary tradition since its beginnings in the 8th century is also taken into account.
Starting in 1913, language material was collected from all Bavarian-speaking regions in Bavaria. Questionnaires were sent out to local informants throughout Bavaria, and contemporary and historical literary sources were excerpted. Today the collection comprises around nine million dialect examples. With the exception of the “Wörterlisten” (word lists), which can be digitally searched and edited, this material consists of index cards, to which corresponding standard German or quasi-standard German keywords have been added, filed alphabetically (see link below for more information).
For detailed information, please see https://www.bwb.badw.de/en/the-project.html and https://www.bwb.badw.de/en/digital-platform.html
A petition for a referendum (called: "Schluss mit Gendersprache in Verwaltung und Bildung" / eng.: "abolition of gender language in administration and education") was formed in Hamburg in February 2023. The project "Empirical Gender Linguistics" at the "Leibniz Institute for the German Language" took this as an opportunity to completely scrap the "https://www.hamburg.de" website (except the list of ships in the Port of Hamburg and the yellow page). The Hamburg.de website is the central digital contact point for citizens. The scraped texts were cleaned, processed and annotated using http://www.CorpusExplorer.de (TreeTagger - POS/Lemma information).
We use the corpus to analyze the use of words with gender signs.
The database currently contains about 1 million dialectal linguistic evidences of the project "The Franconian Dictionary" (German: Das Fränkische Wörterbuch), each of which lemmatized, annotated, and linked to the original questionnaire. The database is work in progress, so there will be more data available regularly.
The Franconian Dictionary was initiated by the Munich office of the Bavarian Dictionary project, sending questionnaires for a dialect survey in Franconia. In the wake of this survey an office in Erlangen was established in 1933 (see link below for more information).
During the course of 90 years thousands of volunteers helped to compile a considerable collection of vernacular examples of usage, drawn from the Bavarian districts of Upper, Middle and Lower Frankonia. For the most part they represent the East Franconian dialect, to the lesser extent also Rhine-Franconian, Swabian and North-Bavarian vernaculars. Between 2007 and 2008 a small selection of the research results was published in three editions of one printed volume by Eberhard Wagner and Alfred Klepsch: “Handwörterbuch von Bayerisch-Franken” (see link below for more information).
Since 2012 the Franconian Dictionary, a project of the Bavarian Academy of Sciences and Humanities, has been entrusted to the Friedrich-Alexander-University in Erlangen and Nuremberg (FAU). The project is supervised by Prof. Dr. Mechthild Habermann, Chair of the Faculty of German Linguistics at the FAU.
For detailed information, please see http://www.wbf.badw.de/en/the-project.html and http://www.wbf.badw.de/en/wbf-digital.html