CEHugeWebCorpus

  Autoři

Rüdiger, Jan Oliver

  Identifikátor

http://hdl.handle.net/11372/LRT-2638

 URL projektu

https://notes.jan-oliver-ruediger.de/korpora/

 Datum vydání

2018-03-01

 Typ

corpus, text

 Velikost

3021000000 tokens

 Jazyky

German

 Popis

This corpus was originally created for performance testing (server infrastructure CorpusExplorer - see: diskurslinguistik.net / diskursmonitor.de). It includes the filtered database (German texts only) of CommonCrawl (as of March 2018). First, the URLs were filtered according to their top-level domain (de, at, ch). Then the texts were classified using NTextCat and only uniquely German texts were included in the corpus. The texts were then annotated using TreeTagger (token, lemma, part-of-speech). 2.58 million documents - 232.87 million sentences - 3.021 billion tokens. You can use CorpusExplorer (http://hdl.handle.net/11234/1-2634) to convert this data into various other corpus formats (XML, JSON, Weblicht, TXM and many more).

 Nakladatel

Rüdiger, Jan Oliver

 Klíčová slova

corpus German Germanistik Web corpus web corpora CorpusExplorer

 Kolekce

LRT + Open Submissions Data & Tools

Zobrazit celý záznam