Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy s podporou

CLARIN Knowledge Centre for Treebanking

logo clarin k-centre INESS LINDAT/CLARIAH-CZ

CLARIN Knowledge Centre for Treebanking is operated jointly by LINDAT/CLARIAH-CZ (Prague, Czechia) and INESS (Bergen, Norway).

It offers an integrated environment for storing, searching and visualizing treebanks. All its functionality can be used online through a normal web browser interface or programatically using REST API.

  •  (renewed until 30 March 2028)

  • The aim of the CLARIN Knowledge Centre for Treebanking is to provide support to researchers interested in the following:

    • building, depositing, and/or disseminating their treebanks;
    • accessing, searching and visualizing syntactically parsed data in various formats.

What is a treebank?

Treebanks are syntactically (and often also morphologically) annotated corpora, i.e., resources for linguistic research and for training models in language processing technology. Syntax of a sentence is usually represented as a hierarchical structure which, given its shape, is called tree. Hence the name treebank – a collection of trees.

There are many different theories that specify how exactly a tree of a particular sentence in a particular language should look like. The main dividing line runs between dependency structures and phrase (or constituency) structures, although some treebanks work with a combination of both. Some of the influential theories are Lexical Functional Grammar (LFG), Head-Driven Phrase Structure Grammar (HPSG), Functional Generative Description (FGD), or Universal Dependencies (UD). There are also various annotation projects that provide treebanks annotated according to the theory they subscribe to. UD has currently the largest collection of treebanks in terms of languages and language families covered.

What is a parser?

A parser is a computational tool with a language model that can take text in natural language as input, analyze it and produce its treebank-like annotation. In most cases, the parsing models are trained on manually annotated data from treebanks, but may also be based on computational grammars consisting of handwritten rules, or a combination of both. With a parser, you can analyze any text in a given language, not just the text that is included in a treebank. However, the parser relies on a computational model, which is never 100% accurate. Usually the accuracy is higher if the model was trained on larger data and if the type (genre, domain) of the training data is closer to the type of the text the parser is applied to. For example, a parser trained on articles from the Wall Street Journal will work well on other articles from Wall Street Journal, its accuracy will not be much worse on other English news texts, but it will be probably much worse when applied to Shakespeare's plays.

If you want to use treebanks…

You can use on-line services to search treebanks for examples of your interest, visualize the tree structures, and obtain aggregated statistics (such as the list of most frequent verbs that occur with a dative object in a treebank). Or you can download the treebank data and process it on your own. Both types of access are provided directly by the partners of this K-centre, and we also offer links to a few other similar sites. Many treebanks are openly accessible, whereas treebanks with more restrictive licenses require authentication via CLARIN SPF, eduGAIN or CLARIN IdP. 

If you want to create treebanks…

You can make your treebank available through this K-centre, either just for download by our users, or it can also be searchable through our search engines. However, if your treebank is a UD-style treebank, publish it through UD instead (see their How to contribute page). We take over all UD-released treebanks automatically.

INESS

INESS is operated by the CLARINO Bergen centre, and transfers knowledge through written documentation, personal contact with experts at the Knowledge Centre, resource hosting and training events (contact: iness@uib.no).

INESS hosts treebanks for many languages and accepts treebanks which have been created by others (if interested, contact us at iness@uib.no). Via the querying system INESS Search, searches can be made on treebanks in several syntactic frameworks, such as Lexical Functional Grammar, Dependency Grammar and Head-Driven Phrase Structure Grammar. After each official release of Universal Dependencies, the UD treebanks are also made available in INESS.

INESS also provides XLE Web, a web interface to the xle parser: It can parse sentences on the fly with one of the LFG grammars that INESS hosts.

LINDAT/CLARIAH-CZ

Treebanks are stored in the LINDAT/CLARIAH-CZ digital library and access is determined according to their licenses; while most treebanks can be freely downloaded due to holding some variant of a Creative Commons licence, some need authentication. Online search is free but logging in increases functionality, such as storing search queries. Authentication is flexible, using user's credentials from their home organisations from CLARIN SPF, eduGAIN or CLARIN IdP.

The download packages of official Universal Dependencies releases are hosted on LINDAT. After each official release, the UD treebanks are also made searchable in LINDAT search engines (most notably PML-TQ and TEITOK). Besides UD, LINDAT hosts a number of other linguistic resources, including treebanks in non-UD frameworks. In particular, there are several members of the “Prague family of treebanks” (based on FGD), such as the Prague Dependency Treebank-Consolidated.

Other related sites outside the K-centre

  • Grew Match is a popular search engine that provides access to all Universal Dependencies treebanks, as well as to their conversion to SUD. They also provide a web application called Arborator-Grew, which helps contributors create SUD and UD treebanks.