Search

Start Over Contributor Ministerstvo školství, mládeže a tělovýchovy České republiky@@LM2015071@@LINDAT/CLARIN: Institut pro analýzu, zpracování a distribuci lingvistických dat@@nationalFunds@@ Creator Hajič, Jan Language Vietnamese

1. Corpus for training and evaluating diacritics restoration systems

Creator:
Náplava, Jakub, Straka, Milan, Hajič, Jan, and Straňák, Pavel
Publisher:
Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics (UFAL)
Type:
text and corpus
Subject:
diacritical marks generation and natural language correction
Language:
Czech, Vietnamese, Romanian, Polish, Slovak, Spanish, Croatian, Irish, Latvian, Hungarian, French, and Turkish
Description:
Corpus of texts in 12 languages. For each language, we provide one training, one development and one testing set acquired from Wikipedia articles. Moreover, each language dataset contains (substantially larger) training set collected from (general) Web texts. All sets, except for Wikipedia and Web training sets that can contain similar sentences, are disjoint. Data are segmented into sentences which are further word tokenized. All data in the corpus contain diacritics. To strip diacritics from them, use Python script diacritization_stripping.py contained within attached stripping_diacritics.zip. This script has two modes. We generally recommend using method called uninames, which for some languages behaves better. The code for training recurrent neural-network based model for diacritics restoration is located at https://github.com/arahusky/diacritics_restoration.
Rights:
Creative Commons - Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0), http://creativecommons.org/licenses/by-nc-sa/4.0/, and PUB

2. Universal Dependencies 2.4

Creator:
Nivre, Joakim, Abrams, Mitchell, Agić, Željko, Ahrenberg, Lars, Aleksandravičiūtė, Gabrielė, Antonsen, Lene, Aplonova, Katya, Aranzabe, Maria Jesus, Arutie, Gashaw, Asahara, Masayuki, Ateyah, Luma, Attia, Mohammed, Atutxa, Aitziber, Augustinus, Liesbeth, Badmaeva, Elena, Ballesteros, Miguel, Banerjee, Esha, Bank, Sebastian, Barbu Mititelu, Verginica, Basmov, Victoria, Bauer, John, Bellato, Sandra, Bengoetxea, Kepa, Berzak, Yevgeni, Bhat, Irshad Ahmad, Bhat, Riyaz Ahmad, Biagetti, Erica, Bick, Eckhard, Bielinskienė, Agnė, Blokland, Rogier, Bobicev, Victoria, Boizou, Loïc, Borges Völker, Emanuel, Börstell, Carl, Bosco, Cristina, Bouma, Gosse, Bowman, Sam, Boyd, Adriane, Brokaitė, Kristina, Burchardt, Aljoscha, Candito, Marie, Caron, Bernard, Caron, Gauthier, Cebiroğlu Eryiğit, Gülşen, Cecchini, Flavio Massimiliano, Celano, Giuseppe G. A., Čéplö, Slavomír, Cetin, Savas, Chalub, Fabricio, Choi, Jinho, Cho, Yongseok, Chun, Jayeol, Cinková, Silvie, Collomb, Aurélie, Çöltekin, Çağrı, Connor, Miriam, Courtin, Marine, Davidson, Elizabeth, de Marneffe, Marie-Catherine, de Paiva, Valeria, Diaz de Ilarraza, Arantza, Dickerson, Carly, Dione, Bamba, Dirix, Peter, Dobrovoljc, Kaja, Dozat, Timothy, Droganova, Kira, Dwivedi, Puneet, Eckhoff, Hanne, Eli, Marhaba, Elkahky, Ali, Ephrem, Binyam, Erjavec, Tomaž, Etienne, Aline, Farkas, Richárd, Fernandez Alcalde, Hector, Foster, Jennifer, Freitas, Cláudia, Fujita, Kazunori, Gajdošová, Katarína, Galbraith, Daniel, Garcia, Marcos, Gärdenfors, Moa, Garza, Sebastian, Gerdes, Kim, Ginter, Filip, Goenaga, Iakes, Gojenola, Koldo, Gökırmak, Memduh, Goldberg, Yoav, Gómez Guinovart, Xavier, González Saavedra, Berta, Grioni, Matias, Grūzītis, Normunds, Guillaume, Bruno, Guillot-Barbance, Céline, Habash, Nizar, Hajič, Jan, Hajič jr., Jan, Hà Mỹ, Linh, Han, Na-Rae, Harris, Kim, Haug, Dag, Heinecke, Johannes, Hennig, Felix, Hladká, Barbora, Hlaváčová, Jaroslava, Hociung, Florinel, Hohle, Petter, Hwang, Jena, Ikeda, Takumi, Ion, Radu, Irimia, Elena, Ishola, Ọlájídé, Jelínek, Tomáš, Johannsen, Anders, Jørgensen, Fredrik, Kaşıkara, Hüner, Kaasen, Andre, Kahane, Sylvain, Kanayama, Hiroshi, Kanerva, Jenna, Katz, Boris, Kayadelen, Tolga, Kenney, Jessica, Kettnerová, Václava, Kirchner, Jesse, Köhn, Arne, Kopacewicz, Kamil, Kotsyba, Natalia, Kovalevskaitė, Jolanta, Krek, Simon, Kwak, Sookyoung, Laippala, Veronika, Lambertino, Lorenzo, Lam, Lucia, Lando, Tatiana, Larasati, Septina Dian, Lavrentiev, Alexei, Lee, John, Lê Hồng, Phương, Lenci, Alessandro, Lertpradit, Saran, Leung, Herman, Li, Cheuk Ying, Li, Josie, Li, Keying, Lim, KyungTae, Li, Yuan, Ljubešić, Nikola, Loginova, Olga, Lyashevskaya, Olga, Lynn, Teresa, Macketanz, Vivien, Makazhanov, Aibek, Mandl, Michael, Manning, Christopher, Manurung, Ruli, Mărănduc, Cătălina, Mareček, David, Marheinecke, Katrin, Martínez Alonso, Héctor, Martins, André, Mašek, Jan, Matsumoto, Yuji, McDonald, Ryan, McGuinness, Sarah, Mendonça, Gustavo, Miekka, Niko, Misirpashayeva, Margarita, Missilä, Anna, Mititelu, Cătălin, Miyao, Yusuke, Montemagni, Simonetta, More, Amir, Moreno Romero, Laura, Mori, Keiko Sophie, Morioka, Tomohiko, Mori, Shinsuke, Moro, Shigeki, Mortensen, Bjartur, Moskalevskyi, Bohdan, Muischnek, Kadri, Murawaki, Yugo, Müürisep, Kaili, Nainwani, Pinkey, Navarro Horñiacek, Juan Ignacio, Nedoluzhko, Anna, Nešpore-Bērzkalne, Gunta, Nguyễn Thị, Lương, Nguyễn Thị Minh, Huyền, Nikaido, Yoshihiro, Nikolaev, Vitaly, Nitisaroj, Rattima, Nurmi, Hanna, Ojala, Stina, Olúòkun, Adédayọ̀, Omura, Mai, Osenova, Petya, Östling, Robert, Øvrelid, Lilja, Partanen, Niko, Pascual, Elena, Passarotti, Marco, Patejuk, Agnieszka, Paulino-Passos, Guilherme, Peljak-Łapińska, Angelika, Peng, Siyao, Perez, Cenel-Augusto, Perrier, Guy, Petrova, Daria, Petrov, Slav, Piitulainen, Jussi, Pirinen, Tommi A, Pitler, Emily, Plank, Barbara, Poibeau, Thierry, Popel, Martin, Pretkalniņa, Lauma, Prévost, Sophie, Prokopidis, Prokopis, Przepiórkowski, Adam, Puolakainen, Tiina, Pyysalo, Sampo, Rääbis, Andriela, Rademaker, Alexandre, Ramasamy, Loganathan, Rama, Taraka, Ramisch, Carlos, Ravishankar, Vinit, Real, Livy, Reddy, Siva, Rehm, Georg, Rießler, Michael, Rimkutė, Erika, Rinaldi, Larissa, Rituma, Laura, Rocha, Luisa, Romanenko, Mykhailo, Rosa, Rudolf, Rovati, Davide, Roșca, Valentin, Rudina, Olga, Rueter, Jack, Sadde, Shoval, Sagot, Benoît, Saleh, Shadi, Salomoni, Alessio, Samardžić, Tanja, Samson, Stephanie, Sanguinetti, Manuela, Särg, Dage, Saulīte, Baiba, Sawanakunanon, Yanin, Schneider, Nathan, Schuster, Sebastian, Seddah, Djamé, Seeker, Wolfgang, Seraji, Mojgan, Shen, Mo, Shimada, Atsuko, Shirasu, Hiroyuki, Shohibussirri, Muh, Sichinava, Dmitry, Silveira, Natalia, Simi, Maria, Simionescu, Radu, Simkó, Katalin, Šimková, Mária, Simov, Kiril, Smith, Aaron, Soares-Bastos, Isabela, Spadine, Carolyn, Stella, Antonio, Straka, Milan, Strnadová, Jana, Suhr, Alane, Sulubacak, Umut, Suzuki, Shingo, Szántó, Zsolt, Taji, Dima, Takahashi, Yuta, Tamburini, Fabio, Tanaka, Takaaki, Tellier, Isabelle, Thomas, Guillaume, Torga, Liisi, Trosterud, Trond, Trukhina, Anna, Tsarfaty, Reut, Tyers, Francis, Uematsu, Sumire, Urešová, Zdeňka, Uria, Larraitz, Uszkoreit, Hans, Vajjala, Sowmya, van Niekerk, Daniel, van Noord, Gertjan, Varga, Viktor, Villemonte de la Clergerie, Eric, Vincze, Veronika, Wallin, Lars, Walsh, Abigail, Wang, Jing Xian, Washington, Jonathan North, Wendt, Maximilan, Williams, Seyi, Wirén, Mats, Wittern, Christian, Woldemariam, Tsegay, Wong, Tak-sum, Wróblewska, Alina, Yako, Mary, Yamazaki, Naoki, Yan, Chunxiao, Yasuoka, Koichi, Yavrumyan, Marat M., Yu, Zhuoran, Žabokrtský, Zdeněk, Zeldes, Amir, Zeman, Daniel, Zhang, Manying, and Zhu, Hanzhi
Publisher:
Universal Dependencies Consortium
Type:
text and corpus
Subject:
treebank, dependency, syntax, morphology, harmonized annotation, interset, universal tagset, and stanford dependencies
Language:
Ancient Greek (to 1453), Arabic, Basque, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Gothic, Modern Greek (1453-), Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Norwegian, Church Slavic, Persian, Polish, Portuguese, Romanian, Slovenian, Spanish, Swedish, Tamil, Catalan, Chinese, Galician, Kazakh, Latvian, Russian, Turkish, Coptic, Sanskrit, Slovak, Ukrainian, Uighur, Vietnamese, Belarusian, Korean, Lithuanian, Urdu, Russia Buriat, Northern Kurdish, Northern Sami, Upper Sorbian, Afrikaans, Yue Chinese, Marathi, Serbian, Swedish Sign Language, Telugu, Amharic, Armenian, Breton, Faroese, Komi-Zyrian, Nigerian Pidgin, Old French (842-ca. 1400), Tagalog, Thai, Warlpiri, Yoruba, Akkadian, Bambara, Erzya, Maltese, Welsh, Wolof, Assyrian Neo-Aramaic, Literary Chinese, Old Russian, Karelian, and Mbyá Guaraní
Description:
Universal Dependencies is a project that seeks to develop cross-linguistically consistent treebank annotation for many languages, with the goal of facilitating multilingual parser development, cross-lingual learning, and parsing research from a language typology perspective. The annotation scheme is based on (universal) Stanford dependencies (de Marneffe et al., 2006, 2008, 2014), Google universal part-of-speech tags (Petrov et al., 2012), and the Interset interlingua for morphosyntactic tagsets (Zeman, 2008).
Rights:
Licence Universal Dependencies v2.4, https://lindat.mff.cuni.cz/repository/xmlui/page/licence-UD-2.4, and PUB

3. Universal Dependencies 2.5

Creator:
Zeman, Daniel, Nivre, Joakim, Abrams, Mitchell, Aepli, Noëmi, Agić, Željko, Ahrenberg, Lars, Aleksandravičiūtė, Gabrielė, Antonsen, Lene, Aplonova, Katya, Aranzabe, Maria Jesus, Arutie, Gashaw, Asahara, Masayuki, Ateyah, Luma, Attia, Mohammed, Atutxa, Aitziber, Augustinus, Liesbeth, Badmaeva, Elena, Ballesteros, Miguel, Banerjee, Esha, Bank, Sebastian, Barbu Mititelu, Verginica, Basmov, Victoria, Batchelor, Colin, Bauer, John, Bellato, Sandra, Bengoetxea, Kepa, Berzak, Yevgeni, Bhat, Irshad Ahmad, Bhat, Riyaz Ahmad, Biagetti, Erica, Bick, Eckhard, Bielinskienė, Agnė, Blokland, Rogier, Bobicev, Victoria, Boizou, Loïc, Borges Völker, Emanuel, Börstell, Carl, Bosco, Cristina, Bouma, Gosse, Bowman, Sam, Boyd, Adriane, Brokaitė, Kristina, Burchardt, Aljoscha, Candito, Marie, Caron, Bernard, Caron, Gauthier, Cavalcanti, Tatiana, Cebiroğlu Eryiğit, Gülşen, Cecchini, Flavio Massimiliano, Celano, Giuseppe G. A., Čéplö, Slavomír, Cetin, Savas, Chalub, Fabricio, Choi, Jinho, Cho, Yongseok, Chun, Jayeol, Cignarella, Alessandra T., Cinková, Silvie, Collomb, Aurélie, Çöltekin, Çağrı, Connor, Miriam, Courtin, Marine, Davidson, Elizabeth, de Marneffe, Marie-Catherine, de Paiva, Valeria, de Souza, Elvis, Diaz de Ilarraza, Arantza, Dickerson, Carly, Dione, Bamba, Dirix, Peter, Dobrovoljc, Kaja, Dozat, Timothy, Droganova, Kira, Dwivedi, Puneet, Eckhoff, Hanne, Eli, Marhaba, Elkahky, Ali, Ephrem, Binyam, Erina, Olga, Erjavec, Tomaž, Etienne, Aline, Evelyn, Wograine, Farkas, Richárd, Fernandez Alcalde, Hector, Foster, Jennifer, Freitas, Cláudia, Fujita, Kazunori, Gajdošová, Katarína, Galbraith, Daniel, Garcia, Marcos, Gärdenfors, Moa, Garza, Sebastian, Gerdes, Kim, Ginter, Filip, Goenaga, Iakes, Gojenola, Koldo, Gökırmak, Memduh, Goldberg, Yoav, Gómez Guinovart, Xavier, González Saavedra, Berta, Griciūtė, Bernadeta, Grioni, Matias, Grūzītis, Normunds, Guillaume, Bruno, Guillot-Barbance, Céline, Habash, Nizar, Hajič, Jan, Hajič jr., Jan, Hämäläinen, Mika, Hà Mỹ, Linh, Han, Na-Rae, Harris, Kim, Haug, Dag, Heinecke, Johannes, Hennig, Felix, Hladká, Barbora, Hlaváčová, Jaroslava, Hociung, Florinel, Hohle, Petter, Hwang, Jena, Ikeda, Takumi, Ion, Radu, Irimia, Elena, Ishola, Ọlájídé, Jelínek, Tomáš, Johannsen, Anders, Jørgensen, Fredrik, Juutinen, Markus, Kaşıkara, Hüner, Kaasen, Andre, Kabaeva, Nadezhda, Kahane, Sylvain, Kanayama, Hiroshi, Kanerva, Jenna, Katz, Boris, Kayadelen, Tolga, Kenney, Jessica, Kettnerová, Václava, Kirchner, Jesse, Klementieva, Elena, Köhn, Arne, Kopacewicz, Kamil, Kotsyba, Natalia, Kovalevskaitė, Jolanta, Krek, Simon, Kwak, Sookyoung, Laippala, Veronika, Lambertino, Lorenzo, Lam, Lucia, Lando, Tatiana, Larasati, Septina Dian, Lavrentiev, Alexei, Lee, John, Lê Hồng, Phương, Lenci, Alessandro, Lertpradit, Saran, Leung, Herman, Li, Cheuk Ying, Li, Josie, Li, Keying, Lim, KyungTae, Liovina, Maria, Li, Yuan, Ljubešić, Nikola, Loginova, Olga, Lyashevskaya, Olga, Lynn, Teresa, Macketanz, Vivien, Makazhanov, Aibek, Mandl, Michael, Manning, Christopher, Manurung, Ruli, Mărănduc, Cătălina, Mareček, David, Marheinecke, Katrin, Martínez Alonso, Héctor, Martins, André, Mašek, Jan, Matsumoto, Yuji, McDonald, Ryan, McGuinness, Sarah, Mendonça, Gustavo, Miekka, Niko, Misirpashayeva, Margarita, Missilä, Anna, Mititelu, Cătălin, Mitrofan, Maria, Miyao, Yusuke, Montemagni, Simonetta, More, Amir, Moreno Romero, Laura, Mori, Keiko Sophie, Morioka, Tomohiko, Mori, Shinsuke, Moro, Shigeki, Mortensen, Bjartur, Moskalevskyi, Bohdan, Muischnek, Kadri, Munro, Robert, Murawaki, Yugo, Müürisep, Kaili, Nainwani, Pinkey, Navarro Horñiacek, Juan Ignacio, Nedoluzhko, Anna, Nešpore-Bērzkalne, Gunta, Nguyễn Thị, Lương, Nguyễn Thị Minh, Huyền, Nikaido, Yoshihiro, Nikolaev, Vitaly, Nitisaroj, Rattima, Nurmi, Hanna, Ojala, Stina, Ojha, Atul Kr., Olúòkun, Adédayọ̀, Omura, Mai, Osenova, Petya, Östling, Robert, Øvrelid, Lilja, Partanen, Niko, Pascual, Elena, Passarotti, Marco, Patejuk, Agnieszka, Paulino-Passos, Guilherme, Peljak-Łapińska, Angelika, Peng, Siyao, Perez, Cenel-Augusto, Perrier, Guy, Petrova, Daria, Petrov, Slav, Phelan, Jason, Piitulainen, Jussi, Pirinen, Tommi A, Pitler, Emily, Plank, Barbara, Poibeau, Thierry, Ponomareva, Larisa, Popel, Martin, Pretkalniņa, Lauma, Prévost, Sophie, Prokopidis, Prokopis, Przepiórkowski, Adam, Puolakainen, Tiina, Pyysalo, Sampo, Qi, Peng, Rääbis, Andriela, Rademaker, Alexandre, Ramasamy, Loganathan, Rama, Taraka, Ramisch, Carlos, Ravishankar, Vinit, Real, Livy, Reddy, Siva, Rehm, Georg, Riabov, Ivan, Rießler, Michael, Rimkutė, Erika, Rinaldi, Larissa, Rituma, Laura, Rocha, Luisa, Romanenko, Mykhailo, Rosa, Rudolf, Rovati, Davide, Roșca, Valentin, Rudina, Olga, Rueter, Jack, Sadde, Shoval, Sagot, Benoît, Saleh, Shadi, Salomoni, Alessio, Samardžić, Tanja, Samson, Stephanie, Sanguinetti, Manuela, Särg, Dage, Saulīte, Baiba, Sawanakunanon, Yanin, Schneider, Nathan, Schuster, Sebastian, Seddah, Djamé, Seeker, Wolfgang, Seraji, Mojgan, Shen, Mo, Shimada, Atsuko, Shirasu, Hiroyuki, Shohibussirri, Muh, Sichinava, Dmitry, Silveira, Aline, Silveira, Natalia, Simi, Maria, Simionescu, Radu, Simkó, Katalin, Šimková, Mária, Simov, Kiril, Smith, Aaron, Soares-Bastos, Isabela, Spadine, Carolyn, Stella, Antonio, Straka, Milan, Strnadová, Jana, Suhr, Alane, Sulubacak, Umut, Suzuki, Shingo, Szántó, Zsolt, Taji, Dima, Takahashi, Yuta, Tamburini, Fabio, Tanaka, Takaaki, Tellier, Isabelle, Thomas, Guillaume, Torga, Liisi, Trosterud, Trond, Trukhina, Anna, Tsarfaty, Reut, Tyers, Francis, Uematsu, Sumire, Urešová, Zdeňka, Uria, Larraitz, Uszkoreit, Hans, Utka, Andrius, Vajjala, Sowmya, van Niekerk, Daniel, van Noord, Gertjan, Varga, Viktor, Villemonte de la Clergerie, Eric, Vincze, Veronika, Wallin, Lars, Walsh, Abigail, Wang, Jing Xian, Washington, Jonathan North, Wendt, Maximilan, Williams, Seyi, Wirén, Mats, Wittern, Christian, Woldemariam, Tsegay, Wong, Tak-sum, Wróblewska, Alina, Yako, Mary, Yamazaki, Naoki, Yan, Chunxiao, Yasuoka, Koichi, Yavrumyan, Marat M., Yu, Zhuoran, Žabokrtský, Zdeněk, Zeldes, Amir, Zhang, Manying, and Zhu, Hanzhi
Publisher:
Universal Dependencies Consortium
Type:
text and corpus
Subject:
treebank, dependency, syntax, morphology, harmonized annotation, interset, universal tagset, and stanford dependencies
Language:
Ancient Greek (to 1453), Arabic, Basque, Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Gothic, Modern Greek (1453-), Hebrew, Hindi, Hungarian, Indonesian, Irish, Italian, Japanese, Latin, Norwegian, Church Slavic, Persian, Polish, Portuguese, Romanian, Slovenian, Spanish, Swedish, Tamil, Catalan, Chinese, Galician, Kazakh, Latvian, Russian, Turkish, Coptic, Sanskrit, Slovak, Ukrainian, Uighur, Vietnamese, Belarusian, Korean, Lithuanian, Urdu, Russia Buriat, Northern Kurdish, Northern Sami, Upper Sorbian, Afrikaans, Yue Chinese, Marathi, Serbian, Swedish Sign Language, Telugu, Amharic, Armenian, Breton, Faroese, Komi-Zyrian, Nigerian Pidgin, Old French (842-ca. 1400), Tagalog, Thai, Warlpiri, Yoruba, Akkadian, Bambara, Erzya, Maltese, Welsh, Wolof, Assyrian Neo-Aramaic, Literary Chinese, Old Russian, Karelian, Mbyá Guaraní, Bhojpuri, Komi-Permyak, Livvi, Moksha, Scottish Gaelic, Skolt Sami, and Swiss German
Description:
Universal Dependencies is a project that seeks to develop cross-linguistically consistent treebank annotation for many languages, with the goal of facilitating multilingual parser development, cross-lingual learning, and parsing research from a language typology perspective. The annotation scheme is based on (universal) Stanford dependencies (de Marneffe et al., 2006, 2008, 2014), Google universal part-of-speech tags (Petrov et al., 2012), and the Interset interlingua for morphosyntactic tagsets (Zeman, 2008).
Rights:
Licence Universal Dependencies v2.5, https://lindat.mff.cuni.cz/repository/xmlui/page/licence-UD-2.5, and PUB