Software and Resources

Neural BERT-like language model for French

Neural GPT-based language model for French

ELMo language model for French

ELMo language models for 5 mid-resource languages (Bulgarian, Catalan, Danish, Finnish, Indonesian)

Neural BERT-like language model for Early Modern French

Neural BERT-like language model for the French biomedical domain

A differentiable tokenizer trained end-to-end with the language model.

A CharacterBERT language model for North-African Arabizi and French user-generated content

Open large multilingual language model

A DeBERTa v3-based French language model

A SOTA French pretrained language model, based on the DeBERTaV3 architecture.

A new updated version of the CamemBERT pretrained language model for french

Huge multilingual web-based corpus

Asynchronous concurrent pipeline for classifying Common Crawl

High-performance pipeline that provides tools to build corpus generation pipelines from CommonCrawl.

Large-scale multilingual, multimodal (text-image) web-crawled corpus

Speech parallel corpus mined from VoxPopuli

A Benchmark and Analysis of Discrete Expressive Speech Resynthesis

KaMI-lib is an HTR and OCR engine agnostic Python package for evaluating transcription models

HTR-United is an open Github ecosystem designed to share training data for HTR and OCR tasks

Dataset for HTR training on Contemporary French

Open and collaborative documentation for eScriptorium

Ground-truth for training HTR models

Large-scale diverse dataset for handwritten text recognition of medieval manuscripts

Handwritten Text Recognition model for medieval manuscripts in Latin scripts

LADaS (Layout Analysis Dataset with SegmOnto) is a diachronic diageneric layout analysis dataset (16th-21st c.)

Contrastive test sets for the evaluation of discourse phenomena in English-to-French machine translation

FR-EN parallel corpus of noisy user-generated content

FR-EN Annotated parallel corpus of noisy user-generated content

Parallel dataset of English-French bilingual dialogues

A multimodal machine translation model

A contrastive evaluation dataset for multimodal (text-image) machine translation.

Robust Challenge Set for Machine Translation

SONAR (Sentence-level multimOdal and laNguage-Agnostic Representations) is a multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders

Approach to cross-modal transfer between speech and text for translation tasks

Controllable Text Simplification Model

Text Simplification Evaluation Dataset

Text Simplification Evaluation Library

Text Simplification Evaluation Library

Free Wordnet for French

Morphological (and sometimes syntactic) lexicons (including the Lefff)

Etymological database extracted from wiktionary

Multilingual collection of morphological lexicons

Online user interface to collaboratively modify and check the OFrLex lexicon

Collection of research use case scenarios illustrating best practices in Digital Humanities and Heritage research

French corpus with surface and deep syntactic annotations

Multi-layered treebank made of questions for French

French social media bank

A multi-layered treebank for the Arabic dialect spoken in North Africa and written in Latin Script

A large-coverage meta-grammar for French

Lexical and syntactic parser generator

Environment for building tabular parsers and programs

Linguistic Wiki for FRMG

Transition-based parser built on top of DyALog

Neural parsing system developed for ALMAnaCH's submission to the CoNLL-18 multilingual parsing shared task

Shallow language pipeline

GROBID module for structuring digitised lexical resources and entry-based documents

Library for extracting, parsing and re-structuring raw documents

Entity recognition and disambiguation

Statistical part-of-speech tagger

GROBID module for extracting and restructuring medical reports from PDF documents into encoded XML/TEI documents

A tool for extracting scholarly documents and visualizing the results on PDF files using GROBID.

Normalisation of Modern (17th c.) French

NerdKid is a tool for grouping Wikidata entities into 27 classes (e.g., ANIMAL, LOCATION, MEDIA, PERSON).

Similarity detection tool for legal texts from the Cour de Cassation

NER model for Early Modern French

POS tagger for Early Modern French

Automatic analysis of answers to open-ended questions in employee surveys

Generation of notes from structured data

Omission detection tool for journalistic content.

Corpora and NLP tools for Early Modern French (16th-18th c.)

3 Minutes Thesis Corpus

An open-source pseudonymized dataset aimed at facilitating research on radicalization detection with NER annotations. It is the first publicly available multilingual dataset for radicalization detection, gathered from diverse sources.

A Cuban Spanish variety identification dataset with common example annotations developed to facilitate more accurate detection of Cuban and Caribbean Spanish varieties. This is the first dataset dedicated to identifying the Cuban (or any other Caribbean) Spanish variety.

A collection of Spanish tweets annotated for hate speech towards immigrants across two different Spanish-speaking regions.