Software and Resources

Neural BERT-like language model for French

Neural GPT-based language model for French

A fully open suite of French-English-coding language models designed to advance transparency and reproducibility in large-scale model training.

ModernCamemBERT is a French language model pretrained on a large corpus of 1T tokens of High-Quality French text. It is the French version of the ModernBERT model.

ELMo language model for French

ELMo language models for 5 mid-resource languages (Bulgarian, Catalan, Danish, Finnish, Indonesian)

A DeBERTa v3-based French language model

A differentiable tokenizer trained end-to-end with the language model.

Neural GLiNER-like language model for the French biomedical domain

Neural BERT-like language model for Early Modern French

Neural BERT-like language model for the French biomedical domain

A CharacterBERT language model for North-African Arabizi and French user-generated content

Open large multilingual language model

A SOTA French pretrained language model, based on the DeBERTaV3 architecture.

A new updated version of the CamemBERT pretrained language model for french

Huge multilingual web-based corpus

Asynchronous concurrent pipeline for classifying Common Crawl

High-performance pipeline that provides tools to build corpus generation pipelines from CommonCrawl.

Large-scale multilingual, multimodal (text-image) web-crawled corpus

Speech parallel corpus mined from VoxPopuli

A Benchmark and Analysis of Discrete Expressive Speech Resynthesis

HTR-United is an open Github ecosystem designed to share training data for HTR and OCR tasks

Large-scale diverse dataset for handwritten text recognition of medieval manuscripts

Dataset for HTR training on Contemporary French

LADaS (Layout Analysis Dataset with SegmOnto) is a diachronic diageneric layout analysis dataset (16th-21st c.)

KaMI-lib is an HTR and OCR engine agnostic Python package for evaluating transcription models

Open and collaborative documentation for eScriptorium

Handwritten Text Recognition model for medieval manuscripts in Latin scripts

Ground-truth for training HTR models

Web application for manual, semi-automatic, and automatic segmentation and transcription of printed or handwritten text documents, with the possibility of training or reusing transcription models.

Kraken is software that can be used to train and utilize models for transcribing, segmenting, and annotating printed or handwritten documents, regardless of language.

Contrastive test sets for the evaluation of discourse phenomena in English-to-French machine translation

FR-EN parallel corpus of noisy user-generated content

FR-EN Annotated parallel corpus of noisy user-generated content

Parallel dataset of English-French bilingual dialogues

SONAR (Sentence-level multimOdal and laNguage-Agnostic Representations) is a multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders

Approach to cross-modal transfer between speech and text for translation tasks

SWELLS makes it possible to assess, in a controlled way, the ability of language models to assimilate specific aspects of an unknown language on the basis of grammar book excerpts added to their prompts.

Generates fine-tuning datasets for preference optimization in machine translation.

A multimodal machine translation model

A contrastive evaluation dataset for multimodal (text-image) machine translation.

Robust Challenge Set for Machine Translation

Controllable Text Simplification Model

Text Simplification Evaluation Dataset

Text Simplification Evaluation Library

Text Simplification Evaluation Library

Free Wordnet for French

Morphological (and sometimes syntactic) lexicons (including the Lefff)

Online user interface to collaboratively modify and check the OFrLex lexicon

Etymological database extracted from wiktionary

Multilingual collection of morphological lexicons

Collection of research use case scenarios illustrating best practices in Digital Humanities and Heritage research

French corpus with surface and deep syntactic annotations

Multi-layered treebank made of questions for French

French social media bank

A multi-layered treebank for the Arabic dialect spoken in North Africa and written in Latin Script

A large-coverage meta-grammar for French

Lexical and syntactic parser generator

Environment for building tabular parsers and programs

Linguistic Wiki for FRMG

Transition-based parser built on top of DyALog

Neural parsing system developed for ALMAnaCH's submission to the CoNLL-18 multilingual parsing shared task

Shallow language pipeline

GROBID module for structuring digitised lexical resources and entry-based documents

Library for extracting, parsing and re-structuring raw documents

Statistical part-of-speech tagger

Entity recognition and disambiguation

GROBID module for extracting and restructuring medical reports from PDF documents into encoded XML/TEI documents

Normalisation of Modern (17th c.) French

A tool for extracting scholarly documents and visualizing the results on PDF files using GROBID.

NerdKid is a tool for grouping Wikidata entities into 27 classes (e.g., ANIMAL, LOCATION, MEDIA, PERSON).

Similarity detection tool for legal texts from the Cour de Cassation

NER model for Early Modern French

POS tagger for Early Modern French

Occitan dialect identification models

Automatic analysis of answers to open-ended questions in employee surveys

Generation of notes from structured data

Omission detection tool for journalistic content.

Corpora and NLP tools for Early Modern French (16th-18th c.)

3 Minutes Thesis Corpus

An open-source pseudonymized dataset aimed at facilitating research on radicalization detection with NER annotations. It is the first publicly available multilingual dataset for radicalization detection, gathered from diverse sources.

A collection of Spanish tweets annotated for hate speech towards immigrants across two different Spanish-speaking regions.

A corpus of annotated instances of Word Meaning Negotiation (sequences in conversation where speakers discuss word meaning) from existing oral and written conversational corpora.

SPOT (Stopping Points in Online Threads) is a French corpus of 43k Facebook comments annotated for the presence of stopping points (critical interventions)

A Cuban Spanish variety identification dataset with common example annotations developed to facilitate more accurate detection of Cuban and Caribbean Spanish varieties. This is the first dataset dedicated to identifying the Cuban (or any other Caribbean) Spanish variety.

OcWikiDialects is a corpus derived from Occitan Wikipedia, featuring diverse metadata, including dialect annotations.