Supporting research into the history of French syntax, the three projects presented here sought to integrate existing tools and develop methodologies for successful management and syntactic annotation of medium to large diachronic corpora. To limit potential documentation biases, all three share a common approach to the selection of data, opting for texts of a given non-literary genre and produced in the same geographical area (Normandy), to control for extraneous variation (Table 1). The ConDÉ project (2018-2021)[5] produced a large digitized corpus of Norman customal law texts (coutumiers normands) from the 13th to the 18th century, based on a method trialed on a set of historical letters (EPELE).[6] Building on the experience of ConDÉ, High-Tech project (2021-2023) gave rise to a collection of chronicles and historical treatises from the 12th to the 19th centuries, with one text per century.[7] In addition, as part of the Franco-German MICLE project (2021-2024), a further medium-sized corpus of legal texts was elaborated which includes trial accounts and styles de procéder from the 12th to the 17th centuries, with one text per half-century.[8] Below, we refer to the corpus of customal law as ConDÉ, to the corpus of chronicles produced as part of the High-Tech project as Chroniques and to the corpus of trial accounts and styles de procéder, created by the French team of the MICLE project, as MICLE-Fr. The semi-automated workflow for lemmatization and annotation developed by the High-Tech project and tested during the annotation of Chroniques and MICLE-Fr corpora is referred to as the HT-CRISCO workflow. The annotated collections can be consulted via the CRISCO Laboratory’s TXM server and two dedicated websites.[9] Most texts are out of copyright and can be consulted either in full or via a concordance, whereas for a small minority of sources only concordance view is available.

All corpora were digitized from reproductions of manuscripts or scans of printed editions using OCR and HTR technologies made available via the Transkribus portal. [10] Transkribus allows the use of pre-trained transcription models and training of new models adapted to particular sources. Creating custom-made models was especially necessary in the case of the ConDÉ project, which processed a mass of material: the two volumes by Basnage (1678), for example, run up to a million characters. As part of MICLE-Fr, one particular challenge was the complete and fully verified transcription of a 16th-century manuscript of witness statements from the island of Guernsey Greffe Crime 1 (more than 41,000 tokens).[11]

The diversity of sources from across several centuries, ranging from medieval manuscripts to modern critical editions, not to mention early printed books, meant that the texts of the corpora presented considerable variation where spelling of individual words were concerned. The decision to remain as close to the original as possible and not to normalize the spelling was taken for all projects as the tools available permitted to accommodate variation. Lemmatization allowed grouping all grammatical forms and all spelling variants of the same word under the same headword. For example, in the Chroniques corpus, the French word king can be encountered under the forms rei, reis, rey, reys, roi, rois, all lemmatized under the headword roi.

Once digitized, the texts of ConDÉ, Chroniques and MICLE-Fr corpora were converted into the XML-TEI format [DeRose 1999]. This format permits encoding editorial interventions such as corrections of the original (e.g., in case of repetition of words). The use of this format facilitated manual correction of the data throughout the annotation pipeline described below and allowed integration of the final versions of the annotated files into the TXM-portal and websites to enable consultation and interrogation of the corpora. Texts were processed and corrected individually, with a different file corresponding to each.

The chosen format allowed reflecting the structure of the original (i.e., the manuscript or printed version that had been digitized) in the structure of the XML document. According to the XML-TEI hierarchy, a text can be divided into books, a book can be divided into sections, a section into chapters, and a chapter into paragraphs; the corresponding tags made it possible to indicate where paragraphs and larger sections of the text begin and end.[12]

For ConDÉ, only XML-TEI format was used throughout all stages of processing. For Chroniques and MICLE-Fr, files needed to be converted into the CONLL-U tabular format that is used by the syntactic parsers twice during the execution of the workflow (see 3d-2 below). To reintegrate the parsed data into the XML-TEI corpus, CONLL-U files were converted back into the XML-TEI format. The synchronization between the XML-TEI version and CONLL-U version of the files was done via the assignment of a unique ID to each sentence of the text. Thus, sentence 1 in paragraph 3 of chapter 34 in section 1 of book 2 would be given sentence ID “2_1_34_3_1”. This number would, in turn, allow finding the place of the sentence in the XML-TEI structure after parsing is completed.

Prior to any further treatment, the texts were tokenized and, in the case of Chroniques and MICLE-Fr, segmented into sentences (using strong punctuation as a prompt for automatic segmentation, followed by manual revision). For the ConDÉ project, annotation was done exclusively on the token level, whereas for High-Tech and MICLE-French sentence segmentation was an essential step towards preparing the data for syntactic parsing using an automatic tool.

Considering the nature of the data and of the research needs as well as the availability of annotation tools, the sequence of projects followed two different yet complementary axes:

1) increased accuracy of the annotation and formalization of the correction process;

2) introduction of more layers of annotation.

Where ConDÉ is lemmatized and annotated in PoS using one set of tags, Chroniques and MICLE-Fr are lemmatized, PoS-annotated in three different systems, adding a layer of morphological annotation, and (automatically) annotated in syntactic functions. At every stage of this process, new tools, resources and methodological approaches were integrated as they became available to the linguistic community.