LEAF-Turning Engine Documentation

The LEAF-Turning Engine (LEAF-TE) is an integrated component of LEAF-Writer Commons that supports easy transformation to and from TEI-XML file formats, designed to enhance the digital editorial workflow.

Importing files into LEAF-Writer

For editors working in Word documents, footnotes, endnotes, links, and basic styling are preserved and converted into valid TEI.

For editors working in Markdown LEAF-Writer converts the file into valid TEI while retaining basic styling and URL links.

For editors using HTR and OCR tools for transcription, LEAF-TE supports import from a range of such tools, including Transkribus, eScriptorium, and OCR4all, with the option to encode with or without facsimile zone markup.

Regardless of your imported format, your file will include a blank TEI header: you will need to add information about your project when you're editing in LEAF-Writer.

Import from Word

This is one of the simplest import paths, and preserves simple styling, URL links, as well as footnotes and endnotes as valid TEI.

1. With your document open in MS Word, choose Save As and select the Word XML Document (.xml) option.
2. In LEAF-Writer, click Import Document and choose the Word option. Drag your file into the box or browse for it on your desktop.
3. Click Import. Your document should open in the LEAF-Writer panel.

Import from Markdown

This path is also simple, and preserves basic styling and URL links as valid TEI.

Because LEAF-Writer requires an .xml file with an XML declaration, you'll need to prepare your Markdown file first:

1. In a text editor, add these two lines at the very top of your document:

   <?xml version="1.0" encoding="utf-8"?>
   <markdown>

   and this line at the very end of your document:

   </markdown>

2. Save the file with an .xml extension.
3. In LEAF-Writer, click Import Document and choose the Markdown option. Drag your file into the box or browse for it on your desktop.
4. Click Import. Your document should open in the LEAF-Writer panel.

Import from HTR/OCR tools

LEAF-TE supports import from a range of handwritten text recognition (HTR) and optical character recognition (OCR) tools widely used in cultural heritage transcription work. There are three paths into LEAF-Writer, depending on which export format your HTR tool or platform produces. All three converge on the same final step — importing the resulting TEI XML file into LEAF-Writer, where you'll choose whether to keep or discard facsimile zone data.

You will need a LEAF-Writer account at leaf-writer.leaf-vre.org for all three paths.

Path A — Transkribus TEI Export (page2tei)

Tools: Transkribus only

This is the simplest HTR path — no script or Python setup is needed. Transkribus can export a fully structured, multi-page TEI file directly using the page2tei stylesheet. This is also the only path that lets you choose between verse and prose formatting at export time, described below.

Step 1: Export from Transkribus

1. Open your document in Transkribus and click Export.
2. Select TEI Export.
3. Under Stylesheet, confirm page2tei is selected.
4. Configure the export options:
   • Under Line Breaks, choose between line tags (<l>...</l>) and line breaks (<lb/>). Line tags produce verse-formatted text in LEAF-Writer, with lines nested inside line groups; line breaks produce prose-formatted text, with divided lines turned into paragraphs. Line breaks is recommended unless your material is verse.
   • Use rectangles will simplify zone coordinates to bounding boxes (optional).
   • Leave other options at their defaults unless you have specific requirements.
5. Click Export and download the TEI XML file — this is a single multi-page file.

Step 2: Import into LEAF-Writer See Completing the import in LEAF-Writer below.

Note: The TEI Export (page2tei) option requires a Scholar, Team, or Organisation plan in Transkribus. It is not available on the free plan. LEAF is not affiliated with Transkribus or READ-COOP and has no control over their subscription practices, so plan requirements may change over time.

Python setup (one-time, needed for Paths B and C)

Editors using other HTR or OCR tools — such as eScriptorium, Kraken, or OCR4all — will need to complete an additional, one-time setup step using Python before importing into LEAF-Writer. Paths B and C use a Python script to merge your per-page export files and convert them to TEI. Complete the following setup once before running the script for the first time.

1. Install Python

Mac: Python 3 is usually already available on macOS. Open Terminal (Applications → Utilities) and check:

python3 --version

If you see a version number, you're ready. Otherwise, install from python.org.

Windows: Open Command Prompt or PowerShell and check:

python --version

If you see a version number, you're ready. Otherwise, install from python.org, and be sure to tick "Add Python to PATH" during installation.

2. Download the script files

Save the following files into the same folder on your computer:

• htr_merge_and_transform.py
• alto-to-tei.xsl
• page-to-tei.xsl

3. Navigate to the script folder in your terminal

Mac:

cd ~/Desktop/htr-scripts

Windows:

cd C:\Users\YourName\Desktop\htr-scripts

4. Install the required libraries

Mac: Recent macOS/Homebrew Python setups require a virtual environment first. Run once:

python3 -m venv venv
source venv/bin/activate
pip install lxml saxonche

Your prompt will show (venv) when active. Reactivate it each time you open a new Terminal window before running the script:

source venv/bin/activate

Windows:

pip install lxml saxonche

Path B — PAGE XML

Tools that export PAGE XML: eScriptorium, OCR4all, Kraken, PERO OCR, Transkribus (PAGE XML export option)

PAGE XML is the native export format of most modern open-source HTR tools and is the recommended export choice where available.

Step 1: Export from your HTR tool

eScriptorium:

1. Open your document and go to the Images tab.
2. Select all pages using the checkboxes.
3. Click Export, choose Pagexml as the format.
4. Download the zip file and unzip it — you'll have a folder of XML files, one per page.

OCR4all:

1. Go to Result Generation in the OCR4all interface.
2. Export the PAGE XML files for your document.
3. Locate the exported folder — one XML file per page.

Kraken / PERO OCR:

1. Run recognition as normal — output PAGE XML files save to your specified output folder, one file per page.

Transkribus (PAGE XML option):

1. Click Export and select Page XML.
2. Download and unzip — one XML file per page.

Step 2: Run the merge and transform script

Mac:

python3 htr_merge_and_transform.py /path/to/page_xml_folder/ /path/to/output.xml --format page

Windows:

python htr_merge_and_transform.py C:\path\to\page_xml_folder\ C:\path\to\output.xml --format page

The script processes files in alphabetical order by filename, producing a single TEI XML file at the output path you specified.

Step 3: Import into LEAF-Writer See Completing the import in LEAF-Writer below.

Path C — ALTO XML

Tools that export ALTO XML: Transkribus (ALTO export option), eScriptorium (ALTO export option), ABBYY FineReader (via conversion), Tesseract (via conversion)

Use this path if your tool doesn't support PAGE XML export, or if you specifically need to work with ALTO XML files.

Step 1: Export from your HTR tool

Transkribus:

1. Click Export and select Alto XML.
2. Download the export — one XML file per page.

eScriptorium:

1. Open your document and go to the Images tab.
2. Select all pages using the checkboxes.
3. Click Export, choose Alto as the format.
4. Download the zip file and unzip it — one XML file per page.

Step 2: Run the merge and transform script

Note that --format page is not needed here — ALTO is the default.

Mac:

python3 htr_merge_and_transform.py /path/to/alto_folder/ /path/to/output.xml

Windows:

python htr_merge_and_transform.py C:\path\to\alto_folder\ C:\path\to\output.xml

The script processes files in alphabetical order by filename, producing a single TEI XML file at the output path you specified.

Step 3: Import into LEAF-Writer See Completing the import in LEAF-Writer below.

Completing the import in LEAF-Writer

All three HTR paths produce (or already have) a single TEI XML file. Import it as follows:

1. Go to leaf-writer.leaf-vre.org.

2. From the opening screen, click Import Document.

3. Choose one of the two HTR import options:

   • HTR import (with zones) — keeps all facsimile zone data linking text to image coordinates. Choose this if you want to use the image viewer in LEAF-Writer, or if your project needs to preserve the link between transcribed text and its location on the page image.
   • HTR import (without zones) — strips the facsimile zone data, keeping only the text and its structure. Choose this for a cleaner file if you don't need image-text linking.

   Both options preserve <lb/> line break elements and all transcribed text.

4. Drag your TEI XML file onto the upload box, or click to browse for it.

5. The file will be transformed and open automatically in the LEAF-Writer editor.

Running into issues with an HTR/OCR import? See Troubleshooting HTR/OCR Imports.

A note on the image viewer: LEAF-Writer's image viewer can only display images hosted at a publicly accessible web URL. Some HTR tools will export page images or indicate where page breaks appear. Images stored locally on your computer will therefore not appear. The TEI files produced by this workflow do not link page breaks to images by default for this reason. If you want to enable image viewing later, you'll need to host your page images online (e.g. via IIIF or a web server) and add a @facs attribute to each <pb/> element pointing to the corresponding image URL.

Exporting files from LEAF-Writer

Completed work can in turn be exported from LEAF-Writer as TEI-XML, Markdown, HTML, or CSV.

Some editors working in TEI have felt limited once their encoding is complete, without easy access to the kind of sophisticated post-processing tools available in more complex publishing workflows. LEAF-TE addresses this by putting some of that post-processing directly in editors' hands: exported files can be used to build a website or static site in HTML or Markdown, or to extract tagged entity data into CSV for further analysis such as mapping or social network analysis — all while retaining the entity authority URLs created in LEAF-Writer and the associated web annotations.

To export your work, look for the three horizontal lines icon at the top left of the screen, next to the LEAF-Writer logo. From the drop-down menu choose Export Document, then select the format you want.

Export to XML

Includes all of the encoding you've done in LEAF-Writer, including RDF, if you've been tagging entities via Markup and Linking.

Export to Markdown

Includes basic formatting and retains links to the entity authorities you've tagged (e.g. Wikidata, VIAF, or a custom authority), making this a useful starting point for building a static site or publishing your edition on the web. Other semantic tags are not retained.

Export to HTML

Includes basic formatting and retains links to the entity authorities you've tagged, in the same way as the Markdown export — useful for building a website directly from your encoded text. Other semantic tags are not retained.

Export to CSV

Unlike the other export formats, CSV export doesn't produce a version of your document — it extracts the entity data you've tagged in LEAF-Writer into a spreadsheet for further analysis, such as mapping or social network analysis.

For each tagged entity, the exported spreadsheet includes:

• Entity type — e.g. person, place, organization, or title of work
• Authority URL — the linked authority record (e.g. Wikidata, VIAF, or a custom authority)
• Tagged text — the text span associated with the entity

This is one of the more unusual features of LEAF-TE: it turns the entity tagging already done as part of encoding into structured data that can be brought into other tools, without requiring editors to write their own extraction script.

LEAF-Writer currently only supports downloading the exported document to your desktop. If you're working in a GitHub repository, you'll need to manually add the exported file to your repository.

If you want to experiment with or adapt any of the XSLTs that LEAF-Writer uses, you'll find the files on the LEAF-VRE GitHub repository.