How to load PDFs

Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems.

This guide covers how to load PDF documents into the LangChain Document format that we use downstream.

LangChain integrates with a host of PDF parsers. Some are simple and relatively low-level; others will support OCR and image-processing, or perform advanced document layout analysis. The right choice will depend on your application. Below we enumerate the possibilities.

Using PyPDF

Here we load a PDF using pypdf into array of documents, where each document contains the page content and metadata with page number.

%pip install --upgrade --quiet pypdf

from langchain_community.document_loaders import PyPDFLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PyPDFLoader(file_path)
pages = loader.load_and_split()

pages[0]

API Reference:PyPDFLoader

Document(page_content='LayoutParser : A Uniﬁed Toolkit for Deep\nLearning Based Document Image Analysis\nZejiang Shen1( \x00), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\nLee4, Jacob Carlson3, and Weining Li5\n1Allen Institute for AI\nshannons@allenai.org\n2Brown University\nruochen zhang@brown.edu\n3Harvard University\n{melissadell,jacob carlson }@fas.harvard.edu\n4University of Washington\nbcgl@cs.washington.edu\n5University of Waterloo\nw422li@uwaterloo.ca\nAbstract. Recent advances in document image analysis (DIA) have been\nprimarily driven by the application of neural networks. Ideally, research\noutcomes could be easily deployed in production and extended for further\ninvestigation. However, various factors like loosely organized codebases\nand sophisticated model conﬁgurations complicate the easy reuse of im-\nportant innovations by a wide audience. Though there have been on-going\neﬀorts to improve reusability and simplify deep learning (DL) model\ndevelopment in disciplines like natural language processing and computer\nvision, none of them are optimized for challenges in the domain of DIA.\nThis represents a major gap in the existing toolkit, as DIA is central to\nacademic research across a wide range of disciplines in the social sciences\nand humanities. This paper introduces LayoutParser , an open-source\nlibrary for streamlining the usage of DL in DIA research and applica-\ntions. The core LayoutParser library comes with a set of simple and\nintuitive interfaces for applying and customizing DL models for layout de-\ntection, character recognition, and many other document processing tasks.\nTo promote extensibility, LayoutParser also incorporates a community\nplatform for sharing both pre-trained models and full document digiti-\nzation pipelines. We demonstrate that LayoutParser is helpful for both\nlightweight and large-scale digitization pipelines in real-word use cases.\nThe library is publicly available at https://layout-parser.github.io .\nKeywords: Document Image Analysis ·Deep Learning ·Layout Analysis\n·Character Recognition ·Open Source library ·Toolkit.\n1 Introduction\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\ndocument image analysis (DIA) tasks including document image classiﬁcation [ 11,arXiv:2103.15348v2  [cs.CV]  21 Jun 2021', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'page': 0})

An advantage of this approach is that documents can be retrieved with page numbers.

Vector search over PDFs

Once we have loaded PDFs into LangChain Document objects, we can index them (e.g., a RAG application) in the usual way:

%pip install --upgrade --quiet faiss-cpu 
# use `pip install faiss-gpu` for CUDA GPU support

import getpass
import os

os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

faiss_index = FAISS.from_documents(pages, OpenAIEmbeddings())
docs = faiss_index.similarity_search("What is LayoutParser?", k=2)
for doc in docs:
    print(str(doc.metadata["page"]) + ":", doc.page_content[:300])

API Reference:FAISS | OpenAIEmbeddings

13: 14 Z. Shen et al.
6 Conclusion
LayoutParser provides a comprehensive toolkit for deep learning-based document
image analysis. The oﬀ-the-shelf library is easy to install, and can be used to
build ﬂexible and accurate pipelines for processing documents with complicated
structures. It also supports hi
0: LayoutParser : A Uniﬁed Toolkit for Deep
Learning Based Document Image Analysis
Zejiang Shen1( ), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain
Lee4, Jacob Carlson3, and Weining Li5
1Allen Institute for AI
shannons@allenai.org
2Brown University
ruochen zhang@brown.edu
3Harvard University

Extract text from images

Some PDFs contain images of text -- e.g., within scanned documents, or figures. Using the rapidocr-onnxruntime package we can extract images as text as well:

%pip install --upgrade --quiet rapidocr-onnxruntime

loader = PyPDFLoader("https://arxiv.org/pdf/2103.15348.pdf", extract_images=True)
pages = loader.load()
pages[4].page_content

'LayoutParser : A Uniﬁed Toolkit for DL-Based DIA 5\nTable 1: Current layout detection models in the LayoutParser model zoo\nDataset Base Model1Large Model Notes\nPubLayNet [38] F / M M Layouts of modern scientiﬁc documents\nPRImA [3] M - Layouts of scanned modern magazines and scientiﬁc reports\nNewspaper [17] F - Layouts of scanned US newspapers from the 20th century\nTableBank [18] F F Table region on modern scientiﬁc and business document\nHJDataset [31] F / M - Layouts of history Japanese documents\n1For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\nvs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\nbackbones [ 13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [ 28] (F) and Mask\nR-CNN [ 12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\nusing the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\nzoo in coming months.\nlayout data structures , which are optimized for eﬃciency and versatility. 3) When\nnecessary, users can employ existing or customized OCR models via the uniﬁed\nAPI provided in the OCR module . 4)LayoutParser comes with a set of utility\nfunctions for the visualization and storage of the layout data. 5) LayoutParser\nis also highly customizable, via its integration with functions for layout data\nannotation and model training . We now provide detailed descriptions for each\ncomponent.\n3.1 Layout Detection Models\nInLayoutParser , a layout model takes a document image as an input and\ngenerates a list of rectangular boxes for the target content regions. Diﬀerent\nfrom traditional methods, it relies on deep convolutional neural networks rather\nthan manually curated rules to identify content regions. It is formulated as an\nobject detection problem and state-of-the-art models like Faster R-CNN [ 28] and\nMask R-CNN [ 12] are used. This yields prediction results of high accuracy and\nmakes it possible to build a concise, generalized interface for layout detection.\nLayoutParser , built upon Detectron2 [ 35], provides a minimal API that can\nperform layout detection with only four lines of code in Python:\n1import layoutparser as lp\n2image = cv2. imread (" image_file ") # load images\n3model = lp. Detectron2LayoutModel (\n4 "lp :// PubLayNet / faster_rcnn_R_50_FPN_3x / config ")\n5layout = model . detect ( image )\nLayoutParser provides a wealth of pre-trained model weights using various\ndatasets covering diﬀerent languages, time periods, and document types. Due to\ndomain shift [ 7], the prediction performance can notably drop when models are ap-\nplied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\ndocument structures and layouts vary greatly in diﬀerent domains, it is important\nto select models trained on a dataset similar to the test samples. A semantic syntax\nis used for initializing the model weights in LayoutParser , using both the dataset\nname and model name lp://<dataset-name>/<model-architecture-name> .'

Using PyMuPDF

PyMuPDF is optimized for speed, and contains detailed metadata about the PDF and its pages. It returns one document per page:

%pip install --upgrade --quiet pymupdf

from langchain_community.document_loaders import PyMuPDFLoader

loader = PyMuPDFLoader(
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
data = loader.load()
data[0]

API Reference:PyMuPDFLoader

Document(page_content='LayoutParser: A Uniﬁed Toolkit for Deep\nLearning Based Document Image Analysis\nZejiang Shen1 (\x00), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\nLee4, Jacob Carlson3, and Weining Li5\n1 Allen Institute for AI\nshannons@allenai.org\n2 Brown University\nruochen zhang@brown.edu\n3 Harvard University\n{melissadell,jacob carlson}@fas.harvard.edu\n4 University of Washington\nbcgl@cs.washington.edu\n5 University of Waterloo\nw422li@uwaterloo.ca\nAbstract. Recent advances in document image analysis (DIA) have been\nprimarily driven by the application of neural networks. Ideally, research\noutcomes could be easily deployed in production and extended for further\ninvestigation. However, various factors like loosely organized codebases\nand sophisticated model conﬁgurations complicate the easy reuse of im-\nportant innovations by a wide audience. Though there have been on-going\neﬀorts to improve reusability and simplify deep learning (DL) model\ndevelopment in disciplines like natural language processing and computer\nvision, none of them are optimized for challenges in the domain of DIA.\nThis represents a major gap in the existing toolkit, as DIA is central to\nacademic research across a wide range of disciplines in the social sciences\nand humanities. This paper introduces LayoutParser, an open-source\nlibrary for streamlining the usage of DL in DIA research and applica-\ntions. The core LayoutParser library comes with a set of simple and\nintuitive interfaces for applying and customizing DL models for layout de-\ntection, character recognition, and many other document processing tasks.\nTo promote extensibility, LayoutParser also incorporates a community\nplatform for sharing both pre-trained models and full document digiti-\nzation pipelines. We demonstrate that LayoutParser is helpful for both\nlightweight and large-scale digitization pipelines in real-word use cases.\nThe library is publicly available at https://layout-parser.github.io.\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\n· Character Recognition · Open Source library · Toolkit.\n1\nIntroduction\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\ndocument image analysis (DIA) tasks including document image classiﬁcation [11,\narXiv:2103.15348v2  [cs.CV]  21 Jun 2021\n', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'file_path': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'page': 0, 'total_pages': 16, 'format': 'PDF 1.5', 'title': '', 'author': '', 'subject': '', 'keywords': '', 'creator': 'LaTeX with hyperref', 'producer': 'pdfTeX-1.40.21', 'creationDate': 'D:20210622012710Z', 'modDate': 'D:20210622012710Z', 'trapped': ''})

Additionally, you can pass along any of the options from the PyMuPDF documentation as keyword arguments in the load call, and it will be pass along to the get_text() call.

Using MathPix

Inspired by Daniel Gross's snippet here: https://gist.github.com/danielgross/3ab4104e14faccc12b49200843adab21

from langchain_community.document_loaders import MathpixPDFLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = MathpixPDFLoader(file_path)
data = loader.load()
data[0]

API Reference:MathpixPDFLoader

Using Unstructured

Unstructured supports a common interface for working with unstructured or semi-structured file formats, such as Markdown or PDF. LangChain's UnstructuredPDFLoader integrates with Unstructured to parse PDF documents into LangChain Document objects.

Please see this page for more information on installing system requirements.

%pip install --upgrade --quiet unstructured

from langchain_community.document_loaders import UnstructuredPDFLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = UnstructuredPDFLoader(file_path)
data = loader.load()
data[0]

API Reference:UnstructuredPDFLoader

Document(page_content='1 2 0 2\n\nn u J\n\n1 2\n\n]\n\nV C . s c [\n\n2 v 8 4 3 5 1 . 3 0 1 2 : v i X r a\n\nLayoutParser: A Uniﬁed Toolkit for Deep Learning Based Document Image Analysis\n\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain Lee4, Jacob Carlson3, and Weining Li5\n\n1 Allen Institute for AI shannons@allenai.org 2 Brown University ruochen zhang@brown.edu 3 Harvard University {melissadell,jacob carlson}@fas.harvard.edu 4 University of Washington bcgl@cs.washington.edu 5 University of Waterloo w422li@uwaterloo.ca\n\nAbstract. Recent advances in document image analysis (DIA) have been primarily driven by the application of neural networks. Ideally, research outcomes could be easily deployed in production and extended for further investigation. However, various factors like loosely organized codebases and sophisticated model conﬁgurations complicate the easy reuse of im- portant innovations by a wide audience. Though there have been on-going eﬀorts to improve reusability and simplify deep learning (DL) model development in disciplines like natural language processing and computer vision, none of them are optimized for challenges in the domain of DIA. This represents a major gap in the existing toolkit, as DIA is central to academic research across a wide range of disciplines in the social sciences and humanities. This paper introduces LayoutParser, an open-source library for streamlining the usage of DL in DIA research and applica- tions. The core LayoutParser library comes with a set of simple and intuitive interfaces for applying and customizing DL models for layout de- tection, character recognition, and many other document processing tasks. To promote extensibility, LayoutParser also incorporates a community platform for sharing both pre-trained models and full document digiti- zation pipelines. We demonstrate that LayoutParser is helpful for both lightweight and large-scale digitization pipelines in real-word use cases. The library is publicly available at https://layout-parser.github.io.\n\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis · Character Recognition · Open Source library · Toolkit.\n\n1\n\nIntroduction\n\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of document image analysis (DIA) tasks including document image classiﬁcation [11,\n\n2\n\nZ. Shen et al.\n\n37], layout detection [38, 22], table detection [26], and scene text detection [4]. A generalized learning-based framework dramatically reduces the need for the manual speciﬁcation of complicated rules, which is the status quo with traditional methods. DL has the potential to transform DIA pipelines and beneﬁt a broad spectrum of large-scale document digitization projects.\n\nHowever, there are several practical diﬃculties for taking advantages of re- cent advances in DL-based methods: 1) DL models are notoriously convoluted for reuse and extension. Existing models are developed using distinct frame- works like TensorFlow [1] or PyTorch [24], and the high-level parameters can be obfuscated by implementation details [8]. It can be a time-consuming and frustrating experience to debug, reproduce, and adapt existing models for DIA, and many researchers who would beneﬁt the most from using these methods lack the technical background to implement them from scratch. 2) Document images contain diverse and disparate patterns across domains, and customized training is often required to achieve a desirable detection accuracy. Currently there is no full-ﬂedged infrastructure for easily curating the target document image datasets and ﬁne-tuning or re-training the models. 3) DIA usually requires a sequence of models and other processing to obtain the ﬁnal outputs. Often research teams use DL models and then perform further document analyses in separate processes, and these pipelines are not documented in any central location (and often not documented at all). This makes it diﬃcult for research teams to learn about how full pipelines are implemented and leads them to invest signiﬁcant resources in reinventing the DIA wheel.\n\nLayoutParser provides a uniﬁed toolkit to support DL-based document image analysis and processing. To address the aforementioned challenges, LayoutParser is built with the following components:\n\n1. An oﬀ-the-shelf toolkit for applying DL models for layout detection, character recognition, and other DIA tasks (Section 3)\n\n2. A rich repository of pre-trained neural network models (Model Zoo) that underlies the oﬀ-the-shelf usage\n\n3. Comprehensive tools for eﬃcient document image data annotation and model tuning to support diﬀerent levels of customization\n\n4. A DL model hub and community platform for the easy sharing, distribu- tion, and discussion of DIA models and pipelines, to promote reusability, reproducibility, and extensibility (Section 4)\n\nThe library implements simple and intuitive Python APIs without sacriﬁcing generalizability and versatility, and can be easily installed via pip. Its convenient functions for handling document image data can be seamlessly integrated with existing DIA pipelines. With detailed documentations and carefully curated tutorials, we hope this tool will beneﬁt a variety of end-users, and will lead to advances in applications in both industry and academic research.\n\nLayoutParser is well aligned with recent eﬀorts for improving DL model reusability in other disciplines like natural language processing [8, 34] and com- puter vision [35], but with a focus on unique challenges in DIA. We show LayoutParser can be applied in sophisticated and large-scale digitization projects\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nthat require precision, eﬃciency, and robustness, as well as simple and light- weight document processing tasks focusing on eﬃcacy and ﬂexibility (Section 5). LayoutParser is being actively maintained, and support for more deep learning models and novel methods in text-based layout analysis methods [37, 34] is planned.\n\nThe rest of the paper is organized as follows. Section 2 provides an overview of related work. The core LayoutParser library, DL Model Zoo, and customized model training are described in Section 3, and the DL model hub and commu- nity platform are detailed in Section 4. Section 5 shows two examples of how LayoutParser can be used in practical DIA projects, and Section 6 concludes.\n\n2 Related Work\n\nRecently, various DL models and datasets have been developed for layout analysis tasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen- tation tasks on historical documents. Object detection-based methods like Faster R-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38] and detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also been used in table detection [27]. However, these models are usually implemented individually and there is no uniﬁed framework to load and use such models.\n\nThere has been a surge of interest in creating open-source tools for document image processing: a search of document image analysis in Github leads to 5M relevant code pieces 6; yet most of them rely on traditional rule-based methods or provide limited functionalities. The closest prior research to our work is the OCR-D project7, which also tries to build a complete toolkit for DIA. However, similar to the platform developed by Neudecker et al. [21], it is designed for analyzing historical documents, and provides no supports for recent DL models. The DocumentLayoutAnalysis project8 focuses on processing born-digital PDF documents via analyzing the stored PDF data. Repositories like DeepLayout9 and Detectron2-PubLayNet10 are individual deep learning models trained on layout analysis datasets without support for the full DIA pipeline. The Document Analysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2] aim to improve the reproducibility of DIA methods (or DL models), yet they are not actively maintained. OCR engines like Tesseract [14], easyOCR11 and paddleOCR12 usually do not come with comprehensive functionalities for other DIA tasks like layout analysis.\n\nRecent years have also seen numerous eﬀorts to create libraries for promoting reproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\n\n6 The number shown is obtained by specifying the search type as ‘code’. 7 https://ocr-d.de/en/about 8 https://github.com/BobLd/DocumentLayoutAnalysis 9 https://github.com/leonlulu/DeepLayout 10 https://github.com/hpanwar08/detectron2 11 https://github.com/JaidedAI/EasyOCR 12 https://github.com/PaddlePaddle/PaddleOCR\n\n3\n\n4\n\nZ. Shen et al.\n\nLayout Data Structure\n\nDocument Images\n\nDIA Pipeline Sharing\n\nCustomized Model Training\n\nEfficient Data Annotation\n\nDIA Model Hub\n\nModel Customization\n\nStorage & Visualization\n\nCommunity Platform\n\nLayout Detection Models\n\nThe Core LayoutParser Library\n\nOCR Module\n\nFig. 1: The overall architecture of LayoutParser. For an input document image, the core LayoutParser library provides a set of oﬀ-the-shelf tools for layout detection, OCR, visualization, and storage, backed by a carefully designed layout data structure. LayoutParser also supports high level customization via eﬃcient layout annotation and model training functions. These improve model accuracy on the target samples. The community platform enables the easy sharing of DIA models and whole digitization pipelines to promote reusability and reproducibility. A collection of detailed documentation, tutorials and exemplar projects make LayoutParser easy to learn and use.\n\nAllenNLP [8] and transformers [34] have provided the community with complete DL-based support for developing and deploying models for general computer vision and natural language processing problems. LayoutParser, on the other hand, specializes speciﬁcally in DIA tasks. LayoutParser is also equipped with a community platform inspired by established model hubs such as Torch Hub [23] and TensorFlow Hub [1]. It enables the sharing of pretrained models as well as full document processing pipelines that are unique to DIA tasks.\n\nThere have been a variety of document data collections to facilitate the development of DL models. Some examples include PRImA [3](magazine layouts), PubLayNet [38](academic paper layouts), Table Bank [18](tables in academic papers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and HJDataset [31](historical Japanese document layouts). A spectrum of models trained on these datasets are currently available in the LayoutParser model zoo to support diﬀerent use cases.\n\n3 The Core LayoutParser Library\n\nAt the core of LayoutParser is an oﬀ-the-shelf toolkit that streamlines DL- based document image analysis. Five components support a simple interface with comprehensive functionalities: 1) The layout detection models enable using pre-trained or self-trained DL models for layout detection with just four lines of code. 2) The detected layout information is stored in carefully engineered\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nTable 1: Current layout detection models in the LayoutParser model zoo\n\nDataset\n\nBase Model1 Large Model Notes\n\nPubLayNet [38] PRImA [3] Newspaper [17] TableBank [18] HJDataset [31]\n\nF / M M F F F / M\n\nM - - F -\n\nLayouts of modern scientiﬁc documents Layouts of scanned modern magazines and scientiﬁc reports Layouts of scanned US newspapers from the 20th century Table region on modern scientiﬁc and business document Layouts of history Japanese documents\n\n1 For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy vs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101 backbones [13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [28] (F) and Mask R-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained using the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model zoo in coming months.\n\nlayout data structures, which are optimized for eﬃciency and versatility. 3) When necessary, users can employ existing or customized OCR models via the uniﬁed API provided in the OCR module. 4) LayoutParser comes with a set of utility functions for the visualization and storage of the layout data. 5) LayoutParser is also highly customizable, via its integration with functions for layout data annotation and model training. We now provide detailed descriptions for each component.\n\n3.1 Layout Detection Models\n\nIn LayoutParser, a layout model takes a document image as an input and generates a list of rectangular boxes for the target content regions. Diﬀerent from traditional methods, it relies on deep convolutional neural networks rather than manually curated rules to identify content regions. It is formulated as an object detection problem and state-of-the-art models like Faster R-CNN [28] and Mask R-CNN [12] are used. This yields prediction results of high accuracy and makes it possible to build a concise, generalized interface for layout detection. LayoutParser, built upon Detectron2 [35], provides a minimal API that can perform layout detection with only four lines of code in Python:\n\n1 import layoutparser as lp 2 image = cv2 . imread ( " image_file " ) # load images 3 model = lp . De t e c tro n2 Lay outM odel (\n\n" lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config " )\n\n4 5 layout = model . detect ( image )\n\nLayoutParser provides a wealth of pre-trained model weights using various datasets covering diﬀerent languages, time periods, and document types. Due to domain shift [7], the prediction performance can notably drop when models are ap- plied to target samples that are signiﬁcantly diﬀerent from the training dataset. As document structures and layouts vary greatly in diﬀerent domains, it is important to select models trained on a dataset similar to the test samples. A semantic syntax is used for initializing the model weights in LayoutParser, using both the dataset name and model name lp://<dataset-name>/<model-architecture-name>.\n\n5\n\n6\n\nZ. Shen et al.\n\nFig. 2: The relationship between the three types of layout data structures. Coordinate supports three kinds of variation; TextBlock consists of the co- ordinate information and extra features like block text, types, and reading orders; a Layout object is a list of all possible layout elements, including other Layout objects. They all support the same set of transformation and operation APIs for maximum ﬂexibility.\n\nShown in Table 1, LayoutParser currently hosts 9 pre-trained models trained on 5 diﬀerent datasets. Description of the training dataset is provided alongside with the trained models such that users can quickly identify the most suitable models for their tasks. Additionally, when such a model is not readily available, LayoutParser also supports training customized layout models and community sharing of the models (detailed in Section 3.5).\n\n3.2 Layout Data Structures\n\nA critical feature of LayoutParser is the implementation of a series of data structures and operations that can be used to eﬃciently process and manipulate the layout elements. In document image analysis pipelines, various post-processing on the layout analysis model outputs is usually required to obtain the ﬁnal outputs. Traditionally, this requires exporting DL model outputs and then loading the results into other pipelines. All model outputs from LayoutParser will be stored in carefully engineered data types optimized for further processing, which makes it possible to build an end-to-end document digitization pipeline within LayoutParser. There are three key components in the data structure, namely the Coordinate system, the TextBlock, and the Layout. They provide diﬀerent levels of abstraction for the layout data, and a set of APIs are supported for transformations or operations on these classes.\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nCoordinates are the cornerstones for storing layout information. Currently, three types of Coordinate data structures are provided in LayoutParser, shown in Figure 2. Interval and Rectangle are the most common data types and support specifying 1D or 2D regions within a document. They are parameterized with 2 and 4 parameters. A Quadrilateral class is also implemented to support a more generalized representation of rectangular regions when the document is skewed or distorted, where the 4 corner points can be speciﬁed and a total of 8 degrees of freedom are supported. A wide collection of transformations like shift, pad, and scale, and operations like intersect, union, and is_in, are supported for these classes. Notably, it is common to separate a segment of the image and analyze it individually. LayoutParser provides full support for this scenario via image cropping operations crop_image and coordinate transformations like relative_to and condition_on that transform coordinates to and from their relative representations. We refer readers to Table 2 for a more detailed description of these operations13.\n\nBased on Coordinates, we implement the TextBlock class that stores both the positional and extra features of individual layout elements. It also supports specifying the reading orders via setting the parent ﬁeld to the index of the parent object. A Layout class is built that takes in a list of TextBlocks and supports processing the elements in batch. Layout can also be nested to support hierarchical layout structures. They support the same operations and transformations as the Coordinate classes, minimizing both learning and deployment eﬀort.\n\n3.3 OCR\n\nLayoutParser provides a uniﬁed interface for existing OCR tools. Though there are many OCR tools available, they are usually conﬁgured diﬀerently with distinct APIs or protocols for using them. It can be ineﬃcient to add new OCR tools into an existing pipeline, and diﬃcult to make direct comparisons among the available tools to ﬁnd the best option for a particular project. To this end, LayoutParser builds a series of wrappers among existing OCR engines, and provides nearly the same syntax for using them. It supports a plug-and-play style of using OCR engines, making it eﬀortless to switch, evaluate, and compare diﬀerent OCR modules:\n\n1 ocr_agent = lp . TesseractAgent () 2 # Can be easily switched to other OCR software 3 tokens = ocr_agent . detect ( image )\n\nThe OCR outputs will also be stored in the aforementioned layout data structures and can be seamlessly incorporated into the digitization pipeline. Currently LayoutParser supports the Tesseract and Google Cloud Vision OCR engines.\n\nLayoutParser also comes with a DL-based CNN-RNN OCR model [6] trained with the Connectionist Temporal Classiﬁcation (CTC) loss [10]. It can be used like the other OCR modules, and can be easily trained on customized datasets.\n\n13 This is also available in the LayoutParser documentation pages.\n\n7\n\n8\n\nZ. Shen et al.\n\nTable 2: All operations supported by the layout elements. The same APIs are supported across diﬀerent layout element classes including Coordinate types, TextBlock and Layout.\n\nOperation Name\n\nDescription\n\nblock.pad(top, bottom, right, left) Enlarge the current block according to the input\n\nblock.scale(fx, fy)\n\nScale the current block given the ratio in x and y direction\n\nblock.shift(dx, dy)\n\nMove the current block with the shift distances in x and y direction\n\nblock1.is in(block2)\n\nWhether block1 is inside of block2\n\nblock1.intersect(block2)\n\nReturn the intersection region of block1 and block2. Coordinate type to be determined based on the inputs.\n\nblock1.union(block2)\n\nReturn the union region of block1 and block2. Coordinate type to be determined based on the inputs.\n\nblock1.relative to(block2)\n\nConvert the absolute coordinates of block1 to relative coordinates to block2\n\nblock1.condition on(block2)\n\nCalculate the absolute coordinates of block1 given the canvas block2’s absolute coordinates\n\nblock.crop image(image)\n\nObtain the image segments in the block region\n\n3.4 Storage and visualization\n\nThe end goal of DIA is to transform the image-based document data into a structured database. LayoutParser supports exporting layout data into diﬀerent formats like JSON, csv, and will add the support for the METS/ALTO XML format 14 . It can also load datasets from layout analysis-speciﬁc formats like COCO [38] and the Page Format [25] for training layout models (Section 3.5). Visualization of the layout detection results is critical for both presentation and debugging. LayoutParser is built with an integrated API for displaying the layout information along with the original document image. Shown in Figure 3, it enables presenting layout data with rich meta information and features in diﬀerent modes. More detailed information can be found in the online LayoutParser documentation page.\n\n3.5 Customized Model Training\n\nBesides the oﬀ-the-shelf library, LayoutParser is also highly customizable with supports for highly unique and challenging document analysis tasks. Target document images can be vastly diﬀerent from the existing datasets for train- ing layout models, which leads to low layout detection accuracy. Training data\n\n14 https://altoxml.github.io\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nFig. 3: Layout detection and OCR results visualization generated by the LayoutParser APIs. Mode I directly overlays the layout region bounding boxes and categories over the original image. Mode II recreates the original document via drawing the OCR’d texts at their corresponding positions on the image canvas. In this ﬁgure, tokens in textual regions are ﬁltered using the API and then displayed.\n\ncan also be highly sensitive and not sharable publicly. To overcome these chal- lenges, LayoutParser is built with rich features for eﬃcient data annotation and customized model training.\n\nLayoutParser incorporates a toolkit optimized for annotating document lay- outs using object-level active learning [32]. With the help from a layout detection model trained along with labeling, only the most important layout objects within each image, rather than the whole image, are required for labeling. The rest of the regions are automatically annotated with high conﬁdence predictions from the layout detection model. This allows a layout dataset to be created more eﬃciently with only around 60% of the labeling budget.\n\nAfter the training dataset is curated, LayoutParser supports diﬀerent modes for training the layout models. Fine-tuning can be used for training models on a small newly-labeled dataset by initializing the model with existing pre-trained weights. Training from scratch can be helpful when the source dataset and target are signiﬁcantly diﬀerent and a large training set is available. However, as suggested in Studer et al.’s work[33], loading pre-trained weights on large-scale datasets like ImageNet [5], even from totally diﬀerent domains, can still boost model performance. Through the integrated API provided by LayoutParser, users can easily compare model performances on the benchmark datasets.\n\n9\n\n10\n\nZ. Shen et al.\n\nFig. 4: Illustration of (a) the original historical Japanese document with layout detection results and (b) a recreated version of the document image that achieves much better character recognition recall. The reorganization algorithm rearranges the tokens based on the their detected bounding boxes given a maximum allowed height.\n\n4 LayoutParser Community Platform\n\nAnother focus of LayoutParser is promoting the reusability of layout detection models and full digitization pipelines. Similar to many existing deep learning libraries, LayoutParser comes with a community model hub for distributing layout models. End-users can upload their self-trained models to the model hub, and these models can be loaded into a similar interface as the currently available LayoutParser pre-trained models. For example, the model trained on the News Navigator dataset [17] has been incorporated in the model hub.\n\nBeyond DL models, LayoutParser also promotes the sharing of entire doc- ument digitization pipelines. For example, sometimes the pipeline requires the combination of multiple DL models to achieve better accuracy. Currently, pipelines are mainly described in academic papers and implementations are often not pub- licly available. To this end, the LayoutParser community platform also enables the sharing of layout pipelines to promote the discussion and reuse of techniques. For each shared pipeline, it has a dedicated project page, with links to the source code, documentation, and an outline of the approaches. A discussion panel is provided for exchanging ideas. Combined with the core LayoutParser library, users can easily build reusable components based on the shared pipelines and apply them to solve their unique problems.\n\n5 Use Cases\n\nThe core objective of LayoutParser is to make it easier to create both large-scale and light-weight document digitization pipelines. Large-scale document processing\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nfocuses on precision, eﬃciency, and robustness. The target documents may have complicated structures, and may require training multiple layout detection models to achieve the optimal accuracy. Light-weight pipelines are built for relatively simple documents, with an emphasis on development ease, speed and ﬂexibility. Ideally one only needs to use existing resources, and model training should be avoided. Through two exemplar projects, we show how practitioners in both academia and industry can easily build such pipelines using LayoutParser and extract high-quality structured document data for their downstream tasks. The source code for these projects will be publicly available in the LayoutParser community hub.\n\n5.1 A Comprehensive Historical Document Digitization Pipeline\n\nThe digitization of historical documents can unlock valuable data that can shed light on many important social, economic, and historical questions. Yet due to scan noises, page wearing, and the prevalence of complicated layout structures, ob- taining a structured representation of historical document scans is often extremely complicated. In this example, LayoutParser was used to develop a comprehensive pipeline, shown in Figure 5, to gener- ate high-quality structured data from historical Japanese ﬁrm ﬁnancial ta- bles with complicated layouts. The pipeline applies two layout models to identify diﬀerent levels of document structures and two customized OCR engines for optimized character recog- nition accuracy.\n\nAs shown in Figure 4 (a), the document contains columns of text written vertically 15, a common style in Japanese. Due to scanning noise and archaic printing technology, the columns can be skewed or have vari- able widths, and hence cannot be eas- ily identiﬁed via rule-based methods. Within each column, words are sepa- rated by white spaces of variable size, and the vertical positions of objects can be an indicator of their layout type.\n\nFig. 5: Illustration of how LayoutParser helps with the historical document digi- tization pipeline.\n\n15 A document page consists of eight rows like this. For simplicity we skip the row\n\nsegmentation discussion and refer readers to the source code when available.\n\n11\n\n12\n\nZ. Shen et al.\n\nTo decipher the complicated layout\n\nstructure, two object detection models have been trained to recognize individual columns and tokens, respectively. A small training set (400 images with approxi- mately 100 annotations each) is curated via the active learning based annotation tool [32] in LayoutParser. The models learn to identify both the categories and regions for each token or column via their distinct visual features. The layout data structure enables easy grouping of the tokens within each column, and rearranging columns to achieve the correct reading orders based on the horizontal position. Errors are identiﬁed and rectiﬁed via checking the consistency of the model predictions. Therefore, though trained on a small dataset, the pipeline achieves a high level of layout detection accuracy: it achieves a 96.97 AP [19] score across 5 categories for the column detection model, and a 89.23 AP across 4 categories for the token detection model.\n\nA combination of character recognition methods is developed to tackle the unique challenges in this document. In our experiments, we found that irregular spacing between the tokens led to a low character recognition recall rate, whereas existing OCR models tend to perform better on densely-arranged texts. To overcome this challenge, we create a document reorganization algorithm that rearranges the text based on the token bounding boxes detected in the layout analysis step. Figure 4 (b) illustrates the generated image of dense text, which is sent to the OCR APIs as a whole to reduce the transaction costs. The ﬂexible coordinate system in LayoutParser is used to transform the OCR results relative to their original positions on the page.\n\nAdditionally, it is common for historical documents to use unique fonts with diﬀerent glyphs, which signiﬁcantly degrades the accuracy of OCR models trained on modern texts. In this document, a special ﬂat font is used for printing numbers and could not be detected by oﬀ-the-shelf OCR engines. Using the highly ﬂexible functionalities from LayoutParser, a pipeline approach is constructed that achieves a high recognition accuracy with minimal eﬀort. As the characters have unique visual structures and are usually clustered together, we train the layout model to identify number regions with a dedicated category. Subsequently, LayoutParser crops images within these regions, and identiﬁes characters within them using a self-trained OCR model based on a CNN-RNN [6]. The model detects a total of 15 possible categories, and achieves a 0.98 Jaccard score16 and a 0.17 average Levinstein distances17 for token prediction on the test set.\n\nOverall, it is possible to create an intricate and highly accurate digitization pipeline for large-scale digitization using LayoutParser. The pipeline avoids specifying the complicated rules used in traditional methods, is straightforward to develop, and is robust to outliers. The DL models also generate ﬁne-grained results that enable creative approaches like page reorganization for OCR.\n\n16 This measures the overlap between the detected and ground-truth characters, and\n\nthe maximum is 1.\n\n17 This measures the number of edits from the ground-truth text to the predicted text,\n\nand lower is better.\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\nFig. 6: This lightweight table detector can identify tables (outlined in red) and cells (shaded in blue) in diﬀerent locations on a page. In very few cases (d), it might generate minor error predictions, e.g, failing to capture the top text line of a table.\n\n5.2 A light-weight Visual Table Extractor\n\nDetecting tables and parsing their structures (table extraction) are of central im- portance for many document digitization tasks. Many previous works [26, 30, 27] and tools 18 have been developed to identify and parse table structures. Yet they might require training complicated models from scratch, or are only applicable for born-digital PDF documents. In this section, we show how LayoutParser can help build a light-weight accurate visual table extractor for legal docket tables using the existing resources with minimal eﬀort.\n\nThe extractor uses a pre-trained layout detection model for identifying the table regions and some simple rules for pairing the rows and the columns in the PDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the LayoutParser Model Zoo can be used for detecting table regions. By ﬁltering out model predictions of low conﬁdence and removing overlapping predictions, LayoutParser can identify the tabular regions on each page, which signiﬁcantly simpliﬁes the subsequent steps. By applying the line detection functions within the tabular segments, provided in the utility module from LayoutParser, the pipeline can identify the three distinct columns in the tables. A row clustering method is then applied via analyzing the y coordinates of token bounding boxes in the left-most column, which are obtained from the OCR engines. A non-maximal suppression algorithm is used to remove duplicated rows with extremely small gaps. Shown in Figure 6, the built pipeline can detect tables at diﬀerent positions on a page accurately. Continued tables from diﬀerent pages are concatenated, and a structured table representation has been easily created.\n\n18 https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\n\n13\n\n14\n\nZ. Shen et al.\n\n6 Conclusion\n\nLayoutParser provides a comprehensive toolkit for deep learning-based document image analysis. The oﬀ-the-shelf library is easy to install, and can be used to build ﬂexible and accurate pipelines for processing documents with complicated structures. It also supports high-level customization and enables easy labeling and training of DL models on unique document image datasets. The LayoutParser community platform facilitates sharing DL models and DIA pipelines, inviting discussion and promoting code reproducibility and reusability. The LayoutParser team is committed to keeping the library updated continuously and bringing the most recent advances in DL-based DIA, such as multi-modal document modeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\n\nAcknowledgements We thank the anonymous reviewers for their comments and suggestions. This project is supported in part by NSF Grant OIA-2033558 and funding from the Harvard Data Science Initiative and Harvard Catalyst. Zejiang Shen thanks Doug Downey for suggestions.\n\nReferences\n\n[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015), https://www.tensorflow.org/, software available from tensorﬂow.org\n\n[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a highly-functional python framework for reproducible experiments. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 423–428. IEEE (2018)\n\n[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic dataset for performance evaluation of document layout analysis. In: 2009 10th International Conference on Document Analysis and Recognition. pp. 296–300. IEEE (2009)\n\n[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9365–9374 (2019)\n\n[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\n\nHierarchical Image Database. In: CVPR09 (2009)\n\n[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with coarse-to-ﬁne attention. In: International Conference on Machine Learning. pp. 980–989. PMLR (2017)\n\n[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. pp. 1180–1189. PMLR (2015)\n\nLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters, M., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640 (2018) (cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P., Grali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in- formation extraction (2020)\n\n[9]\n\n[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classiﬁcation: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd international conference on Machine learning. pp. 369–376 (2006)\n\n[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for document image classiﬁcation and retrieval. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015) [12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\n\nIEEE international conference on computer vision. pp. 2961–2969 (2017)\n\n[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)\n\n[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\n\n2007(159), 2 (Jul 2007)\n\n[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis benchmarking. In: 2011 International Conference on Document Analysis and Recognition. pp. 42–47. IEEE (2011)\n\n[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5 million images. In: Adjunct Publication of the 33rd Annual ACM Sym- posium on User Interface Software and Technology. p. 120–122. UIST ’20 Adjunct, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3379350.3416143, https://doi-org.offcampus. lib.washington.edu/10.1145/3379350.3416143\n\n[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N., Thomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting Headlines and Visual Content from 16 Million Historic Newspaper Pages in Chronicling America, p. 3055–3062. Association for Computing Machinery, New York, NY, USA (2020), https://doi.org/10.1145/3340531.3412767\n\n[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark for image-based table detection and recognition. arXiv preprint arXiv:1903.01949 (2019)\n\n[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)\n\n[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3431–3440 (2015)\n\n[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Suﬁ, S., Williams, A., Wolsten- croft, K.: An experimental workﬂow development platform for historical document digitisation and analysis. In: Proceedings of the 2011 workshop on historical document imaging and processing. pp. 161–168 (2011)\n\n[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach for document segmentation. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\n\n15\n\n16\n\nZ. Shen et al.\n\n[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic diﬀerentiation in pytorch (2017) [24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019) [25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth elements) format framework. In: 2010 20th International Conference on Pattern Recognition. pp. 257–260. IEEE (2010)\n\n[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet: An approach for end to end table detection and structure recognition from image- based documents. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 572–573 (2020)\n\n[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph neural networks. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 142–147. IEEE (2019)\n\n[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. pp. 91–99 (2015)\n\n[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE transactions on neural networks 20(1), 61–80 (2008) [30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning for detection and structure recognition of tables in document images. In: 2017 14th IAPR international conference on document analysis and recognition (ICDAR). vol. 1, pp. 1162–1167. IEEE (2017)\n\n[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents with complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. pp. 548–549 (2020)\n\n[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\n\nbased layout annotation. arXiv preprint arXiv:2010.01762 (2020)\n\n[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer, A., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for historical document image analysis. In: 2019 International Conference on Document Analysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\n\n[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of- the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019) [35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https://\n\ngithub.com/facebookresearch/detectron2 (2019)\n\n[36] Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C., Che, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740 (2020)\n\n[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\n\ntext and layout for document image understanding (2019)\n\n[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\n\nlargest dataset ever for doc- In: 2019 International Conference on Document IEEE (Sep 2019).\n\nument Analysis and Recognition (ICDAR). pp. 1015–1022. https://doi.org/10.1109/ICDAR.2019.00166\n\nlayout analysis.', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'})

Retain Elements

Under the hood, Unstructured creates different "elements" for different chunks of text. By default we combine those together, but you can easily keep that separation by specifying mode="elements".

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = UnstructuredPDFLoader(file_path, mode="elements")

data = loader.load()
data[0]

Document(page_content='1 2 0 2', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf', 'coordinates': {'points': ((16.34, 213.36), (16.34, 253.36), (36.34, 253.36), (36.34, 213.36)), 'system': 'PixelSpace', 'layout_width': 612, 'layout_height': 792}, 'file_directory': '../../docs/integrations/document_loaders/example_data', 'filename': 'layout-parser-paper.pdf', 'languages': ['eng'], 'last_modified': '2023-12-19T13:42:18', 'page_number': 1, 'filetype': 'application/pdf', 'category': 'UncategorizedText'})

See the full set of element types for this particular document:

set(doc.metadata["category"] for doc in data)

{'ListItem', 'NarrativeText', 'Title', 'UncategorizedText'}

Fetching remote PDFs using Unstructured

This covers how to load online PDFs into a document format that we can use downstream. This can be used for various online PDF sites such as https://open.umn.edu/opentextbooks/textbooks/ and https://arxiv.org/archive/

Note: all other PDF loaders can also be used to fetch remote PDFs, but OnlinePDFLoader is a legacy function, and works specifically with UnstructuredPDFLoader.

from langchain_community.document_loaders import OnlinePDFLoader

loader = OnlinePDFLoader("https://arxiv.org/pdf/2302.03803.pdf")
data = loader.load()
data[0]

API Reference:OnlinePDFLoader

Document(page_content='3 2 0 2\n\nb e F 7\n\n]\n\nG A . h t a m\n\n[\n\n1 v 3 0 8 3 0 . 2 0 3 2 : v i X r a\n\nA WEAK (k, k)-LEFSCHETZ THEOREM FOR PROJECTIVE TORIC ORBIFOLDS\n\nWilliam D. Montoya\n\nInstituto de Matem´atica, Estat´ıstica e Computa¸c˜ao Cient´ıﬁca, Universidade Estadual de Campinas (UNICAMP),\n\nRua S´ergio Buarque de Holanda 651, 13083-859, Campinas, SP, Brazil\n\nFebruary 9, 2023\n\nAbstract\n\nFirstly we show a generalization of the (1, 1)-Lefschetz theorem for projective toric orbifolds and secondly we prove that on 2k-dimensional quasi-smooth hyper- surfaces coming from quasi-smooth intersection surfaces, under the Cayley trick, every rational (k, k)-cohomology class is algebraic, i.e., the Hodge conjecture holds on them.\n\n1\n\nIntroduction\n\nIn [3] we proved that, under suitable conditions, on a very general codimension s quasi- smooth intersection subvariety X in a projective toric orbifold Pd Σ with d + s = 2(k + 1) the Hodge conjecture holds, that is, every (p, p)-cohomology class, under the Poincar´e duality is a rational linear combination of fundamental classes of algebraic subvarieties of X. The proof of the above-mentioned result relies, for p ≠ d + 1 − s, on a Lefschetz\n\nDate: February 9, 2023 2020 Mathematics Subject Classiﬁcation: 14C30, 14M10, 14J70, 14M25 Keywords: (1,1)- Lefschetz theorem, Hodge conjecture, toric varieties, complete intersection Email: wmontoya@ime.unicamp.br\n\n1\n\ntheorem ([7]) and the Hard Lefschetz theorem for projective orbifolds ([11]). When p = d + 1 − s the proof relies on the Cayley trick, a trick which associates to X a quasi-smooth hypersurface Y in a projective vector bundle, and the Cayley Proposition (4.3) which gives an isomorphism of some primitive cohomologies (4.2) of X and Y . The Cayley trick, following the philosophy of Mavlyutov in [7], reduces results known for quasi-smooth hypersurfaces to quasi-smooth intersection subvarieties. The idea in this paper goes the other way around, we translate some results for quasi-smooth intersection subvarieties to quasi-smooth hypersurfaces, mainly the (1, 1)-Lefschetz theorem.\n\nAcknowledgement. I thank Prof. Ugo Bruzzo and Tiago Fonseca for useful discus-\n\nsions. I also acknowledge support from FAPESP postdoctoral grant No. 2019/23499-7.\n\n2 Preliminaries and Notation\n\n2.1 Toric varieties\n\nLet M be a free abelian group of rank d, let N = Hom(M, Z), and NR = N ⊗Z R.\n\nA convex subset σ ⊂ NR is a rational k-dimensional simplicial cone if there exist k linearly independent primitive elements e1, . . . , ek ∈ N such that σ = {µ1e1 + ⋯ + µkek}.\n\nDeﬁnition 2.1.\n\nThe generators ei are integral if for every i and any nonnegative rational number µ the product µei is in N only if µ is an integer.\n\nGiven two rational simplicial cones σ, σ′ one says that σ′ is a face of σ (σ′ < σ) if the set of integral generators of σ′ is a subset of the set of integral generators of σ.\n\nA ﬁnite set Σ = {σ1, . . . , σt} of rational simplicial cones is called a rational simplicial complete d-dimensional fan if:\n\n1. all faces of cones in Σ are in Σ;\n\n2. if σ, σ′ ∈ Σ then σ ∩ σ′ < σ and σ ∩ σ′ < σ′;\n\n3. NR = σ1 ∪ ⋅ ⋅ ⋅ ∪ σt.\n\nA rational simplicial complete d-dimensional fan Σ deﬁnes a d-dimensional toric variety Σ having only orbifold singularities which we assume to be projective. Moreover, T ∶= Pd N ⊗Z C∗ ≃ (C∗)d is the torus action on Pd Σ. We denote by Σ(i) the i-dimensional cones\n\n2\n\nof Σ and each ρ ∈ Σ corresponds to an irreducible T -invariant Weil divisor Dρ on Pd Cl(Σ) be the group of Weil divisors on Pd\n\nΣ module rational equivalences.\n\nΣ. Let\n\nThe total coordinate ring of Pd\n\nΣ is the polynomial ring S = C[xρ ∣ ρ ∈ Σ(1)], S has the ρ ∈\n\nCl(Σ)-grading, a Weil divisor D = ∑ρ∈Σ(1) uρDρ determines the monomial xu ∶= ∏ρ∈Σ(1) xuρ S and conversely deg(xu) = [D] ∈ Cl(Σ).\n\nFor a cone σ ∈ Σ, ˆσ is the set of 1-dimensional cone in Σ that are not contained in σ\n\nand xˆσ ∶= ∏ρ∈ˆσ xρ is the associated monomial in S.\n\nΣ is the monomial ideal BΣ ∶=< xˆσ ∣ σ ∈ Σ > and\n\nDeﬁnition 2.2. The irrelevant ideal of Pd the zero locus Z(Σ) ∶= V(BΣ) in the aﬃne space Ad ∶= Spec(S) is the irrelevant locus.\n\nProposition 2.3 (Theorem 5.1.11 [5]). The toric variety Pd Σ is a categorical quotient Ad ∖ Z(Σ) by the group Hom(Cl(Σ), C∗) and the group action is induced by the Cl(Σ)- grading of S.\n\n2.2 Orbifolds\n\nNow we give a brief introduction to complex orbifolds and we mention the needed theorems for the next section. Namely: de Rham theorem and Dolbeault theorem for complex orbifolds.\n\nDeﬁnition 2.4. A complex orbifold of complex dimension d is a singular complex space whose singularities are locally isomorphic to quotient singularities Cd/G, for ﬁnite sub- groups G ⊂ Gl(d, C).\n\nDeﬁnition 2.5. A diﬀerential form on a complex orbifold Z is deﬁned locally at z ∈ Z as a G-invariant diﬀerential form on Cd where G ⊂ Gl(d, C) and Z is locally isomorphic to Cd/G around z.\n\nRoughly speaking the local geometry of orbifolds reduces to local G-invariant geometry. We have a complex of diﬀerential forms (A●(Z), d) and a double complex (A●,●(Z), ∂, ¯∂) of bigraded diﬀerential forms which deﬁne the de Rham and the Dolbeault cohomology groups (for a ﬁxed p ∈ N) respectively:\n\ndR(Z, C) ∶=\n\nH ●\n\nker d im d\n\nand H p,●(Z, ¯∂) ∶=\n\nker ¯∂ im ¯∂\n\nTheorem 2.6 (Theorem 3.4.4 in [4] and Theorem 1.2 in [1] ). Let Z be a compact complex orbifold. There are natural isomorphisms:\n\n3\n\nH ●\n\ndR(Z, C) ≃ H ●(Z, C)\n\nH p,●(Z, ¯∂) ≃ H ●(X, Ωp Z )\n\n3\n\n(1,1)-Lefschetz theorem for projective toric orbifolds\n\nDeﬁnition 3.1. A subvariety X ⊂ Pd Z(Σ).\n\nΣ is quasi-smooth if V(IX ) ⊂ A#Σ(1) is smooth outside\n\nExample 3.2. Quasi-smooth hypersurfaces or more generally quasi-smooth intersection sub- varieties are quasi-smooth subvarieties (see [2] or [7] for more details).\n\nRemark 3.3. Quasi-smooth subvarieties are suborbifolds of Pd Σ in the sense of Satake in [8]. Intuitively speaking they are subvarieties whose only singularities come from the ambient space.\n\nTheorem 3.4. Let X ⊂ Pd class λ ∈ H 1,1(X) ∩ H 2(X, Z) is algebraic\n\nΣ be a quasi-smooth subvariety. Then every (1, 1)-cohomology\n\nProof. From the exponential short exact sequence\n\n0 → Z → OX → O∗ X\n\n→ 0\n\nwe have a long exact sequence in cohomology\n\nX ) → H 2(X, Z) → H 2(OX ) ≃ H 0,2(X)\n\nH 1(O∗\n\nwhere the last isomorphisms is due to Steenbrink in [9]. Now, it is enough to prove the commutativity of the next diagram\n\nH 2(X, Z)\n\nH 2(X, OX )\n\nH 2(X, C)\n\n≃ Dolbeault\n\nde Rham ≃\n\n(cid:15)\n\n(cid:15)\n\nH 2\n\ndR(X, C)\n\n/\n\n/ H 0,2\n\n¯∂ (X)\n\n4\n\n△\n\n△\n\nThe key points are the de Rham and Dolbeault’s isomorphisms for orbifolds. The rest\n\nof the proof follows as the (1, 1)-Lefschetz theorem in [6].\n\nRemark 3.5. For k = 1 and Pd Lefschetz theorem.\n\nΣ as the projective space, we recover the classical (1, 1)-\n\nBy the Hard Lefschetz Theorem for projective orbifolds (see [11] for details) we get an\n\nisomorphism of cohomologies :\n\nH ●(X, Q) ≃ H 2 dim X−●(X, Q)\n\ngiven by the Lefschetz morphism and since it is a morphism of Hodge structures, we have:\n\nH 1,1(X, Q) ≃ H dim X−1,dim X−1(X, Q)\n\nFor X as before:\n\nCorollary 3.6. If the dimension of X is 1, 2 or 3. The Hodge conjecture holds on X.\n\nProof. If the dimCX = 1 the result is clear by the Hard Lefschetz theorem for projective orbifolds. The dimension 2 and 3 cases are covered by Theorem 3.5 and the Hard Lefschetz. theorem.\n\n4 Cayley trick and Cayley proposition\n\nThe Cayley trick is a way to associate to a quasi-smooth intersection subvariety a quasi- smooth hypersurface. Let L1, . . . , Ls be line bundles on Pd Σ be the projective space bundle associated to the vector bundle E = L1 ⊕ ⋯ ⊕ Ls. It is known that P(E) is a (d + s − 1)-dimensional simplicial toric variety whose fan depends on the degrees of the line bundles and the fan Σ. Furthermore, if the Cox ring, without considering the grading, of Pd\n\nΣ and let π ∶ P(E) → Pd\n\nΣ is C[x1, . . . , xm] then the Cox ring of P(E) is\n\nC[x1, . . . , xm, y1, . . . , ys]\n\nMoreover for X a quasi-smooth intersection subvariety cut oﬀ by f1, . . . , fs with deg(fi) = [Li] we relate the hypersurface Y cut oﬀ by F = y1f1 + ⋅ ⋅ ⋅ + ysfs which turns out to be quasi-smooth. For more details see Section 2 in [7].\n\n5\n\n△\n\nWe will denote P(E) as Pd+s−1\n\nΣ,X to keep track of its relation with X and Pd Σ.\n\nThe following is a key remark.\n\nRemark 4.1. There is a morphism ι ∶ X → Y ⊂ Pd+s−1 with y ≠ 0 has a preimage. Hence for any subvariety W = V(IW ) ⊂ X ⊂ Pd W ′ ⊂ Y ⊂ Pd+s−1 Σ,X such that π(W ′) = W , i.e., W ′ = {z = (x, y) ∣ x ∈ W }.\n\nΣ,X . Moreover every point z ∶= (x, y) ∈ Y Σ there exists\n\n△\n\nFor X ⊂ Pd\n\nΣ a quasi-smooth intersection variety the morphism in cohomology induced\n\nby the inclusion i∗ ∶ H d−s(Pd\n\nΣ, C) → H d−s(X, C) is injective by Proposition 1.4 in [7].\n\nDeﬁnition 4.2. The primitive cohomology of H d−s and H d−s prim(X, Q) with rational coeﬃcients.\n\nprim(X) is the quotient H d−s(X, C)/i∗(H d−s(Pd\n\nH d−s(Pd\n\nΣ, C) and H d−s(X, C) have pure Hodge structures, and the morphism i∗ is com-\n\npatible with them, so that H d−s\n\nprim(X) gets a pure Hodge structure.\n\nThe next Proposition is the Cayley proposition.\n\nProposition 4.3. [Proposition 2.3 in [3] ] Let X = X1 ∩⋅ ⋅ ⋅∩Xs be a quasi-smooth intersec- , d+s−3 tion subvariety in Pd 2\n\nΣ cut oﬀ by homogeneous polynomials f1 . . . fs. Then for p ≠ d+s−1\n\n2\n\nH p−1,d+s−1−p\n\nprim\n\n(Y ) ≃ H p−s,d−p\n\nprim (X).\n\nCorollary 4.4. If d + s = 2(k + 1),\n\nH k+1−s,k+1−s\n\nprim\n\n(X) ≃ H k,k\n\nprim(Y )\n\nRemark 4.5. The above isomorphisms are also true with rational coeﬃcients since H ●(X, C) = H ●(X, Q) ⊗Q C. See the beginning of Section 7.1 in [10] for more details.\n\n△\n\n5 Main result\n\nTheorem 5.1. Let Y = {F = y1f1 + ⋯ + ykfk = 0} ⊂ P2k+1 associated to the quasi-smooth intersection surface X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfk ⊂ Pk+2 the Hodge conjecture holds.\n\nΣ,X be the quasi-smooth hypersurface Σ . Then on Y\n\nProof. If H k,k proposition H k,k\n\nprim(X, Q) = 0 we are done. So let us assume H k,k\n\nprim(X, Q) ≠ 0. By the Cayley prim(X, Q) and by the (1, 1)-Lefschetz theorem for projective\n\nprim(Y, Q) ≃ H 1,1\n\n6\n\nΣ, C))\n\ntoric orbifolds there is a non-zero algebraic basis λC1, . . . , λCn with rational coeﬃcients of H 1,1 prim(X, Q) algebraic curves C1, . . . , Cn in X such that under the Poincar´e duality the class in homology [Ci] goes to λCi, [Ci] ↦ λCi. Recall that the Cox ring of Pk+2 is contained in the Cox ring of P2k+1 Σ,X without considering the Σ ) then (α, 0) ∈ Cl(P2k+1 grading. Considering the grading we have that if α ∈ Cl(Pk+2 Σ,X ). So the polynomials deﬁning Ci ⊂ Pk+2 X,Σ but with diﬀerent degree. Moreover, by Remark 4.1 each Ci is contained in Y = {F = y1f1 + ⋯ + ykfk = 0} and furthermore it has codimension k.\n\nprim(X, Q), that is, there are n ∶= h1,1\n\ncan be interpreted in P2k+1\n\nΣ\n\ni=1 is a basis of H k,k It is enough to prove that λCi is diﬀerent from zero in H k,k prim(Y, Q) or equivalently that the cohomology classes {λCi}n i=1 do not come from the ambient space. By contradiction, let us assume that there exists a j and C ⊂ P2k+1 Σ,X , Q) with i∗(λC) = λCj or in terms of homology there exists a (k + 2)-dimensional algebraic subvariety V ⊂ P2k+1 Σ,X such that V ∩ Y = Cj so they are equal as a homology class of P2k+1 Σ,X ,i.e., [V ∩ Y ] = [Cj] . Σ where π ∶ (x, y) ↦ x. Hence It is easy to check that π(V ) ∩ X = Cj as a subvariety of Pk+2 [π(V ) ∩ X] = [Cj] which is equivalent to say that λCj comes from Pk+2 Σ which contradicts the choice of [Cj].\n\nClaim: {λCi}n\n\nprim(Y, Q).\n\nΣ,X such that λC ∈ H k,k(P2k+1\n\nRemark 5.2. Into the proof of the previous theorem, the key fact was that on X the Hodge conjecture holds and we translate it to Y by contradiction. So, using an analogous argument we have:\n\nProposition 5.3. Let Y = {F = y1fs+⋯+ysfs = 0} ⊂ P2k+1 associated to a quasi-smooth intersection subvariety X = Xf1 ∩ ⋅ ⋅ ⋅ ∩ Xfs ⊂ Pd d + s = 2(k + 1). If the Hodge conjecture holds on X then it holds as well on Y .\n\nΣ,X be the quasi-smooth hypersurface Σ such that\n\nCorollary 5.4. If the dimension of Y is 2s − 1, 2s or 2s + 1 then the Hodge conjecture holds on Y .\n\nProof. By Proposition 5.3 and Corollary 3.6.\n\n7\n\n△\n\nReferences\n\n[1] Angella, D. Cohomologies of certain orbifolds. Journal of Geometry and Physics\n\n71 (2013), 117–126.\n\n[2] Batyrev, V. V., and Cox, D. A. On the Hodge structure of projective hypersur-\n\nfaces in toric varieties. Duke Mathematical Journal 75, 2 (Aug 1994).\n\n[3] Bruzzo, U., and Montoya, W. On the Hodge conjecture for quasi-smooth in- tersections in toric varieties. S˜ao Paulo J. Math. Sci. Special Section: Geometry in Algebra and Algebra in Geometry (2021).\n\n[4] Caramello Jr, F. C. Introduction to orbifolds. arXiv:1909.08699v6 (2019).\n\n[5] Cox, D., Little, J., and Schenck, H. Toric varieties, vol. 124. American Math-\n\nematical Soc., 2011.\n\n[6] Griffiths, P., and Harris, J. Principles of Algebraic Geometry. John Wiley &\n\nSons, Ltd, 1978.\n\n[7] Mavlyutov, A. R. Cohomology of complete intersections in toric varieties. Pub-\n\nlished in Paciﬁc J. of Math. 191 No. 1 (1999), 133–144.\n\n[8] Satake, I. On a Generalization of the Notion of Manifold. Proceedings of the National Academy of Sciences of the United States of America 42, 6 (1956), 359–363.\n\n[9] Steenbrink, J. H. M. Intersection form for quasi-homogeneous singularities. Com-\n\npositio Mathematica 34, 2 (1977), 211–223.\n\n[10] Voisin, C. Hodge Theory and Complex Algebraic Geometry I, vol. 1 of Cambridge\n\nStudies in Advanced Mathematics. Cambridge University Press, 2002.\n\n[11] Wang, Z. Z., and Zaffran, D. A remark on the Hard Lefschetz theorem for K¨ahler orbifolds. Proceedings of the American Mathematical Society 137, 08 (Aug 2009).\n\n8', metadata={'source': '/var/folders/z4/1qk27d6n7w59z2h3r31hwxgr0000gn/T/tmpgjpunfou/tmp.pdf'})

Using PyPDFium2

from langchain_community.document_loaders import PyPDFium2Loader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PyPDFium2Loader(file_path)
data = loader.load()
data[0]

API Reference:PyPDFium2Loader

Using PDFMiner

from langchain_community.document_loaders import PDFMinerLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PDFMinerLoader(file_path)
data = loader.load()
data[0]

API Reference:PDFMinerLoader

Document(page_content='1\n2\n0\n2\n\nn\nu\nJ\n\n1\n2\n\n]\n\nV\nC\n.\ns\nc\n[\n\n2\nv\n8\n4\n3\n5\n1\n.\n3\n0\n1\n2\n:\nv\ni\nX\nr\na\n\nLayoutParser: A Uniﬁed Toolkit for Deep\nLearning Based Document Image Analysis\n\nZejiang Shen1 ((cid:0)), Ruochen Zhang2, Melissa Dell3, Benjamin Charles Germain\nLee4, Jacob Carlson3, and Weining Li5\n\n1 Allen Institute for AI\nshannons@allenai.org\n2 Brown University\nruochen zhang@brown.edu\n3 Harvard University\n{melissadell,jacob carlson}@fas.harvard.edu\n4 University of Washington\nbcgl@cs.washington.edu\n5 University of Waterloo\nw422li@uwaterloo.ca\n\nAbstract. Recent advances in document image analysis (DIA) have been\nprimarily driven by the application of neural networks. Ideally, research\noutcomes could be easily deployed in production and extended for further\ninvestigation. However, various factors like loosely organized codebases\nand sophisticated model conﬁgurations complicate the easy reuse of im-\nportant innovations by a wide audience. Though there have been on-going\neﬀorts to improve reusability and simplify deep learning (DL) model\ndevelopment in disciplines like natural language processing and computer\nvision, none of them are optimized for challenges in the domain of DIA.\nThis represents a major gap in the existing toolkit, as DIA is central to\nacademic research across a wide range of disciplines in the social sciences\nand humanities. This paper introduces LayoutParser, an open-source\nlibrary for streamlining the usage of DL in DIA research and applica-\ntions. The core LayoutParser library comes with a set of simple and\nintuitive interfaces for applying and customizing DL models for layout de-\ntection, character recognition, and many other document processing tasks.\nTo promote extensibility, LayoutParser also incorporates a community\nplatform for sharing both pre-trained models and full document digiti-\nzation pipelines. We demonstrate that LayoutParser is helpful for both\nlightweight and large-scale digitization pipelines in real-word use cases.\nThe library is publicly available at https://layout-parser.github.io.\n\nKeywords: Document Image Analysis · Deep Learning · Layout Analysis\n· Character Recognition · Open Source library · Toolkit.\n\n1\n\nIntroduction\n\nDeep Learning(DL)-based approaches are the state-of-the-art for a wide range of\ndocument image analysis (DIA) tasks including document image classiﬁcation [11,\n\n \n \n \n \n \n \n\x0c2\n\nZ. Shen et al.\n\n37], layout detection [38, 22], table detection [26], and scene text detection [4].\nA generalized learning-based framework dramatically reduces the need for the\nmanual speciﬁcation of complicated rules, which is the status quo with traditional\nmethods. DL has the potential to transform DIA pipelines and beneﬁt a broad\nspectrum of large-scale document digitization projects.\n\nHowever, there are several practical diﬃculties for taking advantages of re-\ncent advances in DL-based methods: 1) DL models are notoriously convoluted\nfor reuse and extension. Existing models are developed using distinct frame-\nworks like TensorFlow [1] or PyTorch [24], and the high-level parameters can\nbe obfuscated by implementation details [8]. It can be a time-consuming and\nfrustrating experience to debug, reproduce, and adapt existing models for DIA,\nand many researchers who would beneﬁt the most from using these methods lack\nthe technical background to implement them from scratch. 2) Document images\ncontain diverse and disparate patterns across domains, and customized training\nis often required to achieve a desirable detection accuracy. Currently there is no\nfull-ﬂedged infrastructure for easily curating the target document image datasets\nand ﬁne-tuning or re-training the models. 3) DIA usually requires a sequence of\nmodels and other processing to obtain the ﬁnal outputs. Often research teams use\nDL models and then perform further document analyses in separate processes,\nand these pipelines are not documented in any central location (and often not\ndocumented at all). This makes it diﬃcult for research teams to learn about how\nfull pipelines are implemented and leads them to invest signiﬁcant resources in\nreinventing the DIA wheel.\n\nLayoutParser provides a uniﬁed toolkit to support DL-based document image\nanalysis and processing. To address the aforementioned challenges, LayoutParser\nis built with the following components:\n\n1. An oﬀ-the-shelf toolkit for applying DL models for layout detection, character\n\nrecognition, and other DIA tasks (Section 3)\n\n2. A rich repository of pre-trained neural network models (Model Zoo) that\n\nunderlies the oﬀ-the-shelf usage\n\n3. Comprehensive tools for eﬃcient document image data annotation and model\n\ntuning to support diﬀerent levels of customization\n\n4. A DL model hub and community platform for the easy sharing, distribu-\ntion, and discussion of DIA models and pipelines, to promote reusability,\nreproducibility, and extensibility (Section 4)\n\nThe library implements simple and intuitive Python APIs without sacriﬁcing\ngeneralizability and versatility, and can be easily installed via pip. Its convenient\nfunctions for handling document image data can be seamlessly integrated with\nexisting DIA pipelines. With detailed documentations and carefully curated\ntutorials, we hope this tool will beneﬁt a variety of end-users, and will lead to\nadvances in applications in both industry and academic research.\n\nLayoutParser is well aligned with recent eﬀorts for improving DL model\nreusability in other disciplines like natural language processing [8, 34] and com-\nputer vision [35], but with a focus on unique challenges in DIA. We show\nLayoutParser can be applied in sophisticated and large-scale digitization projects\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n3\n\nthat require precision, eﬃciency, and robustness, as well as simple and light-\nweight document processing tasks focusing on eﬃcacy and ﬂexibility (Section 5).\nLayoutParser is being actively maintained, and support for more deep learning\nmodels and novel methods in text-based layout analysis methods [37, 34] is\nplanned.\n\nThe rest of the paper is organized as follows. Section 2 provides an overview\nof related work. The core LayoutParser library, DL Model Zoo, and customized\nmodel training are described in Section 3, and the DL model hub and commu-\nnity platform are detailed in Section 4. Section 5 shows two examples of how\nLayoutParser can be used in practical DIA projects, and Section 6 concludes.\n\n2 Related Work\n\nRecently, various DL models and datasets have been developed for layout analysis\ntasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen-\ntation tasks on historical documents. Object detection-based methods like Faster\nR-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38]\nand detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also\nbeen used in table detection [27]. However, these models are usually implemented\nindividually and there is no uniﬁed framework to load and use such models.\n\nThere has been a surge of interest in creating open-source tools for document\nimage processing: a search of document image analysis in Github leads to 5M\nrelevant code pieces 6; yet most of them rely on traditional rule-based methods\nor provide limited functionalities. The closest prior research to our work is the\nOCR-D project7, which also tries to build a complete toolkit for DIA. However,\nsimilar to the platform developed by Neudecker et al. [21], it is designed for\nanalyzing historical documents, and provides no supports for recent DL models.\nThe DocumentLayoutAnalysis project8 focuses on processing born-digital PDF\ndocuments via analyzing the stored PDF data. Repositories like DeepLayout9\nand Detectron2-PubLayNet10 are individual deep learning models trained on\nlayout analysis datasets without support for the full DIA pipeline. The Document\nAnalysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2]\naim to improve the reproducibility of DIA methods (or DL models), yet they\nare not actively maintained. OCR engines like Tesseract [14], easyOCR11 and\npaddleOCR12 usually do not come with comprehensive functionalities for other\nDIA tasks like layout analysis.\n\nRecent years have also seen numerous eﬀorts to create libraries for promoting\nreproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\n\n6 The number shown is obtained by specifying the search type as ‘code’.\n7 https://ocr-d.de/en/about\n8 https://github.com/BobLd/DocumentLayoutAnalysis\n9 https://github.com/leonlulu/DeepLayout\n10 https://github.com/hpanwar08/detectron2\n11 https://github.com/JaidedAI/EasyOCR\n12 https://github.com/PaddlePaddle/PaddleOCR\n\n\x0c4\n\nZ. Shen et al.\n\nFig. 1: The overall architecture of LayoutParser. For an input document image,\nthe core LayoutParser library provides a set of oﬀ-the-shelf tools for layout\ndetection, OCR, visualization, and storage, backed by a carefully designed layout\ndata structure. LayoutParser also supports high level customization via eﬃcient\nlayout annotation and model training functions. These improve model accuracy\non the target samples. The community platform enables the easy sharing of DIA\nmodels and whole digitization pipelines to promote reusability and reproducibility.\nA collection of detailed documentation, tutorials and exemplar projects make\nLayoutParser easy to learn and use.\n\nAllenNLP [8] and transformers [34] have provided the community with complete\nDL-based support for developing and deploying models for general computer\nvision and natural language processing problems. LayoutParser, on the other\nhand, specializes speciﬁcally in DIA tasks. LayoutParser is also equipped with a\ncommunity platform inspired by established model hubs such as Torch Hub [23]\nand TensorFlow Hub [1]. It enables the sharing of pretrained models as well as\nfull document processing pipelines that are unique to DIA tasks.\n\nThere have been a variety of document data collections to facilitate the\ndevelopment of DL models. Some examples include PRImA [3](magazine layouts),\nPubLayNet [38](academic paper layouts), Table Bank [18](tables in academic\npapers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and\nHJDataset [31](historical Japanese document layouts). A spectrum of models\ntrained on these datasets are currently available in the LayoutParser model zoo\nto support diﬀerent use cases.\n\n3 The Core LayoutParser Library\n\nAt the core of LayoutParser is an oﬀ-the-shelf toolkit that streamlines DL-\nbased document image analysis. Five components support a simple interface\nwith comprehensive functionalities: 1) The layout detection models enable using\npre-trained or self-trained DL models for layout detection with just four lines\nof code. 2) The detected layout information is stored in carefully engineered\n\nEfficient Data AnnotationCustomized Model TrainingModel CustomizationDIA Model HubDIA Pipeline SharingCommunity PlatformLayout Detection ModelsDocument Images The Core LayoutParser LibraryOCR ModuleStorage & VisualizationLayout Data Structure\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n5\n\nTable 1: Current layout detection models in the LayoutParser model zoo\n\nDataset\n\nBase Model1 Large Model Notes\n\nPubLayNet [38]\nPRImA [3]\nNewspaper [17]\nTableBank [18]\nHJDataset [31]\n\nF / M\nM\nF\nF\nF / M\n\nM\n-\n-\nF\n-\n\nLayouts of modern scientiﬁc documents\nLayouts of scanned modern magazines and scientiﬁc reports\nLayouts of scanned US newspapers from the 20th century\nTable region on modern scientiﬁc and business document\nLayouts of history Japanese documents\n\n1 For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\nvs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\nbackbones [13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [28] (F) and Mask\nR-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\nusing the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\nzoo in coming months.\n\nlayout data structures, which are optimized for eﬃciency and versatility. 3) When\nnecessary, users can employ existing or customized OCR models via the uniﬁed\nAPI provided in the OCR module. 4) LayoutParser comes with a set of utility\nfunctions for the visualization and storage of the layout data. 5) LayoutParser\nis also highly customizable, via its integration with functions for layout data\nannotation and model training. We now provide detailed descriptions for each\ncomponent.\n\n3.1 Layout Detection Models\n\nIn LayoutParser, a layout model takes a document image as an input and\ngenerates a list of rectangular boxes for the target content regions. Diﬀerent\nfrom traditional methods, it relies on deep convolutional neural networks rather\nthan manually curated rules to identify content regions. It is formulated as an\nobject detection problem and state-of-the-art models like Faster R-CNN [28] and\nMask R-CNN [12] are used. This yields prediction results of high accuracy and\nmakes it possible to build a concise, generalized interface for layout detection.\nLayoutParser, built upon Detectron2 [35], provides a minimal API that can\nperform layout detection with only four lines of code in Python:\n\n1 import layoutparser as lp\n2 image = cv2 . imread ( " image_file " ) # load images\n3 model = lp . De t e c tro n2 Lay outM odel (\n\n" lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config " )\n\n4\n5 layout = model . detect ( image )\n\nLayoutParser provides a wealth of pre-trained model weights using various\ndatasets covering diﬀerent languages, time periods, and document types. Due to\ndomain shift [7], the prediction performance can notably drop when models are ap-\nplied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\ndocument structures and layouts vary greatly in diﬀerent domains, it is important\nto select models trained on a dataset similar to the test samples. A semantic syntax\nis used for initializing the model weights in LayoutParser, using both the dataset\nname and model name lp://<dataset-name>/<model-architecture-name>.\n\n\x0c6\n\nZ. Shen et al.\n\nFig. 2: The relationship between the three types of layout data structures.\nCoordinate supports three kinds of variation; TextBlock consists of the co-\nordinate information and extra features like block text, types, and reading orders;\na Layout object is a list of all possible layout elements, including other Layout\nobjects. They all support the same set of transformation and operation APIs for\nmaximum ﬂexibility.\n\nShown in Table 1, LayoutParser currently hosts 9 pre-trained models trained\non 5 diﬀerent datasets. Description of the training dataset is provided alongside\nwith the trained models such that users can quickly identify the most suitable\nmodels for their tasks. Additionally, when such a model is not readily available,\nLayoutParser also supports training customized layout models and community\nsharing of the models (detailed in Section 3.5).\n\n3.2 Layout Data Structures\n\nA critical feature of LayoutParser is the implementation of a series of data\nstructures and operations that can be used to eﬃciently process and manipulate\nthe layout elements. In document image analysis pipelines, various post-processing\non the layout analysis model outputs is usually required to obtain the ﬁnal\noutputs. Traditionally, this requires exporting DL model outputs and then loading\nthe results into other pipelines. All model outputs from LayoutParser will be\nstored in carefully engineered data types optimized for further processing, which\nmakes it possible to build an end-to-end document digitization pipeline within\nLayoutParser. There are three key components in the data structure, namely\nthe Coordinate system, the TextBlock, and the Layout. They provide diﬀerent\nlevels of abstraction for the layout data, and a set of APIs are supported for\ntransformations or operations on these classes.\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n7\n\nCoordinates are the cornerstones for storing layout information. Currently,\nthree types of Coordinate data structures are provided in LayoutParser, shown\nin Figure 2. Interval and Rectangle are the most common data types and\nsupport specifying 1D or 2D regions within a document. They are parameterized\nwith 2 and 4 parameters. A Quadrilateral class is also implemented to support\na more generalized representation of rectangular regions when the document\nis skewed or distorted, where the 4 corner points can be speciﬁed and a total\nof 8 degrees of freedom are supported. A wide collection of transformations\nlike shift, pad, and scale, and operations like intersect, union, and is_in,\nare supported for these classes. Notably, it is common to separate a segment\nof the image and analyze it individually. LayoutParser provides full support\nfor this scenario via image cropping operations crop_image and coordinate\ntransformations like relative_to and condition_on that transform coordinates\nto and from their relative representations. We refer readers to Table 2 for a more\ndetailed description of these operations13.\n\nBased on Coordinates, we implement the TextBlock class that stores both\nthe positional and extra features of individual layout elements. It also supports\nspecifying the reading orders via setting the parent ﬁeld to the index of the parent\nobject. A Layout class is built that takes in a list of TextBlocks and supports\nprocessing the elements in batch. Layout can also be nested to support hierarchical\nlayout structures. They support the same operations and transformations as the\nCoordinate classes, minimizing both learning and deployment eﬀort.\n\n3.3 OCR\n\nLayoutParser provides a uniﬁed interface for existing OCR tools. Though there\nare many OCR tools available, they are usually conﬁgured diﬀerently with distinct\nAPIs or protocols for using them. It can be ineﬃcient to add new OCR tools into\nan existing pipeline, and diﬃcult to make direct comparisons among the available\ntools to ﬁnd the best option for a particular project. To this end, LayoutParser\nbuilds a series of wrappers among existing OCR engines, and provides nearly\nthe same syntax for using them. It supports a plug-and-play style of using OCR\nengines, making it eﬀortless to switch, evaluate, and compare diﬀerent OCR\nmodules:\n\n1 ocr_agent = lp . TesseractAgent ()\n2 # Can be easily switched to other OCR software\n3 tokens = ocr_agent . detect ( image )\n\nThe OCR outputs will also be stored in the aforementioned layout data\nstructures and can be seamlessly incorporated into the digitization pipeline.\nCurrently LayoutParser supports the Tesseract and Google Cloud Vision OCR\nengines.\n\nLayoutParser also comes with a DL-based CNN-RNN OCR model [6] trained\nwith the Connectionist Temporal Classiﬁcation (CTC) loss [10]. It can be used\nlike the other OCR modules, and can be easily trained on customized datasets.\n\n13 This is also available in the LayoutParser documentation pages.\n\n\x0c8\n\nZ. Shen et al.\n\nTable 2: All operations supported by the layout elements. The same APIs are\nsupported across diﬀerent layout element classes including Coordinate types,\nTextBlock and Layout.\n\nOperation Name\n\nDescription\n\nblock.pad(top, bottom, right, left) Enlarge the current block according to the input\n\nblock.scale(fx, fy)\n\nblock.shift(dx, dy)\n\nScale the current block given the ratio\nin x and y direction\n\nMove the current block with the shift\ndistances in x and y direction\n\nblock1.is in(block2)\n\nWhether block1 is inside of block2\n\nblock1.intersect(block2)\n\nblock1.union(block2)\n\nblock1.relative to(block2)\n\nblock1.condition on(block2)\n\nReturn the intersection region of block1 and block2.\nCoordinate type to be determined based on the inputs.\n\nReturn the union region of block1 and block2.\nCoordinate type to be determined based on the inputs.\n\nConvert the absolute coordinates of block1 to\nrelative coordinates to block2\n\nCalculate the absolute coordinates of block1 given\nthe canvas block2’s absolute coordinates\n\nblock.crop image(image)\n\nObtain the image segments in the block region\n\n3.4 Storage and visualization\n\nThe end goal of DIA is to transform the image-based document data into a\nstructured database. LayoutParser supports exporting layout data into diﬀerent\nformats like JSON, csv, and will add the support for the METS/ALTO XML\nformat 14 . It can also load datasets from layout analysis-speciﬁc formats like\nCOCO [38] and the Page Format [25] for training layout models (Section 3.5).\nVisualization of the layout detection results is critical for both presentation\nand debugging. LayoutParser is built with an integrated API for displaying the\nlayout information along with the original document image. Shown in Figure 3, it\nenables presenting layout data with rich meta information and features in diﬀerent\nmodes. More detailed information can be found in the online LayoutParser\ndocumentation page.\n\n3.5 Customized Model Training\n\nBesides the oﬀ-the-shelf library, LayoutParser is also highly customizable with\nsupports for highly unique and challenging document analysis tasks. Target\ndocument images can be vastly diﬀerent from the existing datasets for train-\ning layout models, which leads to low layout detection accuracy. Training data\n\n14 https://altoxml.github.io\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n9\n\nFig. 3: Layout detection and OCR results visualization generated by the\nLayoutParser APIs. Mode I directly overlays the layout region bounding boxes\nand categories over the original image. Mode II recreates the original document\nvia drawing the OCR’d texts at their corresponding positions on the image\ncanvas. In this ﬁgure, tokens in textual regions are ﬁltered using the API and\nthen displayed.\n\ncan also be highly sensitive and not sharable publicly. To overcome these chal-\nlenges, LayoutParser is built with rich features for eﬃcient data annotation and\ncustomized model training.\n\nLayoutParser incorporates a toolkit optimized for annotating document lay-\nouts using object-level active learning [32]. With the help from a layout detection\nmodel trained along with labeling, only the most important layout objects within\neach image, rather than the whole image, are required for labeling. The rest of\nthe regions are automatically annotated with high conﬁdence predictions from\nthe layout detection model. This allows a layout dataset to be created more\neﬃciently with only around 60% of the labeling budget.\n\nAfter the training dataset is curated, LayoutParser supports diﬀerent modes\nfor training the layout models. Fine-tuning can be used for training models on a\nsmall newly-labeled dataset by initializing the model with existing pre-trained\nweights. Training from scratch can be helpful when the source dataset and\ntarget are signiﬁcantly diﬀerent and a large training set is available. However, as\nsuggested in Studer et al.’s work[33], loading pre-trained weights on large-scale\ndatasets like ImageNet [5], even from totally diﬀerent domains, can still boost\nmodel performance. Through the integrated API provided by LayoutParser,\nusers can easily compare model performances on the benchmark datasets.\n\n\x0c10\n\nZ. Shen et al.\n\nFig. 4: Illustration of (a) the original historical Japanese document with layout\ndetection results and (b) a recreated version of the document image that achieves\nmuch better character recognition recall. The reorganization algorithm rearranges\nthe tokens based on the their detected bounding boxes given a maximum allowed\nheight.\n\n4 LayoutParser Community Platform\n\nAnother focus of LayoutParser is promoting the reusability of layout detection\nmodels and full digitization pipelines. Similar to many existing deep learning\nlibraries, LayoutParser comes with a community model hub for distributing\nlayout models. End-users can upload their self-trained models to the model hub,\nand these models can be loaded into a similar interface as the currently available\nLayoutParser pre-trained models. For example, the model trained on the News\nNavigator dataset [17] has been incorporated in the model hub.\n\nBeyond DL models, LayoutParser also promotes the sharing of entire doc-\nument digitization pipelines. For example, sometimes the pipeline requires the\ncombination of multiple DL models to achieve better accuracy. Currently, pipelines\nare mainly described in academic papers and implementations are often not pub-\nlicly available. To this end, the LayoutParser community platform also enables\nthe sharing of layout pipelines to promote the discussion and reuse of techniques.\nFor each shared pipeline, it has a dedicated project page, with links to the source\ncode, documentation, and an outline of the approaches. A discussion panel is\nprovided for exchanging ideas. Combined with the core LayoutParser library,\nusers can easily build reusable components based on the shared pipelines and\napply them to solve their unique problems.\n\n5 Use Cases\n\nThe core objective of LayoutParser is to make it easier to create both large-scale\nand light-weight document digitization pipelines. Large-scale document processing\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n11\n\nfocuses on precision, eﬃciency, and robustness. The target documents may have\ncomplicated structures, and may require training multiple layout detection models\nto achieve the optimal accuracy. Light-weight pipelines are built for relatively\nsimple documents, with an emphasis on development ease, speed and ﬂexibility.\nIdeally one only needs to use existing resources, and model training should be\navoided. Through two exemplar projects, we show how practitioners in both\nacademia and industry can easily build such pipelines using LayoutParser and\nextract high-quality structured document data for their downstream tasks. The\nsource code for these projects will be publicly available in the LayoutParser\ncommunity hub.\n\n5.1 A Comprehensive Historical Document Digitization Pipeline\n\nThe digitization of historical documents can unlock valuable data that can shed\nlight on many important social, economic, and historical questions. Yet due to\nscan noises, page wearing, and the prevalence of complicated layout structures, ob-\ntaining a structured representation of historical document scans is often extremely\ncomplicated.\nIn this example, LayoutParser was\nused to develop a comprehensive\npipeline, shown in Figure 5, to gener-\nate high-quality structured data from\nhistorical Japanese ﬁrm ﬁnancial ta-\nbles with complicated layouts. The\npipeline applies two layout models to\nidentify diﬀerent levels of document\nstructures and two customized OCR\nengines for optimized character recog-\nnition accuracy.\n\nAs shown in Figure 4 (a), the\ndocument contains columns of text\nwritten vertically 15, a common style\nin Japanese. Due to scanning noise\nand archaic printing technology, the\ncolumns can be skewed or have vari-\nable widths, and hence cannot be eas-\nily identiﬁed via rule-based methods.\nWithin each column, words are sepa-\nrated by white spaces of variable size,\nand the vertical positions of objects\ncan be an indicator of their layout\ntype.\n\nFig. 5: Illustration of how LayoutParser\nhelps with the historical document digi-\ntization pipeline.\n\n15 A document page consists of eight rows like this. For simplicity we skip the row\n\nsegmentation discussion and refer readers to the source code when available.\n\n\x0c12\n\nZ. Shen et al.\n\nTo decipher the complicated layout\n\nstructure, two object detection models have been trained to recognize individual\ncolumns and tokens, respectively. A small training set (400 images with approxi-\nmately 100 annotations each) is curated via the active learning based annotation\ntool [32] in LayoutParser. The models learn to identify both the categories and\nregions for each token or column via their distinct visual features. The layout\ndata structure enables easy grouping of the tokens within each column, and\nrearranging columns to achieve the correct reading orders based on the horizontal\nposition. Errors are identiﬁed and rectiﬁed via checking the consistency of the\nmodel predictions. Therefore, though trained on a small dataset, the pipeline\nachieves a high level of layout detection accuracy: it achieves a 96.97 AP [19]\nscore across 5 categories for the column detection model, and a 89.23 AP across\n4 categories for the token detection model.\n\nA combination of character recognition methods is developed to tackle the\nunique challenges in this document. In our experiments, we found that irregular\nspacing between the tokens led to a low character recognition recall rate, whereas\nexisting OCR models tend to perform better on densely-arranged texts. To\novercome this challenge, we create a document reorganization algorithm that\nrearranges the text based on the token bounding boxes detected in the layout\nanalysis step. Figure 4 (b) illustrates the generated image of dense text, which is\nsent to the OCR APIs as a whole to reduce the transaction costs. The ﬂexible\ncoordinate system in LayoutParser is used to transform the OCR results relative\nto their original positions on the page.\n\nAdditionally, it is common for historical documents to use unique fonts\nwith diﬀerent glyphs, which signiﬁcantly degrades the accuracy of OCR models\ntrained on modern texts. In this document, a special ﬂat font is used for printing\nnumbers and could not be detected by oﬀ-the-shelf OCR engines. Using the highly\nﬂexible functionalities from LayoutParser, a pipeline approach is constructed\nthat achieves a high recognition accuracy with minimal eﬀort. As the characters\nhave unique visual structures and are usually clustered together, we train the\nlayout model to identify number regions with a dedicated category. Subsequently,\nLayoutParser crops images within these regions, and identiﬁes characters within\nthem using a self-trained OCR model based on a CNN-RNN [6]. The model\ndetects a total of 15 possible categories, and achieves a 0.98 Jaccard score16 and\na 0.17 average Levinstein distances17 for token prediction on the test set.\n\nOverall, it is possible to create an intricate and highly accurate digitization\npipeline for large-scale digitization using LayoutParser. The pipeline avoids\nspecifying the complicated rules used in traditional methods, is straightforward\nto develop, and is robust to outliers. The DL models also generate ﬁne-grained\nresults that enable creative approaches like page reorganization for OCR.\n\n16 This measures the overlap between the detected and ground-truth characters, and\n\nthe maximum is 1.\n\n17 This measures the number of edits from the ground-truth text to the predicted text,\n\nand lower is better.\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n13\n\nFig. 6: This lightweight table detector can identify tables (outlined in red) and\ncells (shaded in blue) in diﬀerent locations on a page. In very few cases (d), it\nmight generate minor error predictions, e.g, failing to capture the top text line of\na table.\n\n5.2 A light-weight Visual Table Extractor\n\nDetecting tables and parsing their structures (table extraction) are of central im-\nportance for many document digitization tasks. Many previous works [26, 30, 27]\nand tools 18 have been developed to identify and parse table structures. Yet they\nmight require training complicated models from scratch, or are only applicable\nfor born-digital PDF documents. In this section, we show how LayoutParser can\nhelp build a light-weight accurate visual table extractor for legal docket tables\nusing the existing resources with minimal eﬀort.\n\nThe extractor uses a pre-trained layout detection model for identifying the\ntable regions and some simple rules for pairing the rows and the columns in the\nPDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the\nLayoutParser Model Zoo can be used for detecting table regions. By ﬁltering\nout model predictions of low conﬁdence and removing overlapping predictions,\nLayoutParser can identify the tabular regions on each page, which signiﬁcantly\nsimpliﬁes the subsequent steps. By applying the line detection functions within\nthe tabular segments, provided in the utility module from LayoutParser, the\npipeline can identify the three distinct columns in the tables. A row clustering\nmethod is then applied via analyzing the y coordinates of token bounding boxes in\nthe left-most column, which are obtained from the OCR engines. A non-maximal\nsuppression algorithm is used to remove duplicated rows with extremely small\ngaps. Shown in Figure 6, the built pipeline can detect tables at diﬀerent positions\non a page accurately. Continued tables from diﬀerent pages are concatenated,\nand a structured table representation has been easily created.\n\n18 https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\n\n\x0c14\n\nZ. Shen et al.\n\n6 Conclusion\n\nLayoutParser provides a comprehensive toolkit for deep learning-based document\nimage analysis. The oﬀ-the-shelf library is easy to install, and can be used to\nbuild ﬂexible and accurate pipelines for processing documents with complicated\nstructures. It also supports high-level customization and enables easy labeling and\ntraining of DL models on unique document image datasets. The LayoutParser\ncommunity platform facilitates sharing DL models and DIA pipelines, inviting\ndiscussion and promoting code reproducibility and reusability. The LayoutParser\nteam is committed to keeping the library updated continuously and bringing\nthe most recent advances in DL-based DIA, such as multi-modal document\nmodeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\n\nAcknowledgements We thank the anonymous reviewers for their comments\nand suggestions. This project is supported in part by NSF Grant OIA-2033558\nand funding from the Harvard Data Science Initiative and Harvard Catalyst.\nZejiang Shen thanks Doug Downey for suggestions.\n\nReferences\n\n[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado,\nG.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A.,\nIrving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg,\nJ., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J.,\nSteiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V.,\nVi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng,\nX.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015),\nhttps://www.tensorflow.org/, software available from tensorﬂow.org\n\n[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a\nhighly-functional python framework for reproducible experiments. In: 2018 16th\nInternational Conference on Frontiers in Handwriting Recognition (ICFHR). pp.\n423–428. IEEE (2018)\n\n[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic\ndataset for performance evaluation of document layout analysis. In: 2009 10th\nInternational Conference on Document Analysis and Recognition. pp. 296–300.\nIEEE (2009)\n\n[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text\ndetection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and\nPattern Recognition. pp. 9365–9374 (2019)\n\n[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\n\nHierarchical Image Database. In: CVPR09 (2009)\n\n[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with\ncoarse-to-ﬁne attention. In: International Conference on Machine Learning. pp.\n980–989. PMLR (2017)\n\n[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation.\nIn: International conference on machine learning. pp. 1180–1189. PMLR (2015)\n\n\x0cLayoutParser: A Uniﬁed Toolkit for DL-Based DIA\n\n15\n\n[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters,\nM., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language\nprocessing platform. arXiv preprint arXiv:1803.07640 (2018)\n(cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P.,\nGrali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in-\nformation extraction (2020)\n\n[9]\n\n[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal\nclassiﬁcation: labelling unsegmented sequence data with recurrent neural networks.\nIn: Proceedings of the 23rd international conference on Machine learning. pp.\n369–376 (2006)\n\n[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for\ndocument image classiﬁcation and retrieval. In: 2015 13th International Conference\non Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015)\n[12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\n\nIEEE international conference on computer vision. pp. 2961–2969 (2017)\n\n[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.\nIn: Proceedings of the IEEE conference on computer vision and pattern recognition.\npp. 770–778 (2016)\n\n[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\n\n2007(159), 2 (Jul 2007)\n\n[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis\nbenchmarking. In: 2011 International Conference on Document Analysis and\nRecognition. pp. 42–47. IEEE (2011)\n\n[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5\nmillion images. In: Adjunct Publication of the 33rd Annual ACM Sym-\nposium on User\nInterface Software and Technology. p. 120–122. UIST\n’20 Adjunct, Association for Computing Machinery, New York, NY, USA\n(2020). https://doi.org/10.1145/3379350.3416143, https://doi-org.offcampus.\nlib.washington.edu/10.1145/3379350.3416143\n\n[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N.,\nThomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting\nHeadlines and Visual Content from 16 Million Historic Newspaper Pages in\nChronicling America, p. 3055–3062. Association for Computing Machinery, New\nYork, NY, USA (2020), https://doi.org/10.1145/3340531.3412767\n\n[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark\nfor image-based table detection and recognition. arXiv preprint arXiv:1903.01949\n(2019)\n\n[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P.,\nZitnick, C.L.: Microsoft coco: Common objects in context. In: European conference\non computer vision. pp. 740–755. Springer (2014)\n\n[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic\nsegmentation. In: Proceedings of the IEEE conference on computer vision and\npattern recognition. pp. 3431–3440 (2015)\n\n[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Suﬁ, S., Williams, A., Wolsten-\ncroft, K.: An experimental workﬂow development platform for historical document\ndigitisation and analysis. In: Proceedings of the 2011 workshop on historical\ndocument imaging and processing. pp. 161–168 (2011)\n\n[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach\nfor document segmentation. In: 2018 16th International Conference on Frontiers\nin Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\n\n\x0c16\n\nZ. Shen et al.\n\n[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z.,\nDesmaison, A., Antiga, L., Lerer, A.: Automatic diﬀerentiation in pytorch (2017)\n[24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen,\nT., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style,\nhigh-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019)\n[25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth\nelements) format framework. In: 2010 20th International Conference on Pattern\nRecognition. pp. 257–260. IEEE (2010)\n\n[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet:\nAn approach for end to end table detection and structure recognition from image-\nbased documents. In: Proceedings of the IEEE/CVF Conference on Computer\nVision and Pattern Recognition Workshops. pp. 572–573 (2020)\n\n[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph\nneural networks. In: 2019 International Conference on Document Analysis and\nRecognition (ICDAR). pp. 142–147. IEEE (2019)\n\n[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object\ndetection with region proposal networks. In: Advances in neural information\nprocessing systems. pp. 91–99 (2015)\n\n[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph\nneural network model. IEEE transactions on neural networks 20(1), 61–80 (2008)\n[30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning\nfor detection and structure recognition of tables in document images. In: 2017 14th\nIAPR international conference on document analysis and recognition (ICDAR).\nvol. 1, pp. 1162–1167. IEEE (2017)\n\n[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents\nwith complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer\nVision and Pattern Recognition Workshops. pp. 548–549 (2020)\n\n[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\n\nbased layout annotation. arXiv preprint arXiv:2010.01762 (2020)\n\n[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer,\nA., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for\nhistorical document image analysis. In: 2019 International Conference on Document\nAnalysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\n\n[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P.,\nRault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-\nthe-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)\n[35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. https://\n\ngithub.com/facebookresearch/detectron2 (2019)\n\n[36] Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C.,\nChe, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document\nunderstanding. arXiv preprint arXiv:2012.14740 (2020)\n\n[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\n\ntext and layout for document image understanding (2019)\n\n[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\n\nlayout analysis.\n\nument\nAnalysis and Recognition (ICDAR). pp. 1015–1022.\nhttps://doi.org/10.1109/ICDAR.2019.00166\n\nlargest dataset ever for doc-\nIn: 2019 International Conference on Document\nIEEE (Sep 2019).\n\n\x0c', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'})

Using PDFMiner to generate HTML text

This can be helpful for chunking texts semantically into sections as the output html content can be parsed via BeautifulSoup to get more structured and rich information about font size, page numbers, PDF headers/footers, etc.

from langchain_community.document_loaders import PDFMinerPDFasHTMLLoader

file_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PDFMinerPDFasHTMLLoader(file_path)
docs = loader.load()
docs[0]

API Reference:PDFMinerPDFasHTMLLoader

Document(page_content='<html><head>\n<meta http-equiv="Content-Type" content="text/html">\n</head><body>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:50px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:50px;"><a name="1">Page 1</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:263px; width:20px; height:40px;"><span style="font-family: Times-Roman; font-size:10px">1\n<br>2\n<br>0\n<br>2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:308px; width:20px; height:27px;"><span style="font-family: Times-Roman; font-size:10px">n\n<br>u\n<br></span><span style="font-family: Times-Roman; font-size:7px">J\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:341px; width:20px; height:20px;"><span style="font-family: Times-Roman; font-size:10px">1\n<br>2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:371px; width:20px; height:6px;"><span style="font-family: Times-Roman; font-size:6px">]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:377px; width:20px; height:56px;"><span style="font-family: Times-Roman; font-size:14px">V\n<br></span><span style="font-family: Times-Roman; font-size:13px">C\n<br></span><span style="font-family: Times-Roman; font-size:5px">.\n<br></span><span style="font-family: Times-Roman; font-size:7px">s\n<br></span><span style="font-family: Times-Roman; font-size:8px">c\n<br></span><span style="font-family: Times-Roman; font-size:6px">[\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:443px; width:20px; height:166px;"><span style="font-family: Times-Roman; font-size:10px">2\n<br>v\n<br>8\n<br>4\n<br>3\n<br>5\n<br>1\n<br></span><span style="font-family: Times-Roman; font-size:5px">.\n<br></span><span style="font-family: Times-Roman; font-size:10px">3\n<br>0\n<br>1\n<br>2\n<br></span><span style="font-family: Times-Roman; font-size:5px">:\n<br></span><span style="font-family: Times-Roman; font-size:10px">v\n<br></span><span style="font-family: Times-Roman; font-size:5px">i\n<br></span><span style="font-family: Times-Roman; font-size:14px">X\n<br></span><span style="font-family: Times-Roman; font-size:6px">r\n<br></span><span style="font-family: Times-Roman; font-size:8px">a\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:157px; top:164px; width:300px; height:32px;"><span style="font-family: CMTT12; font-size:14px">LayoutParser</span><span style="font-family: CMBX12; font-size:14px">: A Uniﬁed Toolkit for Deep\n<br>Learning Based Document Image Analysis\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:218px; width:345px; height:23px;"><span style="font-family: CMR10; font-size:9px">Zejiang Shen</span><span style="font-family: CMR7; font-size:6px">1 </span><span style="font-family: CMR10; font-size:9px">(</span><span style="font-family: unknown; font-size:9px">(cid:0)</span><span style="font-family: CMR10; font-size:9px">), Ruochen Zhang</span><span style="font-family: CMR7; font-size:6px">2</span><span style="font-family: CMR10; font-size:9px">, Melissa Dell</span><span style="font-family: CMR7; font-size:6px">3</span><span style="font-family: CMR10; font-size:9px">, Benjamin Charles Germain\n<br>Lee</span><span style="font-family: CMR7; font-size:6px">4</span><span style="font-family: CMR10; font-size:9px">, Jacob Carlson</span><span style="font-family: CMR7; font-size:6px">3</span><span style="font-family: CMR10; font-size:9px">, and Weining Li</span><span style="font-family: CMR7; font-size:6px">5\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:207px; top:252px; width:200px; height:109px;"><span style="font-family: CMR6; font-size:5px">1 </span><span style="font-family: CMR9; font-size:8px">Allen Institute for AI\n<br></span><span style="font-family: CMTT9; font-size:8px">shannons@allenai.org\n<br></span><span style="font-family: CMR6; font-size:5px">2 </span><span style="font-family: CMR9; font-size:8px">Brown University\n<br></span><span style="font-family: CMTT9; font-size:8px">ruochen zhang@brown.edu\n<br></span><span style="font-family: CMR6; font-size:5px">3 </span><span style="font-family: CMR9; font-size:8px">Harvard University\n<br></span><span style="font-family: CMSY9; font-size:8px">{</span><span style="font-family: CMTT9; font-size:8px">melissadell,jacob carlson</span><span style="font-family: CMSY9; font-size:8px">}</span><span style="font-family: CMTT9; font-size:8px">@fas.harvard.edu\n<br></span><span style="font-family: CMR6; font-size:5px">4 </span><span style="font-family: CMR9; font-size:8px">University of Washington\n<br></span><span style="font-family: CMTT9; font-size:8px">bcgl@cs.washington.edu\n<br></span><span style="font-family: CMR6; font-size:5px">5 </span><span style="font-family: CMR9; font-size:8px">University of Waterloo\n<br></span><span style="font-family: CMTT9; font-size:8px">w422li@uwaterloo.ca\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:162px; top:388px; width:291px; height:228px;"><span style="font-family: CMBX9; font-size:8px">Abstract. </span><span style="font-family: CMR9; font-size:8px">Recent advances in document image analysis (DIA) have been\n<br>primarily driven by the application of neural networks. Ideally, research\n<br>outcomes could be easily deployed in production and extended for further\n<br>investigation. However, various factors like loosely organized codebases\n<br>and sophisticated model conﬁgurations complicate the easy reuse of im-\n<br>portant innovations by a wide audience. Though there have been on-going\n<br>eﬀorts to improve reusability and simplify deep learning (DL) model\n<br>development in disciplines like natural language processing and computer\n<br>vision, none of them are optimized for challenges in the domain of DIA.\n<br>This represents a major gap in the existing toolkit, as DIA is central to\n<br>academic research across a wide range of disciplines in the social sciences\n<br>and humanities. This paper introduces </span><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">, an open-source\n<br>library for streamlining the usage of DL in DIA research and applica-\n<br>tions. The core </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">library comes with a set of simple and\n<br>intuitive interfaces for applying and customizing DL models for layout de-\n<br>tection, character recognition, and many other document processing tasks.\n<br>To promote extensibility, </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">also incorporates a community\n<br>platform for sharing both pre-trained models and full document digiti-\n<br></span><span style="font-family: CMR9; font-size:8px">zation pipelines. We demonstrate that </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">is helpful for both\n<br>lightweight and large-scale digitization pipelines in real-word use cases.\n<br>The library is publicly available at </span><span style="font-family: CMTT9; font-size:8px">https://layout-parser.github.io</span><span style="font-family: CMR9; font-size:8px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:162px; top:627px; width:289px; height:21px;"><span style="font-family: CMBX9; font-size:8px">Keywords: </span><span style="font-family: CMR9; font-size:8px">Document Image Analysis </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Deep Learning </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Layout Analysis\n<br></span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Character Recognition </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Open Source library </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Toolkit.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:669px; width:6px; height:11px;"><span style="font-family: CMBX12; font-size:11px">1\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:669px; width:74px; height:11px;"><span style="font-family: CMBX12; font-size:11px">Introduction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:692px; width:347px; height:21px;"><span style="font-family: CMR10; font-size:9px">Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n<br></span><span style="font-family: CMR10; font-size:9px">document image analysis (DIA) tasks including document image classiﬁcation [11,\n<br></span></div><span style="position:absolute; border: black 1px solid; left:287px; top:293px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:315px; width:2px; height:0px;"></span>\n<span style="font-family: Times-Roman; font-size:5px"> \n<br> \n<br> \n<br> \n<br> \n<br> \n<br><span style="position:absolute; border: gray 1px solid; left:0px; top:892px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:892px;"><a name="2">Page 2</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:984px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:984px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1009px; width:348px; height:57px;"><span style="font-family: CMR10; font-size:9px">37], layout detection [38, 22], table detection [26], and scene text detection [4].\n<br>A generalized learning-based framework dramatically reduces the need for the\n<br>manual speciﬁcation of complicated rules, which is the status quo with traditional\n<br>methods. DL has the potential to transform DIA pipelines and beneﬁt a broad\n<br>spectrum of large-scale document digitization projects.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1069px; width:348px; height:213px;"><span style="font-family: CMR10; font-size:9px">However, there are several practical diﬃculties for taking advantages of re-\n<br>cent advances in DL-based methods: 1) DL models are notoriously convoluted\n<br>for reuse and extension. Existing models are developed using distinct frame-\n<br>works like TensorFlow [1] or PyTorch [24], and the high-level parameters can\n<br>be obfuscated by implementation details [8]. It can be a time-consuming and\n<br>frustrating experience to debug, reproduce, and adapt existing models for DIA,\n<br>and </span><span style="font-family: CMTI10; font-size:9px">many researchers who would beneﬁt the most from using these methods lack\n<br>the technical background to implement them from scratch. </span><span style="font-family: CMR10; font-size:9px">2) Document images\n<br></span><span style="font-family: CMR10; font-size:9px">contain diverse and disparate patterns across domains, and customized training\n<br></span><span style="font-family: CMR10; font-size:9px">is often required to achieve a desirable detection accuracy. Currently </span><span style="font-family: CMTI10; font-size:9px">there is no\n<br>full-ﬂedged infrastructure for easily curating the target document image datasets\n<br>and ﬁne-tuning or re-training the models. </span><span style="font-family: CMR10; font-size:9px">3) DIA usually requires a sequence of\n<br>models and other processing to obtain the ﬁnal outputs. Often research teams use\n<br>DL models and then perform further document analyses in separate processes,\n<br>and these pipelines are not documented in any central location (and often not\n<br>documented at all). This makes it </span><span style="font-family: CMTI10; font-size:9px">diﬃcult for research teams to learn about how\n<br>full pipelines are implemented </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTI10; font-size:9px">leads them to invest signiﬁcant resources in\n<br>reinventing the DIA wheel</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1284px; width:346px; height:33px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a uniﬁed toolkit to support DL-based document image\n<br>analysis and processing. To address the aforementioned challenges, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">is built with the following components:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1326px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">1. An oﬀ-the-shelf toolkit for applying DL models for layout detection, character\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1338px; width:194px; height:9px;"><span style="font-family: CMR10; font-size:9px">recognition, and other DIA tasks (Section 3)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1350px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">2. A rich repository of pre-trained neural network models (Model Zoo) that\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1362px; width:137px; height:9px;"><span style="font-family: CMR10; font-size:9px">underlies the oﬀ-the-shelf usage\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1373px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">3. Comprehensive tools for eﬃcient document image data annotation and model\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1385px; width:218px; height:9px;"><span style="font-family: CMR10; font-size:9px">tuning to support diﬀerent levels of customization\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1397px; width:343px; height:33px;"><span style="font-family: CMR10; font-size:9px">4. A DL model hub and community platform for the easy sharing, distribu-\n<br>tion, and discussion of DIA models and pipelines, to promote reusability,\n<br>reproducibility, and extensibility (Section 4)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1439px; width:346px; height:69px;"><span style="font-family: CMR10; font-size:9px">The library implements simple and intuitive Python APIs without sacriﬁcing\n<br>generalizability and versatility, and can be easily installed via pip. Its convenient\n<br>functions for handling document image data can be seamlessly integrated with\n<br>existing DIA pipelines. With detailed documentations and carefully curated\n<br>tutorials, we hope this tool will beneﬁt a variety of end-users, and will lead to\n<br>advances in applications in both industry and academic research.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1511px; width:347px; height:46px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is well aligned with recent eﬀorts for improving DL model\n<br>reusability in other disciplines like natural language processing [8, 34] and com-\n<br>puter vision [35], but with a focus on unique challenges in DIA. We show\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can be applied in sophisticated and large-scale digitization projects\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:1734px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:1734px;"><a name="3">Page 3</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:1826px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:1826px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">3\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1851px; width:348px; height:57px;"><span style="font-family: CMR10; font-size:9px">that require precision, eﬃciency, and robustness, as well as simple and light-\n<br>weight document processing tasks focusing on eﬃcacy and ﬂexibility (Section 5).\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is being actively maintained, and support for more deep learning\n<br>models and novel methods in text-based layout analysis methods [37, 34] is\n<br>planned.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1911px; width:347px; height:58px;"><span style="font-family: CMR10; font-size:9px">The rest of the paper is organized as follows. Section 2 provides an overview\n<br>of related work. The core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library, DL Model Zoo, and customized\n<br>model training are described in Section 3, and the DL model hub and commu-\n<br>nity platform are detailed in Section 4. Section 5 shows two examples of how\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can be used in practical DIA projects, and Section 6 concludes.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1990px; width:102px; height:11px;"><span style="font-family: CMBX12; font-size:11px">2 Related Work\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2016px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Recently, various DL models and datasets have been developed for layout analysis\n<br>tasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen-\n<br>tation tasks on historical documents. Object detection-based methods like Faster\n<br>R-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38]\n<br>and detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also\n<br>been used in table detection [27]. However, these models are usually implemented\n<br>individually and there is no uniﬁed framework to load and use such models.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2100px; width:348px; height:189px;"><span style="font-family: CMR10; font-size:9px">There has been a surge of interest in creating open-source tools for document\n<br>image processing: a search of </span><span style="font-family: CMTT10; font-size:9px">document image analysis </span><span style="font-family: CMR10; font-size:9px">in Github leads to 5M\n<br>relevant code pieces </span><span style="font-family: CMR7; font-size:6px">6</span><span style="font-family: CMR10; font-size:9px">; yet most of them rely on traditional rule-based methods\n<br>or provide limited functionalities. The closest prior research to our work is the\n<br>OCR-D project</span><span style="font-family: CMR7; font-size:6px">7</span><span style="font-family: CMR10; font-size:9px">, which also tries to build a complete toolkit for DIA. However,\n<br>similar to the platform developed by Neudecker et al. [21], it is designed for\n<br>analyzing historical documents, and provides no supports for recent DL models.\n<br>The </span><span style="font-family: CMTT10; font-size:9px">DocumentLayoutAnalysis </span><span style="font-family: CMR10; font-size:9px">project</span><span style="font-family: CMR7; font-size:6px">8 </span><span style="font-family: CMR10; font-size:9px">focuses on processing born-digital PDF\n<br>documents via analyzing the stored PDF data. Repositories like </span><span style="font-family: CMTT10; font-size:9px">DeepLayout</span><span style="font-family: CMR7; font-size:6px">9\n<br></span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Detectron2-PubLayNet</span><span style="font-family: CMR7; font-size:6px">10 </span><span style="font-family: CMR10; font-size:9px">are individual deep learning models trained on\n<br>layout analysis datasets without support for the full DIA pipeline. The Document\n<br>Analysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2]\n<br>aim to improve the reproducibility of DIA methods (or DL models), yet they\n<br>are not actively maintained. OCR engines like </span><span style="font-family: CMTT10; font-size:9px">Tesseract </span><span style="font-family: CMR10; font-size:9px">[14], </span><span style="font-family: CMTT10; font-size:9px">easyOCR</span><span style="font-family: CMR7; font-size:6px">11 </span><span style="font-family: CMR10; font-size:9px">and\n<br></span><span style="font-family: CMTT10; font-size:9px">paddleOCR</span><span style="font-family: CMR7; font-size:6px">12 </span><span style="font-family: CMR10; font-size:9px">usually do not come with comprehensive functionalities for other\n<br>DIA tasks like layout analysis.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2291px; width:347px; height:21px;"><span style="font-family: CMR10; font-size:9px">Recent years have also seen numerous eﬀorts to create libraries for promoting\n<br>reproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:2322px; width:295px; height:76px;"><span style="font-family: CMR6; font-size:5px">6 </span><span style="font-family: CMR9; font-size:8px">The number shown is obtained by specifying the search type as ‘code’.\n<br></span><span style="font-family: CMR6; font-size:5px">7 </span><span style="font-family: CMR9; font-size:8px">https://ocr-d.de/en/about\n<br></span><span style="font-family: CMR6; font-size:5px">8 </span><span style="font-family: CMR9; font-size:8px">https://github.com/BobLd/DocumentLayoutAnalysis\n<br></span><span style="font-family: CMR6; font-size:5px">9 </span><span style="font-family: CMR9; font-size:8px">https://github.com/leonlulu/DeepLayout\n<br></span><span style="font-family: CMR6; font-size:5px">10 </span><span style="font-family: CMR9; font-size:8px">https://github.com/hpanwar08/detectron2\n<br></span><span style="font-family: CMR6; font-size:5px">11 </span><span style="font-family: CMR9; font-size:8px">https://github.com/JaidedAI/EasyOCR\n<br></span><span style="font-family: CMR6; font-size:5px">12 </span><span style="font-family: CMR9; font-size:8px">https://github.com/PaddlePaddle/PaddleOCR\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:2320px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:2576px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:2576px;"><a name="4">Page 4</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2668px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">4\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:2668px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2838px; width:348px; height:105px;"><span style="font-family: CMR10; font-size:9px">Fig. 1: The overall architecture of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. For an input document image,\n<br></span><span style="font-family: CMR10; font-size:9px">the core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library provides a set of oﬀ-the-shelf tools for layout\n<br>detection, OCR, visualization, and storage, backed by a carefully designed layout\n<br>data structure. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also supports high level customization via eﬃcient\n<br>layout annotation and model training functions. These improve model accuracy\n<br>on the target samples. The community platform enables the easy sharing of DIA\n<br>models and whole digitization pipelines to promote reusability and reproducibility.\n<br>A collection of detailed documentation, tutorials and exemplar projects make\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">easy to learn and use.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2970px; width:346px; height:81px;"><span style="font-family: CMR10; font-size:9px">AllenNLP [8] and transformers [34] have provided the community with complete\n<br>DL-based support for developing and deploying models for general computer\n<br>vision and natural language processing problems. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, on the other\n<br>hand, specializes speciﬁcally in DIA tasks. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is also equipped with a\n<br>community platform inspired by established model hubs such as </span><span style="font-family: CMTT10; font-size:9px">Torch Hub </span><span style="font-family: CMR10; font-size:9px">[23]\n<br>and </span><span style="font-family: CMTT10; font-size:9px">TensorFlow Hub </span><span style="font-family: CMR10; font-size:9px">[1]. It enables the sharing of pretrained models as well as\n<br>full document processing pipelines that are unique to DIA tasks.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3054px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">There have been a variety of document data collections to facilitate the\n<br>development of DL models. Some examples include PRImA [3](magazine layouts),\n<br>PubLayNet [38](academic paper layouts), Table Bank [18](tables in academic\n<br>papers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and\n<br></span><span style="font-family: CMTT10; font-size:9px">HJDataset </span><span style="font-family: CMR10; font-size:9px">[31](historical Japanese document layouts). A spectrum of models\n<br></span><span style="font-family: CMR10; font-size:9px">trained on these datasets are currently available in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">model zoo\n<br>to support diﬀerent use cases.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3156px; width:202px; height:12px;"><span style="font-family: CMBX12; font-size:11px">3 The Core </span><span style="font-family: CMTT12; font-size:11px">LayoutParser </span><span style="font-family: CMBX12; font-size:11px">Library\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3183px; width:347px; height:57px;"><span style="font-family: CMR10; font-size:9px">At the core of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is an oﬀ-the-shelf toolkit that streamlines DL-\n<br>based document image analysis. Five components support a simple interface\n<br>with comprehensive functionalities: 1) The </span><span style="font-family: CMTI10; font-size:9px">layout detection models </span><span style="font-family: CMR10; font-size:9px">enable using\n<br>pre-trained or self-trained DL models for layout detection with just four lines\n<br></span><span style="font-family: CMR10; font-size:9px">of code. 2) The detected layout information is stored in carefully engineered\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:169px; top:2691px; width:276px; height:136px;"><span style="position:absolute; border: black 1px solid; left:169px; top:2691px; width:276px; height:136px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2701px; width:77px; height:66px;"></span>\n<span style="position:absolute; border: black 1px solid; left:185px; top:2719px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2719px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2736px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:194px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:199px; top:2726px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:199px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2727px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2726px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:213px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:217px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:217px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:222px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:232px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2726px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2727px; width:2px; height:3px;"></span>\n<span style="font-family: unknown; font-size:6px">Efficient Data Annotation<span style="position:absolute; border: black 1px solid; left:185px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:187px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:187px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:196px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2752px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:208px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:208px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2751px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:222px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:233px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:233px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:238px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:238px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:247px; top:2752px; width:1px; height:2px;"></span>\nCustomized Model Training<span style="position:absolute; border: black 1px solid; left:179px; top:2701px; width:77px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2707px; width:77px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2705px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:194px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2705px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:201px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:201px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2707px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2705px; width:0px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:224px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:224px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2708px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:237px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:237px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2706px; width:2px; height:3px;"></span>\nModel Customization<span style="position:absolute; border: black 1px solid; left:358px; top:2701px; width:77px; height:66px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2719px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2719px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2736px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:379px; top:2726px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2726px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2726px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:400px; top:2727px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:411px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2726px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2727px; width:1px; height:2px;"></span>\nDIA Model Hub<span style="position:absolute; border: black 1px solid; left:363px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2751px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:377px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2751px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:387px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:392px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:403px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:403px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2752px; width:1px; height:2px;"></span>\nDIA Pipeline Sharing<span style="position:absolute; border: black 1px solid; left:358px; top:2701px; width:77px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:358px; top:2707px; width:77px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:367px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:367px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:380px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2706px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2706px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2705px; width:0px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2708px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2705px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:416px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2706px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:420px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:421px; top:2706px; width:4px; height:3px;"></span>\nCommunity Platform<span style="position:absolute; border: black 1px solid; left:179px; top:2774px; width:256px; height:43px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2737px; width:83px; height:36px;"></span>\n<span style="position:absolute; border: black 1px solid; left:274px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:277px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:277px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:291px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2751px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:304px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:306px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:306px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:309px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:312px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:312px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2751px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:325px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:328px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2752px; width:2px; height:3px;"></span>\nLayout Detection Models<span style="position:absolute; border: black 1px solid; left:281px; top:2701px; width:52px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2701px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2701px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2719px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2719px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2701px; width:52px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2701px; width:1px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2719px; width:52px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2701px; width:1px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2708px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2709px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:2710px; width:0px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:294px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:297px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:297px; top:2709px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2708px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2708px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2709px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2711px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2709px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2710px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:325px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2709px; width:2px; height:3px;"></span>\nDocument Images <span style="position:absolute; border: black 1px solid; left:179px; top:2805px; width:256px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2805px; width:256px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:266px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:266px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:270px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:270px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:274px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:278px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:278px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2810px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:287px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:296px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2810px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2810px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2809px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2810px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:317px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:322px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:323px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:332px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2810px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2809px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:337px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:338px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:339px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:341px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:343px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:343px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2810px; width:3px; height:4px;"></span>\n</span><span style="font-family: unknown; font-size:6px">The Core LayoutParser Library<span style="position:absolute; border: black 1px solid; left:185px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:203px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:203px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:210px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:210px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2789px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2788px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:219px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2788px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:230px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:230px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2790px; width:1px; height:0px;"></span>\n</span><span style="font-family: unknown; font-size:6px">OCR Module<span style="position:absolute; border: black 1px solid; left:363px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:425px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:425px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:373px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:373px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:374px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:379px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2789px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2790px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2789px; width:0px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:404px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:404px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2788px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:413px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:416px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:417px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:417px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:422px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:422px; top:2789px; width:2px; height:3px;"></span>\nStorage &amp; Visualization<span style="position:absolute; border: black 1px solid; left:274px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:287px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2789px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:316px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:316px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2790px; width:1px; height:0px;"></span>\nLayout Data Structure<span style="position:absolute; border: black 1px solid; left:304px; top:2767px; width:7px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:304px; top:2723px; width:7px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2789px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2788px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:256px; top:2791px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2791px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2752px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2750px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:256px; top:2754px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2754px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2789px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:355px; top:2788px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:346px; top:2791px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2791px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2752px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:355px; top:2750px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:346px; top:2754px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2754px; width:3px; height:2px;"></span>\n</span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:3418px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:3418px;"><a name="5">Page 5</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:3510px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:3510px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">5\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:146px; top:3544px; width:321px; height:10px;"><span style="font-family: CMR10; font-size:9px">Table 1: Current layout detection models in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">model zoo\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:150px; top:3559px; width:26px; height:6px;"><span style="font-family: CMBX9; font-size:6px">Dataset\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:194px; top:3558px; width:118px; height:7px;"><span style="font-family: CMBX9; font-size:6px">Base Model</span><span style="font-family: CMR6; font-size:4px">1 </span><span style="font-family: CMBX9; font-size:6px">Large Model Notes\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:3573px; width:47px; height:46px;"><span style="font-family: CMR9; font-size:6px">PubLayNet [38]\n<br>PRImA [3]\n<br>Newspaper [17]\n<br>TableBank [18]\n<br>HJDataset [31]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:205px; top:3573px; width:18px; height:46px;"><span style="font-family: CMR9; font-size:6px">F / M\n<br>M\n<br>F\n<br>F\n<br>F / M\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:261px; top:3573px; width:6px; height:46px;"><span style="font-family: CMR9; font-size:6px">M\n<br>-\n<br>-\n<br>F\n<br>-\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:293px; top:3573px; width:181px; height:46px;"><span style="font-family: CMR9; font-size:6px">Layouts of modern scientiﬁc documents\n<br>Layouts of scanned modern magazines and scientiﬁc reports\n<br>Layouts of scanned US newspapers from the 20th century\n<br>Table region on modern scientiﬁc and business document\n<br>Layouts of history Japanese documents\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:136px; top:3624px; width:342px; height:48px;"><span style="font-family: CMR6; font-size:4px">1 </span><span style="font-family: CMR9; font-size:6px">For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\n<br>vs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\n<br>backbones [13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [28] (F) and Mask\n<br>R-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\n<br>using the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\n<br>zoo in coming months.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3697px; width:346px; height:81px;"><span style="font-family: CMTI10; font-size:9px">layout data structures</span><span style="font-family: CMR10; font-size:9px">, which are optimized for eﬃciency and versatility. 3) When\n<br></span><span style="font-family: CMR10; font-size:9px">necessary, users can employ existing or customized OCR models via the uniﬁed\n<br>API provided in the </span><span style="font-family: CMTI10; font-size:9px">OCR module</span><span style="font-family: CMR10; font-size:9px">. 4) </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">comes with a set of utility\n<br>functions for the </span><span style="font-family: CMTI10; font-size:9px">visualization and storage </span><span style="font-family: CMR10; font-size:9px">of the layout data. 5) </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">is also highly customizable, via its integration with functions for </span><span style="font-family: CMTI10; font-size:9px">layout data\n<br>annotation and model training</span><span style="font-family: CMR10; font-size:9px">. We now provide detailed descriptions for each\n<br>component.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3797px; width:152px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.1 Layout Detection Models\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3816px; width:347px; height:105px;"><span style="font-family: CMR10; font-size:9px">In </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, a layout model takes a document image as an input and\n<br>generates a list of rectangular boxes for the target content regions. Diﬀerent\n<br>from traditional methods, it relies on deep convolutional neural networks rather\n<br>than manually curated rules to identify content regions. It is formulated as an\n<br>object detection problem and state-of-the-art models like Faster R-CNN [28] and\n<br>Mask R-CNN [12] are used. This yields prediction results of high accuracy and\n<br>makes it possible to build a concise, generalized interface for layout detection.\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, built upon Detectron2 [35], provides a minimal API that can\n<br>perform layout detection with only four lines of code in Python:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:3929px; width:267px; height:30px;"><span style="font-family: CMR5; font-size:4px">1 </span><span style="font-family: CMTT9; font-size:8px">import layoutparser as lp\n<br></span><span style="font-family: CMR5; font-size:4px">2 </span><span style="font-family: CMTT9; font-size:8px">image = cv2 . imread ( &quot; image_file &quot; ) # load images\n<br></span><span style="font-family: CMR5; font-size:4px">3 </span><span style="font-family: CMTT9; font-size:8px">model = lp . De t e c tro n2 Lay outM odel (\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:157px; top:3961px; width:270px; height:8px;"><span style="font-family: CMTT9; font-size:8px">&quot; lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config &quot; )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:3964px; width:166px; height:16px;"><span style="font-family: CMR5; font-size:4px">4\n<br>5 </span><span style="font-family: CMTT9; font-size:8px">layout = model . detect ( image )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3989px; width:347px; height:93px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a wealth of pre-trained model weights using various\n<br>datasets covering diﬀerent languages, time periods, and document types. Due to\n<br>domain shift [7], the prediction performance can notably drop when models are ap-\n<br>plied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\n<br>document structures and layouts vary greatly in diﬀerent domains, it is important\n<br>to select models trained on a dataset similar to the test samples. A semantic syntax\n<br>is used for initializing the model weights in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, using both the dataset\n<br></span><span style="font-family: CMR10; font-size:9px">name and model name </span><span style="font-family: CMTT10; font-size:9px">lp://&lt;dataset-name&gt;/&lt;model-architecture-name&gt;</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:137px; top:3556px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:3569px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:3622px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3928px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3939px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3950px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3961px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3972px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:4260px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:4260px;"><a name="6">Page 6</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4352px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">6\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:4352px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4567px; width:347px; height:69px;"><span style="font-family: CMR10; font-size:9px">Fig. 2: The relationship between the three types of layout data structures.\n<br></span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">supports three kinds of variation; </span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">consists of the co-\n<br>ordinate information and extra features like block text, types, and reading orders;\n<br>a </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">object is a list of all possible layout elements, including other </span><span style="font-family: CMTT10; font-size:9px">Layout\n<br></span><span style="font-family: CMR10; font-size:9px">objects. They all support the same set of transformation and operation APIs for\n<br>maximum ﬂexibility.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4665px; width:347px; height:69px;"><span style="font-family: CMR10; font-size:9px">Shown in Table 1, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">currently hosts 9 pre-trained models trained\n<br>on 5 diﬀerent datasets. Description of the training dataset is provided alongside\n<br>with the trained models such that users can quickly identify the most suitable\n<br>models for their tasks. Additionally, when such a model is not readily available,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also supports training customized layout models and community\n<br>sharing of the models (detailed in Section 3.5).\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4759px; width:144px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.2 Layout Data Structures\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4783px; width:346px; height:141px;"><span style="font-family: CMR10; font-size:9px">A critical feature of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is the implementation of a series of data\n<br></span><span style="font-family: CMR10; font-size:9px">structures and operations that can be used to eﬃciently process and manipulate\n<br>the layout elements. In document image analysis pipelines, various post-processing\n<br>on the layout analysis model outputs is usually required to obtain the ﬁnal\n<br>outputs. Traditionally, this requires exporting DL model outputs and then loading\n<br>the results into other pipelines. All model outputs from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">will be\n<br>stored in carefully engineered data types optimized for further processing, which\n<br>makes it possible to build an end-to-end document digitization pipeline within\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. There are three key components in the data structure, namely\n<br>the </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">system, the </span><span style="font-family: CMTT10; font-size:9px">TextBlock</span><span style="font-family: CMR10; font-size:9px">, and the </span><span style="font-family: CMTT10; font-size:9px">Layout</span><span style="font-family: CMR10; font-size:9px">. They provide diﬀerent\n<br>levels of abstraction for the layout data, and a set of APIs are supported for\n<br></span><span style="font-family: CMR10; font-size:9px">transformations or operations on these classes.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:195px; top:4375px; width:224px; height:181px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:5102px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:5102px;"><a name="7">Page 7</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:5194px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:5194px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">7\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5219px; width:347px; height:177px;"><span style="font-family: CMR10; font-size:9px">Coordinates are the cornerstones for storing layout information. Currently,\n<br>three types of </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">data structures are provided in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, shown\n<br>in Figure 2. </span><span style="font-family: CMTT10; font-size:9px">Interval </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Rectangle </span><span style="font-family: CMR10; font-size:9px">are the most common data types and\n<br>support specifying 1D or 2D regions within a document. They are parameterized\n<br>with 2 and 4 parameters. A </span><span style="font-family: CMTT10; font-size:9px">Quadrilateral </span><span style="font-family: CMR10; font-size:9px">class is also implemented to support\n<br>a more generalized representation of rectangular regions when the document\n<br>is skewed or distorted, where the 4 corner points can be speciﬁed and a total\n<br>of 8 degrees of freedom are supported. A wide collection of transformations\n<br>like </span><span style="font-family: CMTT10; font-size:9px">shift</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">pad</span><span style="font-family: CMR10; font-size:9px">, and </span><span style="font-family: CMTT10; font-size:9px">scale</span><span style="font-family: CMR10; font-size:9px">, and operations like </span><span style="font-family: CMTT10; font-size:9px">intersect</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">union</span><span style="font-family: CMR10; font-size:9px">, and </span><span style="font-family: CMTT10; font-size:9px">is_in</span><span style="font-family: CMR10; font-size:9px">,\n<br>are supported for these classes. Notably, it is common to separate a segment\n<br>of the image and analyze it individually. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides full support\n<br>for this scenario via image cropping operations </span><span style="font-family: CMTT10; font-size:9px">crop_image </span><span style="font-family: CMR10; font-size:9px">and coordinate\n<br>transformations like </span><span style="font-family: CMTT10; font-size:9px">relative_to </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">condition_on </span><span style="font-family: CMR10; font-size:9px">that transform coordinates\n<br></span><span style="font-family: CMR10; font-size:9px">to and from their relative representations. We refer readers to Table 2 for a more\n<br></span><span style="font-family: CMR10; font-size:9px">detailed description of these operations</span><span style="font-family: CMR7; font-size:6px">13</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5399px; width:346px; height:81px;"><span style="font-family: CMR10; font-size:9px">Based on </span><span style="font-family: CMTT10; font-size:9px">Coordinate</span><span style="font-family: CMR10; font-size:9px">s, we implement the </span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">class that stores both\n<br>the positional and extra features of individual layout elements. It also supports\n<br>specifying the reading orders via setting the </span><span style="font-family: CMTT10; font-size:9px">parent </span><span style="font-family: CMR10; font-size:9px">ﬁeld to the index of the parent\n<br>object. A </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">class is built that takes in a list of </span><span style="font-family: CMTT10; font-size:9px">TextBlock</span><span style="font-family: CMR10; font-size:9px">s and supports\n<br>processing the elements in batch. </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">can also be nested to support hierarchical\n<br>layout structures. They support the same operations and transformations as the\n<br></span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">classes, minimizing both learning and deployment eﬀort.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5498px; width:51px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.3 OCR\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5516px; width:346px; height:105px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a uniﬁed interface for existing OCR tools. Though there\n<br>are many OCR tools available, they are usually conﬁgured diﬀerently with distinct\n<br>APIs or protocols for using them. It can be ineﬃcient to add new OCR tools into\n<br>an existing pipeline, and diﬃcult to make direct comparisons among the available\n<br>tools to ﬁnd the best option for a particular project. To this end, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">builds a series of wrappers among existing OCR engines, and provides nearly\n<br>the same syntax for using them. It supports a plug-and-play style of using OCR\n<br>engines, making it eﬀortless to switch, evaluate, and compare diﬀerent OCR\n<br>modules:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:5629px; width:267px; height:30px;"><span style="font-family: CMR5; font-size:4px">1 </span><span style="font-family: CMTT9; font-size:8px">ocr_agent = lp . TesseractAgent ()\n<br></span><span style="font-family: CMR5; font-size:4px">2 </span><span style="font-family: CMTT9; font-size:8px"># Can be easily switched to other OCR software\n<br></span><span style="font-family: CMR5; font-size:4px">3 </span><span style="font-family: CMTT9; font-size:8px">tokens = ocr_agent . detect ( image )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5667px; width:347px; height:45px;"><span style="font-family: CMR10; font-size:9px">The OCR outputs will also be stored in the aforementioned layout data\n<br>structures and can be seamlessly incorporated into the digitization pipeline.\n<br>Currently </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports the Tesseract and Google Cloud Vision OCR\n<br>engines.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5715px; width:346px; height:33px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also comes with a DL-based CNN-RNN OCR model [6] trained\n<br>with the Connectionist Temporal Classiﬁcation (CTC) loss [10]. It can be used\n<br>like the other OCR modules, and can be easily trained on customized datasets.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:5756px; width:271px; height:10px;"><span style="font-family: CMR6; font-size:5px">13 </span><span style="font-family: CMR9; font-size:8px">This is also available in the </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">documentation pages.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:5628px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5639px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5650px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5754px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:5944px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:5944px;"><a name="8">Page 8</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6036px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">8\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:6036px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6070px; width:347px; height:34px;"><span style="font-family: CMR10; font-size:9px">Table 2: All operations supported by the layout elements. The same APIs are\n<br>supported across diﬀerent layout element classes including </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">types,\n<br></span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Layout</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6110px; width:65px; height:7px;"><span style="font-family: CMBX9; font-size:7px">Operation Name\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6110px; width:44px; height:7px;"><span style="font-family: CMBX9; font-size:7px">Description\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6125px; width:310px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.pad(top, bottom, right, left) </span><span style="font-family: CMR9; font-size:7px">Enlarge the current block according to the input\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6146px; width:75px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.scale(fx, fy)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6172px; width:75px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.shift(dx, dy)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6140px; width:129px; height:18px;"><span style="font-family: CMR9; font-size:7px">Scale the current block given the ratio\n<br>in x and y direction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6167px; width:127px; height:18px;"><span style="font-family: CMR9; font-size:7px">Move the current block with the shift\n<br>distances in x and y direction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6193px; width:77px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.is in(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6193px; width:116px; height:7px;"><span style="font-family: CMR9; font-size:7px">Whether block1 is inside of block2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6214px; width:94px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.intersect(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6240px; width:79px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.union(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6266px; width:101px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.relative to(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6292px; width:105px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.condition on(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6208px; width:186px; height:18px;"><span style="font-family: CMR9; font-size:7px">Return the intersection region of block1 and block2.\n<br></span><span style="font-family: CMR9; font-size:7px">Coordinate type to be determined based on the inputs.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6234px; width:186px; height:18px;"><span style="font-family: CMR9; font-size:7px">Return the union region of block1 and block2.\n<br>Coordinate type to be determined based on the inputs.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6261px; width:154px; height:18px;"><span style="font-family: CMR9; font-size:7px">Convert the absolute coordinates of block1 to\n<br>relative coordinates to block2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6287px; width:170px; height:18px;"><span style="font-family: CMR9; font-size:7px">Calculate the absolute coordinates of block1 given\n<br>the canvas block2’s absolute coordinates\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6313px; width:89px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.crop image(image)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6313px; width:158px; height:7px;"><span style="font-family: CMR9; font-size:7px">Obtain the image segments in the block region\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6350px; width:152px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.4 Storage and visualization\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6371px; width:346px; height:129px;"><span style="font-family: CMR10; font-size:9px">The end goal of DIA is to transform the image-based document data into a\n<br>structured database. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports exporting layout data into diﬀerent\n<br>formats like </span><span style="font-family: CMTT10; font-size:9px">JSON</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">csv</span><span style="font-family: CMR10; font-size:9px">, and will add the support for the METS/ALTO XML\n<br>format </span><span style="font-family: CMR7; font-size:6px">14 </span><span style="font-family: CMR10; font-size:9px">. It can also load datasets from layout analysis-speciﬁc formats like\n<br>COCO [38] and the Page Format [25] for training layout models (Section 3.5).\n<br>Visualization of the layout detection results is critical for both presentation\n<br>and debugging. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is built with an integrated API for displaying the\n<br>layout information along with the original document image. Shown in Figure 3, it\n<br>enables presenting layout data with rich meta information and features in diﬀerent\n<br>modes. More detailed information can be found in the online </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">documentation page.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6522px; width:166px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.5 Customized Model Training\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6543px; width:347px; height:45px;"><span style="font-family: CMR10; font-size:9px">Besides the oﬀ-the-shelf library, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is also highly customizable with\n<br>supports for highly unique and challenging document analysis tasks. Target\n<br>document images can be vastly diﬀerent from the existing datasets for train-\n<br>ing layout models, which leads to low layout detection accuracy. Training data\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:6598px; width:111px; height:10px;"><span style="font-family: CMR6; font-size:5px">14 </span><span style="font-family: CMR9; font-size:8px">https://altoxml.github.io\n<br></span></div><span style="position:absolute; border: black 1px solid; left:137px; top:6105px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6108px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6121px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6123px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6136px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6139px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6162px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6165px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6189px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:177px; top:6199px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6191px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6204px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6206px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6230px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6233px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6256px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:6272px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6259px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6283px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:6298px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6285px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6309px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:181px; top:6319px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6312px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6324px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:6596px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:6786px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:6786px;"><a name="9">Page 9</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:6878px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:6878px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">9\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7103px; width:346px; height:69px;"><span style="font-family: CMR10; font-size:9px">Fig. 3: Layout detection and OCR results visualization generated by the\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">APIs. Mode I directly overlays the layout region bounding boxes\n<br>and categories over the original image. Mode II recreates the original document\n<br>via drawing the OCR’d texts at their corresponding positions on the image\n<br>canvas. In this ﬁgure, tokens in textual regions are ﬁltered using the API and\n<br>then displayed.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7212px; width:347px; height:33px;"><span style="font-family: CMR10; font-size:9px">can also be highly sensitive and not sharable publicly. To overcome these chal-\n<br>lenges, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is built with rich features for eﬃcient data annotation and\n<br>customized model training.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7255px; width:347px; height:81px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">incorporates a toolkit optimized for annotating document lay-\n<br>outs using object-level active learning [32]. With the help from a layout detection\n<br>model trained along with labeling, only the most important layout objects within\n<br>each image, rather than the whole image, are required for labeling. The rest of\n<br>the regions are automatically annotated with high conﬁdence predictions from\n<br>the layout detection model. This allows a layout dataset to be created more\n<br></span><span style="font-family: CMR10; font-size:9px">eﬃciently with only around 60% of the labeling budget.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7345px; width:347px; height:105px;"><span style="font-family: CMR10; font-size:9px">After the training dataset is curated, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports diﬀerent modes\n<br>for training the layout models. </span><span style="font-family: CMTI10; font-size:9px">Fine-tuning </span><span style="font-family: CMR10; font-size:9px">can be used for training models on a\n<br></span><span style="font-family: CMTI10; font-size:9px">small </span><span style="font-family: CMR10; font-size:9px">newly-labeled dataset by initializing the model with existing pre-trained\n<br>weights. </span><span style="font-family: CMTI10; font-size:9px">Training from scratch </span><span style="font-family: CMR10; font-size:9px">can be helpful when the source dataset and\n<br>target are signiﬁcantly diﬀerent and a large training set is available. However, as\n<br>suggested in Studer et al.’s work[33], loading pre-trained weights on large-scale\n<br>datasets like ImageNet [5], even from totally diﬀerent domains, can still boost\n<br>model performance. Through the integrated API provided by </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">,\n<br></span><span style="font-family: CMR10; font-size:9px">users can easily compare model performances on the benchmark datasets.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:169px; top:6901px; width:276px; height:191px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:7628px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:7628px;"><a name="10">Page 10</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7720px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">10\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:7720px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7912px; width:346px; height:57px;"><span style="font-family: CMR10; font-size:9px">Fig. 4: Illustration of (a) the original historical Japanese document with layout\n<br>detection results and (b) a recreated version of the document image that achieves\n<br>much better character recognition recall. The reorganization algorithm rearranges\n<br>the tokens based on the their detected bounding boxes given a maximum allowed\n<br>height.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7992px; width:224px; height:12px;"><span style="font-family: CMBX12; font-size:11px">4 </span><span style="font-family: CMTT12; font-size:11px">LayoutParser </span><span style="font-family: CMBX12; font-size:11px">Community Platform\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8016px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Another focus of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is promoting the reusability of layout detection\n<br>models and full digitization pipelines. Similar to many existing deep learning\n<br>libraries, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">comes with a community model hub for distributing\n<br>layout models. End-users can upload their self-trained models to the model hub,\n<br>and these models can be loaded into a similar interface as the currently available\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">pre-trained models. For example, the model trained on the News\n<br>Navigator dataset [17] has been incorporated in the model hub.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8099px; width:347px; height:129px;"><span style="font-family: CMR10; font-size:9px">Beyond DL models, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also promotes the sharing of entire doc-\n<br>ument digitization pipelines. For example, sometimes the pipeline requires the\n<br>combination of multiple DL models to achieve better accuracy. Currently, pipelines\n<br>are mainly described in academic papers and implementations are often not pub-\n<br>licly available. To this end, the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">community platform also enables\n<br></span><span style="font-family: CMR10; font-size:9px">the sharing of layout pipelines to promote the discussion and reuse of techniques.\n<br>For each shared pipeline, it has a dedicated project page, with links to the source\n<br>code, documentation, and an outline of the approaches. A discussion panel is\n<br>provided for exchanging ideas. Combined with the core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library,\n<br>users can easily build reusable components based on the shared pipelines and\n<br>apply them to solve their unique problems.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8247px; width:79px; height:11px;"><span style="font-family: CMBX12; font-size:11px">5 Use Cases\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8270px; width:346px; height:21px;"><span style="font-family: CMR10; font-size:9px">The core objective of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is to make it easier to create both large-scale\n<br></span><span style="font-family: CMR10; font-size:9px">and light-weight document digitization pipelines. Large-scale document processing\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:134px; top:7743px; width:345px; height:158px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:8470px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:8470px;"><a name="11">Page 11</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:8562px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:8562px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">11\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8587px; width:347px; height:117px;"><span style="font-family: CMR10; font-size:9px">focuses on precision, eﬃciency, and robustness. The target documents may have\n<br>complicated structures, and may require training multiple layout detection models\n<br>to achieve the optimal accuracy. Light-weight pipelines are built for relatively\n<br>simple documents, with an emphasis on development ease, speed and ﬂexibility.\n<br>Ideally one only needs to use existing resources, and model training should be\n<br>avoided. Through two exemplar projects, we show how practitioners in both\n<br>academia and industry can easily build such pipelines using </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">and\n<br>extract high-quality structured document data for their downstream tasks. The\n<br>source code for these projects will be publicly available in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">community hub.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8730px; width:330px; height:9px;"><span style="font-family: CMBX10; font-size:9px">5.1 A Comprehensive Historical Document Digitization Pipeline\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8755px; width:347px; height:189px;"><span style="font-family: CMR10; font-size:9px">The digitization of historical documents can unlock valuable data that can shed\n<br>light on many important social, economic, and historical questions. Yet due to\n<br>scan noises, page wearing, and the prevalence of complicated layout structures, ob-\n<br>taining a structured representation of historical document scans is often extremely\n<br>complicated.\n<br>In this example, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">was\n<br>used to develop a comprehensive\n<br>pipeline, shown in Figure 5, to gener-\n<br>ate high-quality structured data from\n<br>historical Japanese ﬁrm ﬁnancial ta-\n<br>bles with complicated layouts. The\n<br>pipeline applies two layout models to\n<br>identify diﬀerent levels of document\n<br>structures and two customized OCR\n<br>engines for optimized character recog-\n<br>nition accuracy.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8947px; width:165px; height:153px;"><span style="font-family: CMR10; font-size:9px">As shown in Figure 4 (a), the\n<br>document contains columns of text\n<br>written vertically </span><span style="font-family: CMR7; font-size:6px">15</span><span style="font-family: CMR10; font-size:9px">, a common style\n<br>in Japanese. Due to scanning noise\n<br>and archaic printing technology, the\n<br></span><span style="font-family: CMR10; font-size:9px">columns can be skewed or have vari-\n<br>able widths, and hence cannot be eas-\n<br>ily identiﬁed via rule-based methods.\n<br>Within each column, words are sepa-\n<br>rated by white spaces of variable size,\n<br>and the vertical positions of objects\n<br>can be an indicator of their layout\n<br>type.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:307px; top:9052px; width:174px; height:33px;"><span style="font-family: CMR10; font-size:9px">Fig. 5: Illustration of how </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">helps with the historical document digi-\n<br>tization pipeline.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9113px; width:347px; height:10px;"><span style="font-family: CMR6; font-size:5px">15 </span><span style="font-family: CMR9; font-size:8px">A document page consists of eight rows like this. For simplicity we skip the row\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9125px; width:308px; height:8px;"><span style="font-family: CMR9; font-size:8px">segmentation discussion and refer readers to the source code when available.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:307px; top:8849px; width:172px; height:182px;"></div><span style="position:absolute; border: black 1px solid; left:134px; top:9111px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:9312px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:9312px;"><a name="12">Page 12</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9404px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">12\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:9404px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:149px; top:9429px; width:148px; height:9px;"><span style="font-family: CMR10; font-size:9px">To decipher the complicated layout\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9441px; width:347px; height:141px;"><span style="font-family: CMR10; font-size:9px">structure, two object detection models have been trained to recognize individual\n<br>columns and tokens, respectively. A small training set (400 images with approxi-\n<br>mately 100 annotations each) is curated via the active learning based annotation\n<br>tool [32] in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. The models learn to identify both the categories and\n<br>regions for each token or column via their distinct visual features. The layout\n<br>data structure enables easy grouping of the tokens within each column, and\n<br>rearranging columns to achieve the correct reading orders based on the horizontal\n<br>position. Errors are identiﬁed and rectiﬁed via checking the consistency of the\n<br>model predictions. Therefore, though trained on a small dataset, the pipeline\n<br>achieves a high level of layout detection accuracy: it achieves a 96.97 AP [19]\n<br>score across 5 categories for the column detection model, and a 89.23 AP across\n<br>4 categories for the token detection model.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9587px; width:346px; height:117px;"><span style="font-family: CMR10; font-size:9px">A combination of character recognition methods is developed to tackle the\n<br>unique challenges in this document. In our experiments, we found that irregular\n<br>spacing between the tokens led to a low character recognition recall rate, whereas\n<br>existing OCR models tend to perform better on densely-arranged texts. To\n<br>overcome this challenge, we create a document reorganization algorithm that\n<br>rearranges the text based on the token bounding boxes detected in the layout\n<br>analysis step. Figure 4 (b) illustrates the generated image of dense text, which is\n<br>sent to the OCR APIs as a whole to reduce the transaction costs. The ﬂexible\n<br>coordinate system in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is used to transform the OCR results relative\n<br>to their original positions on the page.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9710px; width:347px; height:141px;"><span style="font-family: CMR10; font-size:9px">Additionally, it is common for historical documents to use unique fonts\n<br>with diﬀerent glyphs, which signiﬁcantly degrades the accuracy of OCR models\n<br>trained on modern texts. In this document, a special ﬂat font is used for printing\n<br>numbers and could not be detected by oﬀ-the-shelf OCR engines. Using the highly\n<br>ﬂexible functionalities from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, a pipeline approach is constructed\n<br>that achieves a high recognition accuracy with minimal eﬀort. As the characters\n<br>have unique visual structures and are usually clustered together, we train the\n<br>layout model to identify number regions with a dedicated category. Subsequently,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">crops images within these regions, and identiﬁes characters within\n<br>them using a self-trained OCR model based on a CNN-RNN [6]. The model\n<br>detects a total of 15 possible categories, and achieves a 0.98 Jaccard score</span><span style="font-family: CMR7; font-size:6px">16 </span><span style="font-family: CMR10; font-size:9px">and\n<br></span><span style="font-family: CMR10; font-size:9px">a 0.17 average Levinstein distances</span><span style="font-family: CMR7; font-size:6px">17 </span><span style="font-family: CMR10; font-size:9px">for token prediction on the test set.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9856px; width:345px; height:57px;"><span style="font-family: CMR10; font-size:9px">Overall, it is possible to create an intricate and highly accurate digitization\n<br>pipeline for large-scale digitization using </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. The pipeline avoids\n<br>specifying the complicated rules used in traditional methods, is straightforward\n<br>to develop, and is robust to outliers. The DL models also generate ﬁne-grained\n<br>results that enable creative approaches like page reorganization for OCR.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9933px; width:346px; height:10px;"><span style="font-family: CMR6; font-size:5px">16 </span><span style="font-family: CMR9; font-size:8px">This measures the overlap between the detected and ground-truth characters, and\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9945px; width:75px; height:8px;"><span style="font-family: CMR9; font-size:8px">the maximum is 1.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9955px; width:348px; height:10px;"><span style="font-family: CMR6; font-size:5px">17 </span><span style="font-family: CMR9; font-size:8px">This measures the number of edits from the ground-truth text to the predicted text,\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9967px; width:78px; height:8px;"><span style="font-family: CMR9; font-size:8px">and lower is better.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:9931px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:10154px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:10154px;"><a name="13">Page 13</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:10246px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:10246px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">13\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10398px; width:346px; height:45px;"><span style="font-family: CMR10; font-size:9px">Fig. 6: This lightweight table detector can identify tables (outlined in red) and\n<br>cells (shaded in blue) in diﬀerent locations on a page. In very few cases (d), it\n<br>might generate minor error predictions, e.g, failing to capture the top text line of\n<br></span><span style="font-family: CMR10; font-size:9px">a table.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10478px; width:216px; height:9px;"><span style="font-family: CMBX10; font-size:9px">5.2 A light-weight Visual Table Extractor\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10518px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Detecting tables and parsing their structures (table extraction) are of central im-\n<br>portance for many document digitization tasks. Many previous works [26, 30, 27]\n<br>and tools </span><span style="font-family: CMR7; font-size:6px">18 </span><span style="font-family: CMR10; font-size:9px">have been developed to identify and parse table structures. Yet they\n<br>might require training complicated models from scratch, or are only applicable\n<br>for born-digital PDF documents. In this section, we show how </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can\n<br>help build a light-weight accurate visual table extractor for legal docket tables\n<br>using the existing resources with minimal eﬀort.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10606px; width:347px; height:177px;"><span style="font-family: CMR10; font-size:9px">The extractor uses a pre-trained layout detection model for identifying the\n<br>table regions and some simple rules for pairing the rows and the columns in the\n<br>PDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">Model Zoo can be used for detecting table regions. By ﬁltering\n<br>out model predictions of low conﬁdence and removing overlapping predictions,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can identify the tabular regions on each page, which signiﬁcantly\n<br>simpliﬁes the subsequent steps. By applying the line detection functions within\n<br></span><span style="font-family: CMR10; font-size:9px">the tabular segments, provided in the utility module from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, the\n<br>pipeline can identify the three distinct columns in the tables. A row clustering\n<br>method is then applied via analyzing the y coordinates of token bounding boxes in\n<br>the left-most column, which are obtained from the OCR engines. A non-maximal\n<br>suppression algorithm is used to remove duplicated rows with extremely small\n<br>gaps. Shown in Figure 6, the built pipeline can detect tables at diﬀerent positions\n<br>on a page accurately. Continued tables from diﬀerent pages are concatenated,\n<br>and a structured table representation has been easily created.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:10808px; width:315px; height:10px;"><span style="font-family: CMR6; font-size:5px">18 </span><span style="font-family: CMR9; font-size:8px">https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:134px; top:10269px; width:345px; height:118px;"></div><span style="position:absolute; border: black 1px solid; left:134px; top:10806px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:10996px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:10996px;"><a name="14">Page 14</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11088px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">14\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:11088px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11112px; width:84px; height:11px;"><span style="font-family: CMBX12; font-size:11px">6 Conclusion\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11139px; width:346px; height:117px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a comprehensive toolkit for deep learning-based document\n<br>image analysis. The oﬀ-the-shelf library is easy to install, and can be used to\n<br>build ﬂexible and accurate pipelines for processing documents with complicated\n<br>structures. It also supports high-level customization and enables easy labeling and\n<br>training of DL models on unique document image datasets. The </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">community platform facilitates sharing DL models and DIA pipelines, inviting\n<br>discussion and promoting code reproducibility and reusability. The </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">team is committed to keeping the library updated continuously and bringing\n<br>the most recent advances in DL-based DIA, such as multi-modal document\n<br>modeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11279px; width:347px; height:45px;"><span style="font-family: CMBX10; font-size:9px">Acknowledgements </span><span style="font-family: CMR10; font-size:9px">We thank the anonymous reviewers for their comments\n<br>and suggestions. This project is supported in part by NSF Grant OIA-2033558\n<br>and funding from the Harvard Data Science Initiative and Harvard Catalyst.\n<br>Zejiang Shen thanks Doug Downey for suggestions.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11347px; width:62px; height:11px;"><span style="font-family: CMBX12; font-size:11px">References\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11375px; width:341px; height:85px;"><span style="font-family: CMR9; font-size:8px">[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado,\n<br>G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A.,\n<br>Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg,\n<br>J., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J.,\n<br>Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V.,\n<br>Vi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng,\n<br>X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015),\n<br></span><span style="font-family: CMTT9; font-size:8px">https://www.tensorflow.org/</span><span style="font-family: CMR9; font-size:8px">, software available from tensorﬂow.org\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11463px; width:341px; height:41px;"><span style="font-family: CMR9; font-size:8px">[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a\n<br>highly-functional python framework for reproducible experiments. In: 2018 16th\n<br>International Conference on Frontiers in Handwriting Recognition (ICFHR). pp.\n<br>423–428. IEEE (2018)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11507px; width:341px; height:41px;"><span style="font-family: CMR9; font-size:8px">[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic\n<br>dataset for performance evaluation of document layout analysis. In: 2009 10th\n<br></span><span style="font-family: CMR9; font-size:8px">International Conference on Document Analysis and Recognition. pp. 296–300.\n<br></span><span style="font-family: CMR9; font-size:8px">IEEE (2009)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11551px; width:340px; height:30px;"><span style="font-family: CMR9; font-size:8px">[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text\n<br>detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and\n<br>Pattern Recognition. pp. 9365–9374 (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11585px; width:340px; height:8px;"><span style="font-family: CMR9; font-size:8px">[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:11596px; width:200px; height:8px;"><span style="font-family: CMR9; font-size:8px">Hierarchical Image Database. In: CVPR09 (2009)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11607px; width:341px; height:30px;"><span style="font-family: CMR9; font-size:8px">[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with\n<br>coarse-to-ﬁne attention. In: International Conference on Machine Learning. pp.\n<br>980–989. PMLR (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11640px; width:341px; height:19px;"><span style="font-family: CMR9; font-size:8px">[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation.\n<br>In: International conference on machine learning. pp. 1180–1189. PMLR (2015)\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:11838px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:11838px;"><a name="15">Page 15</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:11930px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:11930px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">15\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11956px; width:341px; height:63px;"><span style="font-family: CMR9; font-size:8px">[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters,\n<br>M., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language\n<br>processing platform. arXiv preprint arXiv:1803.07640 (2018)\n<br>(cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P.,\n<br>Grali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in-\n<br>formation extraction (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11989px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">[9]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12022px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal\n<br>classiﬁcation: labelling unsegmented sequence data with recurrent neural networks.\n<br>In: Proceedings of the 23rd international conference on Machine learning. pp.\n<br>369–376 (2006)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12066px; width:344px; height:41px;"><span style="font-family: CMR9; font-size:8px">[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for\n<br>document image classiﬁcation and retrieval. In: 2015 13th International Conference\n<br>on Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015)\n<br>[12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:12110px; width:293px; height:8px;"><span style="font-family: CMR9; font-size:8px">IEEE international conference on computer vision. pp. 2961–2969 (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12121px; width:346px; height:30px;"><span style="font-family: CMR9; font-size:8px">[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.\n<br>In: Proceedings of the IEEE conference on computer vision and pattern recognition.\n<br>pp. 770–778 (2016)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12153px; width:346px; height:8px;"><span style="font-family: CMR9; font-size:8px">[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:12164px; width:99px; height:8px;"><span style="font-family: CMBX9; font-size:8px">2007</span><span style="font-family: CMR9; font-size:8px">(159), 2 (Jul 2007)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12175px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis\n<br>benchmarking. In: 2011 International Conference on Document Analysis and\n<br>Recognition. pp. 42–47. IEEE (2011)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12208px; width:347px; height:64px;"><span style="font-family: CMR9; font-size:8px">[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5\n<br>million images. In: Adjunct Publication of the 33rd Annual ACM Sym-\n<br>posium on User\n<br>Interface Software and Technology. p. 120–122. UIST\n<br>’20 Adjunct, Association for Computing Machinery, New York, NY, USA\n<br>(2020). https://doi.org/10.1145/3379350.3416143, </span><span style="font-family: CMTT9; font-size:8px">https://doi-org.offcampus.\n<br>lib.washington.edu/10.1145/3379350.3416143\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12274px; width:345px; height:53px;"><span style="font-family: CMR9; font-size:8px">[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N.,\n<br>Thomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting\n<br>Headlines and Visual Content from 16 Million Historic Newspaper Pages in\n<br>Chronicling America, p. 3055–3062. Association for Computing Machinery, New\n<br>York, NY, USA (2020), </span><span style="font-family: CMTT9; font-size:8px">https://doi.org/10.1145/3340531.3412767\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12329px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark\n<br>for image-based table detection and recognition. arXiv preprint arXiv:1903.01949\n<br>(2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12362px; width:345px; height:30px;"><span style="font-family: CMR9; font-size:8px">[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P.,\n<br></span><span style="font-family: CMR9; font-size:8px">Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference\n<br></span><span style="font-family: CMR9; font-size:8px">on computer vision. pp. 740–755. Springer (2014)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12395px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic\n<br>segmentation. In: Proceedings of the IEEE conference on computer vision and\n<br>pattern recognition. pp. 3431–3440 (2015)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12427px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Suﬁ, S., Williams, A., Wolsten-\n<br>croft, K.: An experimental workﬂow development platform for historical document\n<br>digitisation and analysis. In: Proceedings of the 2011 workshop on historical\n<br>document imaging and processing. pp. 161–168 (2011)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12471px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach\n<br>for document segmentation. In: 2018 16th International Conference on Frontiers\n<br>in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:12680px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:12680px;"><a name="16">Page 16</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:12772px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">16\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:12772px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12798px; width:345px; height:85px;"><span style="font-family: CMR9; font-size:8px">[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z.,\n<br>Desmaison, A., Antiga, L., Lerer, A.: Automatic diﬀerentiation in pytorch (2017)\n<br>[24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen,\n<br>T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style,\n<br>high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019)\n<br>[25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth\n<br>elements) format framework. In: 2010 20th International Conference on Pattern\n<br>Recognition. pp. 257–260. IEEE (2010)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12886px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet:\n<br>An approach for end to end table detection and structure recognition from image-\n<br>based documents. In: Proceedings of the IEEE/CVF Conference on Computer\n<br>Vision and Pattern Recognition Workshops. pp. 572–573 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12930px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph\n<br>neural networks. In: 2019 International Conference on Document Analysis and\n<br>Recognition (ICDAR). pp. 142–147. IEEE (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12962px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object\n<br>detection with region proposal networks. In: Advances in neural information\n<br>processing systems. pp. 91–99 (2015)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12995px; width:346px; height:63px;"><span style="font-family: CMR9; font-size:8px">[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph\n<br>neural network model. IEEE transactions on neural networks </span><span style="font-family: CMBX9; font-size:8px">20</span><span style="font-family: CMR9; font-size:8px">(1), 61–80 (2008)\n<br>[30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning\n<br>for detection and structure recognition of tables in document images. In: 2017 14th\n<br>IAPR international conference on document analysis and recognition (ICDAR).\n<br>vol. 1, pp. 1162–1167. IEEE (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13061px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents\n<br>with complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer\n<br>Vision and Pattern Recognition Workshops. pp. 548–549 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13094px; width:344px; height:8px;"><span style="font-family: CMR9; font-size:8px">[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13105px; width:261px; height:8px;"><span style="font-family: CMR9; font-size:8px">based layout annotation. arXiv preprint arXiv:2010.01762 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13116px; width:345px; height:41px;"><span style="font-family: CMR9; font-size:8px">[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer,\n<br>A., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for\n<br>historical document image analysis. In: 2019 International Conference on Document\n<br>Analysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13160px; width:346px; height:42px;"><span style="font-family: CMR9; font-size:8px">[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P.,\n<br>Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-\n<br>the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)\n<br>[35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. </span><span style="font-family: CMTT9; font-size:8px">https://\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13204px; width:207px; height:9px;"><span style="font-family: CMTT9; font-size:8px">github.com/facebookresearch/detectron2 </span><span style="font-family: CMR9; font-size:8px">(2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13215px; width:345px; height:30px;"><span style="font-family: CMR9; font-size:8px">[36] </span><span style="font-family: CMR9; font-size:8px">Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C.,\n<br></span><span style="font-family: CMR9; font-size:8px">Che, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document\n<br>understanding. arXiv preprint arXiv:2012.14740 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13247px; width:344px; height:8px;"><span style="font-family: CMR9; font-size:8px">[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13258px; width:234px; height:8px;"><span style="font-family: CMR9; font-size:8px">text and layout for document image understanding (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13269px; width:216px; height:8px;"><span style="font-family: CMR9; font-size:8px">[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:188px; top:13280px; width:67px; height:8px;"><span style="font-family: CMR9; font-size:8px">layout analysis.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:13280px; width:237px; height:30px;"><span style="font-family: CMR9; font-size:8px">ument\n<br>Analysis and Recognition (ICDAR). pp. 1015–1022.\n<br>https://doi.org/10.1109/ICDAR.2019.00166\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:263px; top:13269px; width:219px; height:30px;"><span style="font-family: CMR9; font-size:8px">largest dataset ever for doc-\n<br>In: 2019 International Conference on Document\n<br>IEEE (Sep 2019).\n<br></span></div><div style="position:absolute; top:0px;">Page: <a href="#1">1</a>, <a href="#2">2</a>, <a href="#3">3</a>, <a href="#4">4</a>, <a href="#5">5</a>, <a href="#6">6</a>, <a href="#7">7</a>, <a href="#8">8</a>, <a href="#9">9</a>, <a href="#10">10</a>, <a href="#11">11</a>, <a href="#12">12</a>, <a href="#13">13</a>, <a href="#14">14</a>, <a href="#15">15</a>, <a href="#16">16</a></div>\n</body></html>\n', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'})

from bs4 import BeautifulSoup

soup = BeautifulSoup(docs[0].page_content, "html.parser")
content = soup.find_all("div")

import re

cur_fs = None
cur_text = ""
snippets = []  # first collect all snippets that have the same font size
for c in content:
    sp = c.find("span")
    if not sp:
        continue
    st = sp.get("style")
    if not st:
        continue
    fs = re.findall("font-size:(\d+)px", st)
    if not fs:
        continue
    fs = int(fs[0])
    if not cur_fs:
        cur_fs = fs
    if fs == cur_fs:
        cur_text += c.text
    else:
        snippets.append((cur_text, cur_fs))
        cur_fs = fs
        cur_text = c.text
snippets.append((cur_text, cur_fs))
# Note: The above logic is very straightforward. One can also add more strategies such as removing duplicate snippets (as
# headers/footers in a PDF appear on multiple pages so if we find duplicates it's safe to assume that it is redundant info)

from langchain_core.documents import Document

cur_idx = -1
semantic_snippets = []
# Assumption: headings have higher font size than their respective content
for s in snippets:
    # if current snippet's font size > previous section's heading => it is a new heading
    if (
        not semantic_snippets
        or s[1] > semantic_snippets[cur_idx].metadata["heading_font"]
    ):
        metadata = {"heading": s[0], "content_font": 0, "heading_font": s[1]}
        metadata.update(docs[0].metadata)
        semantic_snippets.append(Document(page_content="", metadata=metadata))
        cur_idx += 1
        continue

    # if current snippet's font size <= previous section's content => content belongs to the same section (one can also create
    # a tree like structure for sub sections if needed but that may require some more thinking and may be data specific)
    if (
        not semantic_snippets[cur_idx].metadata["content_font"]
        or s[1] <= semantic_snippets[cur_idx].metadata["content_font"]
    ):
        semantic_snippets[cur_idx].page_content += s[0]
        semantic_snippets[cur_idx].metadata["content_font"] = max(
            s[1], semantic_snippets[cur_idx].metadata["content_font"]
        )
        continue

    # if current snippet's font size > previous section's content but less than previous section's heading than also make a new
    # section (e.g. title of a PDF will have the highest font size but we don't want it to subsume all sections)
    metadata = {"heading": s[0], "content_font": 0, "heading_font": s[1]}
    metadata.update(docs[0].metadata)
    semantic_snippets.append(Document(page_content="", metadata=metadata))
    cur_idx += 1

print(semantic_snippets[4])

API Reference:Document

page_content='Recently, various DL models and datasets have been developed for layout analysis\ntasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen-\ntation tasks on historical documents. Object detection-based methods like Faster\nR-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38]\nand detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also\nbeen used in table detection [27]. However, these models are usually implemented\nindividually and there is no uniﬁed framework to load and use such models.\nThere has been a surge of interest in creating open-source tools for document\nimage processing: a search of document image analysis in Github leads to 5M\nrelevant code pieces 6; yet most of them rely on traditional rule-based methods\nor provide limited functionalities. The closest prior research to our work is the\nOCR-D project7, which also tries to build a complete toolkit for DIA. However,\nsimilar to the platform developed by Neudecker et al. [21], it is designed for\nanalyzing historical documents, and provides no supports for recent DL models.\nThe DocumentLayoutAnalysis project8 focuses on processing born-digital PDF\ndocuments via analyzing the stored PDF data. Repositories like DeepLayout9\nand Detectron2-PubLayNet10 are individual deep learning models trained on\nlayout analysis datasets without support for the full DIA pipeline. The Document\nAnalysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2]\naim to improve the reproducibility of DIA methods (or DL models), yet they\nare not actively maintained. OCR engines like Tesseract [14], easyOCR11 and\npaddleOCR12 usually do not come with comprehensive functionalities for other\nDIA tasks like layout analysis.\nRecent years have also seen numerous eﬀorts to create libraries for promoting\nreproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\n6 The number shown is obtained by specifying the search type as ‘code’.\n7 https://ocr-d.de/en/about\n8 https://github.com/BobLd/DocumentLayoutAnalysis\n9 https://github.com/leonlulu/DeepLayout\n10 https://github.com/hpanwar08/detectron2\n11 https://github.com/JaidedAI/EasyOCR\n12 https://github.com/PaddlePaddle/PaddleOCR\n4\nZ. Shen et al.\nFig. 1: The overall architecture of LayoutParser. For an input document image,\nthe core LayoutParser library provides a set of oﬀ-the-shelf tools for layout\ndetection, OCR, visualization, and storage, backed by a carefully designed layout\ndata structure. LayoutParser also supports high level customization via eﬃcient\nlayout annotation and model training functions. These improve model accuracy\non the target samples. The community platform enables the easy sharing of DIA\nmodels and whole digitization pipelines to promote reusability and reproducibility.\nA collection of detailed documentation, tutorials and exemplar projects make\nLayoutParser easy to learn and use.\nAllenNLP [8] and transformers [34] have provided the community with complete\nDL-based support for developing and deploying models for general computer\nvision and natural language processing problems. LayoutParser, on the other\nhand, specializes speciﬁcally in DIA tasks. LayoutParser is also equipped with a\ncommunity platform inspired by established model hubs such as Torch Hub [23]\nand TensorFlow Hub [1]. It enables the sharing of pretrained models as well as\nfull document processing pipelines that are unique to DIA tasks.\nThere have been a variety of document data collections to facilitate the\ndevelopment of DL models. Some examples include PRImA [3](magazine layouts),\nPubLayNet [38](academic paper layouts), Table Bank [18](tables in academic\npapers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and\nHJDataset [31](historical Japanese document layouts). A spectrum of models\ntrained on these datasets are currently available in the LayoutParser model zoo\nto support diﬀerent use cases.\n' metadata={'heading': '2 Related Work\n', 'content_font': 9, 'heading_font': 11, 'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'}

PyPDF Directory

Load PDFs from directory

from langchain_community.document_loaders import PyPDFDirectoryLoader

directory_path = (
    "../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf"
)
loader = PyPDFDirectoryLoader("example_data/")

docs = loader.load()

data[0]

API Reference:PyPDFDirectoryLoader

Document(page_content='<html><head>\n<meta http-equiv="Content-Type" content="text/html">\n</head><body>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:50px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:50px;"><a name="1">Page 1</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:263px; width:20px; height:40px;"><span style="font-family: Times-Roman; font-size:10px">1\n<br>2\n<br>0\n<br>2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:308px; width:20px; height:27px;"><span style="font-family: Times-Roman; font-size:10px">n\n<br>u\n<br></span><span style="font-family: Times-Roman; font-size:7px">J\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:341px; width:20px; height:20px;"><span style="font-family: Times-Roman; font-size:10px">1\n<br>2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:371px; width:20px; height:6px;"><span style="font-family: Times-Roman; font-size:6px">]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:377px; width:20px; height:56px;"><span style="font-family: Times-Roman; font-size:14px">V\n<br></span><span style="font-family: Times-Roman; font-size:13px">C\n<br></span><span style="font-family: Times-Roman; font-size:5px">.\n<br></span><span style="font-family: Times-Roman; font-size:7px">s\n<br></span><span style="font-family: Times-Roman; font-size:8px">c\n<br></span><span style="font-family: Times-Roman; font-size:6px">[\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:16px; top:443px; width:20px; height:166px;"><span style="font-family: Times-Roman; font-size:10px">2\n<br>v\n<br>8\n<br>4\n<br>3\n<br>5\n<br>1\n<br></span><span style="font-family: Times-Roman; font-size:5px">.\n<br></span><span style="font-family: Times-Roman; font-size:10px">3\n<br>0\n<br>1\n<br>2\n<br></span><span style="font-family: Times-Roman; font-size:5px">:\n<br></span><span style="font-family: Times-Roman; font-size:10px">v\n<br></span><span style="font-family: Times-Roman; font-size:5px">i\n<br></span><span style="font-family: Times-Roman; font-size:14px">X\n<br></span><span style="font-family: Times-Roman; font-size:6px">r\n<br></span><span style="font-family: Times-Roman; font-size:8px">a\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:157px; top:164px; width:300px; height:32px;"><span style="font-family: CMTT12; font-size:14px">LayoutParser</span><span style="font-family: CMBX12; font-size:14px">: A Uniﬁed Toolkit for Deep\n<br>Learning Based Document Image Analysis\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:218px; width:345px; height:23px;"><span style="font-family: CMR10; font-size:9px">Zejiang Shen</span><span style="font-family: CMR7; font-size:6px">1 </span><span style="font-family: CMR10; font-size:9px">(</span><span style="font-family: unknown; font-size:9px">(cid:0)</span><span style="font-family: CMR10; font-size:9px">), Ruochen Zhang</span><span style="font-family: CMR7; font-size:6px">2</span><span style="font-family: CMR10; font-size:9px">, Melissa Dell</span><span style="font-family: CMR7; font-size:6px">3</span><span style="font-family: CMR10; font-size:9px">, Benjamin Charles Germain\n<br>Lee</span><span style="font-family: CMR7; font-size:6px">4</span><span style="font-family: CMR10; font-size:9px">, Jacob Carlson</span><span style="font-family: CMR7; font-size:6px">3</span><span style="font-family: CMR10; font-size:9px">, and Weining Li</span><span style="font-family: CMR7; font-size:6px">5\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:207px; top:252px; width:200px; height:109px;"><span style="font-family: CMR6; font-size:5px">1 </span><span style="font-family: CMR9; font-size:8px">Allen Institute for AI\n<br></span><span style="font-family: CMTT9; font-size:8px">shannons@allenai.org\n<br></span><span style="font-family: CMR6; font-size:5px">2 </span><span style="font-family: CMR9; font-size:8px">Brown University\n<br></span><span style="font-family: CMTT9; font-size:8px">ruochen zhang@brown.edu\n<br></span><span style="font-family: CMR6; font-size:5px">3 </span><span style="font-family: CMR9; font-size:8px">Harvard University\n<br></span><span style="font-family: CMSY9; font-size:8px">{</span><span style="font-family: CMTT9; font-size:8px">melissadell,jacob carlson</span><span style="font-family: CMSY9; font-size:8px">}</span><span style="font-family: CMTT9; font-size:8px">@fas.harvard.edu\n<br></span><span style="font-family: CMR6; font-size:5px">4 </span><span style="font-family: CMR9; font-size:8px">University of Washington\n<br></span><span style="font-family: CMTT9; font-size:8px">bcgl@cs.washington.edu\n<br></span><span style="font-family: CMR6; font-size:5px">5 </span><span style="font-family: CMR9; font-size:8px">University of Waterloo\n<br></span><span style="font-family: CMTT9; font-size:8px">w422li@uwaterloo.ca\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:162px; top:388px; width:291px; height:228px;"><span style="font-family: CMBX9; font-size:8px">Abstract. </span><span style="font-family: CMR9; font-size:8px">Recent advances in document image analysis (DIA) have been\n<br>primarily driven by the application of neural networks. Ideally, research\n<br>outcomes could be easily deployed in production and extended for further\n<br>investigation. However, various factors like loosely organized codebases\n<br>and sophisticated model conﬁgurations complicate the easy reuse of im-\n<br>portant innovations by a wide audience. Though there have been on-going\n<br>eﬀorts to improve reusability and simplify deep learning (DL) model\n<br>development in disciplines like natural language processing and computer\n<br>vision, none of them are optimized for challenges in the domain of DIA.\n<br>This represents a major gap in the existing toolkit, as DIA is central to\n<br>academic research across a wide range of disciplines in the social sciences\n<br>and humanities. This paper introduces </span><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">, an open-source\n<br>library for streamlining the usage of DL in DIA research and applica-\n<br>tions. The core </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">library comes with a set of simple and\n<br>intuitive interfaces for applying and customizing DL models for layout de-\n<br>tection, character recognition, and many other document processing tasks.\n<br>To promote extensibility, </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">also incorporates a community\n<br>platform for sharing both pre-trained models and full document digiti-\n<br></span><span style="font-family: CMR9; font-size:8px">zation pipelines. We demonstrate that </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">is helpful for both\n<br>lightweight and large-scale digitization pipelines in real-word use cases.\n<br>The library is publicly available at </span><span style="font-family: CMTT9; font-size:8px">https://layout-parser.github.io</span><span style="font-family: CMR9; font-size:8px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:162px; top:627px; width:289px; height:21px;"><span style="font-family: CMBX9; font-size:8px">Keywords: </span><span style="font-family: CMR9; font-size:8px">Document Image Analysis </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Deep Learning </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Layout Analysis\n<br></span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Character Recognition </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Open Source library </span><span style="font-family: SFRM0900; font-size:8px">· </span><span style="font-family: CMR9; font-size:8px">Toolkit.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:669px; width:6px; height:11px;"><span style="font-family: CMBX12; font-size:11px">1\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:669px; width:74px; height:11px;"><span style="font-family: CMBX12; font-size:11px">Introduction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:692px; width:347px; height:21px;"><span style="font-family: CMR10; font-size:9px">Deep Learning(DL)-based approaches are the state-of-the-art for a wide range of\n<br></span><span style="font-family: CMR10; font-size:9px">document image analysis (DIA) tasks including document image classiﬁcation [11,\n<br></span></div><span style="position:absolute; border: black 1px solid; left:287px; top:293px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:315px; width:2px; height:0px;"></span>\n<span style="font-family: Times-Roman; font-size:5px"> \n<br> \n<br> \n<br> \n<br> \n<br> \n<br><span style="position:absolute; border: gray 1px solid; left:0px; top:892px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:892px;"><a name="2">Page 2</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:984px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:984px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1009px; width:348px; height:57px;"><span style="font-family: CMR10; font-size:9px">37], layout detection [38, 22], table detection [26], and scene text detection [4].\n<br>A generalized learning-based framework dramatically reduces the need for the\n<br>manual speciﬁcation of complicated rules, which is the status quo with traditional\n<br>methods. DL has the potential to transform DIA pipelines and beneﬁt a broad\n<br>spectrum of large-scale document digitization projects.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1069px; width:348px; height:213px;"><span style="font-family: CMR10; font-size:9px">However, there are several practical diﬃculties for taking advantages of re-\n<br>cent advances in DL-based methods: 1) DL models are notoriously convoluted\n<br>for reuse and extension. Existing models are developed using distinct frame-\n<br>works like TensorFlow [1] or PyTorch [24], and the high-level parameters can\n<br>be obfuscated by implementation details [8]. It can be a time-consuming and\n<br>frustrating experience to debug, reproduce, and adapt existing models for DIA,\n<br>and </span><span style="font-family: CMTI10; font-size:9px">many researchers who would beneﬁt the most from using these methods lack\n<br>the technical background to implement them from scratch. </span><span style="font-family: CMR10; font-size:9px">2) Document images\n<br></span><span style="font-family: CMR10; font-size:9px">contain diverse and disparate patterns across domains, and customized training\n<br></span><span style="font-family: CMR10; font-size:9px">is often required to achieve a desirable detection accuracy. Currently </span><span style="font-family: CMTI10; font-size:9px">there is no\n<br>full-ﬂedged infrastructure for easily curating the target document image datasets\n<br>and ﬁne-tuning or re-training the models. </span><span style="font-family: CMR10; font-size:9px">3) DIA usually requires a sequence of\n<br>models and other processing to obtain the ﬁnal outputs. Often research teams use\n<br>DL models and then perform further document analyses in separate processes,\n<br>and these pipelines are not documented in any central location (and often not\n<br>documented at all). This makes it </span><span style="font-family: CMTI10; font-size:9px">diﬃcult for research teams to learn about how\n<br>full pipelines are implemented </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTI10; font-size:9px">leads them to invest signiﬁcant resources in\n<br>reinventing the DIA wheel</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1284px; width:346px; height:33px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a uniﬁed toolkit to support DL-based document image\n<br>analysis and processing. To address the aforementioned challenges, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">is built with the following components:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1326px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">1. An oﬀ-the-shelf toolkit for applying DL models for layout detection, character\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1338px; width:194px; height:9px;"><span style="font-family: CMR10; font-size:9px">recognition, and other DIA tasks (Section 3)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1350px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">2. A rich repository of pre-trained neural network models (Model Zoo) that\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1362px; width:137px; height:9px;"><span style="font-family: CMR10; font-size:9px">underlies the oﬀ-the-shelf usage\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1373px; width:341px; height:9px;"><span style="font-family: CMR10; font-size:9px">3. Comprehensive tools for eﬃcient document image data annotation and model\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:151px; top:1385px; width:218px; height:9px;"><span style="font-family: CMR10; font-size:9px">tuning to support diﬀerent levels of customization\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:138px; top:1397px; width:343px; height:33px;"><span style="font-family: CMR10; font-size:9px">4. A DL model hub and community platform for the easy sharing, distribu-\n<br>tion, and discussion of DIA models and pipelines, to promote reusability,\n<br>reproducibility, and extensibility (Section 4)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1439px; width:346px; height:69px;"><span style="font-family: CMR10; font-size:9px">The library implements simple and intuitive Python APIs without sacriﬁcing\n<br>generalizability and versatility, and can be easily installed via pip. Its convenient\n<br>functions for handling document image data can be seamlessly integrated with\n<br>existing DIA pipelines. With detailed documentations and carefully curated\n<br>tutorials, we hope this tool will beneﬁt a variety of end-users, and will lead to\n<br>advances in applications in both industry and academic research.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1511px; width:347px; height:46px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is well aligned with recent eﬀorts for improving DL model\n<br>reusability in other disciplines like natural language processing [8, 34] and com-\n<br>puter vision [35], but with a focus on unique challenges in DIA. We show\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can be applied in sophisticated and large-scale digitization projects\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:1734px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:1734px;"><a name="3">Page 3</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:1826px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:1826px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">3\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1851px; width:348px; height:57px;"><span style="font-family: CMR10; font-size:9px">that require precision, eﬃciency, and robustness, as well as simple and light-\n<br>weight document processing tasks focusing on eﬃcacy and ﬂexibility (Section 5).\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is being actively maintained, and support for more deep learning\n<br>models and novel methods in text-based layout analysis methods [37, 34] is\n<br>planned.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1911px; width:347px; height:58px;"><span style="font-family: CMR10; font-size:9px">The rest of the paper is organized as follows. Section 2 provides an overview\n<br>of related work. The core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library, DL Model Zoo, and customized\n<br>model training are described in Section 3, and the DL model hub and commu-\n<br>nity platform are detailed in Section 4. Section 5 shows two examples of how\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can be used in practical DIA projects, and Section 6 concludes.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:1990px; width:102px; height:11px;"><span style="font-family: CMBX12; font-size:11px">2 Related Work\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2016px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Recently, various DL models and datasets have been developed for layout analysis\n<br>tasks. The dhSegment [22] utilizes fully convolutional networks [20] for segmen-\n<br>tation tasks on historical documents. Object detection-based methods like Faster\n<br>R-CNN [28] and Mask R-CNN [12] are used for identifying document elements [38]\n<br>and detecting tables [30, 26]. Most recently, Graph Neural Networks [29] have also\n<br>been used in table detection [27]. However, these models are usually implemented\n<br>individually and there is no uniﬁed framework to load and use such models.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2100px; width:348px; height:189px;"><span style="font-family: CMR10; font-size:9px">There has been a surge of interest in creating open-source tools for document\n<br>image processing: a search of </span><span style="font-family: CMTT10; font-size:9px">document image analysis </span><span style="font-family: CMR10; font-size:9px">in Github leads to 5M\n<br>relevant code pieces </span><span style="font-family: CMR7; font-size:6px">6</span><span style="font-family: CMR10; font-size:9px">; yet most of them rely on traditional rule-based methods\n<br>or provide limited functionalities. The closest prior research to our work is the\n<br>OCR-D project</span><span style="font-family: CMR7; font-size:6px">7</span><span style="font-family: CMR10; font-size:9px">, which also tries to build a complete toolkit for DIA. However,\n<br>similar to the platform developed by Neudecker et al. [21], it is designed for\n<br>analyzing historical documents, and provides no supports for recent DL models.\n<br>The </span><span style="font-family: CMTT10; font-size:9px">DocumentLayoutAnalysis </span><span style="font-family: CMR10; font-size:9px">project</span><span style="font-family: CMR7; font-size:6px">8 </span><span style="font-family: CMR10; font-size:9px">focuses on processing born-digital PDF\n<br>documents via analyzing the stored PDF data. Repositories like </span><span style="font-family: CMTT10; font-size:9px">DeepLayout</span><span style="font-family: CMR7; font-size:6px">9\n<br></span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Detectron2-PubLayNet</span><span style="font-family: CMR7; font-size:6px">10 </span><span style="font-family: CMR10; font-size:9px">are individual deep learning models trained on\n<br>layout analysis datasets without support for the full DIA pipeline. The Document\n<br>Analysis and Exploitation (DAE) platform [15] and the DeepDIVA project [2]\n<br>aim to improve the reproducibility of DIA methods (or DL models), yet they\n<br>are not actively maintained. OCR engines like </span><span style="font-family: CMTT10; font-size:9px">Tesseract </span><span style="font-family: CMR10; font-size:9px">[14], </span><span style="font-family: CMTT10; font-size:9px">easyOCR</span><span style="font-family: CMR7; font-size:6px">11 </span><span style="font-family: CMR10; font-size:9px">and\n<br></span><span style="font-family: CMTT10; font-size:9px">paddleOCR</span><span style="font-family: CMR7; font-size:6px">12 </span><span style="font-family: CMR10; font-size:9px">usually do not come with comprehensive functionalities for other\n<br>DIA tasks like layout analysis.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2291px; width:347px; height:21px;"><span style="font-family: CMR10; font-size:9px">Recent years have also seen numerous eﬀorts to create libraries for promoting\n<br>reproducibility and reusability in the ﬁeld of DL. Libraries like Dectectron2 [35],\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:2322px; width:295px; height:76px;"><span style="font-family: CMR6; font-size:5px">6 </span><span style="font-family: CMR9; font-size:8px">The number shown is obtained by specifying the search type as ‘code’.\n<br></span><span style="font-family: CMR6; font-size:5px">7 </span><span style="font-family: CMR9; font-size:8px">https://ocr-d.de/en/about\n<br></span><span style="font-family: CMR6; font-size:5px">8 </span><span style="font-family: CMR9; font-size:8px">https://github.com/BobLd/DocumentLayoutAnalysis\n<br></span><span style="font-family: CMR6; font-size:5px">9 </span><span style="font-family: CMR9; font-size:8px">https://github.com/leonlulu/DeepLayout\n<br></span><span style="font-family: CMR6; font-size:5px">10 </span><span style="font-family: CMR9; font-size:8px">https://github.com/hpanwar08/detectron2\n<br></span><span style="font-family: CMR6; font-size:5px">11 </span><span style="font-family: CMR9; font-size:8px">https://github.com/JaidedAI/EasyOCR\n<br></span><span style="font-family: CMR6; font-size:5px">12 </span><span style="font-family: CMR9; font-size:8px">https://github.com/PaddlePaddle/PaddleOCR\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:2320px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:2576px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:2576px;"><a name="4">Page 4</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2668px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">4\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:2668px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2838px; width:348px; height:105px;"><span style="font-family: CMR10; font-size:9px">Fig. 1: The overall architecture of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. For an input document image,\n<br></span><span style="font-family: CMR10; font-size:9px">the core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library provides a set of oﬀ-the-shelf tools for layout\n<br>detection, OCR, visualization, and storage, backed by a carefully designed layout\n<br>data structure. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also supports high level customization via eﬃcient\n<br>layout annotation and model training functions. These improve model accuracy\n<br>on the target samples. The community platform enables the easy sharing of DIA\n<br>models and whole digitization pipelines to promote reusability and reproducibility.\n<br>A collection of detailed documentation, tutorials and exemplar projects make\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">easy to learn and use.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:2970px; width:346px; height:81px;"><span style="font-family: CMR10; font-size:9px">AllenNLP [8] and transformers [34] have provided the community with complete\n<br>DL-based support for developing and deploying models for general computer\n<br>vision and natural language processing problems. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, on the other\n<br>hand, specializes speciﬁcally in DIA tasks. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is also equipped with a\n<br>community platform inspired by established model hubs such as </span><span style="font-family: CMTT10; font-size:9px">Torch Hub </span><span style="font-family: CMR10; font-size:9px">[23]\n<br>and </span><span style="font-family: CMTT10; font-size:9px">TensorFlow Hub </span><span style="font-family: CMR10; font-size:9px">[1]. It enables the sharing of pretrained models as well as\n<br>full document processing pipelines that are unique to DIA tasks.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3054px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">There have been a variety of document data collections to facilitate the\n<br>development of DL models. Some examples include PRImA [3](magazine layouts),\n<br>PubLayNet [38](academic paper layouts), Table Bank [18](tables in academic\n<br>papers), Newspaper Navigator Dataset [16, 17](newspaper ﬁgure layouts) and\n<br></span><span style="font-family: CMTT10; font-size:9px">HJDataset </span><span style="font-family: CMR10; font-size:9px">[31](historical Japanese document layouts). A spectrum of models\n<br></span><span style="font-family: CMR10; font-size:9px">trained on these datasets are currently available in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">model zoo\n<br>to support diﬀerent use cases.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3156px; width:202px; height:12px;"><span style="font-family: CMBX12; font-size:11px">3 The Core </span><span style="font-family: CMTT12; font-size:11px">LayoutParser </span><span style="font-family: CMBX12; font-size:11px">Library\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3183px; width:347px; height:57px;"><span style="font-family: CMR10; font-size:9px">At the core of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is an oﬀ-the-shelf toolkit that streamlines DL-\n<br>based document image analysis. Five components support a simple interface\n<br>with comprehensive functionalities: 1) The </span><span style="font-family: CMTI10; font-size:9px">layout detection models </span><span style="font-family: CMR10; font-size:9px">enable using\n<br>pre-trained or self-trained DL models for layout detection with just four lines\n<br></span><span style="font-family: CMR10; font-size:9px">of code. 2) The detected layout information is stored in carefully engineered\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:169px; top:2691px; width:276px; height:136px;"><span style="position:absolute; border: black 1px solid; left:169px; top:2691px; width:276px; height:136px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2701px; width:77px; height:66px;"></span>\n<span style="position:absolute; border: black 1px solid; left:185px; top:2719px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2719px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2736px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:194px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:199px; top:2726px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:199px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2727px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2726px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:213px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:217px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:217px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:222px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:232px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2729px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2727px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2726px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2727px; width:2px; height:3px;"></span>\n<span style="font-family: unknown; font-size:6px">Efficient Data Annotation<span style="position:absolute; border: black 1px solid; left:185px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:187px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:187px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:196px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:2752px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:205px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:206px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:208px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:208px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:209px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2751px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:222px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:225px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:228px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:233px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:233px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:236px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:238px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:238px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:243px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:247px; top:2752px; width:1px; height:2px;"></span>\nCustomized Model Training<span style="position:absolute; border: black 1px solid; left:179px; top:2701px; width:77px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2707px; width:77px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:188px; top:2705px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:193px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:194px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:195px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:197px; top:2705px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:198px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:201px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:201px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:202px; top:2707px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2705px; width:0px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:212px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:218px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:221px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:224px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:224px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:234px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:235px; top:2708px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:237px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:237px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:240px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:241px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:242px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:245px; top:2706px; width:2px; height:3px;"></span>\nModel Customization<span style="position:absolute; border: black 1px solid; left:358px; top:2701px; width:77px; height:66px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2719px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2719px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2736px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2724px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2732px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2719px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:379px; top:2726px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2726px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2726px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2727px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:400px; top:2727px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2726px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2726px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2727px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:411px; top:2730px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2726px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2727px; width:1px; height:2px;"></span>\nDIA Model Hub<span style="position:absolute; border: black 1px solid; left:363px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:424px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:362px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2751px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:377px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:383px; top:2751px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:387px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:392px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:395px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:399px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:403px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:403px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:414px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2752px; width:1px; height:2px;"></span>\nDIA Pipeline Sharing<span style="position:absolute; border: black 1px solid; left:358px; top:2701px; width:77px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:358px; top:2707px; width:77px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:367px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:367px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:375px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:380px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2706px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:386px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2706px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2705px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:396px; top:2706px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2705px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:402px; top:2706px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:406px; top:2705px; width:0px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2706px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2708px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2706px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2705px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2706px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:416px; top:2707px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2706px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:420px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:421px; top:2706px; width:4px; height:3px;"></span>\nCommunity Platform<span style="position:absolute; border: black 1px solid; left:179px; top:2774px; width:256px; height:43px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2737px; width:83px; height:36px;"></span>\n<span style="position:absolute; border: black 1px solid; left:274px; top:2744px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2744px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2761px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2749px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2757px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2744px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:277px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:277px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2754px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2752px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:291px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2751px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2751px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:304px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:306px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:306px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:309px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2752px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2751px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:312px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:312px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2751px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:325px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2751px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:328px; top:2752px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2752px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2752px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2751px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2755px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2752px; width:2px; height:3px;"></span>\nLayout Detection Models<span style="position:absolute; border: black 1px solid; left:281px; top:2701px; width:52px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2701px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2701px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2719px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2719px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2701px; width:52px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2701px; width:1px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2719px; width:52px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:280px; top:2701px; width:1px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2708px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2709px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:2710px; width:0px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:294px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:297px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:297px; top:2709px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2708px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2708px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:313px; top:2709px; width:4px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2711px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2709px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2710px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2709px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:325px; top:2710px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2712px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:327px; top:2709px; width:2px; height:3px;"></span>\nDocument Images <span style="position:absolute; border: black 1px solid; left:179px; top:2805px; width:256px; height:12px;"></span>\n<span style="position:absolute; border: black 1px solid; left:179px; top:2805px; width:256px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:266px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:266px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:270px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:270px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:274px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:278px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:278px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:282px; top:2810px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:285px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:287px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:288px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:296px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2810px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:301px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2810px; width:3px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:308px; top:2809px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:310px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2809px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2810px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:317px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:319px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:320px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:322px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:323px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2811px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:332px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2810px; width:0px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:333px; top:2809px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:334px; top:2809px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2811px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:337px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:338px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:339px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2810px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:341px; top:2812px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:343px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:343px; top:2810px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2813px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2810px; width:3px; height:4px;"></span>\n</span><span style="font-family: unknown; font-size:6px">The Core LayoutParser Library<span style="position:absolute; border: black 1px solid; left:185px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:250px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:190px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:246px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:184px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:203px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:203px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:207px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:210px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:210px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:211px; top:2789px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:215px; top:2788px; width:4px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:219px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:220px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:223px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:226px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:229px; top:2788px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:230px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:230px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:231px; top:2790px; width:1px; height:0px;"></span>\n</span><span style="font-family: unknown; font-size:6px">OCR Module<span style="position:absolute; border: black 1px solid; left:363px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:429px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:425px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:425px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:363px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:368px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:371px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:372px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:373px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:373px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:374px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:376px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:378px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:379px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:381px; top:2789px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:382px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:384px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:385px; top:2790px; width:1px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:388px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:389px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:390px; top:2789px; width:0px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:393px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:394px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:397px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:398px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:401px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:404px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:404px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:405px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:407px; top:2788px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:408px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:409px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:410px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:412px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:413px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:415px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:416px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:417px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:417px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:418px; top:2788px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:419px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:422px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:422px; top:2789px; width:2px; height:3px;"></span>\nStorage &amp; Visualization<span style="position:absolute; border: black 1px solid; left:274px; top:2782px; width:66px; height:17px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2781px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:340px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:279px; top:2799px; width:56px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2787px; width:1px; height:7px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2794px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:335px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:273px; top:2781px; width:5px; height:5px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:281px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:284px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:287px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:286px; top:2789px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:289px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:2790px; width:1px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:292px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:295px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:298px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2788px; width:3px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:299px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:302px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:303px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:305px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:307px; top:2791px; width:1px; height:1px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:311px; top:2788px; width:2px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:314px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:315px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:316px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:316px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:318px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:321px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:324px; top:2789px; width:1px; height:4px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:326px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:329px; top:2789px; width:1px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:330px; top:2793px; width:0px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2789px; width:2px; height:3px;"></span>\n<span style="position:absolute; border: black 1px solid; left:331px; top:2790px; width:1px; height:0px;"></span>\nLayout Data Structure<span style="position:absolute; border: black 1px solid; left:304px; top:2767px; width:7px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:304px; top:2723px; width:7px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2789px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2788px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:256px; top:2791px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2791px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2752px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:265px; top:2750px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:256px; top:2754px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:255px; top:2754px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2789px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:355px; top:2788px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:346px; top:2791px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2791px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2752px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:355px; top:2750px; width:3px; height:2px;"></span>\n<span style="position:absolute; border: black 1px solid; left:346px; top:2754px; width:12px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:345px; top:2754px; width:3px; height:2px;"></span>\n</span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:3418px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:3418px;"><a name="5">Page 5</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:3510px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:3510px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">5\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:146px; top:3544px; width:321px; height:10px;"><span style="font-family: CMR10; font-size:9px">Table 1: Current layout detection models in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">model zoo\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:150px; top:3559px; width:26px; height:6px;"><span style="font-family: CMBX9; font-size:6px">Dataset\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:194px; top:3558px; width:118px; height:7px;"><span style="font-family: CMBX9; font-size:6px">Base Model</span><span style="font-family: CMR6; font-size:4px">1 </span><span style="font-family: CMBX9; font-size:6px">Large Model Notes\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:3573px; width:47px; height:46px;"><span style="font-family: CMR9; font-size:6px">PubLayNet [38]\n<br>PRImA [3]\n<br>Newspaper [17]\n<br>TableBank [18]\n<br>HJDataset [31]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:205px; top:3573px; width:18px; height:46px;"><span style="font-family: CMR9; font-size:6px">F / M\n<br>M\n<br>F\n<br>F\n<br>F / M\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:261px; top:3573px; width:6px; height:46px;"><span style="font-family: CMR9; font-size:6px">M\n<br>-\n<br>-\n<br>F\n<br>-\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:293px; top:3573px; width:181px; height:46px;"><span style="font-family: CMR9; font-size:6px">Layouts of modern scientiﬁc documents\n<br>Layouts of scanned modern magazines and scientiﬁc reports\n<br>Layouts of scanned US newspapers from the 20th century\n<br>Table region on modern scientiﬁc and business document\n<br>Layouts of history Japanese documents\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:136px; top:3624px; width:342px; height:48px;"><span style="font-family: CMR6; font-size:4px">1 </span><span style="font-family: CMR9; font-size:6px">For each dataset, we train several models of diﬀerent sizes for diﬀerent needs (the trade-oﬀ between accuracy\n<br>vs. computational cost). For “base model” and “large model”, we refer to using the ResNet 50 or ResNet 101\n<br>backbones [13], respectively. One can train models of diﬀerent architectures, like Faster R-CNN [28] (F) and Mask\n<br>R-CNN [12] (M). For example, an F in the Large Model column indicates it has a Faster R-CNN model trained\n<br>using the ResNet 101 backbone. The platform is maintained and a number of additions will be made to the model\n<br>zoo in coming months.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3697px; width:346px; height:81px;"><span style="font-family: CMTI10; font-size:9px">layout data structures</span><span style="font-family: CMR10; font-size:9px">, which are optimized for eﬃciency and versatility. 3) When\n<br></span><span style="font-family: CMR10; font-size:9px">necessary, users can employ existing or customized OCR models via the uniﬁed\n<br>API provided in the </span><span style="font-family: CMTI10; font-size:9px">OCR module</span><span style="font-family: CMR10; font-size:9px">. 4) </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">comes with a set of utility\n<br>functions for the </span><span style="font-family: CMTI10; font-size:9px">visualization and storage </span><span style="font-family: CMR10; font-size:9px">of the layout data. 5) </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">is also highly customizable, via its integration with functions for </span><span style="font-family: CMTI10; font-size:9px">layout data\n<br>annotation and model training</span><span style="font-family: CMR10; font-size:9px">. We now provide detailed descriptions for each\n<br>component.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3797px; width:152px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.1 Layout Detection Models\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3816px; width:347px; height:105px;"><span style="font-family: CMR10; font-size:9px">In </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, a layout model takes a document image as an input and\n<br>generates a list of rectangular boxes for the target content regions. Diﬀerent\n<br>from traditional methods, it relies on deep convolutional neural networks rather\n<br>than manually curated rules to identify content regions. It is formulated as an\n<br>object detection problem and state-of-the-art models like Faster R-CNN [28] and\n<br>Mask R-CNN [12] are used. This yields prediction results of high accuracy and\n<br>makes it possible to build a concise, generalized interface for layout detection.\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, built upon Detectron2 [35], provides a minimal API that can\n<br>perform layout detection with only four lines of code in Python:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:3929px; width:267px; height:30px;"><span style="font-family: CMR5; font-size:4px">1 </span><span style="font-family: CMTT9; font-size:8px">import layoutparser as lp\n<br></span><span style="font-family: CMR5; font-size:4px">2 </span><span style="font-family: CMTT9; font-size:8px">image = cv2 . imread ( &quot; image_file &quot; ) # load images\n<br></span><span style="font-family: CMR5; font-size:4px">3 </span><span style="font-family: CMTT9; font-size:8px">model = lp . De t e c tro n2 Lay outM odel (\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:157px; top:3961px; width:270px; height:8px;"><span style="font-family: CMTT9; font-size:8px">&quot; lp :// PubLayNet / f as t er _ r c nn _ R _ 50 _ F P N_ 3 x / config &quot; )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:3964px; width:166px; height:16px;"><span style="font-family: CMR5; font-size:4px">4\n<br>5 </span><span style="font-family: CMTT9; font-size:8px">layout = model . detect ( image )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:3989px; width:347px; height:93px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a wealth of pre-trained model weights using various\n<br>datasets covering diﬀerent languages, time periods, and document types. Due to\n<br>domain shift [7], the prediction performance can notably drop when models are ap-\n<br>plied to target samples that are signiﬁcantly diﬀerent from the training dataset. As\n<br>document structures and layouts vary greatly in diﬀerent domains, it is important\n<br>to select models trained on a dataset similar to the test samples. A semantic syntax\n<br>is used for initializing the model weights in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, using both the dataset\n<br></span><span style="font-family: CMR10; font-size:9px">name and model name </span><span style="font-family: CMTT10; font-size:9px">lp://&lt;dataset-name&gt;/&lt;model-architecture-name&gt;</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:137px; top:3556px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3558px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:3569px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3572px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3581px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3591px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3601px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:191px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:239px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:290px; top:3611px; width:0px; height:9px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:3622px; width:341px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3928px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3939px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3950px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3961px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:3972px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:4260px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:4260px;"><a name="6">Page 6</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4352px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">6\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:4352px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4567px; width:347px; height:69px;"><span style="font-family: CMR10; font-size:9px">Fig. 2: The relationship between the three types of layout data structures.\n<br></span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">supports three kinds of variation; </span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">consists of the co-\n<br>ordinate information and extra features like block text, types, and reading orders;\n<br>a </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">object is a list of all possible layout elements, including other </span><span style="font-family: CMTT10; font-size:9px">Layout\n<br></span><span style="font-family: CMR10; font-size:9px">objects. They all support the same set of transformation and operation APIs for\n<br>maximum ﬂexibility.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4665px; width:347px; height:69px;"><span style="font-family: CMR10; font-size:9px">Shown in Table 1, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">currently hosts 9 pre-trained models trained\n<br>on 5 diﬀerent datasets. Description of the training dataset is provided alongside\n<br>with the trained models such that users can quickly identify the most suitable\n<br>models for their tasks. Additionally, when such a model is not readily available,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also supports training customized layout models and community\n<br>sharing of the models (detailed in Section 3.5).\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4759px; width:144px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.2 Layout Data Structures\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:4783px; width:346px; height:141px;"><span style="font-family: CMR10; font-size:9px">A critical feature of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is the implementation of a series of data\n<br></span><span style="font-family: CMR10; font-size:9px">structures and operations that can be used to eﬃciently process and manipulate\n<br>the layout elements. In document image analysis pipelines, various post-processing\n<br>on the layout analysis model outputs is usually required to obtain the ﬁnal\n<br>outputs. Traditionally, this requires exporting DL model outputs and then loading\n<br>the results into other pipelines. All model outputs from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">will be\n<br>stored in carefully engineered data types optimized for further processing, which\n<br>makes it possible to build an end-to-end document digitization pipeline within\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. There are three key components in the data structure, namely\n<br>the </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">system, the </span><span style="font-family: CMTT10; font-size:9px">TextBlock</span><span style="font-family: CMR10; font-size:9px">, and the </span><span style="font-family: CMTT10; font-size:9px">Layout</span><span style="font-family: CMR10; font-size:9px">. They provide diﬀerent\n<br>levels of abstraction for the layout data, and a set of APIs are supported for\n<br></span><span style="font-family: CMR10; font-size:9px">transformations or operations on these classes.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:195px; top:4375px; width:224px; height:181px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:5102px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:5102px;"><a name="7">Page 7</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:5194px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:5194px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">7\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5219px; width:347px; height:177px;"><span style="font-family: CMR10; font-size:9px">Coordinates are the cornerstones for storing layout information. Currently,\n<br>three types of </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">data structures are provided in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, shown\n<br>in Figure 2. </span><span style="font-family: CMTT10; font-size:9px">Interval </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Rectangle </span><span style="font-family: CMR10; font-size:9px">are the most common data types and\n<br>support specifying 1D or 2D regions within a document. They are parameterized\n<br>with 2 and 4 parameters. A </span><span style="font-family: CMTT10; font-size:9px">Quadrilateral </span><span style="font-family: CMR10; font-size:9px">class is also implemented to support\n<br>a more generalized representation of rectangular regions when the document\n<br>is skewed or distorted, where the 4 corner points can be speciﬁed and a total\n<br>of 8 degrees of freedom are supported. A wide collection of transformations\n<br>like </span><span style="font-family: CMTT10; font-size:9px">shift</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">pad</span><span style="font-family: CMR10; font-size:9px">, and </span><span style="font-family: CMTT10; font-size:9px">scale</span><span style="font-family: CMR10; font-size:9px">, and operations like </span><span style="font-family: CMTT10; font-size:9px">intersect</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">union</span><span style="font-family: CMR10; font-size:9px">, and </span><span style="font-family: CMTT10; font-size:9px">is_in</span><span style="font-family: CMR10; font-size:9px">,\n<br>are supported for these classes. Notably, it is common to separate a segment\n<br>of the image and analyze it individually. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides full support\n<br>for this scenario via image cropping operations </span><span style="font-family: CMTT10; font-size:9px">crop_image </span><span style="font-family: CMR10; font-size:9px">and coordinate\n<br>transformations like </span><span style="font-family: CMTT10; font-size:9px">relative_to </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">condition_on </span><span style="font-family: CMR10; font-size:9px">that transform coordinates\n<br></span><span style="font-family: CMR10; font-size:9px">to and from their relative representations. We refer readers to Table 2 for a more\n<br></span><span style="font-family: CMR10; font-size:9px">detailed description of these operations</span><span style="font-family: CMR7; font-size:6px">13</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5399px; width:346px; height:81px;"><span style="font-family: CMR10; font-size:9px">Based on </span><span style="font-family: CMTT10; font-size:9px">Coordinate</span><span style="font-family: CMR10; font-size:9px">s, we implement the </span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">class that stores both\n<br>the positional and extra features of individual layout elements. It also supports\n<br>specifying the reading orders via setting the </span><span style="font-family: CMTT10; font-size:9px">parent </span><span style="font-family: CMR10; font-size:9px">ﬁeld to the index of the parent\n<br>object. A </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">class is built that takes in a list of </span><span style="font-family: CMTT10; font-size:9px">TextBlock</span><span style="font-family: CMR10; font-size:9px">s and supports\n<br>processing the elements in batch. </span><span style="font-family: CMTT10; font-size:9px">Layout </span><span style="font-family: CMR10; font-size:9px">can also be nested to support hierarchical\n<br>layout structures. They support the same operations and transformations as the\n<br></span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">classes, minimizing both learning and deployment eﬀort.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5498px; width:51px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.3 OCR\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5516px; width:346px; height:105px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a uniﬁed interface for existing OCR tools. Though there\n<br>are many OCR tools available, they are usually conﬁgured diﬀerently with distinct\n<br>APIs or protocols for using them. It can be ineﬃcient to add new OCR tools into\n<br>an existing pipeline, and diﬃcult to make direct comparisons among the available\n<br>tools to ﬁnd the best option for a particular project. To this end, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">builds a series of wrappers among existing OCR engines, and provides nearly\n<br>the same syntax for using them. It supports a plug-and-play style of using OCR\n<br>engines, making it eﬀortless to switch, evaluate, and compare diﬀerent OCR\n<br>modules:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:126px; top:5629px; width:267px; height:30px;"><span style="font-family: CMR5; font-size:4px">1 </span><span style="font-family: CMTT9; font-size:8px">ocr_agent = lp . TesseractAgent ()\n<br></span><span style="font-family: CMR5; font-size:4px">2 </span><span style="font-family: CMTT9; font-size:8px"># Can be easily switched to other OCR software\n<br></span><span style="font-family: CMR5; font-size:4px">3 </span><span style="font-family: CMTT9; font-size:8px">tokens = ocr_agent . detect ( image )\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5667px; width:347px; height:45px;"><span style="font-family: CMR10; font-size:9px">The OCR outputs will also be stored in the aforementioned layout data\n<br>structures and can be seamlessly incorporated into the digitization pipeline.\n<br>Currently </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports the Tesseract and Google Cloud Vision OCR\n<br>engines.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:5715px; width:346px; height:33px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also comes with a DL-based CNN-RNN OCR model [6] trained\n<br>with the Connectionist Temporal Classiﬁcation (CTC) loss [10]. It can be used\n<br>like the other OCR modules, and can be easily trained on customized datasets.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:5756px; width:271px; height:10px;"><span style="font-family: CMR6; font-size:5px">13 </span><span style="font-family: CMR9; font-size:8px">This is also available in the </span><span style="font-family: CMTT9; font-size:8px">LayoutParser </span><span style="font-family: CMR9; font-size:8px">documentation pages.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:5628px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5639px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5650px; width:345px; height:10px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:5754px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:5944px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:5944px;"><a name="8">Page 8</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6036px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">8\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:6036px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6070px; width:347px; height:34px;"><span style="font-family: CMR10; font-size:9px">Table 2: All operations supported by the layout elements. The same APIs are\n<br>supported across diﬀerent layout element classes including </span><span style="font-family: CMTT10; font-size:9px">Coordinate </span><span style="font-family: CMR10; font-size:9px">types,\n<br></span><span style="font-family: CMTT10; font-size:9px">TextBlock </span><span style="font-family: CMR10; font-size:9px">and </span><span style="font-family: CMTT10; font-size:9px">Layout</span><span style="font-family: CMR10; font-size:9px">.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6110px; width:65px; height:7px;"><span style="font-family: CMBX9; font-size:7px">Operation Name\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6110px; width:44px; height:7px;"><span style="font-family: CMBX9; font-size:7px">Description\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6125px; width:310px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.pad(top, bottom, right, left) </span><span style="font-family: CMR9; font-size:7px">Enlarge the current block according to the input\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6146px; width:75px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.scale(fx, fy)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6172px; width:75px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.shift(dx, dy)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6140px; width:129px; height:18px;"><span style="font-family: CMR9; font-size:7px">Scale the current block given the ratio\n<br>in x and y direction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6167px; width:127px; height:18px;"><span style="font-family: CMR9; font-size:7px">Move the current block with the shift\n<br>distances in x and y direction\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6193px; width:77px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.is in(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6193px; width:116px; height:7px;"><span style="font-family: CMR9; font-size:7px">Whether block1 is inside of block2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6214px; width:94px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.intersect(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6240px; width:79px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.union(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6266px; width:101px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.relative to(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6292px; width:105px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block1.condition on(block2)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6208px; width:186px; height:18px;"><span style="font-family: CMR9; font-size:7px">Return the intersection region of block1 and block2.\n<br></span><span style="font-family: CMR9; font-size:7px">Coordinate type to be determined based on the inputs.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6234px; width:186px; height:18px;"><span style="font-family: CMR9; font-size:7px">Return the union region of block1 and block2.\n<br>Coordinate type to be determined based on the inputs.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6261px; width:154px; height:18px;"><span style="font-family: CMR9; font-size:7px">Convert the absolute coordinates of block1 to\n<br>relative coordinates to block2\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6287px; width:170px; height:18px;"><span style="font-family: CMR9; font-size:7px">Calculate the absolute coordinates of block1 given\n<br>the canvas block2’s absolute coordinates\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:141px; top:6313px; width:89px; height:7px;"><span style="font-family: CMTT9; font-size:7px">block.crop image(image)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:287px; top:6313px; width:158px; height:7px;"><span style="font-family: CMR9; font-size:7px">Obtain the image segments in the block region\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6350px; width:152px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.4 Storage and visualization\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6371px; width:346px; height:129px;"><span style="font-family: CMR10; font-size:9px">The end goal of DIA is to transform the image-based document data into a\n<br>structured database. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports exporting layout data into diﬀerent\n<br>formats like </span><span style="font-family: CMTT10; font-size:9px">JSON</span><span style="font-family: CMR10; font-size:9px">, </span><span style="font-family: CMTT10; font-size:9px">csv</span><span style="font-family: CMR10; font-size:9px">, and will add the support for the METS/ALTO XML\n<br>format </span><span style="font-family: CMR7; font-size:6px">14 </span><span style="font-family: CMR10; font-size:9px">. It can also load datasets from layout analysis-speciﬁc formats like\n<br>COCO [38] and the Page Format [25] for training layout models (Section 3.5).\n<br>Visualization of the layout detection results is critical for both presentation\n<br>and debugging. </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is built with an integrated API for displaying the\n<br>layout information along with the original document image. Shown in Figure 3, it\n<br>enables presenting layout data with rich meta information and features in diﬀerent\n<br>modes. More detailed information can be found in the online </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">documentation page.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6522px; width:166px; height:9px;"><span style="font-family: CMBX10; font-size:9px">3.5 Customized Model Training\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:6543px; width:347px; height:45px;"><span style="font-family: CMR10; font-size:9px">Besides the oﬀ-the-shelf library, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is also highly customizable with\n<br>supports for highly unique and challenging document analysis tasks. Target\n<br>document images can be vastly diﬀerent from the existing datasets for train-\n<br>ing layout models, which leads to low layout detection accuracy. Training data\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:6598px; width:111px; height:10px;"><span style="font-family: CMR6; font-size:5px">14 </span><span style="font-family: CMR9; font-size:8px">https://altoxml.github.io\n<br></span></div><span style="position:absolute; border: black 1px solid; left:137px; top:6105px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6108px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6121px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6123px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6136px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6139px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6162px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6165px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6189px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:177px; top:6199px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6191px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6204px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6206px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6230px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6233px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6256px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:200px; top:6272px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6259px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6283px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:204px; top:6298px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6285px; width:0px; height:22px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6309px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:181px; top:6319px; width:2px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:283px; top:6312px; width:0px; height:11px;"></span>\n<span style="position:absolute; border: black 1px solid; left:137px; top:6324px; width:340px; height:0px;"></span>\n<span style="position:absolute; border: black 1px solid; left:134px; top:6596px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:6786px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:6786px;"><a name="9">Page 9</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:6878px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:475px; top:6878px; width:4px; height:8px;"><span style="font-family: CMR9; font-size:8px">9\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7103px; width:346px; height:69px;"><span style="font-family: CMR10; font-size:9px">Fig. 3: Layout detection and OCR results visualization generated by the\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">APIs. Mode I directly overlays the layout region bounding boxes\n<br>and categories over the original image. Mode II recreates the original document\n<br>via drawing the OCR’d texts at their corresponding positions on the image\n<br>canvas. In this ﬁgure, tokens in textual regions are ﬁltered using the API and\n<br>then displayed.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7212px; width:347px; height:33px;"><span style="font-family: CMR10; font-size:9px">can also be highly sensitive and not sharable publicly. To overcome these chal-\n<br>lenges, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is built with rich features for eﬃcient data annotation and\n<br>customized model training.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7255px; width:347px; height:81px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">incorporates a toolkit optimized for annotating document lay-\n<br>outs using object-level active learning [32]. With the help from a layout detection\n<br>model trained along with labeling, only the most important layout objects within\n<br>each image, rather than the whole image, are required for labeling. The rest of\n<br>the regions are automatically annotated with high conﬁdence predictions from\n<br>the layout detection model. This allows a layout dataset to be created more\n<br></span><span style="font-family: CMR10; font-size:9px">eﬃciently with only around 60% of the labeling budget.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7345px; width:347px; height:105px;"><span style="font-family: CMR10; font-size:9px">After the training dataset is curated, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">supports diﬀerent modes\n<br>for training the layout models. </span><span style="font-family: CMTI10; font-size:9px">Fine-tuning </span><span style="font-family: CMR10; font-size:9px">can be used for training models on a\n<br></span><span style="font-family: CMTI10; font-size:9px">small </span><span style="font-family: CMR10; font-size:9px">newly-labeled dataset by initializing the model with existing pre-trained\n<br>weights. </span><span style="font-family: CMTI10; font-size:9px">Training from scratch </span><span style="font-family: CMR10; font-size:9px">can be helpful when the source dataset and\n<br>target are signiﬁcantly diﬀerent and a large training set is available. However, as\n<br>suggested in Studer et al.’s work[33], loading pre-trained weights on large-scale\n<br>datasets like ImageNet [5], even from totally diﬀerent domains, can still boost\n<br>model performance. Through the integrated API provided by </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">,\n<br></span><span style="font-family: CMR10; font-size:9px">users can easily compare model performances on the benchmark datasets.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:169px; top:6901px; width:276px; height:191px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:7628px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:7628px;"><a name="10">Page 10</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7720px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">10\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:7720px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7912px; width:346px; height:57px;"><span style="font-family: CMR10; font-size:9px">Fig. 4: Illustration of (a) the original historical Japanese document with layout\n<br>detection results and (b) a recreated version of the document image that achieves\n<br>much better character recognition recall. The reorganization algorithm rearranges\n<br>the tokens based on the their detected bounding boxes given a maximum allowed\n<br>height.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:7992px; width:224px; height:12px;"><span style="font-family: CMBX12; font-size:11px">4 </span><span style="font-family: CMTT12; font-size:11px">LayoutParser </span><span style="font-family: CMBX12; font-size:11px">Community Platform\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8016px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Another focus of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is promoting the reusability of layout detection\n<br>models and full digitization pipelines. Similar to many existing deep learning\n<br>libraries, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">comes with a community model hub for distributing\n<br>layout models. End-users can upload their self-trained models to the model hub,\n<br>and these models can be loaded into a similar interface as the currently available\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">pre-trained models. For example, the model trained on the News\n<br>Navigator dataset [17] has been incorporated in the model hub.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8099px; width:347px; height:129px;"><span style="font-family: CMR10; font-size:9px">Beyond DL models, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">also promotes the sharing of entire doc-\n<br>ument digitization pipelines. For example, sometimes the pipeline requires the\n<br>combination of multiple DL models to achieve better accuracy. Currently, pipelines\n<br>are mainly described in academic papers and implementations are often not pub-\n<br>licly available. To this end, the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">community platform also enables\n<br></span><span style="font-family: CMR10; font-size:9px">the sharing of layout pipelines to promote the discussion and reuse of techniques.\n<br>For each shared pipeline, it has a dedicated project page, with links to the source\n<br>code, documentation, and an outline of the approaches. A discussion panel is\n<br>provided for exchanging ideas. Combined with the core </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">library,\n<br>users can easily build reusable components based on the shared pipelines and\n<br>apply them to solve their unique problems.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8247px; width:79px; height:11px;"><span style="font-family: CMBX12; font-size:11px">5 Use Cases\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8270px; width:346px; height:21px;"><span style="font-family: CMR10; font-size:9px">The core objective of </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is to make it easier to create both large-scale\n<br></span><span style="font-family: CMR10; font-size:9px">and light-weight document digitization pipelines. Large-scale document processing\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:134px; top:7743px; width:345px; height:158px;"></div><span style="position:absolute; border: gray 1px solid; left:0px; top:8470px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:8470px;"><a name="11">Page 11</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:8562px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:8562px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">11\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8587px; width:347px; height:117px;"><span style="font-family: CMR10; font-size:9px">focuses on precision, eﬃciency, and robustness. The target documents may have\n<br>complicated structures, and may require training multiple layout detection models\n<br>to achieve the optimal accuracy. Light-weight pipelines are built for relatively\n<br>simple documents, with an emphasis on development ease, speed and ﬂexibility.\n<br>Ideally one only needs to use existing resources, and model training should be\n<br>avoided. Through two exemplar projects, we show how practitioners in both\n<br>academia and industry can easily build such pipelines using </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">and\n<br>extract high-quality structured document data for their downstream tasks. The\n<br>source code for these projects will be publicly available in the </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">community hub.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8730px; width:330px; height:9px;"><span style="font-family: CMBX10; font-size:9px">5.1 A Comprehensive Historical Document Digitization Pipeline\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8755px; width:347px; height:189px;"><span style="font-family: CMR10; font-size:9px">The digitization of historical documents can unlock valuable data that can shed\n<br>light on many important social, economic, and historical questions. Yet due to\n<br>scan noises, page wearing, and the prevalence of complicated layout structures, ob-\n<br>taining a structured representation of historical document scans is often extremely\n<br>complicated.\n<br>In this example, </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">was\n<br>used to develop a comprehensive\n<br>pipeline, shown in Figure 5, to gener-\n<br>ate high-quality structured data from\n<br>historical Japanese ﬁrm ﬁnancial ta-\n<br>bles with complicated layouts. The\n<br>pipeline applies two layout models to\n<br>identify diﬀerent levels of document\n<br>structures and two customized OCR\n<br>engines for optimized character recog-\n<br>nition accuracy.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:8947px; width:165px; height:153px;"><span style="font-family: CMR10; font-size:9px">As shown in Figure 4 (a), the\n<br>document contains columns of text\n<br>written vertically </span><span style="font-family: CMR7; font-size:6px">15</span><span style="font-family: CMR10; font-size:9px">, a common style\n<br>in Japanese. Due to scanning noise\n<br>and archaic printing technology, the\n<br></span><span style="font-family: CMR10; font-size:9px">columns can be skewed or have vari-\n<br>able widths, and hence cannot be eas-\n<br>ily identiﬁed via rule-based methods.\n<br>Within each column, words are sepa-\n<br>rated by white spaces of variable size,\n<br>and the vertical positions of objects\n<br>can be an indicator of their layout\n<br>type.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:307px; top:9052px; width:174px; height:33px;"><span style="font-family: CMR10; font-size:9px">Fig. 5: Illustration of how </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">helps with the historical document digi-\n<br>tization pipeline.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9113px; width:347px; height:10px;"><span style="font-family: CMR6; font-size:5px">15 </span><span style="font-family: CMR9; font-size:8px">A document page consists of eight rows like this. For simplicity we skip the row\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9125px; width:308px; height:8px;"><span style="font-family: CMR9; font-size:8px">segmentation discussion and refer readers to the source code when available.\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:307px; top:8849px; width:172px; height:182px;"></div><span style="position:absolute; border: black 1px solid; left:134px; top:9111px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:9312px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:9312px;"><a name="12">Page 12</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9404px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">12\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:9404px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:149px; top:9429px; width:148px; height:9px;"><span style="font-family: CMR10; font-size:9px">To decipher the complicated layout\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9441px; width:347px; height:141px;"><span style="font-family: CMR10; font-size:9px">structure, two object detection models have been trained to recognize individual\n<br>columns and tokens, respectively. A small training set (400 images with approxi-\n<br>mately 100 annotations each) is curated via the active learning based annotation\n<br>tool [32] in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. The models learn to identify both the categories and\n<br>regions for each token or column via their distinct visual features. The layout\n<br>data structure enables easy grouping of the tokens within each column, and\n<br>rearranging columns to achieve the correct reading orders based on the horizontal\n<br>position. Errors are identiﬁed and rectiﬁed via checking the consistency of the\n<br>model predictions. Therefore, though trained on a small dataset, the pipeline\n<br>achieves a high level of layout detection accuracy: it achieves a 96.97 AP [19]\n<br>score across 5 categories for the column detection model, and a 89.23 AP across\n<br>4 categories for the token detection model.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9587px; width:346px; height:117px;"><span style="font-family: CMR10; font-size:9px">A combination of character recognition methods is developed to tackle the\n<br>unique challenges in this document. In our experiments, we found that irregular\n<br>spacing between the tokens led to a low character recognition recall rate, whereas\n<br>existing OCR models tend to perform better on densely-arranged texts. To\n<br>overcome this challenge, we create a document reorganization algorithm that\n<br>rearranges the text based on the token bounding boxes detected in the layout\n<br>analysis step. Figure 4 (b) illustrates the generated image of dense text, which is\n<br>sent to the OCR APIs as a whole to reduce the transaction costs. The ﬂexible\n<br>coordinate system in </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">is used to transform the OCR results relative\n<br>to their original positions on the page.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9710px; width:347px; height:141px;"><span style="font-family: CMR10; font-size:9px">Additionally, it is common for historical documents to use unique fonts\n<br>with diﬀerent glyphs, which signiﬁcantly degrades the accuracy of OCR models\n<br>trained on modern texts. In this document, a special ﬂat font is used for printing\n<br>numbers and could not be detected by oﬀ-the-shelf OCR engines. Using the highly\n<br>ﬂexible functionalities from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, a pipeline approach is constructed\n<br>that achieves a high recognition accuracy with minimal eﬀort. As the characters\n<br>have unique visual structures and are usually clustered together, we train the\n<br>layout model to identify number regions with a dedicated category. Subsequently,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">crops images within these regions, and identiﬁes characters within\n<br>them using a self-trained OCR model based on a CNN-RNN [6]. The model\n<br>detects a total of 15 possible categories, and achieves a 0.98 Jaccard score</span><span style="font-family: CMR7; font-size:6px">16 </span><span style="font-family: CMR10; font-size:9px">and\n<br></span><span style="font-family: CMR10; font-size:9px">a 0.17 average Levinstein distances</span><span style="font-family: CMR7; font-size:6px">17 </span><span style="font-family: CMR10; font-size:9px">for token prediction on the test set.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:9856px; width:345px; height:57px;"><span style="font-family: CMR10; font-size:9px">Overall, it is possible to create an intricate and highly accurate digitization\n<br>pipeline for large-scale digitization using </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">. The pipeline avoids\n<br>specifying the complicated rules used in traditional methods, is straightforward\n<br>to develop, and is robust to outliers. The DL models also generate ﬁne-grained\n<br>results that enable creative approaches like page reorganization for OCR.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9933px; width:346px; height:10px;"><span style="font-family: CMR6; font-size:5px">16 </span><span style="font-family: CMR9; font-size:8px">This measures the overlap between the detected and ground-truth characters, and\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9945px; width:75px; height:8px;"><span style="font-family: CMR9; font-size:8px">the maximum is 1.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:9955px; width:348px; height:10px;"><span style="font-family: CMR6; font-size:5px">17 </span><span style="font-family: CMR9; font-size:8px">This measures the number of edits from the ground-truth text to the predicted text,\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:144px; top:9967px; width:78px; height:8px;"><span style="font-family: CMR9; font-size:8px">and lower is better.\n<br></span></div><span style="position:absolute; border: black 1px solid; left:134px; top:9931px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:10154px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:10154px;"><a name="13">Page 13</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:10246px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:10246px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">13\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10398px; width:346px; height:45px;"><span style="font-family: CMR10; font-size:9px">Fig. 6: This lightweight table detector can identify tables (outlined in red) and\n<br>cells (shaded in blue) in diﬀerent locations on a page. In very few cases (d), it\n<br>might generate minor error predictions, e.g, failing to capture the top text line of\n<br></span><span style="font-family: CMR10; font-size:9px">a table.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10478px; width:216px; height:9px;"><span style="font-family: CMBX10; font-size:9px">5.2 A light-weight Visual Table Extractor\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10518px; width:347px; height:81px;"><span style="font-family: CMR10; font-size:9px">Detecting tables and parsing their structures (table extraction) are of central im-\n<br>portance for many document digitization tasks. Many previous works [26, 30, 27]\n<br>and tools </span><span style="font-family: CMR7; font-size:6px">18 </span><span style="font-family: CMR10; font-size:9px">have been developed to identify and parse table structures. Yet they\n<br>might require training complicated models from scratch, or are only applicable\n<br>for born-digital PDF documents. In this section, we show how </span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can\n<br>help build a light-weight accurate visual table extractor for legal docket tables\n<br>using the existing resources with minimal eﬀort.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:10606px; width:347px; height:177px;"><span style="font-family: CMR10; font-size:9px">The extractor uses a pre-trained layout detection model for identifying the\n<br>table regions and some simple rules for pairing the rows and the columns in the\n<br>PDF image. Mask R-CNN [12] trained on the PubLayNet dataset [38] from the\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">Model Zoo can be used for detecting table regions. By ﬁltering\n<br>out model predictions of low conﬁdence and removing overlapping predictions,\n<br></span><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">can identify the tabular regions on each page, which signiﬁcantly\n<br>simpliﬁes the subsequent steps. By applying the line detection functions within\n<br></span><span style="font-family: CMR10; font-size:9px">the tabular segments, provided in the utility module from </span><span style="font-family: CMTT10; font-size:9px">LayoutParser</span><span style="font-family: CMR10; font-size:9px">, the\n<br>pipeline can identify the three distinct columns in the tables. A row clustering\n<br>method is then applied via analyzing the y coordinates of token bounding boxes in\n<br>the left-most column, which are obtained from the OCR engines. A non-maximal\n<br>suppression algorithm is used to remove duplicated rows with extremely small\n<br>gaps. Shown in Figure 6, the built pipeline can detect tables at diﬀerent positions\n<br>on a page accurately. Continued tables from diﬀerent pages are concatenated,\n<br>and a structured table representation has been easily created.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:133px; top:10808px; width:315px; height:10px;"><span style="font-family: CMR6; font-size:5px">18 </span><span style="font-family: CMR9; font-size:8px">https://github.com/atlanhq/camelot, https://github.com/tabulapdf/tabula\n<br></span></div><div style="position:absolute; border: figure 1px solid; writing-mode:False; left:134px; top:10269px; width:345px; height:118px;"></div><span style="position:absolute; border: black 1px solid; left:134px; top:10806px; width:56px; height:0px;"></span>\n<span style="position:absolute; border: gray 1px solid; left:0px; top:10996px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:10996px;"><a name="14">Page 14</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11088px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">14\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:11088px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11112px; width:84px; height:11px;"><span style="font-family: CMBX12; font-size:11px">6 Conclusion\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11139px; width:346px; height:117px;"><span style="font-family: CMTT10; font-size:9px">LayoutParser </span><span style="font-family: CMR10; font-size:9px">provides a comprehensive toolkit for deep learning-based document\n<br>image analysis. The oﬀ-the-shelf library is easy to install, and can be used to\n<br>build ﬂexible and accurate pipelines for processing documents with complicated\n<br>structures. It also supports high-level customization and enables easy labeling and\n<br>training of DL models on unique document image datasets. The </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">community platform facilitates sharing DL models and DIA pipelines, inviting\n<br>discussion and promoting code reproducibility and reusability. The </span><span style="font-family: CMTT10; font-size:9px">LayoutParser\n<br></span><span style="font-family: CMR10; font-size:9px">team is committed to keeping the library updated continuously and bringing\n<br>the most recent advances in DL-based DIA, such as multi-modal document\n<br>modeling [37, 36, 9] (an upcoming priority), to a diverse audience of end-users.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11279px; width:347px; height:45px;"><span style="font-family: CMBX10; font-size:9px">Acknowledgements </span><span style="font-family: CMR10; font-size:9px">We thank the anonymous reviewers for their comments\n<br>and suggestions. This project is supported in part by NSF Grant OIA-2033558\n<br>and funding from the Harvard Data Science Initiative and Harvard Catalyst.\n<br>Zejiang Shen thanks Doug Downey for suggestions.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:11347px; width:62px; height:11px;"><span style="font-family: CMBX12; font-size:11px">References\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11375px; width:341px; height:85px;"><span style="font-family: CMR9; font-size:8px">[1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado,\n<br>G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A.,\n<br>Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg,\n<br>J., Man´e, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J.,\n<br>Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V.,\n<br>Vi´egas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng,\n<br>X.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015),\n<br></span><span style="font-family: CMTT9; font-size:8px">https://www.tensorflow.org/</span><span style="font-family: CMR9; font-size:8px">, software available from tensorﬂow.org\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11463px; width:341px; height:41px;"><span style="font-family: CMR9; font-size:8px">[2] Alberti, M., Pondenkandath, V., W¨ursch, M., Ingold, R., Liwicki, M.: Deepdiva: a\n<br>highly-functional python framework for reproducible experiments. In: 2018 16th\n<br>International Conference on Frontiers in Handwriting Recognition (ICFHR). pp.\n<br>423–428. IEEE (2018)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11507px; width:341px; height:41px;"><span style="font-family: CMR9; font-size:8px">[3] Antonacopoulos, A., Bridson, D., Papadopoulos, C., Pletschacher, S.: A realistic\n<br>dataset for performance evaluation of document layout analysis. In: 2009 10th\n<br></span><span style="font-family: CMR9; font-size:8px">International Conference on Document Analysis and Recognition. pp. 296–300.\n<br></span><span style="font-family: CMR9; font-size:8px">IEEE (2009)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11551px; width:340px; height:30px;"><span style="font-family: CMR9; font-size:8px">[4] Baek, Y., Lee, B., Han, D., Yun, S., Lee, H.: Character region awareness for text\n<br>detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and\n<br>Pattern Recognition. pp. 9365–9374 (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11585px; width:340px; height:8px;"><span style="font-family: CMR9; font-size:8px">[5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A Large-Scale\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:11596px; width:200px; height:8px;"><span style="font-family: CMR9; font-size:8px">Hierarchical Image Database. In: CVPR09 (2009)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11607px; width:341px; height:30px;"><span style="font-family: CMR9; font-size:8px">[6] Deng, Y., Kanervisto, A., Ling, J., Rush, A.M.: Image-to-markup generation with\n<br>coarse-to-ﬁne attention. In: International Conference on Machine Learning. pp.\n<br>980–989. PMLR (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11640px; width:341px; height:19px;"><span style="font-family: CMR9; font-size:8px">[7] Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation.\n<br>In: International conference on machine learning. pp. 1180–1189. PMLR (2015)\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:11838px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:11838px;"><a name="15">Page 15</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:237px; top:11930px; width:210px; height:9px;"><span style="font-family: CMTT9; font-size:8px">LayoutParser</span><span style="font-family: CMR9; font-size:8px">: A Uniﬁed Toolkit for DL-Based DIA\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:471px; top:11930px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">15\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11956px; width:341px; height:63px;"><span style="font-family: CMR9; font-size:8px">[8] Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N., Peters,\n<br>M., Schmitz, M., Zettlemoyer, L.: Allennlp: A deep semantic natural language\n<br>processing platform. arXiv preprint arXiv:1803.07640 (2018)\n<br>(cid:32)Lukasz Garncarek, Powalski, R., Stanis(cid:32)lawek, T., Topolski, B., Halama, P.,\n<br>Grali´nski, F.: Lambert: Layout-aware (language) modeling using bert for in-\n<br>formation extraction (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:140px; top:11989px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">[9]\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12022px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[10] Graves, A., Fern´andez, S., Gomez, F., Schmidhuber, J.: Connectionist temporal\n<br>classiﬁcation: labelling unsegmented sequence data with recurrent neural networks.\n<br>In: Proceedings of the 23rd international conference on Machine learning. pp.\n<br>369–376 (2006)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12066px; width:344px; height:41px;"><span style="font-family: CMR9; font-size:8px">[11] Harley, A.W., Ufkes, A., Derpanis, K.G.: Evaluation of deep convolutional nets for\n<br>document image classiﬁcation and retrieval. In: 2015 13th International Conference\n<br>on Document Analysis and Recognition (ICDAR). pp. 991–995. IEEE (2015)\n<br>[12] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:12110px; width:293px; height:8px;"><span style="font-family: CMR9; font-size:8px">IEEE international conference on computer vision. pp. 2961–2969 (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12121px; width:346px; height:30px;"><span style="font-family: CMR9; font-size:8px">[13] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.\n<br>In: Proceedings of the IEEE conference on computer vision and pattern recognition.\n<br>pp. 770–778 (2016)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12153px; width:346px; height:8px;"><span style="font-family: CMR9; font-size:8px">[14] Kay, A.: Tesseract: An open-source optical character recognition engine. Linux J.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:12164px; width:99px; height:8px;"><span style="font-family: CMBX9; font-size:8px">2007</span><span style="font-family: CMR9; font-size:8px">(159), 2 (Jul 2007)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12175px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[15] Lamiroy, B., Lopresti, D.: An open architecture for end-to-end document analysis\n<br>benchmarking. In: 2011 International Conference on Document Analysis and\n<br>Recognition. pp. 42–47. IEEE (2011)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12208px; width:347px; height:64px;"><span style="font-family: CMR9; font-size:8px">[16] Lee, B.C., Weld, D.S.: Newspaper navigator: Open faceted search for 1.5\n<br>million images. In: Adjunct Publication of the 33rd Annual ACM Sym-\n<br>posium on User\n<br>Interface Software and Technology. p. 120–122. UIST\n<br>’20 Adjunct, Association for Computing Machinery, New York, NY, USA\n<br>(2020). https://doi.org/10.1145/3379350.3416143, </span><span style="font-family: CMTT9; font-size:8px">https://doi-org.offcampus.\n<br>lib.washington.edu/10.1145/3379350.3416143\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12274px; width:345px; height:53px;"><span style="font-family: CMR9; font-size:8px">[17] Lee, B.C.G., Mears, J., Jakeway, E., Ferriter, M., Adams, C., Yarasavage, N.,\n<br>Thomas, D., Zwaard, K., Weld, D.S.: The Newspaper Navigator Dataset: Extracting\n<br>Headlines and Visual Content from 16 Million Historic Newspaper Pages in\n<br>Chronicling America, p. 3055–3062. Association for Computing Machinery, New\n<br>York, NY, USA (2020), </span><span style="font-family: CMTT9; font-size:8px">https://doi.org/10.1145/3340531.3412767\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12329px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[18] Li, M., Cui, L., Huang, S., Wei, F., Zhou, M., Li, Z.: Tablebank: Table benchmark\n<br>for image-based table detection and recognition. arXiv preprint arXiv:1903.01949\n<br>(2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12362px; width:345px; height:30px;"><span style="font-family: CMR9; font-size:8px">[19] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll´ar, P.,\n<br></span><span style="font-family: CMR9; font-size:8px">Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference\n<br></span><span style="font-family: CMR9; font-size:8px">on computer vision. pp. 740–755. Springer (2014)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12395px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[20] Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic\n<br>segmentation. In: Proceedings of the IEEE conference on computer vision and\n<br>pattern recognition. pp. 3431–3440 (2015)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12427px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[21] Neudecker, C., Schlarb, S., Dogan, Z.M., Missier, P., Suﬁ, S., Williams, A., Wolsten-\n<br>croft, K.: An experimental workﬂow development platform for historical document\n<br>digitisation and analysis. In: Proceedings of the 2011 workshop on historical\n<br>document imaging and processing. pp. 161–168 (2011)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12471px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[22] Oliveira, S.A., Seguin, B., Kaplan, F.: dhsegment: A generic deep-learning approach\n<br>for document segmentation. In: 2018 16th International Conference on Frontiers\n<br>in Handwriting Recognition (ICFHR). pp. 7–12. IEEE (2018)\n<br></span></div><span style="position:absolute; border: gray 1px solid; left:0px; top:12680px; width:612px; height:792px;"></span>\n<div style="position:absolute; top:12680px;"><a name="16">Page 16</a></div>\n<div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:134px; top:12772px; width:9px; height:8px;"><span style="font-family: CMR9; font-size:8px">16\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:167px; top:12772px; width:54px; height:8px;"><span style="font-family: CMR9; font-size:8px">Z. Shen et al.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12798px; width:345px; height:85px;"><span style="font-family: CMR9; font-size:8px">[23] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z.,\n<br>Desmaison, A., Antiga, L., Lerer, A.: Automatic diﬀerentiation in pytorch (2017)\n<br>[24] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen,\n<br>T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style,\n<br>high-performance deep learning library. arXiv preprint arXiv:1912.01703 (2019)\n<br>[25] Pletschacher, S., Antonacopoulos, A.: The page (page analysis and ground-truth\n<br>elements) format framework. In: 2010 20th International Conference on Pattern\n<br>Recognition. pp. 257–260. IEEE (2010)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12886px; width:346px; height:41px;"><span style="font-family: CMR9; font-size:8px">[26] Prasad, D., Gadpal, A., Kapadni, K., Visave, M., Sultanpure, K.: Cascadetabnet:\n<br>An approach for end to end table detection and structure recognition from image-\n<br>based documents. In: Proceedings of the IEEE/CVF Conference on Computer\n<br>Vision and Pattern Recognition Workshops. pp. 572–573 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12930px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[27] Qasim, S.R., Mahmood, H., Shafait, F.: Rethinking table recognition using graph\n<br>neural networks. In: 2019 International Conference on Document Analysis and\n<br>Recognition (ICDAR). pp. 142–147. IEEE (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12962px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[28] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object\n<br>detection with region proposal networks. In: Advances in neural information\n<br>processing systems. pp. 91–99 (2015)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:12995px; width:346px; height:63px;"><span style="font-family: CMR9; font-size:8px">[29] Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph\n<br>neural network model. IEEE transactions on neural networks </span><span style="font-family: CMBX9; font-size:8px">20</span><span style="font-family: CMR9; font-size:8px">(1), 61–80 (2008)\n<br>[30] Schreiber, S., Agne, S., Wolf, I., Dengel, A., Ahmed, S.: Deepdesrt: Deep learning\n<br>for detection and structure recognition of tables in document images. In: 2017 14th\n<br>IAPR international conference on document analysis and recognition (ICDAR).\n<br>vol. 1, pp. 1162–1167. IEEE (2017)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13061px; width:344px; height:30px;"><span style="font-family: CMR9; font-size:8px">[31] Shen, Z., Zhang, K., Dell, M.: A large dataset of historical japanese documents\n<br>with complex layouts. In: Proceedings of the IEEE/CVF Conference on Computer\n<br>Vision and Pattern Recognition Workshops. pp. 548–549 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13094px; width:344px; height:8px;"><span style="font-family: CMR9; font-size:8px">[32] Shen, Z., Zhao, J., Dell, M., Yu, Y., Li, W.: Olala: Object-level active learning\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13105px; width:261px; height:8px;"><span style="font-family: CMR9; font-size:8px">based layout annotation. arXiv preprint arXiv:2010.01762 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13116px; width:345px; height:41px;"><span style="font-family: CMR9; font-size:8px">[33] Studer, L., Alberti, M., Pondenkandath, V., Goktepe, P., Kolonko, T., Fischer,\n<br>A., Liwicki, M., Ingold, R.: A comprehensive study of imagenet pre-training for\n<br>historical document image analysis. In: 2019 International Conference on Document\n<br>Analysis and Recognition (ICDAR). pp. 720–725. IEEE (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13160px; width:346px; height:42px;"><span style="font-family: CMR9; font-size:8px">[34] Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P.,\n<br>Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-\n<br>the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019)\n<br>[35] Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., Girshick, R.: Detectron2. </span><span style="font-family: CMTT9; font-size:8px">https://\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13204px; width:207px; height:9px;"><span style="font-family: CMTT9; font-size:8px">github.com/facebookresearch/detectron2 </span><span style="font-family: CMR9; font-size:8px">(2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13215px; width:345px; height:30px;"><span style="font-family: CMR9; font-size:8px">[36] </span><span style="font-family: CMR9; font-size:8px">Xu, Y., Xu, Y., Lv, T., Cui, L., Wei, F., Wang, G., Lu, Y., Florencio, D., Zhang, C.,\n<br></span><span style="font-family: CMR9; font-size:8px">Che, W., et al.: Layoutlmv2: Multi-modal pre-training for visually-rich document\n<br>understanding. arXiv preprint arXiv:2012.14740 (2020)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13247px; width:344px; height:8px;"><span style="font-family: CMR9; font-size:8px">[37] Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:155px; top:13258px; width:234px; height:8px;"><span style="font-family: CMR9; font-size:8px">text and layout for document image understanding (2019)\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:135px; top:13269px; width:216px; height:8px;"><span style="font-family: CMR9; font-size:8px">[38] Zhong, X., Tang, J., Yepes, A.J.: Publaynet:\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:188px; top:13280px; width:67px; height:8px;"><span style="font-family: CMR9; font-size:8px">layout analysis.\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:154px; top:13280px; width:237px; height:30px;"><span style="font-family: CMR9; font-size:8px">ument\n<br>Analysis and Recognition (ICDAR). pp. 1015–1022.\n<br>https://doi.org/10.1109/ICDAR.2019.00166\n<br></span></div><div style="position:absolute; border: textbox 1px solid; writing-mode:lr-tb; left:263px; top:13269px; width:219px; height:30px;"><span style="font-family: CMR9; font-size:8px">largest dataset ever for doc-\n<br>In: 2019 International Conference on Document\n<br>IEEE (Sep 2019).\n<br></span></div><div style="position:absolute; top:0px;">Page: <a href="#1">1</a>, <a href="#2">2</a>, <a href="#3">3</a>, <a href="#4">4</a>, <a href="#5">5</a>, <a href="#6">6</a>, <a href="#7">7</a>, <a href="#8">8</a>, <a href="#9">9</a>, <a href="#10">10</a>, <a href="#11">11</a>, <a href="#12">12</a>, <a href="#13">13</a>, <a href="#14">14</a>, <a href="#15">15</a>, <a href="#16">16</a></div>\n</body></html>\n', metadata={'source': '../../docs/integrations/document_loaders/example_data/layout-parser-paper.pdf'})

Using PDFPlumber

Like PyMuPDF, the output Documents contain detailed metadata about the PDF and its pages, and returns one document per page.

from langchain_community.document_loaders import PDFPlumberLoader

loader = PDFPlumberLoader("../../docs/integrations/document_loaders/example_data/")

data = loader.load()
data[0]

API Reference:PDFPlumberLoader

Using AmazonTextractPDFParser

The AmazonTextractPDFLoader calls the Amazon Textract Service to convert PDFs into a Document structure. The loader does pure OCR at the moment, with more features like layout support planned, depending on demand. Single and multi-page documents are supported with up to 3000 pages and 512 MB of size.

For the call to be successful an AWS account is required, similar to the AWS CLI requirements.

Besides the AWS configuration, it is very similar to the other PDF loaders, while also supporting JPEG, PNG and TIFF and non-native PDF formats.

from langchain_community.document_loaders import AmazonTextractPDFLoader

loader = AmazonTextractPDFLoader("example_data/alejandro_rosalez_sample-small.jpeg")
documents = loader.load()

documents[0]

API Reference:AmazonTextractPDFLoader

Using AzureAIDocumentIntelligenceLoader

Azure AI Document Intelligence (formerly known as Azure Form Recognizer) is machine-learning based service that extracts texts (including handwriting), tables, document structures (e.g., titles, section headings, etc.) and key-value-pairs from digital or scanned PDFs, images, Office and HTML files. Document Intelligence supports PDF, JPEG/JPG, PNG, BMP, TIFF, HEIF, DOCX, XLSX, PPTX and HTML.

This current implementation of a loader using Document Intelligence can incorporate content page-wise and turn it into LangChain documents. The default output format is markdown, which can be easily chained with MarkdownHeaderTextSplitter for semantic document chunking. You can also use mode="single" or mode="page" to return pure texts in a single page or document split by page.

Prerequisite

An Azure AI Document Intelligence resource in one of the 3 preview regions: East US, West US2, West Europe - follow this document to create one if you don't have. You will be passing <endpoint> and <key> as parameters to the loader.

%pip install --upgrade --quiet  langchain langchain-community azure-ai-documentintelligence

from langchain_community.document_loaders import AzureAIDocumentIntelligenceLoader

file_path = "<filepath>"
endpoint = "<endpoint>"
key = "<key>"
loader = AzureAIDocumentIntelligenceLoader(
    api_endpoint=endpoint, api_key=key, file_path=file_path, api_model="prebuilt-layout"
)

documents = loader.load()

documents[0]

API Reference:AzureAIDocumentIntelligenceLoader

How to load PDFs

Using PyPDF

Vector search over PDFs

Extract text from images

Using PyMuPDF

Using MathPix

Using Unstructured

Retain Elements

Fetching remote PDFs using Unstructured

Using PyPDFium2

Using PDFMiner

Using PDFMiner to generate HTML text

PyPDF Directory

Using PDFPlumber

Using AmazonTextractPDFParser

Using AzureAIDocumentIntelligenceLoader

Prerequisite

Was this page helpful?

You can also leave detailed feedback on GitHub.

How to load PDFs

Using PyPDF​

Vector search over PDFs​

Extract text from images​

Using PyMuPDF​

Using MathPix​

Using Unstructured​

Retain Elements​

Fetching remote PDFs using Unstructured​

Using PyPDFium2​

Using PDFMiner​

Using PDFMiner to generate HTML text​

PyPDF Directory​

Using PDFPlumber​

Using AmazonTextractPDFParser​

Using AzureAIDocumentIntelligenceLoader​

Prerequisite​

Was this page helpful?

You can also leave detailed feedback on GitHub.

Using PyPDF

Vector search over PDFs

Extract text from images

Using PyMuPDF

Using MathPix

Using Unstructured

Retain Elements

Fetching remote PDFs using Unstructured

Using PyPDFium2

Using PDFMiner

Using PDFMiner to generate HTML text

PyPDF Directory

Using PDFPlumber

Using AmazonTextractPDFParser

Using AzureAIDocumentIntelligenceLoader

Prerequisite