# Unlock the potential of text data. elinor - text as data supports you in analyzing text data and finding patterns and insights that are impossible to see with traditional analytics tools

### **A CLOUD-BASED TOOL TO STRUCTURE TEXT DATA** elinor is a cloud-based annotation tool that lets you organize unstructured text data and turn it into structured data that can be instantly analyzed. It enables you to 1) quickly develop digital maps of your domain expertise (ontologies) and 2) structure your documents around them (annotation). This way, we can combine the best of two worlds in text annotation: You can annotate large gold standard data sets for NLP algorithms like with traditional machine learning annotation tools. You can also create information-rich annotations like with social science coding tools. [**Book a demo now**](https://meetings-eu1.hubspot.com/johannes-mueller) and get a free consultation from our NLP experts!

### HELPING PROJECT MANAGERS AND DATA TEAMS CREATE IMPACT We make data work in the service of the public good. The public sector, welfare organizations, foundations, research institutes, and social businesses want to leverage data and data products to elevate their impact. elinor helps to unlock the potential of text data for good causes. [**Read our whitepaper**](https://www.and-effect.com/publications/2022-10-20_whitepaper_elinor_use_cases.pdf) on "The impact of text data in the public and social sector"!


For Project Managers	elinor helps you to structure text data and turn it into actionable insights. Improve document processing, better understand trends and set data standards in your sector.	If you don't have a dedicated data team, we are offering individual services from data collection to machine learning engineering.	/pages/hmMvCMywRia8R9l5I0Mw
For Data Scientists	Use ontologies and our intuitive annotation interface for natural language processing tasks like document classification, entity recognition, or entity linking.	We host elinor for you, so you can focus on building gold-standard data sets and your machine-learning models!	/pages/8GGSpB7HocRgeyXXYG69

For Project Managers

elinor helps you to structure text data and turn it into actionable insights. Improve document processing, better understand trends and set data standards in your sector.

If you don't have a dedicated data team, we are offering individual services from data collection to machine learning engineering.

/pages/hmMvCMywRia8R9l5I0Mw

For Data Scientists

Use ontologies and our intuitive annotation interface for natural language processing tasks like document classification, entity recognition, or entity linking.

We host elinor for you, so you can focus on building gold-standard data sets and your machine-learning models!

/pages/8GGSpB7HocRgeyXXYG69

We developed elinor based on our experience with text data in the social and public sector.

### WHAT CAN ELINOR DO FOR YOU?

	HubSpot Link
Let's talk about what elinor can do for your organization.	https://meetings-eu1.hubspot.com/johannes-mueller	/files/C16Lu4iJV4xbPun9IFWd
Read more about ontology, project, and annotation management.	/pages/HFqJOWcykpHO8ovV7prK	/files/nWLmgrgbJXeHAQizxwic
What can elinor do for organizations and data science teams?	/pages/CVZ7xltsPQoQhNdXeqnW	/files/iB7bMARM6VkoendA7HaJ

### ABOUT US *elinor - text as data* is a product developed by [**\&effect data solutions GmbH**](https://and-effect.com) and supported by the **Federal Ministry for Economic Affairs and Climate Action**.


Developed by	/files/YtbZbZ8GtVEm6KkvyZrW
Supported by	/files/io0L7Oo5YN8K3KBA55Ut

# Features With elinor, domain knowledge is at the center of your natural language processing pipelines. ## 1. Ontology Management Translate your domain into a structured ontology - a digital representation of your expertise. elinor represents ontologies as linked data using the SKOS standard - making them easy to reuse, adapt and share.

* Rich concept descriptions using preferred labels and alternative labels. * Unlimited nested hierarchical models * Multi-lingual by default * Linked Open Data ready ## 2. Annotation Management Upload and classify thousands of documents effortlessly: Annotate documents, named entities, or word sequences using your domain ontology. Our annotation interface makes your life easy - even with hundreds of concepts.

* Intuitive annotation process * Never lose oversight - even with hundreds of concepts * Annotate entities, spans, or whole documents * Results can immediately be analyzed or used as training data ## 3. Collaboration & Project Management With our flexible and collaborative project structure, you can work in teams and iterate over ontologies and annotation guidelines to achieve scientific-grade results.

* Structure the annotation process using experiments * Automatically distribute tasks for different purposes * Include domain experts in the process to achieve the best results * Develop annotation guidelines in the app [**Book a demo now**](https://meetings-eu1.hubspot.com/johannes-mueller) and get a free consultation from our NLP experts! # elinor for Organisations elinor helps organisations structure their text data, get insights and enable impact > "Data locked away in text, audio, social media, and other unstructured sources can be a competitive advantage for firms that figure out how to use it." > > [MIT Sloan 2021](https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data) ### The impact of text data in organisations By structuring text data, organizations can: * Understand public opinion * Improve the processing of documents * Understand trends * Enrich existing databases * ... [**Read about actionable insights in our whitepaper**](https://www.and-effect.com/publications/2022-10-20_whitepaper_elinor_use_cases.pdf) on "The impact of text data in the public and social sector"! ### From idea to finished product with \&effect While elinor is a stand-alone cloud solution for organizing ontologies and annotating documents, we can help you get from idea to data product with our service packages. [**Discuss your ideas with our NLP experts** ](https://meetings-eu1.hubspot.com/johannes-mueller)and get a quote for a customized project or product.


Data Collection	Not sure if you already have the necessary data? We can help you collect data from the web, documents, from social media, and many more sources.
Ontology Development	We help you translate your domain into an explicit ontology. We help you find existing taxonomies and develop custom workshops with experts.
Project Management	We support your annotation projects with hands-on support during annotation experiments - like coaching annotators, assessing annotation quality, and writing annotation guidelines.
Machine Learning	We develop custom machine-learning pipelines based on your annotation projects: From pre-processing to deployment.
Data Products	We connect your machine learning pipelines to your infrastructure and develop custom web applications that help your organization put data into action.

# elinor for Data Teams Domain-driven natural language processing for better results. ### elinor for Data Science **Understanding your domain**: No matter if you want to classify documents, find mentions of entities in texts or disambiguate entity candidates. Explicitly stating hierarchies, concept relations, and synonyms help you understand what you want to model. Encoding your domain knowledge in an ontology creates transparency and clarity. **Making predictions interoperable**: Using explicit data schemas, you can use predictions from your trained model in other contexts and combine them with other datasets. Ontologies are nothing new (not by a long shot), but linked data is having a revival - for a good reason. **Interpretable and smaller models**: By infusing your domain knowledge into text transformer models (e.g., with siamese neural networks), your models can be considerably smaller than full-blown transfer-learned foundational models. Your results will be as accurate or even better - while being easier to interpret. elinor is fully hosted, so you can concentrate on building better models and collaborating with your team. [**Book a demo now**](https://meetings-eu1.hubspot.com/johannes-mueller) to see what elinor can do for your data team! ### **Use Cases** * Linked Open Data with ontologies * Named Entity Recognition * Span Classification * Text Classification * Entity Linking and Entity Disambiguation # elinor 101 What is elinor about and how does it work? elinor is an annotation and ontology management tool. Whether you’re just interested in elinor and our approach to NLP or are getting started with elinor yourself - this document is for you. Here we cover the design philosophy of elinor and a high-level overview of the annotation process. ## Philosophy We at \&effect developed elinor after we couldn't find any tool that met our needs. There are excellent tools to annotate when you just have a few labels (e.g. Prodigy). However, we often had large corpora to annotate and, at the same time, large codebooks or taxonomies (sometimes with thousands of labels). This is a form of natural language processing that stays as close as possible to the domain and incorporates experts' domain knowledge. With elinor, we want to return the domain to the center of natural language processing. **Understanding your domain**: No matter if you want to classify documents, find mentions of entities in texts or disambiguate entity candidates. Explicitly stating hierarchies, concept relations, and synonyms help you understand what you want to model. Encoding your domain knowledge in an ontology creates transparency and clarity. **Making predictions interoperable**: Using explicit data schemas, you can use predictions from your trained model in other contexts and combine them with other datasets. Ontologies are nothing new (not by a long shot), but linked data is having a revival - for a good reason. **Interpretable and smaller models**: By infusing your domain knowledge into text transformer models (e.g., with siamese neural networks), your models can be considerably smaller than full-blown transfer-learned foundational models. Your results will be as accurate or even better - while being easier to interpret. ## The Project Workflow Here is a prototypical workflow for developing an NLP-product with elinor: 1. **Develop an ontology**: What do you want to model? Which categories are needed, and what concepts do you expect to find in the documents? We believe ontology development is an iterative process, so start with some first ideas. If you want to use an existing ontology or taxonomy, just upload it using .rdf or .json files (coming soo). Read more [Ontologies](/documentation/getting-started/ontologies) 2. **Upload documents:** Upload documents that you are interested in. So far, you can only upload .csv files, but more formats are coming soon. You can structure documents in folders and reorganize them in the document hub. Read more [Documents](/documentation/getting-started/documents) 3. **Create a project**: Create a project that is linked to the ontology. Here you can also start annotation guidelines explaining how annotators structure the documents. These guidelines will probably change as you go as you have to add or change rules to make annotations consistent. Read more [Projects](/documentation/getting-started/projects) 4. **Create experiments and assign documents to annotators**: First, you might want to test your ontology and annotation guidelines. For this purpose, you can give several documents to multiple annotators to check their annotation agreement. Then you can assign larger chunks of documents to different annotators to process them faster. Read more [Experiments](/documentation/getting-started/experiments) 5. Annotate documents: After documents are assigned, the annotators can start processing the documents. You can assign word sequence labels as well as document labels. The project manager can conveniently monitor the progress in the experiment view. Read more [Annotation](/documentation/getting-started/annotation) 6. **Export your results:** You can export annotated documents conveniently in .json format. You can then feed it as training data into your machine-learning model. You can also directly analyze the results to understand how topics are distributed in your data. Read more [Annotation](/documentation/getting-started/annotation) ## Glossary | Term | Description | | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Natural Language Processing (NLP) | Using computational techniques to analyze, structure, and process natural language and speech. | | Annotation | Annotation is the process of attaching labels to words, phrases, or documents. | | Gold standard | A gold standard is a data set of documents for which human annotators have added labels. This data set is usually used for training or fine-tuning machine learning models. | | Ontology | An ontology is a collection of concepts that represent important ideas or topics of your domain. In other tools these might be called codebooks, or taxonomies. Our ontologies are developed as SKOS-compliant and can therefore be used for many use cases other than annotating text. | | SKOS | Simple Knowledge Organization System is a W3C recommendation designed to represent thesauri, classification schemes, taxonomies, subject-heading systems, or any other type of structured controlled vocabulary. [Wikipedia](https://en.wikipedia.org/wiki/Simple_Knowledge_Organization_System) | | Project | Projects structure your annotation workflow. In projects, you can start experiments to achieve specific goals. You can also develop annotation guidelines that help annotators label documents consistently. | | Experiment | An experiment is a collection of task assignments. It also serves as a milestone in the annotation process. | | Task (Assignment) | A task is assigning one document to one annotator within one experiment. This is important as documents might be annotated by different people in different experiments or projects. | # Getting started Creating your organisation and getting started. elinor is a cloud-based service. If you sign up as an **organization** we will set up an instance of elinor just for you. It will be available under **organization.elinor-app.com.** {% hint style="info" %} You don't have your private workspace yet? Contact us and get a demo and a free consultation\ \--> [**Book a demo!**](https://share-eu1.hsforms.com/1H2KmtPjYTOWNqoeYW6h_GQfc3la) {% endhint %} **organization.elinor-app.com** will serve as your private workspace where you can host **unlimited** **ontologies**, **documents,** and **projects.** When your workspace is set up, you can start inviting people and assigning them roles. {% content-ref url="/pages/OWUy9746CfZA2FXjEH58" %} [User Management](/documentation/user-management) {% endcontent-ref %} With people in your organization, you can get to work! Read here our guides to the different aspects of elinor. {% content-ref url="/pages/tyZzka4Ny3jCpnlgkCVp" %} [Ontologies](/documentation/getting-started/ontologies) {% endcontent-ref %} {% content-ref url="/pages/W6ZYckE8iSQQ6jkPZc4q" %} [Documents](/documentation/getting-started/documents) {% endcontent-ref %} {% content-ref url="/pages/IeRfwY5RBhiZmCG3ftdQ" %} [Projects](/documentation/getting-started/projects) {% endcontent-ref %} {% content-ref url="/pages/GKLhx1zyFT45ZMocoYqV" %} [Experiments](/documentation/getting-started/experiments) {% endcontent-ref %} {% content-ref url="/pages/jMWdyEGdw6WX3U1ptgYb" %} [Annotation](/documentation/getting-started/annotation) {% endcontent-ref %} {% hint style="info" %} If you have any questions, please reach out to ! {% endhint %} # Ontologies Ontologies are a digital map of your domain. ## Creating Ontologies An ontology structures topics, themes, and concepts. To create an ontology navigate to the ontology view and click New Ontology

Projects have the following **properties**: * **Name**: A distinctive name for the ontology. * **Description**: A description to describe the ontology. * **Color**: To better distinguish ontologies, you specify a color. * **Tags**: To better organize your ontologies, you can add tags that are shown in the overview. Additionally, you can specify advanced settings.

* **Public**: Should the ontology be available publicly? **This feature is not yet implemented.** * **Preferred Language:** Ontologies are, by default, multi-lingual. Here you can specify the main language of the ontology. ## Create concepts You can create a new concept by clicking Add concept

. For now a concept has three main properties: Preferred Label, Alternative Labels, Description.

Concepts have the following properties: * **Preferred Labels**: The Preferred Label allows one to identify a concept. You can only assign one Preferred Label for any given language within one ontology. * **Alternative Labels**: Alternative Labels can assign multiple labels in the same language to one concept. Alternative Labels can be used to define variations like slang, dialect, synonyms, and other variants of the same concept. * **Color**: The color will be shown when you annotate a word sequence in a document; see [Annotation](/documentation/getting-started/annotation#word-sequence-annotations). Each label has a language attached to it. To change the language click on the language code and select the language from the drop down menu. ## Organize concepts Concepts can be organized by sorting or nesting them. TODO # Documents Upload and manage documents Documents are an essential part of Elinor as they are annotated. After uploading, they are not linked to a specific project but can be used throughout the workspace. ## Upload Documents You can find all documents by navigating there through the menu. To upload documents, click

Elinor currently supports documents in the .csv format. You can either drag a document into the dashed area or click on it to select a file. When a file is selected, you need to fill out three fields: 1. What is the **delimiter** of your dataset? Usually, this is either a comma "," or a semicolon ";". 2. Which **column should be used for the annotation**? All column in your dataset will be read automatically. In elinor we call this column the "full\_text" column. 3. Which column contains the **title of the document**? This property is optional but might help to navigate the documents easier. When uploading documents all properties in your dataset are uploaded as document tags and are available in the annotation view. # Projects ## How Projects work The primary purpose of a project is to bring structure into your work. Project Managers and Researchers can create them by giving them a name and description. Each project contains one ontology the work is centered around. ## Creating a Project The main objective of a project is to serve as a reference point for all experiments and tasks for a specific ontology. It is also the entry point to get to experiments and tasks, create new ones and edit them. Once you are happy with your project description, click ‘Create Project’. Then you will be redirected to an overview of all of your projects. You can search or sort the projects and create new ones via the button on the top right

Projects have the following **properties**: * **Name**: A distinctive name for the project. * **Ontology**: A project has only one ontology associated with it. Note that the connected ontology can not be changed afterward! * **Description**: A description to describe the overall goal of the project ## Organizing your Projects After opening a project, you see several control panes: ![](/files/3gEXPgy0WB0RxT0zU6hh) **Experiments**: Organize your project in smaller modules like "testing an ontology", "testing annotation guidelines", "annotating a gold standard". **Annotation Guidelines**: Write or review the annotation guidelines associated with the project. **Export Annotations**: Select annotations from different experiments to export. **General Settings**: Change the properties of your project or delete it. # Experiments Experiments structure your annotation project: From testing the ontology to annotating a gold standard. ## How Experiments work On an abstract level, experiments are like milestones in your annotation project. If your projects involve multiple annotators and complex ontologies, you want to ensure that annotations are consistent and reliable. Iterating over the ontology and the annotation guideline can help you with that. On a more concrete level, experiments are a collection of assignments that connect a document to an annotator. For example, Annotator A processes documents 1 to 50, and annotator B processes documents 51 to 100. The experiment tracks the progress on the assigned tasks and is finished when all documents are processed. ## Creating an Experiment To create an experiment, go to your project and then open the experiment pane and click Create Experiment

. You are then guided through the process. ### 1. Experiment Setup In the first step you clarify what the experiment is about.

Experiments have the following **properties**: * **Name**: A distinctive name for the experiment. * **Purpose (optional)**: To better structure the annotation project, you can give the experiment a purpose. Check the description below for more information. * **Description (optional)**: A description to describe the experiments' overall goal. * **Due Date (optional)**: Give the experiment a deadline. This helps annotators to gauge when their annotation tasks should be finished. #### Purpose of experiments

Purpose	Description
Enrich an ontology	Annotate the documents to see if your ontology is covering all the concepts you need or if it needs some updates.
Test the annotation rules	Before you start creating a gold standard dataset your want to make sure that all annotators have a clear understanding of the annotation guidelines and concepts. Make sure to assign the exact documents to all annotators.
Annotate a gold standard	If your ontology is set and all annotators have a shared understanding of the project, you can assign documents automatically to different annotators to quickly create a gold standard that can be used for analysis or model training.
Annotate a subset of documents	Are there only a few documents you want to look at in detail? You can manually pick and assign documents.

### 2. Document Selection In the second step, you select the documents you want to be annotated. You can select any document from the document hub by selecting individual documents or folders.

### 3. Task assignment In the last step, you select the annotators that should process the documents.

First, you select the annotators. Here you can select as many annotators as your want from your organization. In the second step, you choose the task assignment strategy. There are two main ways of assigning documents: * **Manual Assignment**: You choose exactly which document should be processed by which annotator. * **Automatic Assignment**: You choose the assignment's parameters: How many documents should be co-assigned (meaning: to assign a document to multiple annotators) and how many annotators each co-assigned document should go.

{% hint style="info" %} Generally, it is a good idea to have some co-assignment. This way, you can check if annotators understand the guidelines and if the ontology is explicit enough. If your goal is to test the guidelines, you want a high co-assignment ratio - so that all annotators process the same documents. Later in the process, when you are confident in the rules, you can drop the co-assignment to 20% or less. {% endhint %} # Annotation ## Starting an annotation task As an annotator, you find all your tasks on your Dashboard under **Your Current Annotation Assignments.** ![Open experiment](/files/uaIlyA9LoDQSAj9YZ7fG) ## Annotating ### Annotation View After opening the annotation view, you will see:

* The **document** is in the middle. * The **concepts** are on the right side. * The **control pane** is on the left side. You can navigate to the following document by clicking Next

. By clicking "next", you complete the task. If you want to continue with the document later, you can click Skip

, which doesn't complete the task. You can also select another document from the list in the **control pane**. In the **control pane,** you also find details about the experiment as well as all tags that were uploaded with the documents (read more about tags here: [Documents](/documentation/getting-started/documents)). ### Word sequence annotations To annotate word sequences, you can click on any word in the document - which highlights it. By clicking on another word, the sequence is extended. You can unselect the highlighting by clicking on the first or last word in the sequence. After selecting a word sequence, you can select a label from the ontology on the right side. This annotates the word sequence. You can see the annotation by hovering over the underlined word sequence with the mouse. ![Example annotation](/files/cqhzNvMA8uUWmmSqCdGO) You can easily assign more labels by clicking on **Select same span again**. You can delete the annotation by clicking on the little trash can button on the side. ### Document annotations In addition to word sequence annotations, you can assign labels directly to the whole document. This can be helpful if you want to develop a text classification model or add meta information to your documents. Click on Add document label

above the document, select a label from the list on the right side. You can click on the "x" next to the label to remove a document annotation. remove label

## Annotation Guidelines Annotation guidelines can help annotators to clarify the rules and things to look out for. Annotation guidelines can be viewed by clicking on

in the **control pane**. Annotation guidelines are connected to the project and not the experiment, because they might change over the experiments but should be considered the ground rules for the whole project. They can be edited in the project view [Projects](/documentation/getting-started/projects#organizing-your-projects). ## Exporting Annotations After finishing the annotation tasks, you can export them via the project view [Projects](/documentation/getting-started/projects). So far, you can only export annotations in the .json format, but new export options will come soon.

You can specify which annotations you want to download. # User Management {% content-ref url="/pages/I0OCnEREVuLxAmUowDzS" %} [Setting roles](/documentation/user-management/setting-roles) {% endcontent-ref %} {% content-ref url="/pages/OEelq1FHf6HCLQWAZ86Z" %} [Inviting Members](/documentation/user-management/inviting-members) {% endcontent-ref %} # Setting roles How the permission system works. elinor has a role-based permission system. It is focused on creating easy workflows and collaboration in the workspace. Within one organization there are no ontology- or project-specific permissions. ## Permissions There are 3 roles of permission levels in the product.

Role	Description
Annotator	An annotator can be from your organization or an external expert. Their primary role in the process is annotating documents. They can see all projects and ontologies but can't create or edit them.
Researcher	A researcher can create ontologies and projects. They can also annotate. However, they can not invite users to the workspace.
Workspace Admin	The workspace admin can add new members to the workspace, delete users from the workspace, and change users' roles.

## Changing roles Only workspace admins can change roles. To make adjustments go to the user management view and click on the user role you want to change. Then you can select a new position. ![change roles](/files/1gNDtnkauLfs5tCUdL7K) **Necessary**: As a workspace admin, you can only change your role to researcher or annotator if there is another admin. # Inviting Members Invite new members to your workspace. Inviting new members to your workspace is easy. Navigate to the user management view and click

You can invite users by sending them an email with a one-time-token with which they can create a profile. # Building an efficient annotation workflow. Coming soon. # Creating ontologies using SKOS. Coming soon.