Privacy statement (GDPR)
With these data protection notices, Bielefeld University fulfils its obligation to provide information in accordance with Articles 13 & 14 of the EU General Data Protection Regulation (GDPR) on the above-mentioned processing of personal data. Terms such as "personal data", "processing", "data controller", "third party", etc. are used as defined in Article 4 GDPR.
Bielefeld University, a legal entity under public law established by the state of North Rhine-Westphalia (NRW), is responsible for processing the data. It is represented by its rector, Prof. Dr. Angelika Epple.
§ 1.1. Contact details of the data controller
- Data protection officer of the Faculty of History
- Prof. Dr. Stefan Gorißen
- Universitätsstraße 25
- D-33615 Bielefeld
- Phone: +49 521 / 106-3152
- Email: stefan.gorissen@uni-bielefeld.de
- Web: https://www.uni-bielefeld.de
§ 1.2. Technical contact person
- Dr. Johanna Vompras
- Email: nopaque@uni-bielefeld.de
- Web.: https://www.uni-bielefeld.de/sfb/sfb1288/projektbereiche/inf/
§ 1.2. Contact details of the data protection officer
- The data protection officer responsible is:
- Phone: +49 521 106-5225
- Email: datenschutzbeauftragte@uni-bielefeld.de
We process the personal data of our users only to the extent necessary to provide a functioning website and its functionalities. Collecting this information enables us to better diagnose problems with the application, provide support more effectively as well as ensure the continuous functionality of the service.
The following (personal) data is collected and stored within the system:
Master Data
Within the scope of user authentication, the following personal data is collected and processed:
- User name
The registration of users is required for the provision of access to services within NOPAQUE. The freely selectable username and the corresponding email address is used to persistently identify you in NOPAQUE. The provided email address might be used to contact you in case we noticed some malfunction, to announce maintenance, or to spread important information regarding Nopaque. If you reset your password, Nopaque will use your email address to send you reset instructions via email. Nopaque does not show your email address and the username to other Nopaque users per default – except, if user gave the permission to do so, according to § 7 paragraph 1 of the General Terms of Use for the use of NOPAQUE.
Protocol and administrative data
In general, when a website is visited, for technical reasons information is automatically sent from the browser to the server and stored there in access protocols. When using a web application, additional protocol data is also generated, which is necessary for tracking technical errors. This information includes:
- IP address
- Complete HTTP request URL
- HTTP action (e.g. GET: call up a page, POST: send form data)
- Access status (HTTP status code)
- Date and time of the action
Protocol data will be used by the development team in order to debug and improve the included tools. This data can only be viewed by the technical administration and by the employees responsible for the NOPAQUE platform. Protocol data is not directly associated with a user account. NOPAQUE deletes all this information when it gets more than six months old.
Further administrative data, which is essentially needed – to enable a proper user registration and user management functionalities:
- Attributes member_since and last_seen: for system cleaning purposes and detection of fake accounts
- Attribute confirmed: This attribute indicates whether the account has been activated via e-mail. This type of activation ensures that the stored e-mail address actually exists.
- Attribute terms_of_use_accepted: To verify that the terms of use have been accepted by the user. This attribute is set to false when the terms of use has changed. After the user has logged in again, the user is asked to accept the new terms of use.
Cookies
Cookies: Browsers store so-called cookies. Cookies are files that can be stored by the provider of a website in the directory of the browser program on the user's computer. These files contain text information and can be read again by the provider when the page is called up again. The provider can use these cookies, for example, to always deliver pages in the theme selected by the user.
The storage of cookies can be switched off in the browser settings or provided with an expiry time. By deactivating cookies, however, some functions that are controlled by cookies can then only be used to a limited extent or not at all.
NOPAQUE uses functionality cookies which deliver certain functions and allow to remember important information and user’s preferences:
- session: Recognition of a user during a session in order to assign personal content and other user-defined settings. The session cookie is deleted after closing the browser session.
- remember_token: Login script with ‘remember me’ feature allowing the user to preserve their logged in status. When the user checks the Remember Me option, then the logged in status is serialized in the session and stored in cookies in an encrypted way.
Cookies collected by NOPAQUE do not collect personal information of the users.
Content Data
The content data includes all data that is entered or created by users themselves in the system. This data is listed here because it is assigned to individual authors and may contain personal data. This may include: uploaded files, images, text documents, other media files and (interim) results after data processing and computations. Please note that files and scans submitted to NOPAQUE are safely stored on the NOPAQUE server in order to allow persistent access during a work session and between work sessions.
According to § 4 paragraph 1 - 3 of the General Terms of Use for the use of NOPAQUE at Bielefeld University, the users themselves are responsible for the content they upload and must comply with the legal provisions of data protection and copyright law. This includes in particular the deletion of personal data that may no longer be processed.
User-added Information (optional)
NOPAQUE also stores optionally user-added personal information, like user’s profile information (full name, affiliation) and user’s added profile photo (avatar).
The legal basis for the processing of personal data for user authentication is Article 6 (1) letter e GDPR. The processing is carried out within the framework of the fulfilment of the tasks of Bielefeld University in accordance with HG NRW (NRW Higher Education Act), if necessary in connection with an order of the university to be named or by a special law, e.g. University Statistics Act, State Civil Servants Act, Staff Representation Act, Equal Opportunities Act.
The collection of personal data for user authentication is based on the consent of the data subjects as stated in Article 6 (1) letter a GDPR. The legal basis for the transmission of personal data is Article 6 (1) letter c GDPR.
Your personal data, which are processed by Bielefeld University for the purposes mentioned under 2 A - D will not be transferred to third parties.
In individual cases, data may also be legally transmitted to third parties, for example, to law enforcement authorities for the investigation of criminal offences within the framework of the Code of Criminal Procedure (StPO). If technical service providers are given access to personal data, this is done on the basis of a contract in accordance with Article 28 GDPR.
NOPAQUE centrally bundles sophisticated data processing functionalities related to Digital Humanities (for example: pattern recognition or text mining) on its platform which are offered by third parties on external platforms. No personal data is passed on to third parties - only content data which is sent anonymously to the APIs of the third-party tools. The results of the calculations are then sent back to NOPAQUE and saved as results in user’s account.
For personal data mentioned under 2 E the following applies: we do not share your personal data with third parties unless you have given your consent. This consent is given if the user actively makes his profile public in NOPAQUE. As consequence, user-added information (as listed under point 2E of this declaration) and, if applicable, a listing of public corpora with metadata is then made visible to other NOPAQUE users.
Data processed for user authentication, all personal data and contents uploaded by the user (listed in 2) are deleted immediately after account deletion.
As a data subject, you have certain rights under GDPR that you may assert at any time:
- the right to access information about whether or not personal data concerning you is processed, and if so, what categories of data are being processed (Article 15 GDPR),
- the right to demand the rectification or completion of data concerning you (Article 16 GDPR),
- the right to erasure of your personal data in accordance with Article 17 GDPR,
- the right to demand the restriction of the processing of your data per Article 18 GDPR,
- the right to withdraw your consent at any time. The withdrawal of consent does not affect the lawfulness of the processing based on consent before its withdrawal (Article 7 (3) GDPR),
- the right to object to the future processing of your data in accordance of Article 21 GDPR,
- the right to receive personal data concerning you and your account in a structured, common and machine-readable format in accordance of Article 20 GDPR.
In addition to the aforementioned rights, you have the right to lodge a complaint with the data protection supervisory authority (Article 77 GDPR); for example, the university is under the supervision of the
- North Rhine-Westphalia State Commissioner
- for Data Protection and Freedom of Information
- (Landesbeauftragte für Datenschutz und
- Informationsfreiheit Nordrhein-Westfalen)
- Kavalleriestraße 2-4
- 40213 Düsseldorf, German
Manual
- Introduction
- Registration and Log in
- Dashboard
- Services
- A closer look at the Corpus Analysis
- CQP Query Language
- Query Builder
- Tagsets
Introduction
nopaque is a web-based digital working environment. It implements a workflow based on the research process in the humanities and supports its users in processing their data in order to subsequently apply digital analysis methods to them. All processes are implemented in a specially provided cloud environment with established open source software. This always ensures that no personal data of the users is disclosed.
Registration and Log in
Before you can start using the web platform, you need to create a user account. This requires only a few details: just a user name, an e-mail address and a password are needed. In order to register yourself, fill out the form on the registration page. After successful registration, the created account must be verified. To do this, follow the instructions given in the automatically sent e-mail. Afterwards, you can log in as usual with your username/email address and password in the log-in form located next to the registration button.
Dashboard
The dashboard provides a central overview of all resources assigned to the user. These are corpora and created jobs. Corpora are freely composable annotated text collections and jobs are the initiated file processing procedures. One can search for jobs as well as corpus listings using the search field displayed above them.
A corpus is a collection of texts that can be analyzed using the Corpus Analysis service. All texts must be in the verticalized text file format, which can be obtained via the Natural Language Processing service. It contains, in addition to the text, further annotations that are searchable in combination with optional metadata that can be added during your analysis.
A job is a construct that represents the execution of a service. It stores input files, output files, processing status, and options selected during creation. After submitting a job, you get redirected to a job overview page. This can be accessed again via the job list on the dashboard. Jobs will be deleted three months after creation, so we encourage you to download the results after a job is completed.
Services
nopaque was designed from the ground up to be modular. This modularity
means that the offered workflow provides variable entry and exit points,
so that different starting points and goals can be flexibly addressed.
Each of these modules are implemented in a self-contained service, each of
which represents a step in the workflow. The services are coordinated in
such a way that they can be used consecutively. The order can either be
taken from the listing of the services in the left sidebar or from the
roadmap (accessible via the pink compass in the upper right corner). All
services are versioned, so the data generated with nopaque is always
reproducible.
File Setup
nopaque was designed from the ground up to be modular. This modularity means that the offered workflow provides variable entry and exit points, so that different starting points and goals can be flexibly addressed. Each of these modules are implemented in a self-contained service, each of which represents a step in the workflow. The services are coordinated in such a way that they can be used consecutively. The order can either be taken from the listing of the services in the left sidebar or from the roadmap (accessible via the pink compass in the upper right corner). All services are versioned, so the data generated with nopaque is always reproducible.
The File Setup Service bundles image data, such as scans and photos, together in a handy PDF file. To use this service, use the job form to select the images to be bundled, choose the desired service version, and specify a title and description. Please note that the service sorts the images into the resulting PDF file based on the file names. So naming the images correctly is of great importance. It has proven to be a good practice to name the files according to the following scheme: page-01.png, page-02.jpg, page-03.tiff, etc. In general, you can assume that the images will be sorted in the order in which the file explorer of your operating system lists them when you view the files in a folder sorted in ascending order by file name.
Optical Character Recognition (OCR)
Coming soon...
Handwritten Text Recognition (HTR)
Coming soon...
Natural Language Processing (NLP)
Coming soon...
Corpus Analysis
With the corpus analysis service, it is possible to create a text corpus and then explore it in an analysis session. The analysis session is realized on the server side by the Open Corpus Workbench software, which enables efficient and complex searches with the help of the CQP Query Language.
A closer look at the Corpus Analysis
Create a corpus
To create a corpus, you can use the "New Corpus" button, which can be found on both the Corpus Analysis Service page and the Dashboard below the corpus list. Fill in the input mask to Create a corpus. After you have completed the input mask, you will be automatically taken to the corpus overview page (which can be called up again via the corpus lists) of your new and accordingly still empty corpus.
Now you can add texts in vrt format (results of the NLP service) to your new corpus. To do this, use the "Add Corpus File" button and fill in the form that appears. You will get the possibility to add metadata to each text. After you have added all the desired texts to the corpus, the corpus must be prepared for the analysis, this process can be initiated by clicking on the "Build" button. On the corpus overview page you can always see information about the current status of the corpus in the upper right corner. After the build process the status should be "built".
Analyze a corpus
After you have created and built a corpus, it can be analyzed. To do this, use the button labeled Analyze. The corpus analysis currently offers two modules, the Reader and the Concordance module. The reader module can be used to read your tokenized corpus in different ways. You can select a token representation option, it determines the property of a token to be shown. You can for example read your text completly lemmatized. You can also change the way of how a token is displayed, by using the text style switch. The concordance module offers some more options regarding the context size of search results. If the context does not provide enough information you can hop into the reader module by using the magnifier icon next to a match.
CQP Query Language
Within the Corpus Query Language, a distinction is made between two types of annotations: positional attributes and structural attributes. Positional attributes refer to a token, e.g. the word "book" is assigned the part-of-speech tag "NN", the lemma "book" and the simplified part-of-speech tag "NOUN" within the token structure. Structural attributes refer to text structure-giving elements such as sentence and entity markup. For example, the markup of a sentence is represented in the background as follows:
<s> structural attribute
word pos lemma simple_pos positional attribute
<ent type="PERSON"> structural attribute
word pos lemma simple_pos positional attribute
word pos lemma simple_pos positional attribute
</ent> structural attribute
word pos lemma simple_pos positional attribute
</s> structural attribute
Positional attributes
Before you can start searching for positional attributes (also called tokens), it is necessary to know what properties they contain.
- word: The string as it is also found in the original text
-
pos: A code for the word type, also called POS tag
- IMPORTANT: POS tags are language-dependent to best reflect language-specific properties.
- The codes (= tagsets) can be taken from the Corpus Analysis Concordance page.
- lemma: The lemmatized representation of the word
-
simple_pos: A simplified code for the word type that covers fewer categories than the pos property, but is the same across languages.
- The codes (= tagsets) can be taken from the Corpus Analysis Concordance page.
Searching for positional attributes
Token with no condition on any property (also called wildcard token)
[]; Each token matches this patternToken with a condition on its word property
[word="begin"]; “begin”[word="begin" %c]; same as above but ignores caseToken with a condition on its lemma property
[lemma="begin"]; “begin”, “began”, “beginning”, …[lemma="begin" %c]; same as above but ignores caseToken with a condition on its simple_pos property
[simple_pos="VERB"]; “begin”, “began”, “beginning”, …Token with a condition on its pos property
[pos="VBG"]; “begin”, “began”, “beginning”, …
Look for words with a variable character (also called wildcard character)
[word="beg.n"]; “begin”, “began”, “begun” ^ the dot represents the wildcard characterToken with two conditions on its properties, where both must be fulfilled (AND operation)
[lemma="be" & simple_pos="VERB"]; Lemma “be” and simple_pos is Verb ^ the ampersand represents the and operationToken with two conditions on its properties, where at least one must be fulfilled (OR operation)
[simple_pos="VERB" | simple_pos="ADJ"]; simple_pos VERB or simple_pos ADJ (adjective) ^ the line represents the or operationSequences
[simple_pos="NOUN"] [simple_pos="VERB"]; NOUN -> VERB[simple_pos="NOUN"] [] [simple_pos="VERB"]; NOUN -> wildcard token -> VERB
Incidence modifiers
Incidence Modifiers are special characters or patterns, that control how often a character/token that stands in front of it should occur.
- +: One or more occurrences of the character/token before
- *: Zero or more occurrences of the character/token before
- ?: Zero or one occurrences of the character/token before
- {n}: Exactly n occurrences of the character/token before
- {n,m}: Between n and m occurrences of the character/token before
[word="beg.+"]; “begging”, “begin”, “began”, “begun”, …[word="beg.*"]; “beg”, “begging”, “begin”, “begun”, …[word="beg?"]; “be”, “beg”[word="beg.{2}"]; “begin”, “begun”, …[word="beg.{2,4}"]; “begging”, “begin”, “begun”, …[word="beg{2}.*"]; “begged”, “beggar”, …[simple_pos="NOUN"] []? [simple_pos="VERB"]; NOUN -> wildcard token (x0 or x1) -> VERB[simple_pos="NOUN"] []* [simple_pos="VERB"]; NOUN -> wildcard token (x0 or x1) -> VERB
Option groups
Find character sequences from a list of options.
[word="be(g|gin|gan|gun)"]; “beg”, “begin”, “began”, “begun” ^ ^ the braces indicate the start and end of an option groupStructural attributes
nopaque provides several structural attributes for query. A distinction is made between attributes with and without value.
- s: Annotates a sentence
-
ent: Annotates an entity
-
*ent_type: Annotates an entity and has as value a code that identifies the type of the entity.
- The codes (= tagsets) can be taken from the Corpus Analysis Concordance page.
-
*ent_type: Annotates an entity and has as value a code that identifies the type of the entity.
-
text: Annotates a text
- Note that all the following attributes have the data entered during the corpus creation as value.
- *text_address
- *text_author
- *text_booktitle
- *text_chapter
- *text_editor
- *text_institution
- *text_journal
- *text_pages
- *text_publisher
- *text_publishing_year
- *text_school
- *text_title
Searching for structural attributes
<ent> [] </ent>; A one token long entity of any type<ent_type="PERSON"> [] </ent_type>; A one token long entity of type PERSON<ent_type="PERSON"> []* </ent_type>; Entity of any length of type PERSON<ent_type="PERSON"> []* </ent_type> []* [simple_pos="VERB"] :: match.text_publishing_year="1991";Arbitrarily long entity of type PERSON -> Arbitrarily many tokens -> VERB but only within texts with publication year 1991Query Builder Tutorial
Overview
The query builder can be accessed via "My Corpora" or "Corpus Analysis" in the sidebar options. Select the desired corpus and click on the "Analyze" and then "Concordance" buttons to open the query builder.
The query builder uses the Corpus Query Language (CQL) to help you make a query for analyzing your texts. In this way, it is possible to filter out various types of text parameters, for example, a specific word, a lemma, or you can set part-of-speech tags (pos) that indicate the type of word you are looking for (a noun, an adjective, etc.). In addition, you can also search for structural attributes, or specify your query for a token (word, lemma, pos) via entity typing. And of course, the different text parameters can be combined.
Tokens and structural attributes can be added by clicking on the "+" button
(the "input marker") in the input field or the labeled buttons below it. Elements
added are shown as chips. These can be reorganized using drag and drop. The input
marker can also be moved in this way. Its position shows where new elements will be added.
A "translation" of your query into Corpus Query Language (CQL) is shown below.
Advanced users can make direct use of the Corpus Query Language (CQL) by switching to "expert mode" via the toggle button.
The entire input field can be cleared using the red trash icon on the right.
Content
Add new token to your Query
If you are only looking for a specific token, you can click on the left button and select the type of token you are looking for from the drop-down menu. By default "Word" is selected.
Word and Lemma
If you want to search for a specific word or lemma and the respective category is selected in the drop-down menu, you can type in the word or lemma of your choice in the input field. You can confirm your entry by clicking the Submit button on the right. You can also use the options below to modify your token request before pressing the submit button. These options are explained further here.
English pos, german pos or simple_pos
You can choose between the options "english pos", "german pos" and "simple_pos" to search for different parts-of-speech. You can find an overview of all tags under the "Tagsets" tab.
Empty Token
Here you can search for a token with unspecified attributes (also called wildcard token). This selection should never stand alone and should always be extended with an incidence modifier or stand in a larger query, because otherwise all possible tokens would be searched for and the program would crash.
Options for editing your query
You have the possibility to extend or specify the token you are searching for with certain factors. For this the query builder offers some fixed options. You can find more information about the options in the Corpus Query Language Tutorial.
Wildcard Character
A wildcard character replaces any character and is represented in the form of a dot.
Option Group
With an option group you can search for different variants of a token. The variants are not limited, so you can manually enter more options in the same format. "Option1" and "option2" must be replaced accordingly.
Incidence Modifiers
With the Incidence Modifiers you can determine the occurrence of single
tokens. For example you can use "?" to indicate that the token occurs either
not at all or once:
[word = "is"] [word="it"] [word="your"] [word="litte"]? [word = "dog"]
Here the word "little" should occur either once or not at all. With
[word="dogs?"] the search is for "dog "or "dogs".
Ignore Case
With the check mark at Ignore Case the upper and lower case is ignored. This is marked with a "%c". By default (if not checked) it is case sensitive.
"or" & "and"
"Or" ( | ) and "and" ( & ) are conditions you can put on a token. With "or"
one of the two conditions must be fulfilled, with "and" both conditions must be
fulfilled. For example, the word must be called "will" AND be a verb, only then
it will be displayed. Note that "and" is not responsible for lining up tokens in
this case. For this you can simply string them together:
[word="I"] [word="will" & simple_pos="VERB"] [word="go"].
Tokens that have already been added can also be modified by clicking on the corresponding pen icon. Click on the "ignore case" box, for example, and the query builder will not differentiate between upper- and lower- case letters for that respective token. New conditions added apply to the most recent token information.
Add structural attributes to your query
You can use structural attributes to search specifically for structures in the text or to further narrow down your previous search query.
Sentence
With "Sentence" (Click on Sentence to add the sentence chips:
Entities
With entities, i.e. units of meaning, you can search for text sections that
contain more specific information, for example, persons, dates, or events. The
codes for these categories can be selected using the drop-down menus. You can find an explanation of
these abbreviations under the tab "Tagsets".
You can also search for unspecified entities by selecting "Add entity of any type".
The entity type can be changed by clicking on the pen symbol on the chip. When the Entity attribute is added, the input marker will automatically be moved between the entity chips. Use drag-and-drop as needed to continue your query at a different position.
Meta Data (currently unavailable)
The meta data function is being worked on and cannot currently be used!
With the meta data you can annotate your text and add specific conditions.
You can select a category on the left and enter your desired value on the right.
The selected metadata will apply to your entire request and will be added at the end.
General Options of the query builder
You have several options to edit your query after adding it to the preview.
Editing the elements
You can edit your query chips by clicking on the pen icon.
Deleting the elements
You can delete the added elements from the query by clicking the X behind the respective content.
Move the elements of your query
You can drag and drop elements to customize your query.
Setting an incidence modifier
With the incidence modifier option, you can specify the amount of times a token should appear in your query. This is particularly relevant for empty tokens (tokens with unspecified attributes). Click on a token (blue chip) and select the desired option from the list to add an incidence modifier. To close the list without adding anything, click on the token again.
Switching between Query Builder and Expert mode
To work with the plain Corpus Query Language instead of using the Query Builder, click on the "expert mode" switch. Your query can be entered into the input field. All elements previously added will be carried over into expert mode. Click on the switch again to switch back to the Query Builder if desired. All recognized elements will be parsed into chips; those not recognized will be deleted from the query.
