Linking archival entities and resources with SPARQL

Linked open data methodologies have an important role to play in the future of archival description.

With this in mind, I am moving EADitor and xEAC further in this direction. Both frameworks already support serialization of EAD or EAC-CPF into RDF. xEAC supports three different RDF models, in fact, depending on which community the data are intended to serve. EADitor transforms EAD finding aids into the Arch ontology. There are no true standards yet for representing archival resources as RDF, but I am hopeful that some will emerge. Since EADitor is now capable of embedding xEAC URIs for archival entities into EAD finding aids, the next step in linking resources together is the implementation of an RDF triplestore and SPARQL endpoint into both xEAC and EADitor.

An Example: Resource Relations

EAC-CPF records two primary types of relations: links to other corporate, personal, or familial entities in the form of CPF Relations and Resource Relations, or links to other resources by or about an entity that may be available on the web. It makes a lot of sense to me to store CPF relations within the EAC-CPF record, especially if these related entities are stored in the same information system (like xEAC). On the other hand, I don't think it makes sense to store resource relations within the EAC, mainly because I think that it's far too complicated to maintain a growing list of relationships.

Let's suppose you have an entity, Thomas Jefferson, defined by a URI, http://example.org/thomas_jefferson. A significant portion of his collection is contained at the University of Virginia and Monticello, but he corresponded with many of his other contemporaries. Therefore, the papers of John Adams or George Washington may also contain letters from Jefferson. He also corresponded with numerous prominent Europeans, so some of his materials may be contained in archives overseas. There may be dozens or hundreds of institutions which contain at least one article by or about Jefferson. If each of these archives adopts a stable URI that defines Jefferson, then it is much easier to accept RDF derived from EAD, MODS, or a relational database into an RDF triplestore, and use SPARQL to gather these materials dynamically from the triplestore when a researcher accesses the http://example.org/thomas_jefferson entity record.

This is the approach that I am implementing in xEAC and EADitor. For example, if the <origination> within the <did> of a finding aid (or individual component) contains an entity URI, the RDF derivative of the finding aid will link the archival resource to the archival entity through the dcterms:creator property. In EADitor, once the SPARQL endpoint URLs for querying, publishing, and updating data have been established, the RDF will be posted into the triplestore when the finding aid has been designated for publication on the web. Likewise, if these endpoint URLs have been added into the xEAC configuration file, the XSLT template for generating HTML from EAC-CPF will query the SPARQL endpoint to list related resources that have been pushed into the triplestore from EADitor. Furthermore, EADitor itself isn't necessary for this functionality in xEAC. RDF may be pushed into the triplestore by other means--from an institutional repository, from ArchivesSpace, or something else. You could even feed data from Europeana into a triplestore to build a prosopography of Impressionistic artists. The sky is the limit.

The SPARQL query looks something like this:

SELECT ?uri ?title WHERE {
?uri dcterms:creator <URI> ;
dcterms:title ?title
}

There is still work to be done in the UI, but the underlying technological functionality is now available in the Github repository for both applications.

Technical mumbo jumbo

The functionality for linking EADitor and xEAC to SPARQL endpoints is identical in both applications. The URLs are added through the Settings page. Under the SPARQL heading, the user clicks the "Connect" button, which launches a popup window requiring the user to input three separate URLs: the query URL, the Graph Store URL, and the SPARQL/Update URL. These URLs may vary from application to application, especially if the configurations have been changed. Note that a SPARQL 1.1-compliant endpoint is required.

After ending these URLs and clicking "Connect," the XForms engine will test each one individually. First, it will attempt a basic query. Then, it will post RDF into the Graph Store URL, and if successful, an XForms submission will be executed to delete the graph through SPARQL/Update. If all three processes complete successfully, the URLs will be added into the EADitor/xEAC config, and then the config can be saved. The user is also presented with the option to post all records that have been slated for publication (in Solr) into the RDF triplestore.

From the admin page, when a user deletes a record from the eXist database or removes the record from publication, the triples will be purged from the endpoint as well as the docs being deleted from Solr. When a user publishes a record, the record will be serialized into RDF and posted into the triplestore in addition to the user Solr publication. Likewise, triples will be updated when the user saves a published record from the EAD/EAC-CPF editing pages.

Ergo, enterprise archival linked open data publishing.

There is still work to do here: the ontology and RDF data models still need work, which is more of a community effort. And of course, I have a lot of plans for enhancing the user experience.

Once this new publication model is fully functional, I will begin SPARQL-based visualizations of social networks and relations between entities and their archival resources.

Incorporating xEAC entities into EAD finding aids

xEAC and EADitor were both conceived as standalone applications. This is what separates xEAC, especially, from other authority management modules that come packaged in larger archival suites like ICA-AtoM and ArchivesSpace. xEAC is applicable toward LAM authority control, for whom EAC-CPF is the primary audience, but can also be applied to scholarly prosopographies (and eventually support social network analysis built upon linked open data methodologies).

It is now possible to hook xEAC and EADitor together through an intermediate, optional, RDF triplestore and SPARQL endpoint. This will be discussed in greater detail in a later blog post. This particular post will detail the more immediate connection between entities defined in xEAC and personal, family, and corporate names within EAD finding aids.

Since its inception, xEAC has provided a Solr-based Atom feed for published EAC-CPF records. The Atom feed returns results based on the Lucene query syntax. A number of fields are available to narrow the search. For example, the entityType_facet Solr field allows a user to search for a name of a particular entity type, which is defined in the EAC-CPF schema as being either "person," "family," or "corporateBody." See http://admin.numismatics.org/xeac/feed/?q=augustus%20AND%20entityType_facet:person for example. The results are machine readable, and therefore the EADitor XForms application can read and process the search results. The interface for persname, corpname, and famname in EADitor has been adjusted to include xEAC lookups, and the functionality is practically identical to the VIAF lookup mechanism (which returns results in RSS as opposed to Atom).

Integrating a xEAC lookup mechanism into EADitor was incredibly easy as a result, and I managed to implement it in about 30 minutes. The Settings page for EADitor now includes an input for the xEAC home page URL. An XForms submission will process this URL and append 'feed/' to it to ascertain whether Atom XML is available at that resource. If so, the xEAC URL will be committed into the EADitor configuration file. If the xEAC URL is in the config, the persname, corpname, and famname element XBL components within the XForms application will include a radio button to select the xEAC lookup, in addition to VIAF and local vocabulary. When you select an entity after performing the lookup, the entity's URI will be embedded in the @authfilenumber attribute. In EAD3, this attribute will be @vocabularysource (I think), following linked data advancements in EAC-CPF.

When linking the EAD finding aid to an entity defined in xEAC, the @type attribute will be set for the persname, corpname, or famname to 'xeac:entity.' Ideally, I would like to avoid system-defined attributes, but I think they are very useful in this case, as it will indicate to the EADitor UI XSLT stylesheets that the EAC-CPF XML can be extracted programmatically by appending '.xml' to the entity URI, and therefore biographical information may be extracted directly from the EAC-CPF entity record. This, I believe, is the dream of the creators of EAC-CPF. Authority information and archival/biographical context are stored separately from the finding aid, but yet the information is made available through the finding aid user interface by means of linked open data methodologies.

I have not yet built these hooks into EADitor's finding aid user interface, but expect them to be available when the new version of EADitor is released later this summer. This feature represents a major advancement in the publication of archival materials.

But wait, there's more.

xEAC in London: SNAP 2014

I was recently in London to discuss ANS participation in the Standards for Networking Ancient Prosopographies meeting, hosted by King's College London, March 31 - April 1. Most Roman imperial people related to coins have already had URIs minted about them on nomisma.org, and we are about to create nomisma IDs for Roman Republican moneyers. Additionally, we will be creating a large number of Greek authorities in the coming months. The as-of-now institutionally unaffiliated kerameikos.org is another thesaurus which will contain numerous URIs for Greek potters and painters which may supplement SNAP with biographical information or objects of cultural heritage. Lastly, I discussed the Roman Imperial Social Network (RISN) project, for which we have requested a grant to develop further. This is a prosopography of the Roman Empire built on EAC-CPF from existing open resources on the web (such as Tom Elliott's PIR RDF and DBpedia) and embellished with xEAC to include important life events, occupations, and places (linked to Pleiades).

Meeting Overview

The meeting was a great experience, and I always find these sorts of gatherings (like LAWDI) useful for learning what other people are working on. There was, of course, some overlap from previous LAWDIs, in terms of participants or projects. The Syriac Reference Portal once again made an appearance, as did Tresmigestos. Maggie Robb is working on a prosopography of the Roman Republic, and so there's a possibility for collaboration between her project and nomisma.org, as we are going to create nomisma IDs for Republican moneyers in the coming weeks, and will certainly incorporate her URIs into our system. Day 1 included presentations and discussions by SNAP organizers as well as a selected group of participants which have either potential datasets to include in SNAP or are using tools or other methodologies for prosopographies. My presentation contained a bit of both tools/methodologies and datasets. In Day 2, the meeting participants split into smaller groups for focused break-out sessions on various topics.

Incorporating RDF relationship ontologies into xEAC

Several months ago, just after presenting the latest developments in xEAC at MARAC, I wrote on the application's enhanced relationship maintenance capabilities. The new system required manual entry of relationships into the system. One of the questions I received at MARAC was, basically, will xEAC be able to harvest from existing ontologies? Now, the answer is "yes."

While this is still very much a prototype (because there may be numerous ways of constructing a relationship ontology in RDF), I have successfully implemented an RDF (XML) upload mechanism. The xEAC relationship maintenance section will parse the RDF/XML provided http://vocab.org/relationship/. The XForms processor will read the relationship properties in the file, creating symmetrical or inverse relationships when applicable. It allow you to select the prefix you would like to use to define the ontology and will create the localTypeDeclaration that contains the abbreviation (the prefix) and citation (URI) if it does not already exist in the config.

Therefore, it will take some RDF that looks like this:

<rdf:Description rdf:about="http://purl.org/vocab/relationship/grandchildOf">
 <rdf:type rdf:resource="http://www.w3.org/1999/02/22-rdf-syntax-ns#Property"/>
 <owl:equivalentClass rdf:resource="http://www.perceive.net/schemas/relationship/grandchildOf"/>
 <owl:inverseOf rdf:resource="http://purl.org/vocab/relationship/grandparentOf"/>
 <rdfs:subPropertyOf rdf:resource="http://xmlns.com/foaf/0.1/knows"/>
 <rdfs:subPropertyOf rdf:resource="http://www.w3.org/2002/07/owl#differentFrom"/>
 <rdfs:label xml:lang="en">Grandchild Of</rdfs:label>
 <rdfs:label>Grandchild Of</rdfs:label>
 <skos:definition xml:lang="en">A person who is a child of any of this person's children.</skos:definition>
 <rdfs:domain rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
 <rdfs:range rdf:resource="http://xmlns.com/foaf/0.1/Person"/>
 <rdfs:isDefinedBy rdf:resource="http://purl.org/vocab/relationship/"/>
 <skos:historyNote rdf:nodeID="mor53341b13853b8"/>
</rdf:Description>

And turn it into this:

Once you've saved xEAC's settings, these relationships will be available through the @xlink:arcole in CPF Relations in the EAC-CPF form.

Of course, after you establish a relationship between your source and target person, family, or corporate body record, the target EAC-CPF record will be updated with the symmetrical/inverse relationship which points back to the source. These relationships will be expressed in RDF output generated by xEAC.

Serializing EAC-CPF into CIDOC CRM

xEAC supports a fairly rudimentary RDF/XML output by appending '.rdf' onto a URI for an entity. There is an RDF ontology based on EAC-CPF, but I am not sure it has seen wide usage (it will eventually be implemented in xEAC, regardless). The RDF model employed in xEAC out of the box is little more than a proof of concept, a placeholder until a more standard model emerges from the archival community. It is based slightly on Aaron Rubinstein's arch ontology and contains little more than labels for name entries, relations (from CPF relations that contain an RDF predicate in the @xlink:arcole), and a dcterms:abstract derived from the EAC-CPF abstract element.

There has been some use of CIDOC CRM to model people. Much of this work has been done by Michele Pasin and John Bradley at King's College London (see their paper). I am heading to London next week for the first meeting of the Standards for Networking Ancient Prosopographies project, and I suspect I will hear much more about their work in this regard there. In order to reach the broadest audience, I am making EAC-CPF data available through xEAC, serialized into CIDOC CRM. This is no easy task, but I have gotten the ball rolling a little bit. I will make more progress once I learn more about the model at the SNAP meeting.

The great advantage of the CRM is that since it is very generalizable, it can be used to model anything. This is a double edged sword, however, since it can be so generalizable that a complicated model is sometimes necessary to communicate a relatively simple concept.

Exist Dates

In EAC-CPF, the date range of existence occupies about four lines of XML, and the @standardDate, @notBefore, and @notAfter attributes communicate ISO standard dates and some semantic certaintly (or uncertainty). These can be modeled in CRM, but in a more complicated fashion. First, a person (or family or organization) was P92i_was_brought_into_existence_by an E63_Beginning_of_Existence which P4_has_time-span designated by a E52_Time-Span which has a human-readable rdfs:label and machine readable P82a_begin_of_the_begin and P82a_end_of_the_begin. @notBefore is P82a_begin_of_the_begin and @notAfter is P82a_end_of_the_begin. That's for a eac:fromDate. An eac:toDate has all of this stuff in a slightly different manner, with End_of_Existence and begin_of_the_end and end_of_the_end. The creation and end of an event can have a place as well, but there are some difficulties in translating birth and death places from EAC-CPF into CRM in this regard.

First, the semantic of an exist date is a bit fuzzy. By definition the existDates are "The dates of existence of the entity being described, such as dates of establishment and dissolution for corporate bodies and dates of birth and death or flourit for persons." The only way to determine between the birth and death dates of a person and the flourit is by using the localType attribute, and the values of @localType may vary from project to project. Therefore, if the entity being described is a person and the existDates are of his or her birth and death, then I should be using properties related to E67_Birth instead of the more generic E63_Beginning_of_Existence (of which E67_Birth is a subclass). Instead, I must opt for the more generic class. The same goes for organizations. Ultimately, the solution to this problem is to implement in the xEAC editing interface a checkbox for inserting a @localType designating whether the existDates are of the life or flourit of the person or organization (e.g., @localType='xeac:birth' or 'xeac:death'). The same goes for the place of birth or death. That way the XSLT stylesheets can read the system-based localType attribute and construct the CIDOC CRM model accordingly, and allow for variation between the exist dates for persons or corporate bodies.

This is something I will continue to wrestle with over the coming weeks, but eventually I hope to have a fully compatible crosswalk between EAC-CPF and both CIDOC CRM and TEI. CRM includes properties for relating children with parents, but arguably these types of relationships should be maintained in a separate ontology build specifically for relations. In fact, there could be many relationship ontologies, depending on the needs of the project. This I am sure will be a topic of discussion at SNAP.

Resources

• http://admin.numismatics.org/xeac/api/get?id=alexander_the_great&model=cidoc-crm
• XSLT: https://github.com/ewg118/xEAC/blob/master/ui/xslt/serializations/eac/rdf-templates.xsl (follow the mode="crm")

Exporting EAC-CPF to TEI

As indicated in the TO-DO list in the recent xEAC beta announcement, and as part of the design specifications for our IMLS grant application for the further development of xEAC (and the creation of a prosopographical datset of the Roman Empire), I have implemented a basic EAC-CPF-to-TEI transformation. It isn't yet a complete crosswalk, but it handles the following:

• name entries
• biographical description
• the generation of a chronological list of events (including descriptions with normalized dates and places that link to either Pleiades or Geonames)
• a list of relations, implementing semantic relationships defined in the @xlink:arcrole of the cpfRelation element.

There is now a link on the HTML page for an entity record to the TEI export. One can access this alternate model by appending '.tei' to the URI, e.g., http://admin.numismatics.org/xeac/id/id/alexander_the_great.tei.

The XSLT stylesheet is available at Github at https://github.com/ewg118/xEAC/blob/master/ui/xslt/serializations/eac/tei.xsl.

The model is based upon the TEI Prosopography documentation and some examples in the Lexicon of Greek Personal Names.

Further AAT Integration into EAC-CPF and EAD

xEAC

Like occupations, the function element in EAC-CPF has also been hooked into the Getty AAT via XForms-based SPARQL queries. The top concept for the function facet in the AAT is http://vocab.getty.edu/aat/300054593. According to Kathleen Roe on the EAD listserv:

The Getty functions vocabulary was built from an analysis of government functions in the U.S. as part of the Government Records Description project (early 1990s) undertaken by 12 or so state archives, including Utah (Jeff Johnson was the state archivist at the time).

 The query appears as follows:

SELECT ?c ?label {
?c a gvp:Concept; skos:inScheme aat: ;
gvp:broaderTransitive aat:300054593 ;
gvp:prefLabelGVP/xl:literalForm ?label ;
luc:term "SEARCH_QUERY*"
} LIMIT 25

 

EADitor

I extended these functionalities to EADitor, with the code practically copied and pasted from the genreform XBL into the function and occupation components (just replacing genreform with occupation/function and updating the SPARQL query). EADitor now supports lookup mechanisms for the following controlled access terms:

• geogname: Geonames (modern)/Pleiades (ancient)
• genreform: AAT/LCGFT
• function: AAT
• occupation: AAT
• subject: LCSH
• persname: VIAF
• corpname: VIAF

EADitor will eventually hook into SNAC (or whatever evolves from it) for persname, corpname, and famname, and I will extend it to hook into xEAC for linking finding aids with EAC-CPF records. xEAC already has a REST query mechanism that returns results in the form of Atom XML, so this will be pretty easy.