Separating Links from Content using XML, XLink and XPointer
ABSTRACT
During the first few years of the web, publishers usually developed web sites by authoring content in HTML. However, as competition between web sites intensified, pressure mounted on publishers to build web sites that provided a richer user-interface and a more personalized experience for their users. It was recognized that using HTML to store content and style information in the same place was beginning to cause problems. This presentation explains the problems associated with inline linking using HTML linking as an example. It then explains how out-of-line linking with XLink and XPointer can help and the extra power and flexibility these technologies offer.
Table of Contents
1. Introduction
This paper explains inline and out-of-line linking. It discusses the most important issues to consider when authoring and managing link information, comparing the inline approach to the out-of-line approach.
In addition to walk throughs of linking with HTML and XLink, this paper discusses some of the architectural issues related to building out-of-line XLink based solutions.
2. What is a Link?
Let us start by defining what we mean by the word link. In a very general sense, we can think of a link as a connection or relationship between two or more things. These things could be located anywhere in space and time. They may be explicitly linked together or implicitly linked by applying a set of rules or assumptions.
Conceptually, a link connects at least two things together. In linking terminology, these points or locations are referred to as anchors. The simplest kind of link consists of two anchors. These are often referred to as the source anchor and the target anchor. When a user traverses a link, the traversal is started at the source anchor and terminates at the target anchor. In fact, certain environments, including XLink, allow links to be defined with multiple anchors and traversal rules.
Although this paper focuses on hypertext, the linking of text and other media types for delivery to end users, many of the issues and technologies discussed are also relevant to other types of linking.
3. Inline Linking
Inline links are the most common type of link. There are literally billions of these, with most of them in the HTML pages that make up the web. In fact, every HTML link is an inline link.
But what do we mean when we say a link is inline? In a nutshell, an inline link is a link that is defined within at least a part of the content that it links together. In most cases, the link is defined within the content that acts as the source anchor.
Let's take a look at an example of how an inline link is created. Below is a simple HTML document:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> <title>A Small Piece of HTML</title> </head> <body> <p>To learn about vegetarianism, visit the Vegetarian Society</p> </body> </html>
Now, let's say we wanted to create a link from the phrase Vegetarian Society to the corresponding web site. To do this, we need to insert an HTML anchor element with an href attribute. The result will look something like this:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>A Small Piece of HTML</title>
</head>
<body>
<p>To learn about vegetarianism, visit the
<highlight style="bital">
<a href="http://www.vegsoc.org">
</highlight>Vegetarian Society
<highlight style="bital">
</a>
</highlight>
</p>
</body>
</html>
The source anchor is defined by enclosing the phrase Vegetarian Society with the start and end tags of the HTML anchor element. This is what the user will click to traverse to the target anchor.
The target anchor is defined by adding an href attribute. This contains the address of the target resource (the URL of the Vegetarian Society's web site).
To create the link, we have had to modify the HTML that acts as the source of the link. In doing this, we have modified part of the content being linked and have thus created an inline link.
So, as can be seen, creating an inline link is quite a straightforward procedure. However, as we shall see, there are assumptions and problems inherent in this approach to linking. In the next section, we will take a closer look at some of the issues with creating and managing links in this way.
4. Problems with Inline Linking
You can quite happily get by with inline linking when you have a small number of resources and a small number of links. However, in most scenarios, we are dealing with many links across many resources. As these links and resource increase in number, they become increasingly difficult to manage. In the following sections we'll find out why.
4.1. Link Validation
If I create a link from resource A to resource B and resource B is subsequently deleted, my link is broken. If a user attempts to traverse the link, the link will fail. On the web, this is recognizable as the familiar HTTP error "404 Not Found". This is probably the most common problem associated with managing and using links.
Broken links are costly, because users will view your content as being out of date and poorly maintained. Users will become frustrated and opportunities will be missed.
With inline linking, there are several ways of dealing with the potential for broken links:
-
One option is to leave it to luck and hope your links are working. You may rely on users to report broken links, but you then have to manage these reports and go through the process of manually locating, checking and fixing these links. This is a haphazard approach to link validation and can prove to be very time consuming.
-
Another option is to periodically open each of your documents, locate each link and traverse it to see if it is broken. With a small number of links and resources this is no problem. However, this is fairly rare.
-
A related option, is to execute some utility that automates the manual process of locating and traversing links. With a few hundred resources this can be relatively painless. But with thousands or millions of resources the process will become very slow. This is because each resource will need to be parsed to find whether it contains any links and then each link will need to be traversed to check whether it is broken.
As can be seen, none of the options explained are very satisfactory. So with inline linking, their isn't really a practical and scalable way to validate your links.
4.2. Modifying Content
If you want to link from one resource to another, with inline linking, you need to insert link information into at least one of the resources you want to link together. This means modifying the content of at least one of the resources. Typically this is the resource that acts as the source of the link. Regardless, it still means you need write access to one of the resources. The problem is, if you don't have write access to at least one of the resources you will be unable to create any links.
This is quite inhibiting. For example, say you wanted to build a portal web site where you wanted to provide access to resources on third party web sites. Perhaps you would like to display links from third-party content to your own content. You could easily create links from your content to the third-party content, but you could not create links from the third-party content to your own content because would not have write access to the content. Given that you have reached some kind of agreement with the owner of the third-party web site, you have two options:
-
You could gain write access to the third-party web site so that you can edit their resources to create links from their content to your own. This is very unlikely because they probably wouldn't want someone else editing their data. Even if this could be done, anyone viewing their content would see your links, which is unacceptable in most cases. If the third party wanted to make their content available to someone else, they probably wouldn't want your links to appear.
-
You could make a separate copy of their data on your web server and add your own links. However, this is difficult to maintain. The next time they modify their content, assuming you wish to keep up-to-date, you'll need to get a new copy of their content and manually check and re-apply all of your links. Even for small changes you would have to go through this process.
Clearly both of the above options are impractical. Inline linking is just not flexible enough to allow this kind of application. Again, it boils down to the fact that with inline linking, the link information is embedded within the content it links together.
4.3. Multiple Link Sets
Sometimes it is desirable to deliver the same content to different types of user, but with different sets of links. For example, a novice may wish to see links to definitions of basic terminology, whereas an advanced user may wish to see links to content that requires a greater level of understanding
With inline linking, the only way to maintain different sets of links is to have multiple copies of the same content. A different set of links would be embedded within each copy. The problems start to occur when you decide to modify your content. Your options are:
-
One option is to modify each copy of your content. This would be tedious and prove inflexible. Also, this introduces scope for errors in your content authoring. What's to ensure that you replicate the changes precisely across each copy?
-
The other option is to maintain an unlinked master copy of your content. You would change your content only in this copy. Then, you would create new linked copies of your master copy by re-applying links from the old version of your linked content. This could be even more time consuming and error prone.
And again, we can see that inline linking prohibits the practical implementation of multiple link sets. It would become completely unmanageable.
4.4. Bi-directional Links
A bi-directional link has two anchors and allows traversal in both directions. These are useful when two things relate to each other or have a one to one mapping.
It is a common misconception that bi-directional linking is provided by a web browser's Back button. This is not the case because a Back button takes you back one-step in the browser's navigation history.
Say you had a unidirectional link with two anchors, anchor A and anchor B. If you traversed from anchor A to anchor B and then pressed the "Back" button, you would be taken back to anchor A. However, if you had started at anchor B, there would be no way of traversing to anchor A. This is because the link is unidirectional and not bi-directional.
With a truly bi-directional link, you can traverse between the anchors in both directions. So in our example, this would mean you could traverse from anchor A to anchor B and from anchor B to anchor A.
With inline linking, it is possible to mimic a bi-directional. You simply define two unidirectional links, each addressing the other as the target anchor. The problem is that from a link management perspective, I cannot tell by examining one of these links that it is intended to mimic a bi-directional link. The only way I could tell is by traversing both of the unidirectional links. Even then, it could be coincidental. Perhaps you can guess from the context of the content. But really, the bi-directional link is the mind of the person who authored the link.
So, even though it is possible to deliver the effect of a bi-directional link to an end-user, with inline linking, there is no concept of a bidirectional link. From a link management perspective there are two unidirectional links, no bidirectional link.
5. Out-of-line Linking
An out-of-line link is the opposite of an inline link. In essence, an out-of-line link is stored outside the content it links together. Out-of-line links are commonly referred to as third-party links. I've chosen to use the term out-of-line link to emphasise the polarity with inline linking.
The idea of out-of-line linking isn't new. In fact, for many years, there have been a number of proprietary software applications that have supported out-of-line linking. Also, in the SGML world, there is HyTime ISO/IE 10744, which supports constructs for out-of-line linking.
However, in the XML world, there has been no way of linking XML unless you use a proprietary system or XML ID/IDREFs. The ID/IDREF system can be viewed as a very simple inline linking mechanism that only works across an individual XML resource. This means links cannot be created between different XML resources.
So there has been a great need for a standard that allows XML to be linked. The answer to this need is W3C's XLink standard. It supports both inline and out-of-line linking. In XLink terminology, out-of-line links are called third-party links. Another important feature of XLink is it's support for defining the link semantics. Although important, this is not discussed here because in XLink, semantic information can be expressed for both inline and out-of-line links.
Often, the first question that springs to mind with out-of-line linking is this: If the links are outside the content, where are they defined? With XLink the answer is that the links are stored in a standalone XML resource referred to as a linkbase. Essentially, a linkbase is a collection of out-of-line links.
A linkbase is not much good on it's own without something that can interpret and act upon the links it defines. For this we need an XLink application. This is a piece of software that does useful things with XLink documents. But before we dive into XLink applications, let's take a look at how we can define an out-of-line link using XLink. Below we have two relatively simple XML resources:
anthony.xml:
<?xml version="1.0"?> <!DOCTYPE person SYSTEM "person.dtd"> <person> <name> <lastname>Duhig</lastname> <firstname>Anthony</firstname> </name> <notes>Anthony is a vegetarian.</notes> </person>
glossary.xml:
<?xml version="1.0"?> <!DOCTYPE glossary SYSTEM "glossary.dtd"> <glossary> <term ID="t1"> <name>Vegetarian</name> <definition>A vegetarian is someone who does not eat fish, game, meat, or poultry and who also avoids slaughterhouse by-products.</definition> </term> </glossary>
Now, lets define an out-of-line link from the word vegetarian in anthony.xml to the corresponding definition in glossary.xml. To do this, we need to define a linkbase using a separate XML resource:
<?xml version="1.0" ?>
<mylinkbase xmlns:xlink="http://www.w3.org/1999/xlink">
<mylink xlink:type="extended">
<myanchor xlink:type="locator" xlink:label="anchorA"
xlink:href="anthony.xml#xpointer(/child::person[
position()=1]/child::notes[
position()=1]/string-range(//*,'vegetarian'))"/>
<myanchor xlink:type="locator" xlink:label="anchorB"
xlink:href="glossary.xml#xpointer(id('t1'))"/>
<xlink:arc xlink:from="anchorA" xlink:to="anchorB"/>
</mylink>
</mylinkbase>
Let's take a look at each part of the linkbase, starting with the mylinkbase element:
<mylinkbase xmlns:xlink="http://www.w3.org/1999/xlink">
I decided to name this element mylinkbase, but I could have given it any name. The important thing to see is the namespace declaration. This declares that the mylinkbase element will be using the XLink namespace. Essentially, this declares that the element (and it's descendants) will use a set of XLink attributes that having meaning to an XLink processor. In other words, a software application that reads this XML will recognize and interpret these attributes according to the rules of the XLink specification.
The link is defined by an element I decided to call mylink as shown below:
<mylink xlink:type="extended">
<myanchor xlink:type="locator" xlink:label="anchorA"
xlink:href="anthony.xml#xpointer(/child::person[position()=1]/child::notes[
position()=1]/string-range(//*,'vegetarian'))"/>
<myanchor xlink:type="locator" xlink:label="anchorB"
xlink:href="glossary.xml#xpointer(id('t1'))"/>
<xlink:arc xlink:from="anchorA" xlink:to="anchorB"/>
</mylink>
It has an xlink:type attribute with a value of extended. It is this type of link that we use to define out-of-line links.
The other type of link supported by XLink is the simple link. An element acting as a simple link would have an attribute xlink:type="simple". A simple link essentially provides the same functionality as an HTML link.
However, the extended link is a more powerful linking construct. Not only does it allow links to be defined out-of-line, but it also enables a link to be defined with any number of anchors and arcs. It can also be used for defining inline links, but it is the ability to define out-of-line links that is of interest here.
The next significant part of the link is it's locators. A locator represents an anchor. In out-of-line links, it defines the anchor by addressing it rather than enclosing it with a start and end tag. Let's take a look at each of the locators. The first locator looks like this:
<myanchor xlink:type="locator" xlink:label="anchorA" xlink:href="anthony.xml#xpointer(/child::person[position()=1]/child::notes[ position()=1]/string-range(//*,'vegetarian'))"/>
Again, the element is declared as being a locator with the attribute xlink:type="locator". The locator consists of two parts. The first is the attribute xlink:label. The value of this attribute is used to reference the locator from elsewhere in the link. Specifically this is used by arcs, more on this later.
The other part of the locator is the href attribute. This defines the address of the anchor. This is divided into two parts separated by a # symbol. The first part is the resource address. This obviously addresses the XML resource.
The second part, the sub-address, is optional and is used to drill down into the XML resource to address an XML fragment. This address is sometimes referred to as the fragment identifier.
XLink uses another W3C standard called XPointer to address sub-resources. This provides a rich language for addressing parts of an XML document. It is essentially a superset of XPath and was developed for the general addressing needs of XML applications.
An XPointer expression is evaluated from left to right. As each part of the expression is evaluated, it takes the result of the previous part as it's input. The resultant node or nodes are known as the context node.
I'm not going to cover all of the XPointer addressing constructs here. But let's take a quick look at the components of this particular XPointer to see how it is evaluated:
xpointer(/child::person[position()=1]/child::notes[ position()=1]/string-range(//*,'vegetarian'))
-
The whole XPointer expression is enclosed within xpointer(). This tells the XLink processor to evaluate the expression as an XPointer.
-
The first part of the expression is the forward slash. This makes the root node of the document the context node.
-
The next part of the expression is /child::person[position()=1]. This selects the first person element that is a child of the context node (the document's root node).
-
The next part is /child::notes[position()=1]. This is evaluated as the first notes element of the context node (now the person element).
-
The final part is /string-range(//*,'vegetarian')). This is evaluated as the string vegetarian within the character content of the context node (now the notes element).
The second locator contains a different XPointer. This addresses the target anchor located in glossary.xml:
xpointer(id('t1'))
This addresses the element with an id attribute with a value of t1. Put another way, this addresses the term element containing the word Vegetarian.
The last part of the link is the arc:
<myarc xlink:type="arc" xlink:from="anchorA" xlink:to="anchorB"/>
The arc defines a specific traversal from one anchor to another. The source anchor is defined by the xlink:from attribute. The value of this attribute must match the value of the xlink:label attribute on one or more of the locator elements defined by the link. So in our example, the arc defines traversal from the word vegetarian in anthony.xml to the corresponding term in glossary.xml. Traversal from the target anchor to the source anchor is not allowed, so in this case we have created a unidirectional link. If we want to allow traversal both ways, in other words, to create a bi-directional link, we could define another arc, so our link would have the following arcs:
<myarc xlink:type="arc" xlink:from="anchorA" xlink:to="anchorB"/> <myarc xlink:type="arc" xlink:from="anchorB" xlink:to="anchorA"/>
In fact, with XLink, this bi-directionality can be achieved by specifying no arcs. In this case the omission of explicitly defined arcs is interpreted as meaning traversal is allowed between any two anchors defined by the link. In our case, this would create a bi-directional link. However, if we had created a link with say, three anchors and we did not define any arcs, this would mean traversal is allowed between any two anchors.
So, in summary we have created an out-of-line link defined in a XML linkbase that is separate to the XML resources that is links together. It is important to note that a linkbase can define any number of links and that in our example we have chosen to define just one.
5.1. Link Resolution
The next part of any XLink based solution is an XLink application. Without an XLink application, your XLinks are not much use. But what is an XLink application and what should it do?
In a very general sense, an XLink application is one that can interpret XML containing elements and attributes from the XLink namespace and that also complies with the rules of the XLink specification.
To bring our links to life, we need an XLink application that enables end-users to view and navigate through our XML resources using the links we have defined. In my view the most practical and flexible approach to this it to view the XLink application as a service, or more specifically a link resolution service. Basically, we ask the service for a specific XML resource. In response to this request, it returns the XML with the relevant links.
To be useful, I believe a link resolution service needs to return a copy of the original XML and that this should contain a set of inline links corresponding to the out-of-line links that address the resource. In this way, the XML can be easily transformed for delivery to a user agent such as a web browser. This approach allows for browser independent XLinking. For example, in a web environment, the following steps could take place:
-
Web browser requests XML resource
-
Web server forwards request to link resolution service
-
Link resolution service returns XML resource with inline XLinks representing applicable out-of-line links
-
An XSL processor transforms the XML and it's inline XLinks into well formed HTML and HTML links
-
Web server returns resultant HTML to web browser
-
Web browser renders HTML
In the above scenario, the web browser does not need to be XLink aware or even XML aware. At the server end, the XSL transforms the XML to HTML. Part of this transformation will transform the inline XLinks to inline HTML links.
Of course, if you know the web browser is XSL aware, you could move the XSL transformation to the browser by adding an XSL declaration to the top of your XML.
Although one can envisage an XLink compliant browser, in my view a more practical solution is to perform XLink processing at the server as previously explained. This way you can start building out-of-line XLink solutions right now, without having to wait for browser vendors to implement XLink compliant browsers.
5.2. High Performance Link Resolution
If you are dealing with a small number of links, performing link resolution by reading an XLink linkbase may be sufficient. But most solutions will need to handle hundreds, thousands or even millions of links.
In these cases, you'll need a link engine that can provide a fast link resolution service. To achieve this, the link engine needs to store links in a proprietary way optimised for quick access.
Despite the internals of the link engine, it would need to have the capability to import and export link information as XLink. In this way, even though the links may be stored in a proprietary way, your links could still be exchanged with other XLink compliant applications. Not only does this allow interoperation between applications but it also prevents vendor lock-in. Of course, this is the reason for having a standard in the first place. This is what XLink does for XML linking. Ultimately the purpose of having XLink is simply to communicate links in a standard way.
5.3. Batch vs. Dynamic Link Resolution
We have just covered link resolution. In particular, we covered what I like to refer to as dynamic link resolution. This is where link resolution is performed on the fly, in other words at the point in time when a particular resource is requested in a user environment.
The other way of performing link resolution is by periodically running a batch process, perhaps towards the end of a publishing cycle, to resolve all of the links for every resource. The result of this would be a completely linked copy of all your XML resources. These resources could be published to a web server or perhaps to CD or some other permanent storage medium.
As to which approach to choose, it very much depends upon the requirements and priorities of your application. Here are some points worth considering:
-
Dynamic resolution allows for more flexibility because it allows links to be selected on a per request basis, perhaps dictated by a user profile or a query. Link sets can be merged in any combination, as and when required.
-
Dynamic resolution requires less storage space because only the original resources need to be stored.
-
Batch resolution requires no extra processing time in a delivery environment because the link resolution is performed during the publishing stage
-
Batch resolution requires more storage space, especially if you need to deliver your content with different link sets.
6. How Out-of-line Linking Helps
When we examined inline linking, we noted some of the problems that this approach to linking can cause. Out-of-line linking doesn't solve everything, but it does help. Lets take another look at the problems we identified with inline linking and see how out-of-line linking helps.
With out-of-line linking, your links can be found quicker than with inline linking because your links are stored together in one place. There is no need to parse every single resource building a list of links.
Additionally, because link information is readily available, it becomes practical to perform some kind of impact analysis. For example, before deleting or changing a resource, you could quickly determine which links might be affected. With inline linking this would be totally impractical. You wouldn't want to search through a large document set to find all the links that address a specific resource.
Interestingly, one could envisage a link authoring system being tightly integrated with a content authoring system to the extent that the impact of content changes on a link set could be immediately drawn to a user's attention. This would be possible because the links are held together in a central place and are not scattered throughout the resources they link together.
6.1. Modifying Content
By definition, to create an inline link, we have to modify the content being linked. As we identified earlier, this is a major problem if you want to create links from content to which you do not have write-access. The only way to get around this with inline linking is to take a copy of the data and modify it. But this is impractical.
With out-of-line linking, the links are stored outside the content they link together. This means we no longer require write-access. We only need to have read-access. So, with out-of-line linking, write access is not an issue. This allows us to link any addressable information.
Going back to our portal example. If you wanted to create a portal web site that provided content from third-party web sites, you could create links from third-party content to your own content using out-of-line links. To resolve links in a given resource, a link resolution service would create a temporary copy of the original XML resource. It would then insert inline links into this resource and return this to the web browser. The inline links would represent the out-of-line links contained in your linkbase. The original XML resource would not be modified.
6.2. Multiple Link Sets
When we looked at managing multiple link sets with inline linking, we identified that we would need to maintain a separate copy of our content for each link set we wanted to publish.
Again, this problem is caused by the fact that inline links are by nature embedded within the content they link together. This is a similar to the problem we had before stylesheets, where style information was embedded within content.
With the advent of stylesheets, style information could be separated from content. This allowed content to be authored in a style independent way. If you wanted to deliver your content to a different device or provide a different view, all you had to do was to create a new stylesheet. This gave content publishers great flexibility. As with stylesheets, by separating links from content we gain similar benefits.
Going back to our example of multiple link sets. If we wanted to deliver the same content but with different sets of links, we could author our content once and just create a new set of out-of-line links for each type of user. This means that to create a new set of links, we don't have to create a new copy of our data. We have no additional content to maintain, just a new set of links. This means we don't have to change the way we author our content and at the same time it gives us greater flexibility with the way in which we deliver our links.
6.3. Bi-directional Links
With inline linking we cannot really create a bi-directional link. With inline linking, the link information is spread across the content it links together. There isn't really anything that ties both ends of a link together. However, with out-of-line linking there is. This is because the link information is held together in a linkbase outside of the content. Each link is defined in one place.
So, when we retrieve a link from a linkbase we immediately know how many anchors it has, how to locate the anchors and the traversals that are allowed between the anchors. In XLink, an out-of-line bi-directional link is represented by a link with two anchors and no arcs. This implies traversal both ways between the anchors. So, out-of-line linking and XLink deliver true bi-directional linking.
7. Conclusion
In this paper we have seen that by separating our links from our content by using out-of-line linking we can gain some significant benefits over inline linking. The main benefits are flexibility and being able to manage links in a centrally efficient way.
With the advent of XLink and the technology available on the market today, it is now possible to build applications that link XML content in powerful new ways.