How to Use XML and PDM in Document Production
ABSTRACT
In the modern business it is crucial to get products into market in as short time as possible. That is why configurable products, that are composed of interchangeable modules, are of often used to create new product variants quickly. Configurable products have proven efficient for creating new one-of-a-kind products. This creates new requirements for documentation processes also: it is necessary to rapidly create unique documents describing each product variant
This paper describes experiences gathered while implementing a machine manuals production system that was built by VTT Information Technology for Metso Paper, biggest paper machine manufacturer in world. Each paper machine delivered to the customer and corresponding manual production project is unique. However, only small part of the documentation is completely new in each project and therefore by re-using old document components, significant savings in documentation costs can be achieved
Product data management (PDM) system is used for keeping track of the variants of the product and changes in the product design. It can also be used for managing the documents describing product properties, e.g. machine operating and maintenance manuals. Since changes in the product design normally affect changes in the corresponding parts of the user manuals, it is natural to link the document components to the corresponding product components.
PDM systems do not generally set any restrictions on the format of the documents stored there but use of XML makes it possible to create sophisticated tools for publishing documents automatically for different output devices. The manuals production system implemented offers automatic routines for creating e.g. a set of HTML files or a complete paper publication.
The project started from an application that automatically combined a set of SGML files into complete manuals, according to the product structure modelled in our own product structure modelling tool. Currently the system has transformed into web based manuals production environment where the authors can view and edit the documents using web browser user interface, and create the combined manuals using product structures defined by product designers in PDM. The paper tries to show the project phases and describe the experiences gathered in each phase.
Table of Contents
1. Introduction
Metso Paper (former Valmet) is the world's leading supplier of technology for pulp and paper industries. Its products are extremely complicated, highly automated machinery whose complexity can be compared only to such products as an aeroplane or an off-shore oil drilling platform. Designing and documenting such products is an difficult task that needs well-defined processes and tools in order to maintain good product and document quality.
Complexity of the paper machine documentation process is not caused only by complexity of the product itself but also by the fact that each paper machine delivered is unique - based on standard components used in earlier deliveries, and additional new component variations. This means that manuals are also mainly composed of standard components which vary in each customer delivery. Therefore the need to use and manage re-usable manual components is evident.
Another level of complexity is added because the paper industry is highly international. This means that the manuals are translated to several languages, so far more 20 languages have been supported.
The goal of the project was to create a manuals production system that would cover all the dimensions of manual creation and configuration process, that is:
-
history versions
-
language versions and
-
document variants.
History versions contain same information but have minor revisions, such as spelling error corrections. Language versions contain same information but translated in different languages. Each language version has its own history versions. Document variants describe different product variants of the same product component. Each document variation contains slightly different information. Each document variation has its own history and language versions.
Next sections explain shortly basics of managing configurable products and how the same idea can be used for managing manual configurations. We also describe some technical details of the Metso Paper manuals production system (more detailed description can be found in earlier articles [Siltanen98] [Siltanen99]). The system, implemented by VTT Information Technology, allows the authors to view and edit manuals on any computer connected to intranet and create automatically paper printings and web documentation of the exact product configuration that was delivered to the customer. The main goal of this paper is not to provide technical description of the system but to show how it was built in several phases and describe experiences gathered during the process.
2. Configurable products and manuals production
Configurable products, i.e. products that are composed of interchangeable modules, have become important while business environment more and more favors rapid progress in product development. Configurable products can be managed by first creating a generic product model, that contains all the possible variants of the product. Each variant corresponds to specific technical or business demands, i.e. each car must have four wheels for technical reasons, and it is quite difficult to sell luxury car that has ecological, low power engine. These demands are expressed as configuration rules. The task of creating a product variant satisfying the rules is called configuration.
Product data management (PDM) systems are used to manage all the data that is linked to product design, maintenance etc. Product data management applications represent products as a set of data objects, each representing a product component. Each data object contains links to the documents describing the components, such as CAD drawings or product property lists. It is therefore very tempting to try to control product manuals similarly to the other product data: each product component contains link to the part of the manual that describes its usage and the documentation of the whole product is the combination of all the manual components.
3. Document component technologies in manuals creation
Using XML for representing the document components offers a fairly simple technical solution to the automatic document combining process. Documentation can be divided into small components that are each managed and authored separately. It is a simple task to automatically collect components and create the whole manual.
However, there are several problems to be solved in addition to technical solution, because the manuals are descriptive text documents, not just lists of data [Overby99]. We tried to solve problems such as:
-
How to organize manual components automatically so that the result is as readable as possible?
-
How to avoid confusion caused by combining texts by multiple authors that have different writing styles?
-
How to manage documents describing several product components? After all, the product is more that just collection of its components!
Product data management offers an obvious tool for solving these problems. PDM based manuals production system organises document components according to the product structures. Product structure is a hierarchy describing product from e.g. assembly viewpoint, i.e. describing part-of structure of the product.
The product structure seems ideal tool for organising document components. It can be used as a familiar tree structure user interface for web client, and the paper documentation is easily divided into sections and subsections according to the product structure.
The problem of combining different writing styles is often caused by using too small granularity for document components, i.e. re-using document components that are too small: for example combining single paragraphs is normally doomed. Typical problem caused by too small granularity is non-formal referencing within the document. Writers often use expressions such as "As I described in previous section..." etc. which causes confusion if the text is re-used in another context.
These kind of problems cannot always be avoided. The problem can be diminished by classifying the documentation according to technical discipline (mechanical, automation etc.) and configuring the document combining process so that only documents of same discipline can be combined. The writers can specialise in one or few disciplines which unifies terminology and document style. Also, there is need for training to emphasise usage of XML linking mechanism instead of non-formal referencing.
It is not always possible to break the documentation down to components even if PDM or ERP system manages component lists. Imagine car dash board containing different meters and a clock etc. One can describe use of each meter individually in own documents but there must be a document describing the whole dash board and how meters relate to each other. That is why product structure hierarchy is needed: product component dash board contains child components, meters and clock etc. Documents describing child components can be linked to the children and documents describing the whole dash board are linked to the dash board itself.
4. Implementing the Metso paper manuals production system
4.1. Phase 1: Basic research (1995 - 1996)
VTT Information Technology started 1995 a research project, " Document management utilising product models", which concentrated on Metso paper manuals production processes. The objective of the project was to design a prototype for the production of customer manuals based on a product structure. Another goal of the project was to gather information about product data standards such as SGML and STEP [ISO94] and distribute the information to the participating companies, 15 Finnish industry organisations.
The project results were promising: a first version of SGML document type definition for paper machine manuals was created and first product data models defined. The DTD was based on FMV GRUND-DTD, that is DTD developed for technical document creation. The first product data models definitions were based on ideas of STEP AP203, an ISO standard for representing product models.
In addition to the modeling issues we were able to produce a working prototype during the project. The prototype contained a product structure modeling and configuration tool, and tools for displaying and combining paper and electronic manuals. Tools used in this phase were InContext for authoring, Multidoc Pro for SGML viewing, Omnimark for transformations and TCL/TK for modeling tool implementation.
The research project was funded by Tekes (The National Technology Agency in Finland) and the participating companies.
4.2. Experiences from phase 1
The basic research phase can be considered crucial for implementation of the whole system. First of all the whole idea of structured documentation was not very well known in industry and project served very well its purpose of introducing basic ideas to the Finnish industry. Also the project was a starting point for a co-operation in information exchange between the participating companies in following years.
The most concrete experience of the project was the importance of the prototype built during the project. Idea of structured documentation and automatic document composition is often too abstract to be demonstrated without concrete examples. The graphical user interface of the product modelling and configuration tool provided us with good way to show the results of the project and get approval to start building real production application.
4.3. Phase 2: Implementing the first version of the production application (1996 -1998)
The goal of the next phase was to expand the prototype built in the research project into real production application. A goal was automatic creation of manuals that are unique for each customer delivery. At that time such applications were being built also elsewhere [Bartlett97], but they were not common.
The application consists of authoring environment, allowing technical writers to create SGML document components, assembly tools for creating combined documents from document components and output transformation tools for creating HTML documents for web and layout definitions for printing in paper.
The assembly tool selects and organises the document components according to the delivery specific product structure. Delivery specific structure contains those product modules that the product designers have selected for this delivery. Output transformation tools take care of creating links to the right documents: the technical writer uses symbolic identifiers to create link to another document and the application ensures that the link end is a correct variant of the document. This also means that linkage is always complete: if a document refers to the component that is not included in the delivery, the output transformation demands the writer to correct the link.
We did not try to implement an application that would be used right away in all Metso Paper units, but concentrated on doing an pilot implementation: manuals production of the products by Metso Paper Winders unit. Reason for concentrating on problems of one unit was evident: product design processes in the all units were not similar and each unit decided their methods for managing documents independently. We were hoping that the success of the implementation would encourage other units to start using similar methods.
Project started in summer 1996 and in the beginning of 1997 the authoring environment was ready and the technical writers really started writing SGML documents. We did not want to create any conversion routines for converting old documents to SGML, because one of the main goals of the project was to improve quality of the documents, which in many cases demanded writing a completely new document based on new document structure. It must be noted that this means that during the system implementation period there were two parallel documentation processes: one for delivering customer documents for other products and the new system for creating SGML manuals.
During the winter 1997 we also implemented the first versions of paper documentation layout definitions. The paper documentation layout was created according to Metso Paper old layout definitions which proved to be more difficult than expected, but in the autumn 1997 we had a style sheets that allowed almost automatic layout of paper documents. Almost automatic system means that someone must check the documents and in some case make some minor changes, mostly picture repositioning. Autumn 1997 we also had implemented conversions allowing automatic HTML conversions from SGML documents.
At that time the system did not have any version management or storage database implementation. The first version of the system was completed in 1997 and first complete document set was created during autumn 1997 first in Finnish and translated into German, English and French. It contained about 300 document components, total page number was about 1000 pages.
The project also contained profound study of SGML databases and linguistic tools to help translations. Linguistic tools were not integrated to the system because most of the translations were made by external translators that had poor capabilities of adopting the tools. We were planning to use SGML database in document version management, but it turned out soon that SGML database were not really suitable for our application because the main thing they could have offered, assembly mechanism, was made by our own tools. Also it seemed obvious that PDM would replace separate document databases after few years.
Tools used in this phase were InContext for authoring, Multidoc Pro for SGML viewing and TCL/TK for modeling tool implementation. In document transformation we begun using Balise because its programming language seemed more familiar to non-SGML experts than Omnimark
The research project was partially funded by Tekes and EU Telematics Program (DocSTEP II -project).
4.4. Experiences from phase 2
Implementation phase confirmed us even more that the basic idea of the system was correct. We were able to create implement the system according to the time tables and the customer documentation were delivered as planned. The documentation group consisted of six writers and a system manager. Training for the system was given in two days course and 1-2 hours of individual teaching was given to each of the writers.
From the beginning it was obvious that use SGML documentation guided the document writers to write more accurate documents: the DTD contained mandatory elements that had sometimes been forgotten using old tools. This was shown in the resulting documentation: there was about 10% more content than in corresponding earlier projects. Creation of the first document set took about 3300 work hours totally, second customer delivery was about 20% larger but because of well defined re-using mechanism, it took only about 670 work hours.
When project ended at end of 1998 there had been 4 customer deliveries using the system, delivered in 4 languages. Work hours used for the fourth customer delivery created by the system was estimated to be clearly more than 50% less than using the old system. In the old system the HTML documentation was made by subcontractor that increased even more costs compared to the automatic routines in the new system.
We were able to create a working prototype of a translation management system that links XML elements to corresponding translated elements. However, in the final production application linguistic tools were replaced by commercial translation memory tools.
One of the biggest drawbacks noticed in the project was that we found out that SGML/XML databases did not offer services that were needed in our application. Our experience was that one could probably build excellent and complicated system using the SGML/XML databases but the implementation means that the database basically offers only document repository and everything else must be programmed yourself. It may be that this has changed since 1998.
Also, it turned out that the system was not simplified enough to be distributed to other units yet. There was a lot of polishing, documenting etc. to be done before other units could implement the system in their own environment.
We also learned that it some cases it was difficult to introduce this kind of system to other unit within the company. Structured documentation techniques were quite at that time and new working habits were not easily accepted. We also used quite a lot of time on details, such as following old layout standards that did not fit on automatic layout process very well.
4.5. Phase 3. Using WWW technologies and PDM (1998 - 2000)
Main objective of the phase was to study how XML and WWW technology can be implemented into product model based technical document management. The project was divided in two sub-projects starting with general review of technologies and prototype building. The second sub-projects was to implement this prototype in real PDM environment.
In summer and fall 1998 requirement planning and reviews of current technology were executed. The prototype defined would automatically create specific views to user from the document database of XML documents. The views were based on the product model and the user's group and role. The prototype system was implemented as three-tier model.
Client system/application was typical Internet browser (in real life IE5 because other browsers at that time did not have XML support), with some plug-ins for drawing rendering. Purpose of the client was to act as user interface to the whole application package. The user interface was built using Java applets that create a hierarchy tree from XML product structure.
Middle Tier includes all the application logic and its the main role is integrating the data from different sources, i.e. product data and manual components. Application server can also be used as a web server. Middle Tier provides a configurable Document Transformation Framework for XML manuals. Middle tier makes dynamic modifications into documents like converting drawing identifiers into database queries, fetching right drawing from database etc.. Middle tier implementation started in fall 1998 and was under development during the whole prototype project. Main implementation tools were Java and XSLT.
Back ends are traditional repository applications like PDM, document management, ERP systems, other application servers etc. Application server connects all back ends together. The prototype contained a simple document management system which was operated through software interface implement in Java. The interface to the XML document repository enabled importing and exporting of the different language technical documents to and from the XML document repository for authoring, translating, viewing and publishing. Also, the XML document database management routines were handled via this interface.
The goal of the XML document repository interface design was generality: it served as highest-level interface to the applications and application server. Thus, the applications are not dependent of the actual XML document repository and the XML document repository can be changed without modifying the applications.
After the prototype was finished by the end of 1999, the real production application implementation started. The biggest challenge compared to the prototype was that the back end was changed to real product data management system. Because of the generic software interface between application server and the repository the change was easily implemented.
Another difference to the prototype is that the production implementation uses PDM interface to control document authoring, i.e. document checkout and checkin functions. Also the prototype product structure modelling and configuration tool was replaced by PDM (eMatrix PDM) functionality as planned already in phase 2. We are also able to use the Phase 2 publication tools with minor changes: we export the manual components belonging to selected configuration from PDM and exported files can be used as input for the publishing transformation tools.
4.6. Experiences from phase 3
While making the PDM integration it came apparent that PDM system did not have well enough readymade product configuration techniques and sophisticated configuration must be done using external tools. However, using PDM both for product design and documentation offers so much co-operation benefits that minor inconveniences can be dismissed.
Moving the manual creation application into PDM environment was one of the first pilot projects taking advantage of Metso new PDM strategy. We noticed that the implementation was very useful not only for documentation processes but also as a way to show benefits of the whole product data management implementation.
4.7. Phase 4. Future
Implementation of the current system assumes that the documents are stored in one system, i.e. internal PDM. However, the a complicated product is constructed from components delivered by several subcontractors. Therefore, we do not have just one product structure that is managed by a single company but a virtual product structure, distributed in several companies and data management systems. Our next goal is to create a simple XML Schema that allows us to create virtual product structures by combining structures from several sources.
Currently the customer electronic documentation is delivered as a set of HTML documents. Next step will be XML document delivery that is expected to be more user friendly, i.e. offering better search capabilities.
Currently the system is at least partially used or being implemented in several Metso Paper units. In the future we see that general interest in XML increases demands of implementing such a system in other units. Of course enthusiastic people are needed to push forward the implementation: our experience is that persons of information technology background are more likely to understand benefits of the approach, rather than e.g. technical writers.
5. Summary
Implementing the first production application took about eighteen months from finishing the first prototype to first customer deliveries. Implementing web interfaces and integration to the real PDM system has taken over two years and is still in progress. However, it must be remembered that the PDM integration project was one of the first pilot projects in the Metso Paper new product data management strategy and a lot of time has been used in general product data modelling issues. We believe that if a company has already well defined product data management, similar system could be implemented in notable shorter time. However, time required to implement a new system that requires changes in product and document production strategies should not be underestimated.
The key for successful implementation is the careful definition of the data models, both document structure and product structure. This means that you need to have good knowledge (or experienced partners) on both document and product modelling.
XML and product data technologies offer advanced technology and it seems that the biggest obstacle when automating document creation processes is not technology but organisation. Changes in existing processes do not happen immediately and process of transforming old legacy document into new format takes time and money. That is why we see that it is necessary to start such a project by creating a prototype using which the benefits of the system can be demonstrated. It is also question of politics: it is easier to get funding and approval for a process that advances step by step.
In the technical implementation, one should not try to implement exactly same document structures as before, but try to make it better. Also, it may be necessary to define new layout rules that are easily implemented using new document format and automated tools.
Product data management offers a solid ground for implementing manuals production system. Because the product data management is normally based on breaking the product down to the assembly hierarchy it offers a natural way of managing document components. Web technologies can be used to create a common user interface for all the users that want to view document within the company intranet.
Manuals production system can also benefit product data management implementation project. It is often difficult to find pilot projects that can rapidly show benefits of PDM. Implementing XML manuals production system as product data management pilot project offers easily understood example how product data management is used efficiently.