Vocabulary Harmonization Tool

Background & First Steps

At the September 2007 WGM, Vocabulary & M&M "upped the ante" on the Vocabulary Harmonization Tool, by requiring that facilitators use this tool, if at all possible in preparing submissions for the November 2007 Harmonization Meeting and subsequent meetings. In exchange, we promised to put up a support page on the Wiki for this tool.

This is the beginning of that documentation. The initial contribution is a minor revision and posting of the instructions and examples for using the previous Vocabulary Submission tool that required preparing XML files that drove the updates.

The current Eclipse-based tool is, for the most part, a GUI application that creates and processes the XML submission files. Therefore the old documentation is a solid start for this documentation. It can be downloaded as "Vocabulary Tool Documentation" from the Vocabulary Tooling project on Gforge. Subsequent updating and editing will move the primary source to this site. GWBeeler 00:32, 23 September 2007 (CDT)

HL7 Vocabulary Maintenance Language - Documentation of XML Submission Format

This section lists the "sections" of html documentation of the XML Vocabulary Submission capability and format that is the foundation of the Eclipse-based Harmonization Tool. The various document sections have been posted in individual Wiki pages for ease of maintenance which

Introduction to the Original VML and to the 2014 Extension

This document describes a formal, XML based language that can be used to create and maintain the HL7 Version 3 reference vocabulary. The immediate purpose of this language is to provide vocabulary facilitators a consistent and rigorous mechanism that can be used to specify HL7 vocabulary additions and updates. In 22014, this is the primary means of preparing vocabulary changes for inclusion in the HL7 Vocabulary. The files are processed from VML through the Access data base and to MIF using the VML Processing Widget and RoseTree.

using these tools, content changes expressed in these proposals are applied to the repository of vocabulary content in an Access data base in two ways:

1. Processing with a Java vocabulary update program first created in 2006-7, and
2. Execution of pre-defined Access update queries drawing data from source tables generated by XSLT transforms that convert the "extension" properties that have been defined (below).

The formal representation of the vocabulary content is expressed in HL7 Model Interchange Format (MIF) files. This expression is undertaken by the RoseTree application working from the Access data base as its sole, primary source.

In this specification, a vocabulary maintenance description consists of a sequential list of parametrized function calls such as RegisterCodeSystem, AddCodes, CreateValueSet, etc. Further, the approach here is to separate the maintenance task into three separate parts:

1. Code Systems – A code system contains a set of unique concept codes. Each concept code serves as a token to represent a useful category or class as viewed from a particular perspective. The definition and organization of the tokens within a code system represents assertions about the organization of the corresponding categories and classes within a real world. A code system may also carry information about the various ways that the categories or classes are identified in different situations and languages, as well as additional defining and identifying information that serves to clarify the intended meaning of the tokens
2. Value Sets – A value set represents a list of concept codes. Value sets are used to specify a set of possible values for one or more RIM-derived coded attributes.
3. Concept Domains - A concept domain represents an abstract conceptual space that can be associated with RIM-derived coded attributes. A concept domain can be represented by one or more value sets, where each associated value set applies in a given context. Further, sub-sets of concept domains may, themselves, be represented as concept domains in a parent-child semantic hierarchy.

Each of the above parts is maintained separately, with the revisions to the code system(s) occurring first followed by changes to the value sets followed by any revisions to concept domain/value set associations that might be necessary.

VML Extension in 2013-14

Over time, it became clear that couple of functions were "missing" from the existing VML. Specifically, the ability to "update" or "remove" a property from a code. As these were required, the update to the Access tables was taken as a manually created update, which while quick was prone to errors.

Beginning about 2009, the need arose to add properties to coded concepts, value sets, concept domains, etc. that could not be documented in the structure of the Access data base. Because the existing Java-based update tool is tightly bound to the data structures of the existing data base, and because no current volunteeers were familiar with the code of the Java application, the "toolsmiths" were reluctant to change the data content existing tables. As an alternative, the extension properties were added through two sets of tables:

1. A pair of tables that treat these properties as name/value pairs, and that identify the target of these properties by their type (such as "vakueSet") and their fully qualified name. These tables are:
   • VCS_property_definition that defines the purpose and type of each if the properties represented in this fashion.

   • VCS_object_property That contains the name/value pairs and the formal identifier of the object to which they apply.

   For several years, the management of the content of these tables required manual updates to the their content in Access.

2. A pair of tables that document the version history of the value sets and code systems in the data base. Each time there is a change to a particular code system or value set, its history records will document in which release-index the change occurred, the formal published release in which the change first appeared, who made the change, and a brief summary of the changes. These data will allow proper release and version management for these code systems and value sets when these updates are completed in the second half of 2014. The tables in which the version history are documented are:
   • VOC_value_set_history. and

   • VCS_code_system_history

The VML Extension, begun in 2013 and completed (we hope) in 2014, works withe the VML Processing Widget to automate the manual update processes of properties (done in 2013) and automate the capture of value set and code system history. In both cases this is realized by adding new "process elements" to the VML. The "automation" with the VML Processing Widget is provided by combination of:

• ANT scripts (run from the command line)

• XSLT Transforms to split the "extensions" from the original VML to create a Vocabulary Update Table that can be imported into Access; and

• Predefined update queries (programmed in Access Visual Basic For Applications (VBA)) that use the update table to change content in the appropriate vocabulary tables.

As the each VML is processed by ANT:

1. the VML file is split to isolate the "extensions" from the "traditional" VML and to create the update table from the extensions;

2. Access is activated to process the SQL-based Update and then shut down

3. The isolated traditional VML is processed to the same Access data base using the Java update program to establish the "traditional" updates.

Recognizing Extension Elements

For those that were familiar with VML, the majority of this document is "old news". In order to make the recognition of extension elements easier, the graphics that have been updated to include extension elements have a red or maroon star somewhere on the right of the diagram; the extension elements are outlined in red, and the textual headers are in red font.

Vocabulary Revisions Proposal

A Vocabulary Revision represents a related collection of updates to one or more code systems, value sets and/or concept domains. Each vocabulary revision proposal comes from a single HL7 committee and is represented by one primary contact person. The VocabularyRevision element has to be the outermost (first) element in any vocabulary submission.

Vocabulary Revision

A VocabularyRevision always begins with an editDescription element that describes the intent and purpose of the proposal. There are several textual elements in editDescription as shown below. The actual description should be recorded in the description element.

Description elements in editDescription

The attributes of the editDescription are listed below:

editDescription attributes

creationDate	when the document was first created (format: yyyy-mm-dd).
proposalId	id should be unique within the committee submitting it. A committee id will be concatenated to get a "unique" id.
primaryContact	the name of the person responsible for the document
committee|	the committee identifier primarily responsible for the changes
documentStatus|	[Not exposed in application, and should be] see table below:

documentStatus values

Status	Meaning
Proposed	The document is in the initial stages
Submitted	The document has been submitted for review by the appropriate committees.
Reviewed	The document has been reviewed by the appropriate committees and is awaiting harmonization.
Harmonized	The document has been reviewed and voted on by the RIM Harmonization Committee
Final	The document has been recorded in the official RIM database and can undergo no further changes.
Rejected	The document was not accepted at some stage in the process.

The tools used to update the actual vocabulary tables won’t process Rejected status documents. In addition, warnings will be generated if Harmonized or Final documents contain Proposed ballotStatus action entries (described below).

Final and Harmonized submissions must also supply OID’s for all new code system registrations.

Example XML for editDescription

The sample above describes a new edit created on September 25, 2007 by Woody Beeler for the Methodology & Modeling committee.

NOTE: the application will not process a proposal that has no content in the <description/> element of editDescription.

Jump to top of page

Edit Versions

[Not exposed in application]

A vocabulary revision may also include one or more editVersion entries that track the history of the submission. editVersion is optional and must immediately follow the editDescription entry.

Attributes in editVersion

changeDate	date that the submission was changed
author	name of the person making the change

Example XML for editVersion

Jump to top of page

Ballot Status

[Not exposed in application]

ballotStatus reflects the current status of the document in terms of the RIM Harmonization process. A ballotStatus element can occur at multiple points throughout a vocabulary submission document, with the “innermost” status taking precedence. A ballotStatus has an optional note element that can be used to clarify the current state. ballotStatus has the following attributes:

Attributes in ballotStatus

The first component of a registerCodeSystem entry is the ballot status of the action. The ballotStatus entry is optional. If it is omitted, it defaults to the innermost surrounding ballotStatus. If, for example, the ballotStatus for addCodesForCodeSystem is omitted, the surrounding registerCodeSystem or selectCodeSystem ballotStatus would apply. If there is no surrounding ballotStatus, the status defaults to Proposed.

action	the current status on this particular part of the submission as described by the table below
vote	the actual vote (format: yy-nn-aa) where yy is the number of yes votes, nn the number of no votes and aa the number of abstentions. The vote attribute applies to Passed, PassedWithChanges and Withdrawn actions

ballotStatus action values

Action	Meaning
Proposed	The proposed revision has been proposed (default).
Passed	The revision has passed RIM harmonization
PassedWithChanges	The revision has passed RIM harmonization subject to the changes outlined in the note.
Tabled	The revision has been set aside and will be reconsidered at a future time.
Withdrawn	The revision has been withdrawn, voted down or otherwise has not been accepted. It cannot be reconsidered in this context.
NonVotingItem	The revision consists of technical changes that do not need to be voted on.

Tabled and Withdrawn entries will not be processed by the tools used to update the RIM vocabulary database. In addition, Proposed items will be considered in error when the vocabularyRevision.documentStatus is Final.

Jump to top of page

Revisions

A list of revision entries follows the edit description and edit versions (if any) . Revision entries are applied in the order that they occur. A revision entry can create or modify a code system, a value set or a concept domain. The following tags identify revision entities:

codeSystemRevision - [Not fully exposed in application] Register (create) a new code system or modify the contents of an existing code system.

valueSetRevision- Create a new value set or modify the contents of an existing value set.

vocabularyDomainRevision – Create or modify a concept domain.

The contents of these revision records are described in more detail in the following sections.

Revision records will be applied to the vocabulary database in sequential order. A typical sequence of revision records might consist of:

1. Register a new code system and define its contents
2. Create the value set (or sets) drawn from the code system
3. Create the concept domain and connect it to the value sets.

Jump to top of page

Concept Domain Revision

---

Not exposed in application

Capability that is listed as Not exposed in application are functions that are not (as of this writing) supported by the Harmonization Tooling application.

Not fully exposed in application

Capability that is listed as Not fully exposed in application represents a set of capabilities, some, but not all of which is implemented in the Harmonization Tooling application.

Exposed under VS in application

Code System capability that is listed as Exposed under VS in application are functions that cannot be invoked directly, but that are present in processing value set changes on Value Sets whose Code System is established by a previous binding

Concept Domain

The term Concept Domain was formally adopted by the HL7 Vocabulary Technical Committee as the name for an abstract conceptual space that may be represented by (bound to) a set of concepts found in one or more specific code systems. Previous to this adoption, the preferred name for the same abstract conceptual space was Vocabulary Domain. In editing this document, the term "vocabulary domain" has been replaced with "concept domain", except where the term is part of the XML schema. In order to avoid "breaking" software tools that were built to the previous version of the schema, the XML attribute and element names retain the phrase "vocabularyDomain".

Harmonization Tool Documentation

work in progress