This wiki has undergone a migration to Confluence found Here
<meta name="googlebot" content="noindex">

Difference between revisions of "VocMnt-Intro"

From HL7Wiki
Jump to navigation Jump to search
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Introduction==
+
==Introduction to the Original VML and to the <font color="bb0000">2014 Extension</font> ==
  
This document describes a formal, XML based language that can be used to create and maintain the HL7 Version 3 reference vocabulary.  The immediate purpose of this language is to provide vocabulary facilitators a consistent and rigorous mechanism that can be used to specify HL7 vocabulary additions and updates. This mechanism will be supplemented with graphical maintenance tools that may partially or completely replace this language as the primary form of inputAs this occurs, it is anticipated that the language specified here will evolve a form that will be used to record and transmit vocabulary changes between systems and tools.
+
This document describes a formal, XML based language that can be used to create and maintain the HL7 Version 3 reference vocabulary.  The immediate purpose of this language is to provide vocabulary facilitators a consistent and rigorous mechanism that can be used to specify HL7 vocabulary additions and updates. In 22014, this is the '''primary''' means of preparing vocabulary changes for inclusion in the HL7 VocabularyThe files are processed from VML through the Access data base and to MIF using the <big>'''[[VML Processing Widget]]''' and '''RoseTree'''</big>.  
  
This specification takes a different approach to vocabulary maintenance than that of its Excel-based predecessor.  One of the major differences is that this language is procedural vs. table-driven.  A vocabulary maintenance description consists of a sequential list of parameterized function calls such as <tt>RegisterCodeSystem, AddCodes, CreateValueSet</tt>, etc.
+
using these tools, content changes expressed in these proposals are applied to the repository of vocabulary content in an Access data base in two ways:
 +
# Processing with a Java vocabulary update program first created in 2006-7, and
 +
# Execution of pre-defined Access update queries drawing data from source tables generated by XSLT transforms that convert the "extension" properties that have been defined (below).
  
A second major difference between this approach and its predecessor is the underlying model. The Excel-based maintenance system blurred the distinction between value sets, concept codes and concept domains. The approach taken in this document is to separate the maintenance task into three separate parts:
+
The '''formal representation''' of the vocabulary content is expressed in HL7 Model Interchange Format (MIF) files.  This expression is undertaken by the RoseTree application working from the Access data base as its sole, primary source.
 +
 
 +
In this specification, a vocabulary maintenance description consists of a sequential list of parametrized function calls such as <tt>RegisterCodeSystem, AddCodes, CreateValueSet</tt>, etc. Further, the approach here is to separate the maintenance task into three separate parts:
 
#'''<u>Code Systems</u>''' – A code system contains a set of unique '''''concept codes'''''. Each '''''concept code''''' serves as a token to represent a useful category or class as viewed from a particular perspective.  The definition and organization of the tokens within a code system represents assertions about the organization of the corresponding categories and classes within a real world.  A code system may also carry information about the various ways that the categories or classes are identified in different situations and languages, as well as additional defining and identifying information that serves to clarify the intended meaning of the tokens
 
#'''<u>Code Systems</u>''' – A code system contains a set of unique '''''concept codes'''''. Each '''''concept code''''' serves as a token to represent a useful category or class as viewed from a particular perspective.  The definition and organization of the tokens within a code system represents assertions about the organization of the corresponding categories and classes within a real world.  A code system may also carry information about the various ways that the categories or classes are identified in different situations and languages, as well as additional defining and identifying information that serves to clarify the intended meaning of the tokens
 
#'''<u>Value Sets</u>''' – A value set represents a list of concept codes.  Value sets are used to specify a set of possible values for one or more RIM-derived coded attributes.
 
#'''<u>Value Sets</u>''' – A value set represents a list of concept codes.  Value sets are used to specify a set of possible values for one or more RIM-derived coded attributes.
#'''<u>Concept Domains</u>''' (4) – A concept domain represents an abstract conceptual space that can be associated with RIM-derived coded attributes.  A concept domain can be represented by one or more value sets, where each associated value set applies in a given context. Further, sub-sets of concept domains may, themselves, be represented as concept domains in a parent-child semantic hierarchy.
+
#'''<u>[[#Nt4|Concept Domains]]</u>''' - A concept domain represents an abstract conceptual space that can be associated with RIM-derived coded attributes.  A concept domain can be represented by one or more value sets, where each associated value set applies in a given context. Further, sub-sets of concept domains may, themselves, be represented as concept domains in a parent-child semantic hierarchy.
  
 
Each of the above parts is maintained separately, with the revisions to the code system(s) occurring first followed by changes to the value sets followed by any revisions to concept domain/value set associations that might be necessary.
 
Each of the above parts is maintained separately, with the revisions to the code system(s) occurring first followed by changes to the value sets followed by any revisions to concept domain/value set associations that might be necessary.
  
==Vocabulary Revisions Proposal==
+
==<font color="#bb0000">VML Extension in 2013-14</font>==
 
+
Over time, it became clear that couple of functions were "missing" from the existing VML. Specifically, the ability to "update" or "remove" a property from a code. As these were required, the update to the Access tables was taken as a manually created update, which while quick was prone to errors.  
A vocabulary revision represents a related collection of updates to one or more code systems, value sets and/or concept domains.  Each vocabulary revision comes from a single HL7 committee and is represented by one primary contact person.  The vocabulary revision element has to be the outermost (first) element in any vocabulary submission.
 
 
 
[[Image:VocMnt010.gif|thumb|center|384px|Vocabulary Revision]]
 
 
 
A '''vocabularyRevision''' always begins with an '''editDescription''' element that describes the intent and purpose of the proposal.  There are several textual elements in '''editDescription''' as shown below. The actual description should be recorded in the description element.
 
 
 
[[Image:VocMnt015.gif|thumb|center|384px|Description elements in '''editDescription''']]
 
 
 
The attributes of the '''editDescription''' are listed below:
 
 
 
[[Image:VocMnt020.gif|thumb|center|384px|'''editDescription''' attributes]]
 
 
 
{| class="wikitable" style="text-align:left"
 
!width="15%"|
 
!
 
|-
 
!valign="top"|creationDate
 
|when the document was first created (format: yyyy-mm-dd).
 
|-
 
!valign="top"|proposalId
 
|id should be unique within the committee submitting it.  A committee id will be concatenated to get a "unique" id.
 
|-
 
!valign="top"|primaryContact
 
|the name of the person responsible for the document
 
|-
 
!valign="top"|committee||the committee identifier primarily responsible for the changes
 
|-
 
!valign="top"|documentStatus||[Not exposed in application, and should be (1)] see table below:
 
 
 
|}
 
:
 
:
 
{| border="1" cellpadding="5" cellspacing="0"
 
|+'''<u>documentStatus</u> values'''
 
!width="15%"|Status
 
!Meaning
 
|-
 
|valign="top"|Proposed       
 
|The document is in the initial stages
 
|-
 
|valign="top"|Submitted
 
|The document has been submitted for review by the appropriate committees.
 
|-
 
|valign="top"|Reviewed
 
|The document has been reviewed by the appropriate committees and is awaiting harmonization.
 
|-
 
|valign="top"|Harmonized
 
|The document has been reviewed and voted on by the RIM Harmonization Committee
 
|-
 
|valign="top"|Final
 
|The document has been recorded in the official RIM database and can undergo no further changes.
 
|-
 
|valign="top"|Rejected
 
|The document was not accepted at some stage in the process.
 
|}
 
 
 
The tools used to update the actual vocabulary tables won’t process '''''Rejected''''' status documents.  In addition, warnings will be generated if '''''Harmonized''''' or '''''Final documents''''' contain '''''Proposed''''' '''ballotStatus''' action entries (described below).
 
 
 
'''''Final''''' and '''''Harmonized''''' submissions must also supply OID’s for all new code system registrations.
 
 
 
[[Image:VocMnt031.gif|thumb|center|640px|Example XML for '''editDescription''']]
 
 
   
 
   
The sample above describes a new edit created on September 25, 2007 by Woody Beeler for the Methodology & Modeling committee. '''NOTE: the <u>application will not process</u> a proposal that has no content in the &lt;description/&gt; element of editDescription'''.
+
Beginning about 2009, the need arose to add properties to coded concepts, value sets, concept domains, etc. that could not be documented in the structure of the Access data base. Because the existing Java-based update tool is tightly bound to the data structures of the existing data base, and because no current volunteeers were familiar with the code of the Java application, the "toolsmiths" were reluctant to change the data content existing tables.  As an alternative, the extension properties were added through two sets of tables:
 
+
# A pair of tables that treat these properties as name/value pairs, and that identify the target of these properties by their type (such as "vakueSet") and their fully qualified name.  These tables are:
===Edit Versions [Not exposed in application (1)]===
+
#* '''VCS_property_definition''' that defines the purpose and type of each if the properties represented in this fashion.
 
+
#*:
A vocabulary revision may also include one or more '''editVersion''' entries that track the history of the submission. '''editVersion''' is optional and must immediately follow the '''editDescription''' entry.
+
#* '''VCS_object_property''' That contains the name/value pairs and the formal identifier of the object to which they apply.  
 
+
#*:
[[Image:VocMnt040.gif|thumb|center|512px|Attributes in '''editVersion''']]
+
#:For several years, the management of the content of these tables required manual updates to the their content in Access.
 
+
#:
{| class="wikitable" style="text-align:left"
+
# A pair of tables that document the version history of the value sets and code systems in the data baseEach time there is a change to a particular code system or value set, its history records will document in which release-index the change occurred, the formal published release in which the change first appeared, who made the change, and a brief summary of the changes. These data will allow proper release and version management for these code systems and value sets when these updates are completed in the second half of 2014. The tables in which the version history are documented are:
!width="15%"|
+
#* '''VOC_value_set_history'''. and
!
+
#*:
|-
+
#* '''VCS_code_system_history'''
!valign="top"|changeDate
 
|date that the submission was changed
 
|-
 
!valign="top"|author
 
|name of the person making the change
 
|}
 
 
 
[[Image:VocMnt051.gif|thumb|center|640px|Example XML for '''editVersion''']]
 
 
 
 
 
===Ballot Status[[VocMnt-Intro#Nt1|''[Not exposed in application]'']]===
 
 
 
'''ballotStatus''' reflects the current status of the document in terms of the RIM Harmonization process.  A '''ballotStatus''' can occur at multiple points throughout a vocabulary submission document, with the “innermost” status taking precedence.  A '''ballotStatus''' has an optional '''note''' element that can be used to clarify the current state. '''ballotStatus''' has the following attributes:
 
 
 
[[Image:VocMnt060.gif|thumb|center|512px|Attributes in '''ballotStatus''']]
 
 
 
The first component of a '''registerCodeSystem''' entry is the ballot status of the action.  The '''ballotStatus''' entry is optional.  If it is omitted, it defaults to the innermost surrounding '''ballotStatus'''. If, for example, the '''ballotStatus''' for '''addCodesForCodeSystem''' is omitted, the surrounding '''registerCodeSystem''' or '''selectCodeSystem ballotStatus''' would apply.  If there is no surrounding '''ballotStatus''', the status defaults to '''''Proposed'''''.
 
 
 
{| class="wikitable" style="text-align:left"
 
!width="15%"|
 
!
 
|-
 
!valign="top"|action
 
|the current status on this particular part of the submission as described by the table below
 
|-
 
!valign="top"|vote
 
|the actual vote (format: '''''yy-nn-aa''''') where '''''yy''''' is the number of yes votes, '''''nn''''' the number of no votes and '''''aa''''' the number of abstentionsThe vote attribute applies to '''Passed''', '''PassedWithChanges''' and '''Withdrawn''' actions
 
|}
 
 
 
:
 
:
 
{| border="1" cellpadding="5" cellspacing="0"
 
|+'''<u>ballotStatus</u> action values'''
 
!width="15%"|Action
 
!Meaning
 
|-
 
|valign="top"|Proposed     
 
|The proposed revision has been proposed (default).
 
|-
 
|valign="top"|Passed
 
|The revision has passed RIM harmonization
 
|-
 
|valign="top"|PassedWithChanges
 
|The revision has passed RIM harmonization subject to the changes outlined in the note.
 
|-
 
|valign="top"|Tabled
 
|The revision has been set aside and will be reconsidered at a future time.
 
|-
 
|valign="top"|Withdrawn
 
|The revision has been withdrawn, voted down or otherwise has not been accepted.  It cannot be reconsidered in this context.
 
|-
 
|valign="top"|NonVotingItem
 
|The revision consists of technical changes that do not need to be voted on.
 
|}
 
 
 
'''''Tabled''''' and '''''Withdrawn''''' entries will not be processed by the tools used to update the RIM vocabulary database.  In addition, '''''Proposed''''' items will be considered in error when the '''vocabularyRevision.documentStatus''' is Final.
 
  
 +
The VML Extension, begun in 2013 and completed (we hope) in 2014, works withe the [[VML Processing Widget]] to automate the manual update processes of properties (done in 2013) and automate the capture of value set and code system history.  In both cases this is realized by '''adding new "process elements" to the VML'''. The "automation"  with the [[VML Processing Widget]] is provided by combination of:
 +
*ANT scripts (run from the command line)
 +
*:
 +
* XSLT Transforms to split the "extensions" from the original VML to create a Vocabulary Update Table that can be imported into Access; and
 +
*:
 +
* Predefined update queries (programmed in Access Visual Basic For Applications (VBA)) that use the update table to change content in the appropriate vocabulary tables.
 
   
 
   
===Revisions===
+
As the each VML is processed by ANT:
 
+
# the VML file is split to isolate the "extensions" from the "traditional" VML and to create the update table from the extensions;
A list of revision entries follows the edit description and edit versions (if any) .  Revision entries are applied in the order that they occur. A revision entry can create or modify a code system, a value set or a concept domain. The following tags identify revision entities:
+
#:
 
+
# Access is activated to process the SQL-based Update and then shut down
:'''codeSystemRevision''' - [[VocMnt-Intro#Nt2|''[Not fully exposed in application]'']] Register (create) a new code system or modify the contents of an existing code system.
+
#:
:'''valueSetRevision'''- Create a new value set or modify the contents of an existing value set.
+
# The isolated traditional VML is processed to the same Access data base using the Java update program to establish the "traditional" updates.
:'''vocabularyDomainRevision''' – Create or modify a concept domain.
+
===Recognizing Extension Elements===
 
+
For those that were familiar with VML, the majority of this document is "old news".  In order to make the recognition of extension elements easier, the graphics that have been updated to include extension elements have a red or maroon star somewhere on the right of the diagram; the extension elements are outlined in red, and the textual headers are in red font.
The contents of these revision records are described in more detail in the following sections.
 
 
 
Revision records will be applied to the vocabulary database in sequential order.  A typical sequence of revision records might consist of:
 
#Register a new code system and define its contents
 
#Create the value set (or sets) drawn from the code system
 
#Create the concept domain and connect it to the value sets.
 
 
 
==Code System Revisions [[VocMnt-Intro#Nt2|''[Not fully exposed in application]'']]==
 
==Value Set Revisions==
 
==Concept Domain (4) Revision [[VocMnt-Intro#Nt2|''[Not fully exposed in application]'']]==
 
 
 
==Notes==
 
=====Nt1=====
 
Capability that is listed as '''Not exposed in application''' are functions that are not (as of this writing) supported by the Harmonization Tooling application.
 
=====Nt2=====
 
Capability that is listed as '''Not fully exposed in application''' represents a set of capabilities, some, but not all of which is implemented in the Harmonization Tooling application.
 
=====Nt3=====
 
Code System capability that is listed as '''Exposed under VS in application''' are functions that cannot be invoked directly, but that are present in processing value set changes on Value Sets whose Code System is established by a previous binding
 
=====Nt4=====
 
The term '''''Concept Domain''''' was formally adopted by the HL7 Vocabulary Technical Committee as the name for an abstract conceptual space that may be represented by (bound to) a set of concepts found in one or more specific code systems.  Previous to this adoption, the preferred name for the same abstract conceptual space was '''''Vocabulary Domain'''''.  In editing this document, the term "vocabulary domain" has been replaced with "concept domain", '''except where the term is part of the XML schema'''.  In order to avoid "breaking" software tools that were built to the previous version of the schema, the XML attribute and element names retain the phrase "vocabularyDomain".
 

Latest revision as of 00:54, 11 June 2014

Introduction to the Original VML and to the 2014 Extension

This document describes a formal, XML based language that can be used to create and maintain the HL7 Version 3 reference vocabulary. The immediate purpose of this language is to provide vocabulary facilitators a consistent and rigorous mechanism that can be used to specify HL7 vocabulary additions and updates. In 22014, this is the primary means of preparing vocabulary changes for inclusion in the HL7 Vocabulary. The files are processed from VML through the Access data base and to MIF using the VML Processing Widget and RoseTree.

using these tools, content changes expressed in these proposals are applied to the repository of vocabulary content in an Access data base in two ways:

  1. Processing with a Java vocabulary update program first created in 2006-7, and
  2. Execution of pre-defined Access update queries drawing data from source tables generated by XSLT transforms that convert the "extension" properties that have been defined (below).

The formal representation of the vocabulary content is expressed in HL7 Model Interchange Format (MIF) files. This expression is undertaken by the RoseTree application working from the Access data base as its sole, primary source.

In this specification, a vocabulary maintenance description consists of a sequential list of parametrized function calls such as RegisterCodeSystem, AddCodes, CreateValueSet, etc. Further, the approach here is to separate the maintenance task into three separate parts:

  1. Code Systems – A code system contains a set of unique concept codes. Each concept code serves as a token to represent a useful category or class as viewed from a particular perspective. The definition and organization of the tokens within a code system represents assertions about the organization of the corresponding categories and classes within a real world. A code system may also carry information about the various ways that the categories or classes are identified in different situations and languages, as well as additional defining and identifying information that serves to clarify the intended meaning of the tokens
  2. Value Sets – A value set represents a list of concept codes. Value sets are used to specify a set of possible values for one or more RIM-derived coded attributes.
  3. Concept Domains - A concept domain represents an abstract conceptual space that can be associated with RIM-derived coded attributes. A concept domain can be represented by one or more value sets, where each associated value set applies in a given context. Further, sub-sets of concept domains may, themselves, be represented as concept domains in a parent-child semantic hierarchy.

Each of the above parts is maintained separately, with the revisions to the code system(s) occurring first followed by changes to the value sets followed by any revisions to concept domain/value set associations that might be necessary.

VML Extension in 2013-14

Over time, it became clear that couple of functions were "missing" from the existing VML. Specifically, the ability to "update" or "remove" a property from a code. As these were required, the update to the Access tables was taken as a manually created update, which while quick was prone to errors.

Beginning about 2009, the need arose to add properties to coded concepts, value sets, concept domains, etc. that could not be documented in the structure of the Access data base. Because the existing Java-based update tool is tightly bound to the data structures of the existing data base, and because no current volunteeers were familiar with the code of the Java application, the "toolsmiths" were reluctant to change the data content existing tables. As an alternative, the extension properties were added through two sets of tables:

  1. A pair of tables that treat these properties as name/value pairs, and that identify the target of these properties by their type (such as "vakueSet") and their fully qualified name. These tables are:
    • VCS_property_definition that defines the purpose and type of each if the properties represented in this fashion.
    • VCS_object_property That contains the name/value pairs and the formal identifier of the object to which they apply.
    For several years, the management of the content of these tables required manual updates to the their content in Access.
  2. A pair of tables that document the version history of the value sets and code systems in the data base. Each time there is a change to a particular code system or value set, its history records will document in which release-index the change occurred, the formal published release in which the change first appeared, who made the change, and a brief summary of the changes. These data will allow proper release and version management for these code systems and value sets when these updates are completed in the second half of 2014. The tables in which the version history are documented are:
    • VOC_value_set_history. and
    • VCS_code_system_history

The VML Extension, begun in 2013 and completed (we hope) in 2014, works withe the VML Processing Widget to automate the manual update processes of properties (done in 2013) and automate the capture of value set and code system history. In both cases this is realized by adding new "process elements" to the VML. The "automation" with the VML Processing Widget is provided by combination of:

  • ANT scripts (run from the command line)
  • XSLT Transforms to split the "extensions" from the original VML to create a Vocabulary Update Table that can be imported into Access; and
  • Predefined update queries (programmed in Access Visual Basic For Applications (VBA)) that use the update table to change content in the appropriate vocabulary tables.

As the each VML is processed by ANT:

  1. the VML file is split to isolate the "extensions" from the "traditional" VML and to create the update table from the extensions;
  2. Access is activated to process the SQL-based Update and then shut down
  3. The isolated traditional VML is processed to the same Access data base using the Java update program to establish the "traditional" updates.

Recognizing Extension Elements

For those that were familiar with VML, the majority of this document is "old news". In order to make the recognition of extension elements easier, the graphics that have been updated to include extension elements have a red or maroon star somewhere on the right of the diagram; the extension elements are outlined in red, and the textual headers are in red font.