Invalid x Value Sets in NE2008

Background

On February 11, 2009 Keith Boone identified an error in value set "x_ActMoodIntent" as it was represented in the vocabulary (and therefore in the vocabulary schema) published as part of Normative Edition 2008 (NE2008). This page is being built to document the problem, its analysis and intended resolution.

The simplest step, looking at the value set in any way (ballot, MIF2 file, data base, RoseTree) showed, as Keith had found, that this value set contains only one code - EVN (event) - and the intent codes that had previously been part of the value set were all missing.

The remainder of this document provides:

• report or analysis as to how this happened both so as to avoid a repetition and to identify other value sets that were similarly affected
• a list of the five value sets that were affected and how they were affected,
• a summary of the impact of this error on packages such as Normative Edition 2008, and
• the resolution of this issue in the form of new releases and patches to NE 2008

Analysis

GW Beeler reviewed the history of the vocabulary content and found the error was introduced between the release of design repository "hl7_rimrepos-2.18.2.zip" in December 2007, and "hl7_rimrepos-2.18.3.zip" in February 2008. (See releases onn gForge:

Found the error in logic.In January/February, we progressed through several stages (each with multiple steps).

The first stage created code system hierarchies in all code systems based upon the value set hierarchies that previously existed.

The second stage involved re-aligning the value sets of the the CS RIM attributes (ActClass, ActMood, etc., but not ActCode, RoleCode and so forth). The task was to add or edit value sets that would align with each of the codes in the affected code system. This included "correcting" the value set constructors (elements that include value sets in other value sets) so that they matched the hierarchy in the code system. This is the stage that introduced the error Keith Boone found.

For each of these stages, we created a source "table" (in a data base) that "drove" the process, and an program process and/or set of queries to execute it. All of these have been retained, along with textual documentation of the whole process in the "MasterVocabularyQuery" data base.

Stage 2 logic

The detailed logic summary for the second stage process, written at the time of the changes in February of 2008, says:

Executes function "VS_ProcessVsChangesForStrucuralCs" which does the following:

1. Opens record sets on tables: "zVS_VsChangesForStructCs" (the source), "VOC_value_set", "VOC_code_reference", and "VOC_value_set_constructor". (It also has a query for deleting selected rows from "VOC_value_set_property").
2. Steps through the source data one row (concept code) at a time. For EACH code, it:
   • a) Determines whether there is an existing Value Set defined for that code. If there is, it:
     • i) Steps through all Voc_value_set_constructor records for which the existing VS is the "containing" VS. For each:
       • 1) if the contained VS is non-taxonomic, reset the containing VS to the "root" VS for this code system; if the contained VS is taxonomic, the constructor entry is deleted.
     • ii) Steps through all Voc_value_set_constructor records for which the existing VS is the "contained" VS and deletes these entries. (This could also be done with a simpler Delete query.)
     • iii) Deletes all Voc_Code_reference entries for which the existing VS is the "usedToBuildVs" (including the head code)
     • iv) Deletes all Voc_Value_set_property entries for which this VS is the "usedToBuildVs"
     • v) Determines whether the existing VS name matches the "style" for VS names of structural attributes. If it does not, it computes a corrected name and should (but does not yet) document that the old name is being "retired."
     • vi) Edits the Voc_value_set record to assert the correct name, definingExpression (null), allCodes(false), isTaxonomicSet (true), and basedOnCodeSystem for this VS
     • x)If there is no existing VS, the routine creates a new VS entry in Voc_value_set using the naming style and the concept's print name to arrive at a name for the VS.
     • xi) In either case, the value set id is stored in a "collection" keyed by the head code internalId (to allow later construction of the hierarchy.
   • b) Creates (or recreates) an entry in Voc_code_reference for the head code of the VS
3. Steps through each row (concept code) of the source table a second time. It uses the parent head code internalId to look up the parent value set id in the collection created in step 2-a-xi. Then adds a row in Voc_value_set_constructor to show this relation ship IF the child VS has children of its own, but not if it is a leaf term.
4. invokes "UpdateVocabularyVersionNumber" to add a new release version for these changes.

Logic Error

The problem is in step 2a-ii. The deletions documented here were fine for the "taxonomic" value sets because the constructors were added back in in step 3. However step 2a-ii should have excluded the cases where the "containing" value set (the "usedToBuildVs") was non-taxonomic (an "x_" value set), because these might not align with the code hierarchy and were not added back in as part of step 3. The algorithm had no such check and therefore executed the deletions that caused the problem Keith Boone found.

This analysis allowed the construction of queries to identify other value sets that might also have been affected. Specifically a query on the DB from a year ago that showed which "non-taxonomic" value sets included a taxonomic value set in their construction. This query revealed the list of value sets ion the following section, and inspection of all five from the current vocabulary content verify that they were indeed affected in a similar way.

Resulting Affected Value Sets

In all, it affected five "x_" value sets:

• x_ActMoodIntentEvent - Deleted inclusion of the ActMoodIntent value set, which is based on the "INT" hierarchy in the ActMood code system

• x_ActOrderableOrBillable - Deleted inclusion of the "ActClassProcedure" and "ActClassObservation" value sets, which are based on the "PROC" and "OBS" hierarchies in the ActClass code system.

• x_EntityClassDocumentReceiving - Deleted inclusion of of the "EntityClassOrganization" value set which is based on the "ORG" hierarchy in the EntityClass code system.

• x_EntityClassPersonOrOrgReceiving - Deleted inclusion of of the "EntityClassOrganization" value set which is based on the "ORG" hierarchy in the EntityClass code system.

• x_RoleClassCredentialedEntity - Deleted Inclusion of "LicensedEntityRole" value set (renamed to "RoleClassLicensedEntity" by step 2a-v above), which is based on the "LIC" hierarchy in the RoleClass code system.