Interaction Conformance and Versioning - Implementation Issues

At a high level of abstraction the question is how we can ensure that the receiver of a message is aware (in as precise a way as possible) what the semantic intent of the sender was. This includes the static model and vocabulary, the dynamic model, and constraints.

If we were to specify a checklist of what a receiver should look for in order to establish the semantic intent of the sender the checklist would look as follows:

• Message(or Batch).InteractionId: the interactionID identifies the static models (wrappers and payload(s)) and constraints placed upon them. The interactionId for normative and DSTU artifacts includes a version component which in turn refers to specific versions of the static models.
  • Note: in the "old" HL7 v3 dynamic model the interactionID also identifies the Receiver Responsibilities. In the new dynamic model interactionId doesn't have dynamic model aspects associated with it anymore.
• Message(or Batch).realmCode: this attribute signals the imposition of Realm-specific (contextual) constraints/substitutions. The value of this attribute identifies the context in question. The constraints/substitutions defined by a realm may not conflict with the constraints as specified by the InteractionId. The realmCode manages the bindings/substitutions of Valuesets and CMETs (and templates and datatype specialisations if and when substitution rules for these are agreed). Each realm has a realm owner: the owner must be an HL7 Affiliate.
  • Note that in the Dutch context this attribute won’t be used anywhere else in an interaction – if the realmCode is used at the root class of the interaction it applies to all classes in the interaction. However, in some cases, realmCode may switch within an instance. E.g. A Canadian claim message referencing a U.S. realm clinical act.
  • (Lloyd) I think there needs to be a rule that realmCode is mandatory on the root data element.
  • If a message instance contains an unknown realm, then the receiver wont be able to determine whether or not vocabulary or other parts of the instance are valid against realm constraints.
  • If an application rejects a message because because the instance asserts a realm the application supports but in a place the application didn't expect it to, that application would be considered non-conformant.
• Message(or Batch).profileId: states the constraints (or potentially the extensions) from the standard static model (Composite Message Type) as well as the dynamic model (Communication Process Model) definition that the instance complies to and implements. When multiple profiles are specified, the message instance must be valid against all of them. However, a receiver may choose to validate against only the first one recognized. For this reason, 'preferred' or more-rigorous profiles should be listed first. The constraints defined by a Profile may not conflict with the constraints as specified by the InteractionId or the Realm.
  • Note: for v3, profiles can’t be (fully) expressed in a computable fashion yet; a MIF based mechanism has been defined but not yet confirmed and there is no tooling which yet supports it
  • A conformance profile definitely can indicate support for one or more realms. The open question is whether it should allow restricting where realm can transition.
• Message(or Batch).versionCode: infrastructure versioning (CMETs, datatypes, vocabulary -particularly structural codes-, transports) is communicated using the versionCode attribute and has values such as "2006NormativeEdition". The versionCode indicates "all infrastructure stuff that was normative as of the point and time the normative edition was released". For early adopters, it could be one of the development editions or an equivalent "packaged release" by a particular realm.
  • See Interaction Versioning for details.
  • In the case of vocabulary, some domains may be marked as "fixed at the time of publication" (e.g. all structural vocabularies) while others may be marked as "mutable", meaning that they change over time. E.g. The set of drug codes expected to be supported are those published by the local regulatory agency at a given time, rather than those which happened to exist at the time the publication was issued. The specific mechanism by which "fixed at the time of publication" vs. "mutable" will be documented in the vocabulary specifications remains to be resolved.
  • Note: the UK MIM uses this attribute to identify their publication edition, e.g. their equivalent of “2005B”. In the UK a reference to a HL7 Edition doesn’t make sense given that artifacts are inspired by, but not constrained versions of, universal artifacts as contained in an HL7 Edition.
• InfrastructureRoot.templateID: states the constraints (or potentially the extensions) from the standard static artefact model definition that the instance complies to and implements. The constraint applies from the "point of occurrence" in the model. A sender can assert any templateID anywhere they like. It can be ignored by receivers.
  • Note: Profile ids always apply from the root of the message, templateId apply to the "point of occurence".
  • There's no reason to prohibit the declaration of any templates at all, because the declaration of non-recognized templates has no impact on the receiver. Prohibiting custom templates would be similar to prohibiting local extensions, which you're also not allowed to do.
  • If an application rejects a message because that message contains a template the application doesn't recognize, that application would be considered non-conformant.
• InfrastructureRoot.typeId: This identifies the actual message type (wrapper, CMET, etc.) used by the sender in constructing the message. The message type applies from the "point of occurrence" in the model. A sender can assert any typeID anywhere they like. It can be ignored by receivers.

(Charlie McCay) Can a conformance profile mandate that particular values of these attributes are present. For example: If you send a weight, you will always include the profiles chosen "weight" templateId. All instances for an interaction will include the "UK" realmCode. TypeId will be populated for all payloads in the profile. My answer to this would be that the profile must be able to express such constraints, but I do not think that we have a balloted or even committee voted resolution to that effect.

InteractionId as a pre-coordinated value for most of the above

During the January 2008 WGM (MnM MON Q1) the use of InteractionId in relationship to versioning and compliance was discussed:

• CA and UK are effectively using the InteractionId as a pre-coordinated identifier for {static model, profile, version, realm}. The UV realm is never used, local realm identifiers are used for all constrained artefacts (i.e. in conformace terms: constrainable profiles, implementable profiles). Each and every publication cycle of the implementation guides the version number increases.
• The Netherlands are using the universal interaction identifier. Each and every publication cycle the profileId changes (the UV model is constrained in a slightly different way, but all instances will still be UV compliant), but the interactionId does not.

Pro's of new artifact:

• places all constraints in a single specification, simplifies testing

Pro's of UV version:

• enables determination of interoperability possibility between cross-realm applications
• simplifies determination of forward and backward compatibility by separating constraints from syntax

Interaction versioning vs using a new Interaction

The following is true in general:

1. A non Semantically Backward Compatible change to an artefact has to result in the use of a new artefact identifier.
2. If instances created against the realm constraint would not validate against the UV specification (for any ITS), it must be treated as a realm artifact (a new artifact id must be issued for that realm code). NOTE: "extension" elements using foreign namespaces or other flags do not impact the above analysis because conformant applications are required to ignore them during validation.
   • In other circumstances, you MAY create a new artifact or MAY reference the UV artifact as well as a profileId with realm constraints
3. When dependent models of the Composite Message model (e.g. CMETs, Vocabulary, Datatypes) have been assigned a new version, the interaction must be assigned a new version as well.

Open Issues

Schema versions of an interaction

Note: for the purpose of this discussion, the word "schema" is used for XML schema as well as associated schematron scripts.

Consider the following scenario:

• The 2005B publication of AORTA contains the description of an interaction ABCD_IN000001NL and the constraints placed upon it. A schema ABCD_IN000001NL.xsd is published as well, and is published jointly with the 2005B publication. The schema is not normative, vendors are free to use other schema as long as their messages also validate against the published schema.
• The schema may not enforce (all of) the constraints as defined in the 2005B publication (e.g. the schema is not able to enfore everything, the universal model used to generate the schema had not been localized, there is no constraints language). After publication of the 2005B cycle some changes are made to the schema to enforce some of the constraints. Note: the constraints as present in the documentation haven't changed - only the schema have been changed.

Question: do we need versioning of the schema, and to inform the receiver of the interaction what version of the schema was used?

Answer: No. The sender defines conformance against abstract models. If the 2005B specification specified that attribute A shall not be used, and schema version X allows it, a sender is not allowed to populate the attribute. Creating a schema version Y which disallows the use of attribute A serves as a technical aid to enforce the constraint, a change in use from schema X to Y doesn’t have any impact on systems which conform to the 2005B specification, and therefore identification of the schema version is of no relevance.

It is certainly important to have schema version information in the schema, and it may be useful to include this in the instance in a comment, maybe only in diagnostic mode -- senders using the wrong specification and/or old weaker schemas is a likely source of trouble in an implementation - so telling the receiver what you are using in a comment does at least provide reassurance / the opportunity to be told that there is something better that you could do. However this should certainly NOT be done using xsi:schemalocation as this may be dereferenced by receiving processors which then try to access the schema file and cause all sorts of problems (file not found errors / processing delays / extra network activity / ...) Charliemccay 05:33, 2 Jun 2006 (CDT)

Another use case is when the older version of the schema contained a bug, which is fixed in the newer version. This will not change the interaction (assuming the bug was purely in the schema). However, receivers might want to check the version of the schema used by sender, in some cases they might even refuse instances produced with the older, buggy schema. Of course having the schema version in a comment has the disadvantage that comments are harder to parse. The argument that schemas are non-normative and therefore need no versioning is in itself not valid; it is common practice in software development to do versioning of all artefacts, requirements, design documents, source code, test scripts etc. Versioning doesn't just apply to normative artefacts, but may apply to all artefacts. Marc de Graauw 9 Jun 2006

I don't think the issue is with schema versioning - any party that releases multiple versions of an object should consider a versioning system. Question is a) do we have to communicate schema versions in interaction instaces? b) if there is a need to communicate schema versions in a manner other than as an XML-comment, what element/attribute should they be in, taking into account that there may be MT-schema versions as well as IN-schema versions. Rene spronk 06:58, 9 Jun 2006 (CDT)

As for a), I strongly believe it should be communicated, for the reasons Charlie and I gave. If you do not, and you run into trouble in production, there is no quick way to rule out the use of a different (weak or buggy) schema as the source of trouble. If there is no schema version attribute in the instance, the only way to rule out schema version problems is use the phone, get some implementer at the other side, have him or her dive into their sources and find out which schemas were used. Theoretically you are right in stating conformance in HL7 is against abstract models (and communicated with versionId), but this stance is also highly unpractical. There is no automated way to check conformance against the HL7 models, nor are the HL7 models expressed in such a way that a HL7 parser can take an HL7 model, and an HL7 instance, and report (non-)conformance. In the absence of such mechanisms there are two possibilities i) abstain from automated conformance checking at all, ii) delegate those parts of conformance checking which can be done by XML Schema to XML Schema (maybe Schematron too). For users of the XML ITS, ii) seems the obvious choice to me. Why have humans dig up documents and hand-check instances against written specs when there is an automated possibility for most of this tedious work? Now if we do ii) (which we all do) then the schema versions in practice become quite important, even if the final word in conformance still lies with the HL7 models, not the schema's. With a schema version attribute it is all of a sudden possible for a receiver to return: "ERROR: You used an old schema version which I cannot accept" and "ERROR: Your instance does not validate against your indicated schema version.". Without the schema version atrribute in the instance the only possibility on this level is: "ERROR: Your instance does not validate against any of the schema versions which I accept", which is significantly less expressive. Besides, as we all know it is quite common for an instance to validate with one schema parser where validation fails with another. If both sides know directly the same schema was used, it is much easier to go straight to the heart of the problem. To quote your example: "Creating a schema version Y which disallows the use of attribute A serves as a technical aid to enforce the constraint, a change in use from schema X to Y doesn’t have any impact on systems which conform to the 2005B specification, and therefore identification of the schema version is of no relevance." You are right, when both systems conform to the 2005B spec, schema version is not relevant. But the problems occur when one of the systems involved does not conform to the 2005B spec. In practice, the receiver will find the instance does not validate against receivers' schema and return an error. The receiver will not inspect the non-validating instance to report "ERROR: You used attribute A, which is not allowed." Consequently the sender gets an error message which does not help much in locating the problem. Software is full of bugs, even in production, even in medical care, so software interactions should handle buggy situations as gracefully as possible. So yes, I do think schema version should be communicated in the instance, not just diagnostic, but always. As for b), I think putting information in a XML-comment which can and should be used by an application is bad XML design, so it should be in an XML element or attribute. I'm not so sure which one, and I don't consider myself enough of an HL7 expert to say which one. I guess distributing versioned schemas in named and versioned schemasets, and communicating the schemaset version in the instance should be enough, so an instance would claim to validate against the relevant schemas in this schemaset. Marc de Graauw 8 Jul 2006

Adding the schemaversion as a processable item would require a change to the XML ITS (versioning of schema isn't an abstract model issue, it is only relevant to the XML ITS). Pre-adoption would mean adding a new namespace and adding an attribute for schemaVersion.

Alternatively, to avoid having to do this, if we were to only release a schemaset with each publication release (e.g. 2005A, 2005B): given that profileId identifies the publication release, it would also identify the schemaset contained therein. Each instance contains a profileID (it's mandatory in AORTA) and therefore the schemaversion. Rene spronk 02:03, 8 Jul 2006 (CDT)

If there are fixes/improvements in the schemas without changes to the model, so there relationship between publication and schemaset could be 1:n. I see the point with the processable item, so it seems there are two solutions:

• use profileId to indicate the publication + schemaset, so a new schemaset (within the same publication) would mean a new profileId
• take schemaversion out of the interaction into a SOAP Header. Looks nice, this SOAP Header actually describes the schema used in making it's body. Marc de Graauw 8 Jul 2006

You're missing the point - if one only ever releases schemasets in conjunction with a publication, then there is a 1:1 relationship between the profileId and the schemaset. Its means one would never release a schemaset in between publication cycles. Rene spronk 03:50, 8 Jul 2006 (CDT)

I get the point I think, and I've been re-contemplating it for a while, and I appreciate the simplicity of the approach, but I still have doubts whether it will work in practice. There's no doubt sooner or later there will be an (urgent) fix in the schemas without a corresponding fix in the publications. A 1:1 approach means a new publication. Even when the publication is affected, and a single (small but important) aspect of the models is changed, with a single schema, rolling out an entire new publication seems overkill. What I'm looking for is a versioning policy with finer granularity, especially for patches. I seems to me profileId is perfect for this, since multiple can occur, with strongest first. So if we make a new and better schema, why could't instances claim conformance to it in a profileId: 'I conform to publication 2005B with improved schemaset 2005B-2' and next claim 'I conform to publication 2005B with the default schemaset'? Marc de Graauw 9 Jul 2006, 15:00 CDT

Simple - conformance is never expressed in terms of schema. Yes, an instance can conform to schema, but that's not what conformance is all about in HL7. Conformance is ITS independent. Therefore we either need an ITS-level solution, or put it in a SOAP header, or we have to use the above approach where the schema version can be implied by the publication version. Rene spronk 10:16, 10 Jul 2006 (CDT)