Flagging extensions in the RIM ITS

Summary

In the RIM ITS, do we need to have the ability to explicitely flag some things as being "extensions"? (see RLC document, section 5.1.2).

Discussion

• (Rene Spronk): Whatever you send conforms to the RIM (note that an extension in the old XML ITS may be XML that does not conform to the RIM, but that's not the topic of this discussion).
  • If you claim, using a templateId of a 'closed template' (e.g. the id an R-MIM) that the instance complies with that model, then that's what it should comply with. Adding other stuff violates a 'closed template'. If you wish to send CDA (closed template R-MIM) with extra stuff, don't claim conformance with the CDA R-MIM, but create a new templateId that covers the new ovella model.
  • If you wish to identify some minimal data set, with potential extra stuff (an extension would be a useful word in this case), use an 'open ended template'. However, an R-MIM is not an open ended template, and we have no mechanism at this point in time for defining open ended templates. Most CDA templates (expressed in human language) are open ended in nature.

• (Lloyd McKenzie): It's not true that sending extra content when you claim compliance with a 'closed template' is non-compliant. Refer to 5.1.2 Informal Extensions in RLC. Implementations are allowed to include local extensions in their instances that meet the constraints listed in 5.1.2.1. And "Isolation" of informal extensions is accomplished on an Implementation Technology Specific (ITS) basis and is described in the relevant ITS.. The RIM ITS needs to define such a mechanism.
  • Note: The fact that the 'expressed' model happens to be the RIM has *no* bearing on the expectations for conformance. RIM ITS instances aren't any more "RIM" instances than XML ITS or UML ITS instances. The only difference is the expressed model, not the rules for conformance.

• (Grahame):
  • RIM instances *may* be more RIM instances that XML ITS, because the RIM ITS is capable of expressing RIM instances that are not further constrained, which the XML ITS is not capable of doing
  • It's true that that RIM ITS doesn't allow you to flag the fact that it's an extension explicitly - I anticipated that it would be done by implication
  • The problem here is that content may only be an extension in one constraint, and not another. Very likely this is the case - that the extension has formal expression in another model. So it's not an extension at the RIM level
  • therefore, the extension mechanism *is* relative to the expressed template, and has to built into the templates attribute value in the QNames in RIM ITS

• (Lloyd):
  • The XML ITS is perfectly capable of sending full RIM instances. I created the RMIM for Bob over a year ago. It's ugly, but fully possible.
  • Implication doesn't suffice. There's a need to be able to distinguish between deliberate, ignorable "extensions" and non-deliberate, non-ignorable errors that result from incorrect implementation, divergent versions or some other reason. It's not safe to just ignore what you don't expect. That may be permissible in v2, but is not and never has been permissible in v3. RLC has very clear rules about extensions, and one of the expectations is that they be flagged.
  • Well, by HL7 methodology, every instance has a single "primary" model. That's true for services, message interactions and documents. And also by HL7 methodology, that's what "extensions" are expressed in terms of. And what ITSs are expected to be able to flag extensions in terms of. Whether they're an extension of the RIM is irrelevant. The RIM isn't the primary model. It's just the expression model.
  • If you don't want to take the perspective of a single primary model (which doesn't make much sense, given that there's no place in any HL7 paradigm for an instance without a single primary model), then we could adopt a pattern that allows flagging extensions on a per-template basis. But we *must* have some sort of means of flagging that an element is an extension.

• (Rene) Reading section 5.1.2 of RLC - that's about expressed models [it doesn't state so explicietly, but it is clearly the intent]. So would we ever need extensions in the expressed model is the RIM? Well, one could have the circumstance where one would have to flag something that's part of a "future RIM version" in an expression that uses an older RIM. Or to identify some non-v3 XML stuff as being an informal extension. So yes, we do need some flagging mechanism if the ITS is to be compliant with RLC.
  • The extension mechanism is entirely unrelated to implied models.

• (Lloyd):
  • Don't agree it's about expressed models. It's about the "conformance" model. i.e. the model that's the basis for conformance. When RLC was written, this was always synonymous with the expressed model. With RIM ITS, it's no longer synonymous. However, conformance is still driven by the focal model - whether that's the model bound to your service, the set of models bound to your interaction, or the single model used as the basis for conformance for all CDA.
  • It may be that Grahame's categorization of 3 types of models missed a level:
    • Expressed - model that drives the wire format
    • Applied - models declared to apply from within the instance (via profileId and/or templateId)
    • Implied - models that either expressed or applied models are derived from
    • Conformance - the model that is asserted as part of the behavioral contract the model is constructed against (CDA HMD for CDA instances, Interaction wrappers and payloads for messages, bound models for services)
  • In the RIM ITS, you may treat the Conformance model as just another applied model, but it's different. The conformance model is mandatory. All other applied models are optional. An instance doesn't have to reference (or comply) with any. An instance *must* comply with its conformance model.
  • Agree the extension mechanism is unrelated to implied models (those inherited through the constraint derivation of either the expressed or the applied models)

• (Grahame)
  • So, defining the RIM ITS has unbundled some things RCL assumes are related
  • understand the notion of Conformance Model as separate from the other types, but do not understand the grounds for saying that there must be one
  • accept that indicated extensions from the conformance model by implication doesn't meet the requirements, but don't want this requirement to be imposed on all instances

• (Lloyd)
  • In all current HL7 methodologies, there's no reason/purpose/mechanism to create an instance that does not have a Conformance Model. I.e. If you're creating a v3 instance and you're not creating an instance for a CDA document or a messaging instance or a service instance, what the heck are you doing?
  • *If* you have an instance that happens to not have a Conformance Model for some reason, then I'm fine with not having extensions noted. But if there is a conformance model, we definitely need the extensions to that model flagged in the instance

• (Grahame)
  • well, you are the heck using the RIM as a model of meaning per RIMBAA. ("Some HL7 member organizations have reported using the RIM as a source of input to their enterprise information architectures or as a starting place for systems analysis and design. The RIM may indeed be useful for such purposes; however, HL7 provides no assurance that the RIM is useful for anything other than as a reference model for HL7 standards development. ")
  • Basically, we now have the proposal that the RIM ITS is able to indicate that a particular model is the "conformance model" and that extensions to this model be indicated explicitly

• (Lloyd)
  • Ok, that's a fair example of a use of v3 models that's outside one of the 3 paradigms, and in such a case I agree that "extensions" wouldn't apply, except as Rene has noted as a means to deal with cross-RIM version instances. (And that should ideally be handled via a separate non-conflicting mechanism.)
  • I see it as a bit more than a proposal, but a fairly strong requirement of the existing methodology :>

• (Lloyd)
  • A further point on Rene's requirement. In circumstances where RIMBAA is using the RIM ITS to convey RIM representations, the "conformance" model would actually be the RIM. However, the mechanism to support "extensions" in a pure RIM serialization caused by conveying RIM data elements from a past or future version of the RIM would not be able to be handled by the standard extension mechanism by nature of the fact that these wouldn't be "valid" RIM elements and thus wouldn't validate against the ITS schema. So for these, there'd need to be a foreign namespace or other similar mechanism to allow them to be excluded before validation, unlike RIM-valid extensions which could (and should) be validated using the regular schema.

Original Skype thread

• [7:20:29 AM] Lloyd McKenzie: How are RIM extensions handled in the RIM ITS? Do you put them in a foreign namespace like ITS 1.0, flag them with an attribute like ITS 1.1, or pretend they aren't extensions at all?
• [7:19:53 AM] Grahame Grieve: what extensions?
• [7:20:29 AM] Lloyd McKenzie: If you want to extend an instance with RIM elements, you put them in a foreign namespace.
• [7:20:40 AM] Lloyd McKenzie: How does that happen in RIM ITS?
• [7:20:55 AM] Grahame Grieve: it's not an extension. it's RIM elements
• [7:29:08 AM] Lloyd McKenzie: But you're defining based on a template
• [7:29:16 AM] Lloyd McKenzie: (i.e. an RMIM)
• [7:29:24 AM] Lloyd McKenzie: And sometimes you include stuff that isn't part of the RMIM
• [7:29:45 AM] Grahame Grieve: so? it's still part of the RIM
• [7:30:22 AM] Lloyd McKenzie: So from the perspective of the RIM ITS, there's no such thing as extensions?
• [7:30:27 AM] Grahame Grieve: no
• [7:30:36 AM] Grahame Grieve: just rim content without any templating information
• [7:32:27 AM] Lloyd McKenzie: How do you "untemplate" an item? I.e. If you have an RMIM defined in a parent node, but one of the child nodes is an extension, how do we identify that?
• [7:32:45 AM] Grahame Grieve: it doesn't have any template information on it
• [7:33:30 AM] Lloyd McKenzie: But its siblings won't necessarily either.
• [7:33:40 AM] Lloyd McKenzie: Or do we have to declare the template on every single node?
• [7:34:54 AM] Grahame Grieve: well, depends what you are trying to do
• [7:35:12 AM] Lloyd McKenzie: I'm trying to send an instance :)
• [7:35:30 AM] Grahame Grieve: I wouldn't bother, myself. But if you want to know whether it's valid RMIM, then you have to mark up every node
• [7:35:33 AM] Grahame Grieve: well you don't
• [7:35:53 AM] Grahame Grieve: but if you aren't, then you have to determine how the RIM matches the RMIM, and you would spot extensions
• [7:35:59 AM] Grahame Grieve: so I don't know what you are asking
• [7:48:38 AM] Lloyd McKenzie: The rules for the RIM ITS say that you only need to declare the template when first invoking it or when necessary for differentiation.
• [7:48:50 AM] Grahame Grieve: there you go. When necessary for differentiation
• [7:48:53 AM] Lloyd McKenzie: If you follow that approach, there's no way to flag something as being an extension.
• [7:49:21 AM] Lloyd McKenzie: There's no way to say "this element is not part of template X
• [7:49:52 AM] Grahame Grieve: no I suppose not
• [7:50:06 AM] Grahame Grieve: except that counts as "when necessary for differentation"
• [7:50:42 AM] Lloyd McKenzie: Omitting a templateId doesn't say "this isn't part of the template"
• [7:51:14 AM] Lloyd McKenzie: Even if I put the templateId on everything except the extension, someone parsing the instance would assume the extension was part of the template.
• [7:52:48 AM] Grahame Grieve: well, so what?
• [7:53:01 AM] Grahame Grieve: I'm kind of at a loss. If you care so much about the RMIM, use it
• [7:53:24 AM] Lloyd McKenzie: So in the old ITS, there was a clear way to validate an instance with extensions:
• [7:53:33 AM] Lloyd McKenzie: - strip off any you don't recognize
• [7:53:39 AM] Lloyd McKenzie: The RIM ITS doesn't seem to provide that capability
• [7:53:50 AM] Lloyd McKenzie: Unless the assumption is we use the same approach - foreign namespace or special attribute
• [7:55:35 AM] Grahame Grieve: well, extensions are RIM extensions
• [7:55:52 AM] Grahame Grieve: and not described. Don't think I mentioned it
• [7:56:05 AM] Lloyd McKenzie: I noticed. That's why I'm asking :)
• [7:56:26 AM] Grahame Grieve: well, extensions are RIM extensions. RMIMs are templates
• [7:56:27 AM] Lloyd McKenzie: There needs to be a way to identify that something's an extension so they can yank it before valdiating
• [7:56:54 AM] Grahame Grieve: I'm lost as to why. it's not an extension if it's valid RIM content
• [7:57:10 AM] Lloyd McKenzie: It's an extension to the model being conveyed.
• [7:57:28 AM] Lloyd McKenzie: The fact the serialization is based on the RIM doesn't change the fact the content is restricted by the declared models
• [7:59:12 AM] Grahame Grieve: The important question is what happens to rim classes that do not identify their conformance to any particular (or recogized) template. Different implementation contexts will have different requirements and rules in this regard.
• [7:59:18 AM] Grahame Grieve: The templates attribute is optional, though its presence, and/or it's use with particular template models may be required by trading partner agreement.
• [8:00:06 AM] Lloyd McKenzie: If they don't declare a template, then the RIM is the only basis for validation and there's no such thing as an extension.
• [8:00:22 AM] Grahame Grieve: no you lost me
• [8:00:33 AM] Grahame Grieve: it says you can make trading partner agreements about this stuff
• [8:00:41 AM] Grahame Grieve: you want to make a trading partner agreement
• [8:01:02 AM] Lloyd McKenzie: We don't require trading partner agreements to mark extensions in other ITSs, why should we do that here?
• [8:01:24 AM] Lloyd McKenzie: Can't we just add an addendum that says the same extension mechanisms work? Put it in a separate namespace or flag it with an attribute?
• [8:01:25 AM] Grahame Grieve: because this is not fucntionality equivalent to the RIM ITS
• [8:01:44 AM] Grahame Grieve: and we do require trading partner agreements concerning the use of templateId
• [8:01:47 AM] Grahame Grieve: so it's the same
• [8:01:54 AM] Lloyd McKenzie: It's not.
• [8:02:02 AM] Lloyd McKenzie: templateId allows you to say a template applies.
• [8:02:07 AM] Lloyd McKenzie: There's no way to say a template doesn't apply
• [8:02:13 AM] Grahame Grieve: and no. It's not the same- you can't put rim content in a different namespace, because it's rim content
• [8:03:05 AM] Lloyd McKenzie: Then only allow the ITS 1.1 mechanism.
• [8:03:37 AM] Lloyd McKenzie: Or even adopt a convention that allows you to send an "excludesTemplate" or something on an element
• [8:05:18 AM] Grahame Grieve: what 1.1 extension?
• [8:05:31 AM] Grahame Grieve: the problem here is that I can't figure out what you're trying to do
• [8:06:34 AM] Lloyd McKenzie: I'm sending an instance over the wire. I'm declaring conformance to CDA. But I've got a few RIM extensions that aren't allowed in CDA. How do I flag that?
• [8:08:05 AM] Lloyd McKenzie: ITS 1.1 introduced using the "HL7extension" attribute to mark extensions
• [8:08:34 AM] Grahame Grieve: well, you're not conformant to CDA, you're a transform away. And the transform gets done one of two ways, depending on your trading partner agreement
• [8:08:54 AM] Grahame Grieve: from first principles. using the RMIM and speculative matching. In which case, extension content is so marked
• [8:09:11 AM] Lloyd McKenzie: Why should I need a trading partner agreement?
• [8:09:16 AM] Grahame Grieve: or because every element has the template marked. in which case extension ontent is easily determined
• [8:09:20 AM | Edited 8:09:21 AM] Grahame Grieve: because we said so
• [8:09:25 AM] Lloyd McKenzie: The rules in the main ITS say strip all extensions you don't want
• [8:09:32 AM] Lloyd McKenzie: There's no less of a requirement for that in RIM ITS
• [8:09:43 AM] Lloyd McKenzie: We still need to be able to identify extensions
• [8:11:20 AM] Lloyd McKenzie: And CDA is the exception for being non-conformant. For all other v3 instances, you're fully conformant to send extensions, so long as they're properly flagged
• [8:11:28 AM] Lloyd McKenzie: The ITS needs to give a way to flag them.
• [8:12:05 AM] Grahame Grieve: thre are no extensions in RIM ITS
• [8:12:41 AM] Grahame Grieve: CDA allows extensions. but you're using RIM its which is obviously not conformant CDA
• [8:13:25 AM] Lloyd McKenzie: Extensions are driven by the declared model, not the ITS.
• [8:13:47 AM] Lloyd McKenzie: If you're sending a pharmacy interaction, and you include RIM elements not part of that interaction definition, those are extensions.
• [8:14:02 AM] Lloyd McKenzie: They're extensions whether you're using XML ITS 1.0, 1.1, 2.0, UML ITS or RIM ITS.
• [8:14:27 AM] Lloyd McKenzie: CDA is the only specification we have which binds to a single ITS.
• [8:14:38 AM] Lloyd McKenzie: For all others, you can use any ITS you like.
• [8:14:44 AM] Grahame Grieve: so, let me see if I understand.
• [8:14:46 AM] Lloyd McKenzie: And the ITS must handle distinguishing extensions.
• [8:14:59 AM] Grahame Grieve: really? where does it say that?
• [8:15:40 AM] Grahame Grieve: your problem is not that the ITS doesn't allow it, but that it doesn't impose such behaviour on everyone, even if they wouldn't really understand the concept of an extension, because it's all RIM content
• [8:15:51 AM] Lloyd McKenzie: We don't have an abstract ITS anymore, so it's not documented anywhere.
• [8:16:10 AM] Lloyd McKenzie: My problem is that the ITS doesn't define a mechanism for it at all.
• [8:16:28 AM] Lloyd McKenzie: Though if it had such a mechanism and left it to site-specific negotiation, I still wouldn't be happy.
• [8:16:34 AM] Lloyd McKenzie: Site-specific negotiation is evil.
• [8:16:47 AM] Grahame Grieve: beats me what you think you want.
• [8:16:53 AM] Grahame Grieve: go back to what I said:
• [8:16:56 AM] Grahame Grieve: it's all valid RIM content
• [8:17:06 AM] Grahame Grieve: if you want to compare it to a RMIM, there's two ways to get there.
• [8:17:12 AM] Grahame Grieve: And both handle extensions
• [8:17:39 AM] Lloyd McKenzie: An ITS is intended to send model instances that comply with a particular specification.
• [8:18:04 AM] Lloyd McKenzie: HL7 defines specifications and then allows implementers to choose which ITS they want to send the instances over.
• [8:18:28 AM] Lloyd McKenzie: One of the things HL7's RLC rules say is that you can send extensions, so long as you flag that they're extensions and the receiver can safely ignore them.
• [8:18:45 AM] Lloyd McKenzie: The ITS needs to provide that flagging mechanism.
• [8:19:03 AM] Lloyd McKenzie: The fact the wire format is RIM instead of something else doesn't mean that the extensions aren't extensions anymore.
• [8:19:27 AM] Lloyd McKenzie: Using the RIM ITS doesn't mean that your constraint model becomes the RIM, merely that your expressed model is the RIM.
• [8:19:54 AM] Grahame Grieve: all of which I agree to
• [8:20:11 AM] Grahame Grieve: but as I said, comparing to the RMIM means that you have to work it out one way or the other
• [8:20:14 AM] Lloyd McKenzie: So what's the flagging mechanism provided by the RIM ITS?
• [8:20:39 AM] Lloyd McKenzie: I have to pop out for an hour. Were you able to dig up the transform that creates a RIM ITS schema?
• [8:21:12 AM] Grahame Grieve: the flagging mechanism is that it doesn't match the RMIM or it's not mapped to it
• [8:21:28 AM] Grahame Grieve: I haven't looked but it was java code. Jean was going to do a xslt based one
• [8:23:24 AM] Lloyd McKenzie: "it doesn't match" is not a flagging mechanism.
• [8:24:01 AM] Lloyd McKenzie: The point of a flagging mechanism is to distinguish content the sender thinks is valid and expects the receiver to process from content the sender thinks is "extra" that they know the receiver might ignore.
• [8:24:35 AM] Lloyd McKenzie: If a receiver validates after throwing out all the extensions and they encounter an unexpected element, they raise an error.
• [8:24:44 AM] Lloyd McKenzie: With your approach, they can't do that anymore.