Datatypes R2 Issue 103
Contents
NullFlavor considerations
outcomes
New definition for NI:
The value is exceptional (missing, incomplete, improper). No information as to the reason for being an exceptional value is provided. This is the most general exceptional value. It is also the default exceptional value.
New definition for OTH:
The actual value is not an element in the constrained value domain of a variable. (e.g., concept not provided by required code system).
nullFlavor is still linked to conformance, but there is only one conformance model. When templates are used, the conformance constraints are evaluated differently. This has a series of consequences for templates which are to be documented in the templates specification.
CD root code is the one that meets the conformance constraints expressed in the root model.
A section is to be added to the nullFlavor property explaining the similarity and differences between HL7 null and other concepts of null.
The existing nullFlavors UNC and OTH are to be reorganised and a new nullFlavor EXPR is to be added.
Domain & conformance differences (what is "proper"?)
The definition of nullFlavor is when the value isn't "proper"
There is a problem - is proper related to the domain definition of the type or to the attribute definition. Here's a list of circumstances where this matters (or might):
- If attribute is INT.Positive Mandatory, is PINF an acceptable value?
- is QS conformant or not? it's clearly not a "proper" value from the perspective of the PQ domain, but it's the appropriate information for making a formulation. If a modeller makes a value of a formulation mandatory, they mean, you must provide enough information so I can make the formulation(?), so QS is good enough
- is a null value with an EXPR that evaluates to non-Null a legitimate fulfillment of a mandatory field?
- CD. The root CD isn't valid, but one of the translations is.
Introduction
There is a series of issues around the nullFlavor definition and the way it is used. These all come together around the CD datatype.
The definition of nullFlavor:
Indicates that a value is an exceptional value, or a NULL-value. A null value means that the information does not exist, is not available or cannot be expressed in the data type's normal value set. Every data element has either a proper value or it is considered NULL. If (and only if) it is NULL, the isNull provides more detail as to in what way or why no proper value is supplied.
The definition of NI:
No information whatsoever can be inferred from this exceptional value. This is the most general exceptional value. It is also the default exceptional value
The definition of OTH:
The actual value is not an element in the value domain of a variable. (e.g., concept not provided by required code system)
The instruction regarding the use of OTH in CD:
a CD value without a value for the code property, or with a value that is not from the cited coding system is an exceptional value (NULL of flavor other)
The definition of mandatory:
if an attribute or association is described as mandatory in a model, then in all message instances that are derived from this model, the element must be present and non-null
Some problems arise from these definitions.
- The definition of nullFlavor is internally inconsistent
- There is confusion between "null" and "incomplete"
- The CD use of OTH is not consistent with the definition of OTH
- linking conformance to nullFlavor has unexpected interactions with the above issues.
Each of these will be explored below. There is some overlap between the different issues; fixing one may fix others, but it is useful to consider each independently.
Internal inconsistencies in definition
The definition for nullFlavor itself says: "A null value means that the information does not exist, is not available or cannot be expressed in the data type's normal value set.", but the definition for the root nullFlavor is "No information whatsoever can be inferred from this exceptional value."
The problem may arise from the term "this exceptional value" - does it mean that you have no information about the reason this is exceptional, or do you have no information about the value that is exceptional. The second interpretation - which would appear to be the technically correct interpretation - is problematic because there is many situations where quite a bit of information is actually provided.
The addendum to the definition muddies the waters a little more, when it says "Every data element has either a proper value or it is considered NULL." The problem is that there is gulf between "no information" and "not a proper value" - these are very different things. But perhaps the biggest problem here is that the question of what constitutes a proper value is left ambiguous - is it proper in respect of the rules laid out in the datatypes specification itself, or proper in regard to the constraints upon the attribute where the datatype is used? (In general, this is a trivial distinction, but there is a few cases where this turns out not to be trivial at all, and these cases are common and important).
More generally, there is confusion between the notion of "null" - a value where there is no information provided, and the notion of exceptional value - a value that falls outside the scope of the definition of the datatype (or it's constraints, depending on the meaning of 'proper'. The confusion may be in the specification, or in the readers, depending on perspective, but it is there, and significant.
confusion between "null" and "incomplete"
In classical computing, a null value is special value that has no other information. It is sometimes referred to as a "sentinel value", and more often by the less descriptive "special value".
HL7 defines the concept of nullFlavor, which extends the concept of null to specify why a concept is null. In fact, the technical definition is "The null flavors are a general domain extension of all normal data types" - in other words any data value may have either a nullFlavor value, or a value defined in the domain of the type of that value.
This is a departure from classical computing practices, with attendant implementation issues, but these are just a price which HL7 chose to accept.
The problem comes when data values that are labelled as null carry some information that illuminates why they are null. Now in the datatypes as defined at this time, this is of critical semantic importance - we need to be able to use AD.use, for instance, on an AD that is null, to explain that we didn't ask what the home address was. But this use of nullFlavor - where we call it null, but still provide more information - is not only not conformant to the definition given above (by the strict technical interpretation of the meaning of domain values), it violates the general semantic definition of null: a partially populated information object is not null.
And in the case of CD, there can be quite a bit of information provided on a null CD.
CD use of OTH inconsistent with the definition of OTH
The definition of OTH mentions CD specifically, and says that CD takes a nullFlavor of OTH is it doesn't provide a concept from the required code System. but CD says, in rather more detail, that a CD has the nullFlavor of OTH if the code is blank and the codeSystem is provided. In other words, if you can specify a codeSystem but not a code (or if you make your own code up), then the nullFlavor is OTH.
There is a subtle but important distinction between the two - over the use of the word "required". The CD definition does not make any reference to "required", only to the specified codeSystem. If the value is not in the specified codeSystem, then the nullFlavor is OTH. This has nothing to do with what is the required codeSystem.
This slight difference is magnified because of conformance
conformance is linked to nullFlavor
Mandatory is explicitly linked to nullFlavor: The requirement that an attribute is mandatory is met by the attribute containing a non-null value. In other words, that the attribute contains a "proper" value.
At first glance, this seems like a reasonable definition. However by this point it should be clear that there is some differences between what is a null value and what is an improper value. There is cases where a proper value has a nullFlavor, and cases where an improper value does not have a nullFlavor.
As things stand at present, "proper" is abandoned in favor of use of null, but in several regards we are not consistent in our practice. Part of the problem is that "proper" has no formal definition - and it would probably not be appropriate anyway.
CD
All these problems meet at CD. Imagine the following scenario:
The model specifies that the code is mandatory, and must come from a SNOMED value set. The originating system actually stores the diagnosis as plain text, and a medical records coder encodes the diagnosis in ICD-10, and SNOMED, in order to send the code. The text is "right arm wound on upper arm", which is encoded in ICD-10 as S41.1, and the SNOMED expression 283369005<272741003:24028007>.
R1
In this case, the XML ITS fragment for the act.code (say), based on Datatypes R1, is
< code nullFlavor="OTH" codeSystem="[oid for ?]">
<originalText>right arm wound on upper arm</originalText>
<translation code="S41.1" codeSystem="2.16.840.1.113883.6.3"
codeSystemName="ICD-10" displayName="Open wound of forearm, part unspecified"/>
<translation code="283369005" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Laceration of upper arm (disorder)">
<qualifier>
<value code="272741003" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Laterality (attribute)"/>
</qualifier>
<qualifier>
<value code="24028007" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Right (qualifier value)"/>
</qualifier>
<translation>
</ code>
So, the first problem is whether the root carries a nullFlavor or not.
In this case, it was given a nullFlavor, since the model expects something
from snomed, and this is not it. So we expect to have to give it a nullFlavor
of OTH. And because we did that, we must give it a codeSystem - that's
a constraint in the abstract spec:
invariant(CD x)
where x.other {
x.code.other;
x.codeSystem.nonNull;
};
But that's not right - we don't even have a coding system. So we can't use nullFlavor OTH. We could try for a non-nullFlavor. In this case it might work - no nulFlavor and no codeSystem. But whoops, there's this constraint:
invariant(CD x)
where x.nonNull {
x.code.nonNull;
};
I guess we'll have to stick to nullFlavor OTH and make a codeSystem up. If we change the use case so that the system is using it's own internal coding system, then this works a little better - at least we're not making a codeSystem up that we didn't use.
Now we turn to conformance. The model says that the code is mandatory and must be encoded from SNOMED, but the CD says it is OTH, and provides some other oid which is not SNOMED, which by some advanced logic (poor implementors!), from the valueSet, we can tell is required.
So this instance is not conformant, and cannot be made to be conformant. So this is definitely not what we anticipated when we set out to allow translations and require a code from a particular valueSet.
R2
In datatypes R2, we have made an important change, by changing the rule that the translation nesting must reflect the sequence of translation order, to a new rule that says that the "" See [[]]. Under this rule, the XML ITS would look like this:
< code id="x1" code="283369005" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Laceration of upper arm (disorder)">
<originalText>right arm wound on upper arm</originalText>
<translation code="S41.1" codeSystem="2.16.840.1.113883.6.3" source="x1"
codeSystemName="ICD-10" displayName="Open wound of forearm, part unspecified"/>
<qualifier>
<value code="272741003" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Laterality (attribute)"/>
</qualifier>
<qualifier>
<value code="24028007" codeSystem="2.16.840.1.113883.6.96"
codeSystemName="SNOMED-CT" displayName="Right (qualifier value)"/>
</qualifier>
</ code>
actually, there's a little bit of getting ahead of ourselves here, in that we haven't discussed how to do the source references, and I have used ID/IDREF to illustrate the concept - but let's put that aside
Anyhow, it's obvious that the fix we propose to make for CE also makes CD work - for basically the same reason, it's related to conformance. This fix simplifies the task of assessing conformance, and finding the [?] code - but the key question is, what code goes in the root?
The sender choses, and the sender's choice mightn't align with the receivers choice. What if no valueSet is specified? what if there is more than one code given that is valid in the valueSet (for instance, if there was multiple snomed renderings). So receivers may have to prowl through translations anyawy. And because of valueSets, there is some complexity relating the codeSystem (which is the most important basis for checking which translation to use) to what you want. The point of this is that this is complicated, and the practical outcome of saying to put a code of the senders choice, partly guided by conformance, might not be as big as we'd like to the implementors.
Multiple codeSystems
So R2 works, if and only if there is only one codeSystem specified. But this not always the case. There is a variety of reasons why one might specify that the code might be sent in more than one codingSystem or valueSet, and a multitude of ways to do it.
A typical example of this is in Australia, with regard to sending lab results. Each lab system has it's own internal terminology for describing lab tests, and has been sending out reports based on this terminology for years. Some 5 years ago or so, the Australian standards body recommended that everything also be LOINC encoded, and standardised the use of CE4-6 for this in V2. But a new decision is to standardize on SNOMED and add new pathology tests to SNOMED. So Australia would like to require all three coding systems for the moment (and has discussed asking for 3 new components on v2 CE for this purposes). There's a number of other reasons for this as well.
--LLoyd - stick a Canadian use case in here.
In the UK for GP2GP there is a requirement to send both the Read code that was originally used and the Snomed code that this maps to. This requirement to send both the new "standard code system that policy requires, and the code from the existing system(s) that were close to de-facto standards will be with us for ever. There are many other examples in the UK.
There's a number of ways to specify multiple valuesets. It could be done by using the realm machinery, though it probably doesn't really give you fine enough control to pull that off. It could be done by specifying templates or a profile with a mandatory valueset different to the underlying model. or it may be done directly in the model itself - to specify multiple required valueSets.
Going back to the use case, and adding to it that the combination of models (however done) specify that this code must contain both a SNOMED and a ICD-10 code, then how does this look in the XML?
The problem is that the concept of "root" code no longer really applies - there is multiple codes provided to meet the conformance requirements, and there can no longer be a single nullFlavor to deal with meeting conformance in this regard.
One possible response to this problem is to say that the nullFlavor relates to all the requirements - you are null unless they are all satisfied. If this is the case, what does the XML look like for this CD use case? whichever code you put in the root, what are the nullFlavors on them? There is two main possibilities, and they are both broken in regard to conformance.
Another possible response is to say that we cannot support more than one constraint on a CD (or any type, as this is a specific case of a general problem). But Given that we allow multiple encodings to be sent, and given the importance of multiple encodings, it would not be an acceptable outcome if the consequence of requiring more than one encoding was that an instance couldn't be conformant.
Another solution is to redesign CD. You could go for a model like this:
type ConceptDescriptor alias CD specializes ANY {
ED originalText;
SET(CODE) encodings;
};
type CodedConcept alias CODE specializes ANY {
ST code;
ST displayName;
UID codeSystem;
ST codeSystemName;
ST codeSystemVersion;
LIST<CR> qualifier;
SET<CS> rationale;
CD source;
}
But this gives you a different (but related) problem - what is the nullFlavor on the root? is it some combination of the nullFlavors of the encodings? So this is just moving the deckchairs on the titanic (though it does have some merit in it's own right from other perspectives).
Other Examples
Earlier I said that there are cases where "proper" != null. I've illustrated some good ones. Here's some others:
- nullFlavor QS - say, add enough to fill the beaker. If you are dealing with a formula, you specify that the value : PQ is mandatory, and require a proper value. the user says, a bit of this, a bit of that, and fill it out with Normal Saline. Is this a proper value? Should it be conformant? Interesting question, probably with a not simple answer.
- A classic case where non-null is not proper is an address with a nonnull flavor and a single del. This should fail conformance in most cases, but doesn't, since it's not technically null.
- if a coded attribute is CWE mandatory, it shouldn't be given a nullFlavor just cause the codingSystem is extended, but by the rules of CD, it must be. So it would be non-conformant by the rules of conformance even though it isn't.
Solution
Though there are some minor internal problems with the nullFlavor definitions, it's only when you introduce the notion of conformance that the real problems start. And though the problems have mostly been illustrated by CD, they don't actually only apply to CD.
I propose that the solution to this problem is to separate conformance and nullness. Nullness should be a property of the datatypes themselves - and in fact, only a few things here and there need to be changed to make this happen, such as the removal of the word "required" from the definition of OTH.
This complicates conformance somewhat, and this will be discussed below. But it does clearly define the scope of "proper" - it's a proper value if it can be described by the unconstrained domain of the datatype itself.
Implications
To discuss:
- not using OTH if code is not in codeSystem
- how to represent content in instances if it fails to conform?
- which attributes can be populated if the value is null?
- how to make conformance work properly without null (ii.root case)
Skype thread archive - Grahame / Gunther / Lloyd
[2:51:23 PM] Lloyd McKenzie says: Would the rules be clear that this behavior would be allowed on non-mandatory codes, but not on mandatory ones?
[9:50:04 PM] Lloyd McKenzie says: Effectively, the code behaves as "null" because it has no real value.
[9:50:21 PM] Grahame Grieve says: well, it's still OTH - you're just saying why
[9:50:35 PM] Lloyd McKenzie says: Technically, it's not OTH
[9:50:55 PM] Grahame Grieve says: why not?
[9:51:21 PM] Lloyd McKenzie says: OTH says "the value is not expressible in the allowed constraints"
[9:51:26 PM] Lloyd McKenzie says: That's not the case here
[9:51:40 PM] Lloyd McKenzie says: It's expressible, it just hasn't actually been expressed . . .
[9:51:56 PM] Grahame Grieve says: oh that
[9:52:07 PM] Lloyd McKenzie says: :)
[9:52:15 PM] Grahame Grieve says: how do you know it's expressible?
[9:52:57 PM] Lloyd McKenzie says: Ok, it's "potentially expressible"
[9:53:06 PM] Lloyd McKenzie says: As opposed to "definitely not expressible" for OTH
[9:55:10 PM] Lloyd McKenzie says: I think we have the head of a different problem here too
[9:55:30 PM] Lloyd McKenzie says: If I have a code that's CNE, mandatory, I expect to find a code from my valueset.
[9:56:04 PM] Lloyd McKenzie says: If I declare 2 templates that are also effective, and the attribute is CNE, mandatory but they have different value-sets, I expect them to be there too.
[9:53:44 PM] Grahame Grieve says: I don't think that the OTH definition was written with that in mind. I think it should be "not expressed"
[9:54:01 PM] Grahame Grieve says: and "expressibility unknown" and "not expressible" should be specialisations
[9:54:52 PM] Grahame Grieve says: specially since what we did approve as a staging solution for R1 was that
[whatever the code is] is subsumed by OTH
[9:56:31 PM] Lloyd McKenzie says: If I send an instance that satisfies some of the valuesets, but not others, is the attribute null or not?
[9:56:57 PM] Grahame Grieve says: yes, last time we discussed this we agreed that conformance was utterly broken on CD
[9:57:01 PM] Lloyd McKenzie says: I could go for the specializations
[9:57:07 PM] Lloyd McKenzie says: I'm not sure it's just CD
[9:57:46 PM] Lloyd McKenzie says: Imagine an instance with three different template constraints on GTS.
[9:58:00 PM] Lloyd McKenzie says: Is a violation of a constraint considered to make something null?
[9:59:03 PM] Lloyd McKenzie says: Only 380 e-mails. I got off easy :)
[10:05:30 PM] Lloyd McKenzie says: 335 when I toast spam . . .
[10:11:07 PM] Grahame Grieve says: Is a violation of a constraint considered to make something null? - no
[10:11:30 PM] Lloyd McKenzie says: Except possibly for OTH
[10:11:42 PM] Lloyd McKenzie says: OTH says "I can't express this within the constraints provided"
[10:11:57 PM] Lloyd McKenzie says: For CD, that's usually a valueset + CNE
[10:12:19 PM] Lloyd McKenzie says: For GTS, it might be a constraint that says "this needs to be a continuous interval", but perhaps in some situation it isn't
[10:12:31 PM] Lloyd McKenzie says: For II, maybe I've got the extension, but no root
[10:27:45 PM] Grahame Grieve says: OTH says "I can't express this within the constraints provided" - that's not the definition
[10:29:03 PM] Lloyd McKenzie says: It says it in other words
[10:29:11 PM] Grahame Grieve says: what words?
[10:29:26 PM] Lloyd McKenzie says: "Not in the value domain of a variable"
[10:29:44 PM] Lloyd McKenzie says: The value domain is effectively the allowed set of values, given the constraints applied to the variable
[10:30:38 PM] Grahame Grieve says: ok. so now we have a situation where we can express full information in the instance, but we still claim no information, because some model makes some rule somewhere?
[10:30:53 PM] Grahame Grieve says: even though we do express full information
[10:31:28 PM] Lloyd McKenzie says: The challenge is that "full information" is dependent on compliance with the value domain.
[10:31:39 PM] Grahame Grieve says: where is this defined?
[10:31:42 PM] Lloyd McKenzie says: If I send you a valid code that's outside the domain, you can't process with it.
[10:34:24 PM] Grahame Grieve says: what's that got to do with the definition or meaning of No Information
[10:34:43 PM] Grahame Grieve says: I think we really need a different flag - one for nullness and one for compliance
[10:35:09 PM] Lloyd McKenzie says: They're intrinsically linked
[10:35:34 PM] Grahame Grieve says: linked? or related?
[10:35:37 PM] Lloyd McKenzie says: Linked.
[10:35:54 PM] Grahame Grieve says: I have a concept. What I have is the coding system - mine - in which the original concept was encoded
[10:35:54 PM] Lloyd McKenzie says: You can't execute many operations on an attribute that is null
[10:36:07 PM] Lloyd McKenzie says: You can't test if it's a specialization.
[10:36:29 PM] Grahame Grieve says: I translate it to a different coding system, but it doesn't have a concept for my concept
[10:36:56 PM] Grahame Grieve says: so I end up with two CD's, one which is fully encoded, and one which is nullFlavor OTH with a codingSystem and originalText
[10:37:08 PM] Grahame Grieve says: agree?
[10:37:15 PM] Lloyd McKenzie says: (having a codingSystem is dumb, but yes)
[10:37:40 PM] Lloyd McKenzie says: And the one that's OTH is null, even though you have information
[10:37:51 PM] Grahame Grieve says: ok. So then when I send it, whether it is null or not depends on what the constraints from the model are
[10:38:28 PM] Lloyd McKenzie says: Yup.
[10:38:43 PM] Lloyd McKenzie says: If it were CWE, it wouldn't need to be OTH
[10:38:45 PM] Grahame Grieve says: so sometimes I am claiming that "No information whatsoever can be inferred from this value", and sometimes I am not. Even though the value itself is unchanged.
[10:38:56 PM] Lloyd McKenzie says: You're misreading NI
[10:39:00 PM] Grahame Grieve says: it would still need to be OTH, whether it was CWE or not.
[10:39:09 PM] Lloyd McKenzie says: NI says "you can't tell anything about the reason why this is null"
[10:39:17 PM] Grahame Grieve says: I just quoted the definition
[10:39:25 PM] Lloyd McKenzie says: No information whatsoever can be inferred from this exceptional value. This is the most general exceptional value. It is also the default exceptional value.
[10:39:39 PM] Lloyd McKenzie says: You can't infer information *from the null value*
[10:39:48 PM] Lloyd McKenzie says: It doesn't say anything about the other properties
[10:40:39 PM] Grahame Grieve says: I thnk you are wrong
[10:40:40 PM] Grahame Grieve says: and right
[10:40:46 PM] Grahame Grieve says: you are wrong because the exceptional value is the whole value
[10:40:52 PM] Grahame Grieve says: for instance: a CD value without a value for the code property, or with a value that is not from the cited coding system is an exceptional value
[10:40:58 PM] Grahame Grieve says: proves the other thing wrong.
[10:41:07 PM] Grahame Grieve says: so reading the spec, you are wrong. BUt you are making sense
[10:41:41 PM] Lloyd McKenzie says: Always good when I make sense ;)
[10:42:54 PM] Lloyd McKenzie says: Fundamentally, a null value is not processable the way a non-null value is processable
[10:43:11 PM] Lloyd McKenzie says: If you make something mandatory, you're saying "this needs to be processable"
[10:43:26 PM] Lloyd McKenzie says: (where processability is based on the type of data you're dealing with)
[10:43:26 PM] Grahame Grieve says: I think you are building an edifice on a deck of cards
[10:43:37 PM] Grahame Grieve says: a null value is not processable the way a non-null value is processable - where is this stated?
[10:43:53 PM] Grahame Grieve says: the datatype spec rather blurs this concept in places
[10:43:58 PM] Lloyd McKenzie says: I'm saying what I think makes sense and is useful.
[10:44:27 PM] Grahame Grieve says: oh? really? me, I'm just working with V3 - I lost the link to sensible and useful long ago ;)
[10:44:33 PM] Lloyd McKenzie says: If I get an II, and it's mandatory, then I expect the value I receive to have certain characteristics and to be able to do things with it.
[10:44:43 PM] Lloyd McKenzie says: :P
[10:45:10 PM] Lloyd McKenzie says: I'm not necessarily pushing sensible, but at least I want useful. (And sensible if possible ;))
[10:45:35 PM] Grahame Grieve says: There's some real problems in that regard
[10:45:42 PM] Lloyd McKenzie says: Certainly the use of OTH fits what I'm describing.
[10:45:56 PM] Grahame Grieve says: I cannot think of any other specification or context where null doesn't mean completely null
[10:46:23 PM] Grahame Grieve says: the idea that you can have something that is null, and then actually have it present data - this has many troublesome connotations.
[10:46:30 PM] Lloyd McKenzie says: I'm happy to say that there are two different axis here.
[10:46:46 PM] Lloyd McKenzie says: Basically you can have:
[10:46:51 PM] Lloyd McKenzie says: Complete information
[10:46:53 PM] Lloyd McKenzie says: Sufficient information
[10:46:56 PM] Lloyd McKenzie says: Insufficient information
[10:46:59 PM] Lloyd McKenzie says: No information
[10:47:13 PM] Lloyd McKenzie says: If we make something mandatory, it means we need at least "sufficient information"
[10:47:43 PM] Lloyd McKenzie says: Most of the null flavors talk about reasons for having "no information"
[10:48:14 PM] Lloyd McKenzie says: OTH talks about why we have insufficient information
[10:49:19 PM] Grahame Grieve says: actually, but your wider definition, it could be "inappropriate information" rather than insufficient
[10:49:34 PM] Grahame Grieve says: but maybe this is "insufficient allowable information"
[10:49:50 PM] Lloyd McKenzie says: Fair enough
[10:50:01 PM] Lloyd McKenzie says: Shall I add Gunther?
[10:50:03 PM] Grahame Grieve says: there's a fine line - what information do I leave out because of constraints before what is left becomes insufficient?
[10:50:09 PM] Grahame Grieve says: I guess.
[10:50:12 PM] Grahame Grieve says: he
[10:50:15 PM] Grahame Grieve says: he's got nothing to do anyway
[10:50:23 PM] Lloyd McKenzie added Gunther Schadow to this chat
[10:50:36 PM] Grahame Grieve says: I'll copy and paste where we have been
[2:51:41 PM] Grahame Grieve says: we are arguing about the definition of null and it's relationship to conformance
[2:52:02 PM] Grahame Grieve says: huh. well, read that. We've all got nothing to do
[3:05:39 PM] Grahame Grieve says: I guess I don't care what I have to leave off to meet the constraints, as long as I can meet the constraints.
[3:06:15 PM] Grahame Grieve says: But what about this CWE/OTH issue - the datatypes spec says CD must be OTH if it doesn't have a code, but you don't think it should be null in this case
[3:06:58 PM] Lloyd McKenzie says: If it has to have a code whether it's CWE or CNE, then what's the point of coding strength?
[3:08:42 PM] Gunther Schadow says: Can u summarize the issue in one sentence? Is it about OTH or NULL in general?
[3:09:17 PM] Grahame Grieve says: yes, that's a good question
[3:09:21 PM] Lloyd McKenzie says: Null tackles two different issues:
[3:09:35 PM] Lloyd McKenzie says: "No information" and "Insufficient appropriate information"
[3:09:53 PM] Lloyd McKenzie says: MSK, NASK deal with the former. OTH deals with the latter
[3:09:54 PM] Grahame Grieve says: and OTH sits on the fence between the two
[3:10:38 PM] Gunther Schadow says: Why is that fine distinction important?
[3:11:09 PM] Lloyd McKenzie says: "Insufficient appropriate information" causes grief because "what's appropriate" changes based on the spec. If I've got an attribute asserting 2 templates, I might have 3 valuesets, but only comply with one.
[3:11:42 PM] Lloyd McKenzie says: Thus I have "insufficient appropriate information" according to 2 specs, but am fully valid against the 3rd.
[3:13:11 PM] Grahame Grieve says: this also runs into translations - the root is null, but the concept is correctly provided within - that's another problem - cause the concept isn't actually null. we sort of fixed that by adopting your (G) recommended solution to the CE problem, but Lloyd keeps positing multiple constraints, and we just made a mess of that
[3:14:01 PM] Gunther Schadow says: But how is that different from "temporarily unavailable" or "masked" (your thing Lloyd?). Even "unknown" and hence "no information"? It is all bound to change as the world evolves.
[3:15:33 PM] Lloyd McKenzie says: The difference is that for a single INSTANCE, if it's UNK, it's UNK against all specs.
[3:15:47 PM] Lloyd McKenzie says: For OTH, it might be null against some specs and not null against others
[3:16:17 PM] Lloyd McKenzie says: And we don't have the ability to say "are you null against spec A?", we can only ask "are you null?"
[3:17:33 PM] Gunther Schadow says: Who cares about the spec? MSK is different depending on purpose. What does "single instance" mean? Why is that important? You are always null because of something. Who is asking for being able to know why?
[3:17:55 PM] Lloyd McKenzie says: Single instance matters because it's the basis of conformance
[3:17:57 PM] Gunther Schadow says: If you spec does not include the last menstural period, then that is unknown to me unless you send something else.
[3:18:43 PM] Lloyd McKenzie says: If null is based on the spec, we need to expose that.
[3:18:54 PM] Lloyd McKenzie says: If it's not based on a spec, then we can't use OTH as a null flavor
[3:19:06 PM] Gunther Schadow says: I hear Grahame doing his spiel about v3 not being sensible, but how is this distinction going to make it more reasonable?
[3:19:24 PM] Lloyd McKenzie says: Right now, it's broken
[3:19:24 PM] Gunther Schadow says: If null is based on the weather we don't need to expose that either?
[3:19:29 PM] Gunther Schadow says: What is broken?
[3:20:01 PM] Lloyd McKenzie says: I can have a single instance that is null according to some specs and not null according to others, but the test to say "is this null" doesn't allow me to test the spec
[3:20:24 PM] Grahame Grieve says: I think the real problem here is that nullness should only be linked to the data itself, not to anything about conformance to some model
[3:20:36 PM] Grahame Grieve says: CD OTH links the 2.
[3:20:55 PM] Grahame Grieve says: if we broke those two things apart, then we would get a clearer picture
[3:20:55 PM] Lloyd McKenzie says: Except that there's a need for that link
[3:20:59 PM] Grahame Grieve says: why?
[3:21:23 PM] Lloyd McKenzie says: Because you want to be able to say "This attribute is mandatory, CNE" and be able to count on it being a valid code from the value-set
[3:21:36 PM] Grahame Grieve says: what has that got to do with nullflavor?
[3:22:10 PM] Lloyd McKenzie says: Mandatory says "can't be null". If OTH is a null flavor, then "incomplete" information is still considered null
[3:22:11 PM] Grahame Grieve says: already I must ignore nullFlavor to evaluate that in my validator
[3:22:23 PM] Grahame Grieve says: so we need to change the definitions in conformance, and make something that works
[3:22:38 PM] Grahame Grieve says: Mandatory should mean, must be properly populated
[3:22:51 PM] Lloyd McKenzie says: If OTH is not a null flavor and null only means no data at all, then I can't use mandatory to ensure I have complete data
[3:22:50 PM] Gunther Schadow says: Again, what is "instance" here? You have two meanings of it and that confuses you. There may be an object and you create different views of it for different purposes. One purpose only wants the age if less than 18 years old. So you conform to it. The object is the same, the view may be different. But the view is always different, because sometimes it allows you to send some extra info other times it doesn't. What you send depends on the spec. Yes. The spec might require you to mask out patient name (MSK). Fine, I don't see a problem.
[3:23:03 PM] Lloyd McKenzie says: Instance means "what goes over the wire"
[3:23:15 PM] Lloyd McKenzie says: Object is irrelevant to 95% of the HL7 world
[3:23:35 PM] Lloyd McKenzie says: Objects don't conform, only XML instances (or other wire formats) conform
[3:23:42 PM] Gunther Schadow says: but it's not in reality. If HL7 doesn't care about reality, I can't help it.
[3:23:44 PM] Lloyd McKenzie says: HL7 is a data exchange standard
[3:23:55 PM] Lloyd McKenzie says: All we care about is what's exchanged when we talk conformance
[3:23:57 PM] Grahame Grieve says: Gunther starting to sound like me a little
[3:24:05 PM] Gunther Schadow says: And you send what the spec allows. You can know more. What you send is not all you have.
[3:24:36 PM] Lloyd McKenzie says: So question 1 is:
[3:24:51 PM] Lloyd McKenzie says: Is mandatory == nullFlavor exists?
[3:24:58 PM] Gunther Schadow says: ?
[3:25:13 PM] Gunther Schadow says: mandatory means not null, doesn't it?
[3:25:19 PM] Lloyd McKenzie says: Sorry, yes
[3:25:27 PM] Grahame Grieve says: that's the standing definition, but it's broken
[3:25:34 PM] Lloyd McKenzie says: So mandatory means null flavor doesn't exist
[3:25:47 PM] Grahame Grieve says: not for CD, at least, not in practice.
[3:25:51 PM] Lloyd McKenzie says: Ok, so you're saying that mandatory means more than just null flavor doesn't exist?
[3:26:06 PM] Grahame Grieve says: it must. try validating CD.
[3:26:12 PM] Lloyd McKenzie says: CD is the prime example, but there are others (e.g. an II where I've got an extension but no clue what the root is)
[3:26:17 PM] Grahame Grieve says: you have to look in translations and ignore the root nullFlavor
[3:26:27 PM] Lloyd McKenzie says: Null flavor applies to the whole type
[3:26:33 PM] Grahame Grieve says: I reject you II case. it's going to be non-conformant some how
[3:26:35 PM] Lloyd McKenzie says: Translations are irrelevant.
[3:26:48 PM] Grahame Grieve says: maybe NullFlavor applies to the whole type, but that means that CD is done wrong
[3:27:16 PM] Lloyd McKenzie says: CD is definitely done wrong in terms of differentiating root from translations. However, that's a side issue
[3:27:35 PM] Grahame Grieve says: if the root nullFlavor is the nullFlavor for the whole concept, not just the root concept, then it's designed wrong
[3:27:55 PM] Grahame Grieve says: because they are conflated in the current design.
[3:28:17 PM] Lloyd McKenzie says: At the moment, OTH seems to allow for the null flavor being present to work for "mandatory", until you dig beneath the covers and think about multiple specs.
[3:29:03 PM] Grahame Grieve says: well, that too. But it doesn't even work with a single spec - unless you account for our R2 CE fix.
[3:29:06 PM] Grahame Grieve says: which you reject anyway
[3:29:28 PM] Grahame Grieve says: anyhow, as soon as you have multiple specs, linking mandatory to nullFlavor breaks down
[3:29:46 PM] Grahame Grieve says: the assessment has to be more sophisticated.
[3:29:48 PM] Lloyd McKenzie says: Agree.
[3:30:59 PM] Grahame Grieve says: we could make some fixes here.
[3:31:01 PM] Grahame Grieve says: The actual value is not an element in the value domain of a variable. (e.g., concept not provided by required code system).
[3:31:02 PM] Grahame Grieve says: make it
[3:31:06 PM] Grahame Grieve says: The actual value is not an element in the value domain of a variable. (e.g., concept not provided by specified code system).
[3:31:15 PM] Grahame Grieve says: bingo - broken the link.
[3:31:45 PM] Grahame Grieve says: leaves mandatory hanging in the wind, but we have to do something about that anyway
[3:31:51 PM] Gunther Schadow says: Why is mandatory for CD not mean not null?
[3:31:56 PM] Lloyd McKenzie says: Um, I don't see a difference in the sentences . . .
[3:32:09 PM] Grahame Grieve says: no. right.
[3:32:25 PM] Grahame Grieve says: The actual value is not an element in the value domain of a variable. (e.g., concept not provided by the codeSystem property).
[3:32:31 PM] Grahame Grieve says: that's what I meant to say
[3:32:52 PM] Gunther Schadow says: Why does CD make an exception for you? It doesn't.
[3:33:03 PM] Grahame Grieve says: Gunther, mandatory for CD does not mean not null because the translation that meets the conformance requirement may not be the root
[3:33:10 PM] Grahame Grieve says: or the one that fails to meet it
[3:33:12 PM] Gunther Schadow says: If you make some CD attribute mandatory, it can't be null. It can't be OTH.
[3:33:37 PM] Grahame Grieve says: but if the originalCode is not coded, and then a translation is the correct code...
[3:33:41 PM] Lloyd McKenzie says: Right, but what happens if an attribute is valid against some specs, but not others - do you assert OTH or not?
[3:33:47 PM] Gunther Schadow says: Then you can't send the message.
[3:33:58 PM] Grahame Grieve says: Lloyd, let's stick to a single spec for the moment
[3:33:59 PM] Lloyd McKenzie says: Pretend it's not mandatory
[3:34:04 PM] Grahame Grieve says: you can't send the message?
[3:34:04 PM] Gunther Schadow says: Mandatory means: if you don't send me this bit of info, I just can't process anything.
[3:34:14 PM] Lloyd McKenzie says: If we've got a single spec, there's no problem.
[3:34:19 PM] Grahame Grieve says: yes, but you do send it, only in one of the translations
[3:34:35 PM] Lloyd McKenzie says: Null flavor at the root applies to the root and all translations against the spec being asserted against
[3:34:37 PM] Gunther Schadow says: Then it's not null.
[3:34:52 PM] Grahame Grieve says: but it is null, because of this rule in the CD definition:
[3:35:02 PM] Lloyd McKenzie says: That rule is crap
[3:35:10 PM] Lloyd McKenzie says: It didn't take into account translations
[3:35:12 PM] Gunther Schadow says: It is not null.
[3:35:14 PM] Grahame Grieve says: Conversely, a CD value without a value for the code propertyis an exceptional value (NULL of flavor other)
[3:35:21 PM] Lloyd McKenzie says: It's similar to the rule that you have to identify a code system when you say OTH
[3:35:32 PM] Lloyd McKenzie says: Which doesn't take into account multi-code system valuesets
[3:35:36 PM] Gunther Schadow says: But aren't we going to redo CD so you put the expected thing on the outside anyway?
[3:35:55 PM] Gunther Schadow says: Either way, it is not null. Tranlations need to be taken into account here.
[3:35:57 PM] Grahame Grieve says: I have a root CD which has no code in some arbitrary codeSystem. I have a translation which has a code in the right codeSystem
[3:36:11 PM] Lloyd McKenzie says: Which is why we need to talk about multiple specs (i.e. templates) where you may have different expected codes
[3:36:19 PM] Lloyd McKenzie says: (Which means you can't posisbly always put it on the outside)
[3:36:23 PM] Gunther Schadow says: Then the CD as a whole is not null.
[3:36:29 PM] Grahame Grieve says: if we redo CD like that, it works for a single spec. Then Lloyd invokdes multiple specs and we are back to where we started
[3:36:54 PM] Grahame Grieve says: either we redo CD so we can express a nullFlavor for the concept at the root, not just the root attempt to encode the concept
[3:36:59 PM] Lloyd McKenzie says: If it's valid against one spec and not against another (it's OTH), is it null as a whole or not?
[3:37:01 PM] Grahame Grieve says: or we come up with some other solution
[3:37:15 PM] Grahame Grieve says: but Lloyd's multiple spec conformance case is worse than that
[3:37:18 PM] Gunther Schadow says: Not really. The rule would be: put the expected codeSystem/code on the outside depending on what the spec says.
[3:37:28 PM] Grahame Grieve says: but there's more than one....
[3:37:42 PM] Lloyd McKenzie says: Exactly - you've got 3 specs applying simultaneously
[3:37:54 PM] Gunther Schadow says: More than one, of course, but different specs, different messages.
[3:38:06 PM] Lloyd McKenzie says: 1 message - multiple templates
[3:38:11 PM] Gunther Schadow says: then they'd be inconsistent.
[3:38:15 PM] Grahame Grieve says: this is why I want to break null and conformance apart. Define null semantics clearly based on the data, and tell conformance to go and sort itself out based on the properly expressed data and the models
[3:38:34 PM] Lloyd McKenzie says: Templates are allowed to be inconsistent. I can have 3 different valuesets - that's what translations gives me
[3:39:01 PM] Lloyd McKenzie says: I think I agree with the split, and mandatory needs to tie to both concepts
[3:39:39 PM] Grahame Grieve says: Lloyd - what do you think you agree with?
[3:39:50 PM] Gunther Schadow says: The rule to put preferred code on the outside can only work with everyone agreeing. You can't have one spec saying you neen LOINC here and the other saying you need SNOMED. They can't apply simultaneously if they're inconsistent.
[3:40:15 PM] Lloyd McKenzie says: Which means "preferred code on the outside" is a really dumb idea
[3:40:27 PM] Gunther Schadow says: no its not.
[3:40:42 PM] Lloyd McKenzie says: It is. Because it makes you say one spec is more important than any other.
[3:40:45 PM] Gunther Schadow says: It's a really dumb idea to use two templates simultaneously that disagree.
[3:40:47 PM] Grahame Grieve says: but it seems to me at a semantic level we should be able to make that work - we can send multiple encoding - so we should be able to ask for more than one
[3:41:11 PM] Lloyd McKenzie says: Requiring that SNOMED and ICD10 codes both be present is perfectly reasonable.
[3:41:18 PM] Gunther Schadow says: You have to choose as an implementer. I use spec A or spec B, but not both. Only both if they are consistent.
[3:41:22 PM] Lloyd McKenzie says: No reason to care which is the root and which is not
[3:41:35 PM] Lloyd McKenzie says: As an implementer I can easily send both codes
[3:41:47 PM] Gunther Schadow says: It is reasonable to require both. But noone would in practice do that.
[3:41:55 PM] Lloyd McKenzie says: You bet they would
[3:42:03 PM] Gunther Schadow says: Who would?
[3:42:14 PM] Gunther Schadow says: Who is actually now asking for it?
[3:42:25 PM] Lloyd McKenzie says: We'll have some jurisdictions in Canada that require SNOMED, others that require ICD10 and applications that want to create single instances that comply with both.
[3:42:38 PM] Lloyd McKenzie says: I.e. I don't care which I'm sending to, I produce the same instance
[3:42:57 PM] Lloyd McKenzie says: Forcing me to swap the codes I can send around just based on recipient is dumb.
[3:43:19 PM] Grahame Grieve says: we're doing this in Australia with Lab results- we currently require every lab to send it's own internal code system, LOINC
[3:43:21 PM] Gunther Schadow says: In that case, none of your folks can benefit from the preferred-code-outside trick. Tough luck. But you're not hurt by it.
[3:43:28 PM] Gunther Schadow says: You just have to process all translations.
[3:43:49 PM] Grahame Grieve says: and we'd like to ask for snomed too, if we weren't stuck with V2. though we might go for more components in CE yet
[3:43:51 PM] Lloyd McKenzie says: I don't think anyone should expect their code in the root
[3:44:02 PM] Lloyd McKenzie says: I think it's dangerous and it hurts interoperability
[3:44:17 PM] Gunther Schadow says: Still, if the spec says LOINC is mandatory (and null not allowed) then you better have a LOINC code there. If it says local code mandatory you better have that. If it says nothing mandatory then there is no problem.
[3:44:28 PM] Grahame Grieve says: so, are we in agreement that it's reasonable to asked to send moer than encoding at once?
[3:44:36 PM] Grahame Grieve says: so, are we in agreement that it's reasonable to asked to send more than one encoding at once?
[3:45:03 PM] Gunther Schadow says: It's reasonable to suggest more codes, but only one will work for actual interoperability.
[3:45:35 PM] Grahame Grieve says: why will only one work?
[3:45:36 PM] Lloyd McKenzie says: Yes, but if the spec says "send LOINC" and the template says "send SNOMED" and you have a root code of SNOMED and no translation (because the concept doesn't exist in LOINC), do you send a null flavor of OTH for the root or not?
[3:46:00 PM] Grahame Grieve says: which do you send in the root? let's figure that out first
[3:47:24 PM] Lloyd McKenzie says: I don't think it should matter. I think either should be valid in the root. Or even neither - just send originalText in the root and have two translations.
[3:47:46 PM] Grahame Grieve says: semantically I agree. So the problem is with conformance, not interoperability
[3:48:35 PM] Lloyd McKenzie says: If we agree that they can go either place, then the interoperability issue goes away
[3:49:25 PM] Grahame Grieve says: it seems reasonable to me - instead of picking the root, you scan the root and translations looking for the right codeSystem.
[3:49:44 PM] Lloyd McKenzie says: Right.
[3:49:55 PM] Lloyd McKenzie says: Or more particularly, for a code from the desired valueset
[3:50:06 PM] Grahame Grieve says: but how do you know that?
[3:50:13 PM] Grahame Grieve says: all you have is codeSystem.
[3:50:47 PM] Lloyd McKenzie says: You have code + codeSystem. You can ask CTS whether that combo fits within a given value-set
[3:51:18 PM] Grahame Grieve says: oh. All of a sudden this is not so fun and easy. The performance hit for that operation is nasty
[3:51:19 PM] Gunther Schadow says: "Yes, but if the spec says "send LOINC" and the template says "send SNOMED" and you have a root code of SNOMED and no translation (because the concept doesn't exist in LOINC), do you send a null flavor of OTH for the root or not?" only one can be mandatory.
[3:51:36 PM] Grahame Grieve says: why? is that for interoperability or conformance?
[3:51:40 PM] Gunther Schadow says: It's mandatory with respect to either SNOMED or LOINC. Likely neither of them in practice.
[3:51:58 PM] Gunther Schadow says: If conformance is not for interoperability then I don't care.
[3:52:07 PM] Lloyd McKenzie says: Pretend it's not mandatory for a second
[3:52:16 PM] Lloyd McKenzie says: (in either the base spec or the template)
[3:52:28 PM] Gunther Schadow says: Mandatory with CD doesn't mean squad if you don't say what code system. I can always send you a made up code.
[3:52:36 PM] Gunther Schadow says: O.K. not mandatory.
[3:52:54 PM] Gunther Schadow says: If it's not mandatory your OTH problem goes away.
[3:52:56 PM] Lloyd McKenzie says: It's OTH against the template and not OTH against the base spec
[3:53:03 PM] Lloyd McKenzie says: Do I send the null flavor or not?
[3:53:09 PM] Grahame Grieve says: following this argument, btw, a single model should be able to say mandatory in two valueSets
[3:53:45 PM] Grahame Grieve says: what's it? I've gotten lost
[3:54:15 PM] Grahame Grieve says: Gunther, did you have any comment about the practicalities of plowing through the translations looking for the actual code you want to use?
[3:55:22 PM] Lloyd McKenzie says: I have an instance that declares support for a template. Base spec requires LOINC, template requires SNOMED. I send just a SNOMED code, because no LOINC translation exists. As far as the base spec is concerned, the instance is OTH, as far as the template is concerned, the instance is not null. So do I send OTH as a null flavor or not?
[3:56:05 PM] Grahame Grieve says: Gunther is claiming that you cannot meet those constraints.
[3:56:27 PM] Grahame Grieve says: I think that this is very disappointing, to accept such a limitation because we cannot define conformance away from the properties of the data
[3:57:01 PM] Lloyd McKenzie says: I think it's really bad to say we can't - we could even do it in v2 with the CE, CWE and CNE datatypes
[3:57:14 PM] Lloyd McKenzie says: That's the whole point of having translations
[4:00:23 PM] Grahame Grieve says: back to the definition of OTH - it's wrong to link it with required. The definition of CD itself is quite specific - CD acquires a nullFlavor of OTH if there is no code but a codeSystem is provided.
[4:00:28 PM] Grahame Grieve says: nothing to do with required there.
[4:01:24 PM] Lloyd McKenzie says: Not sure what you mean by required. If you mean "Requred" in the conformance sense, then yes, that's orthogonal. Where I used the term 'required', read "draws from a valueset based on"
[4:01:41 PM] Lloyd McKenzie says: The definition sucks
[4:01:57 PM] Lloyd McKenzie says: codeSystem is useless with a multi-codesystem valueset
[4:02:22 PM] Lloyd McKenzie says: And not having a root code shouldn't be a problem if you've got the desired code in a translation
[4:02:44 PM] Lloyd McKenzie says: I'm not terribly concerned with how things are - I'm interested in how they should be and what it'll take to get them there :)
[4:02:55 PM] Grahame Grieve says: yeah, well we agreed to add valueSet - but you can only use it with OTH
[4:03:08 PM] Grahame Grieve says: maybe we should split CD and things up and do it differently
[4:03:23 PM] Grahame Grieve says: hang on
[4:03:48 PM] Gunther Schadow says: It's not OTH against anything. You send LOINC or SNOMED. If you can't send either one it's OTH for that one.
[4:04:42 PM] Gunther Schadow says: "Gunther, did you have any comment about the practicalities of plowing through the translations looking for the actual code you want to use?" - few would do it in practice. You get shot for that by the practitioners.
[4:04:46 PM] Lloyd McKenzie says: Against the base spec, I haven't sent a LOINC code - because I can't. That sounds like OTH to me . . .
[4:04:58 PM] Grahame Grieve says: type ConceptDescriptor alias CD specializes ANY {
ED originalText; SET(CODE) encodings;
};
type CodedConcept alias CODE specializes ANY {
ST code;
ST displayName;
UID codeSystem;
ST codeSystemName;
ST codeSystemVersion;
LIST<CR> qualifier;
}
[4:05:12 PM] Grahame Grieve says: need to add rationale to CODE
[4:05:26 PM] Lloyd McKenzie says: BEAUTIFUL :)
[4:05:47 PM] Lloyd McKenzie says: Throw valueset as a set on codedConcept
[4:05:55 PM] Grahame Grieve says: glad you like it. Doesn't actually solve your multiple thingy yet
[4:05:59 PM] Gunther Schadow says: ???
[4:06:09 PM] Gunther Schadow says: What does that do?
[4:06:21 PM] Lloyd McKenzie says: It gets rid of the silly concept of "root code"
[4:06:30 PM] Grahame Grieve says: means that we clearly have a nullFlavor for the concept itself, and also one for each encoding
[4:06:34 PM] Lloyd McKenzie says: And removes originalText from all the translations
[4:06:37 PM] Gunther Schadow says: it's not silly. People wanted to know what is translated from what.
[4:06:48 PM] Grahame Grieve says: oh. yes. hang on
[4:06:48 PM] Lloyd McKenzie says: Root code doesn't give you that
[4:06:49 PM] Gunther Schadow says: If you can go down a list you can go inside.
[4:06:54 PM] Gunther Schadow says: There is no value in the difference.
[4:07:30 PM] Lloyd McKenzie says: (any more - it used to, but they took the whole tree structure away). However, Grahame has a new funky way to handle the what was translated from what aspect - not shown here.
[4:07:53 PM] Grahame Grieve says: type ConceptDescriptor alias CD specializes ANY {
ED originalText; SET(CODE) encodings;
};
type CodedConcept alias CODE specializes ANY {
ST code;
ST displayName;
UID codeSystem;
ST codeSystemName;
ST codeSystemVersion;
LIST<CR> qualifier;
SET<CS> rationale;
CD source;
}
[4:07:57 PM] Gunther Schadow says: root used to. I agree it does not if things go according to Grahame and me. But the set of CV does not help it.
[4:08:06 PM] Gunther Schadow says: "but they took the whole tree structure away" who is they?
[4:08:10 PM] Gunther Schadow says: And how took they?
[4:08:16 PM] Grahame Grieve says: what set of CV?
[4:08:34 PM] Grahame Grieve says: source points you back to the one you were translated from, if you care
[4:08:47 PM] Lloyd McKenzie says: They = vocab committee
[4:08:52 PM] Gunther Schadow says: When?
[4:09:00 PM] Lloyd McKenzie says: 2 WGMs ago at least
[4:09:10 PM] Gunther Schadow says: Ask again you get another opinion.
[4:09:21 PM] Gunther Schadow says: Doesn't mean anything.
[4:09:32 PM] Grahame Grieve says: this also allows for more options in regard to CE, because you no longer have the root problem. You can make a constraint that any encodings that have rationale != O have no qualifiers
[4:10:01 PM] Grahame Grieve says: this meets Lloyd's other problem and still enables proper translations
[4:10:04 PM] Lloyd McKenzie says: Motion passed - everything is subject to change, but at the moment it's policy. Believe it was a joint InM Vocab meeting
[4:10:30 PM] Gunther Schadow says: whatever
[4:11:32 PM] Lloyd McKenzie says: Anyhow, I can accept that people are scared of trees and that the complexity of exposing the history of translations was more than many care about. The new structure gives the ability to convey it when necessary. while being simpler.
[4:11:37 PM] Grahame Grieve says: Lloyd, you liked my proposal, but it doesn't solve your problem
[4:12:08 PM] Lloyd McKenzie says: No, it still doesn't tell me whether the attribute should be OTH or not
[4:12:17 PM] Grahame Grieve says: I like it, because it enables me to say clearly that the concept as a whole is encoded or not
[4:12:28 PM] Grahame Grieve says: so we still have to separate conformance and nullness
[4:12:47 PM] Grahame Grieve says: unless we go for SET<CS> for nullFlavor
[4:12:53 PM] Grahame Grieve says: man, that's a good idea. I'm on fire today
[4:13:15 PM] Lloyd McKenzie says: I'm trying to come up with the appropriate way to curse you . . . ;)
[4:13:50 PM] Grahame Grieve says: but to be serious, if you entertain the concept of multiple models to conform to, then you must entertain the concept of separating nullFlavor from mandatory
[4:14:15 PM] Lloyd McKenzie says: The issue isn't merely one of compliance. If something is CNE, not mandatory and I send originalText, I'm compliant, it's just not a valid value
[4:14:28 PM] Grahame Grieve says: huh?
[4:14:30 PM] Lloyd McKenzie says: I'm ok with separating nullFlavor from mandatory
[4:14:41 PM] Grahame Grieve says: how can you be compliant in that case/
[4:14:42 PM] Grahame Grieve says: how can you be compliant in that case?
[4:14:46 PM] Lloyd McKenzie says: It's only non-conformant if it's CNE and mandatory
[4:14:58 PM] Lloyd McKenzie says: If it's not mandatory, I can send nulls
[4:15:01 PM] Lloyd McKenzie says: Including OTH
[4:15:05 PM] Grahame Grieve says: it's also non conformant if it's not valid
[4:15:13 PM] Lloyd McKenzie says: Or more specifically "not null, but not fully specified"
[4:15:39 PM] Lloyd McKenzie says: I can use OTH with CNE
[4:16:28 PM] Grahame Grieve says: really? pl
[4:16:30 PM] Grahame Grieve says: really? ok
[4:19:26 PM] Lloyd McKenzie says: that's the only circumstance where OTH is needed
[4:19:57 PM] Grahame Grieve says: no, you use it anytime you don't have a code. CNE/CWE is irrelevant
[4:19:59 PM] Lloyd McKenzie says: With CWE, you're saying "this is perfectly valid even if no code from the preferred value set exists, so long as you send a local code or original text"
[4:20:13 PM] Lloyd McKenzie says: Then what does CWE/CNE do?
[4:20:59 PM] Grahame Grieve says: tells you whether you can define your own thing, so far as I can tell
[4:21:09 PM] Grahame Grieve says: your own code, I mean
[4:21:16 PM] Lloyd McKenzie says: Or send original text
[4:21:26 PM] Lloyd McKenzie says: CWE *definitely* says you can send originalText
[4:21:50 PM] Grahame Grieve says: so you can with CNE unless it is mandatory, right?
[4:22:09 PM] Grahame Grieve says: and you can send originalText anyway, with CWE, mandatory or not?
[4:22:22 PM] Lloyd McKenzie says: That's my understanding
[4:22:55 PM] Lloyd McKenzie says: Which means that you can't have to send OTH if it's CWE
[4:23:01 PM] Grahame Grieve says: but you have to use the nullFlavor OTH if you don't provide a code.
[4:23:04 PM] Grahame Grieve says: so you do.
[4:23:13 PM] Lloyd McKenzie says: But that rule is broken
[4:23:17 PM] Grahame Grieve says: yeap.'
[4:26:40 PM] Grahame Grieve says: not sure that my design is better, really. if you have more than one encoding, and they have different nullFlavors, what is the nullFlavor on the concept?
[4:27:37 PM] Lloyd McKenzie says: I think we move away from null flavor.
[4:27:44 PM] Lloyd McKenzie says: If you've got any properties, it's not null
[4:27:52 PM] Lloyd McKenzie says: However, it may not be "complete"
[4:28:11 PM] Lloyd McKenzie says: Complete means "meets the requirements of the specification"
[4:28:23 PM] Grahame Grieve says: This is what I've been working towards for a long time: If you've got any properties, it's not null
[4:28:41 PM] Lloyd McKenzie says: Mandatory says "must not be null and must be complete"
[4:28:53 PM] Grahame Grieve says: but, well, we designed around nullFlavor with propertoes so strongly, I don't think it can be done
[4:29:21 PM] Lloyd McKenzie says: Anything can be done :). What's so strong about null flavor with properties?
[4:29:38 PM] Lloyd McKenzie says: I don't think we see it many places (yet) except CD
[4:29:41 PM] Grahame Grieve says: huh
[4:29:44 PM] Lloyd McKenzie says: And we're gutting and rebuilding it anyhow
[4:29:47 PM] Grahame Grieve says: unknown home address
[4:29:52 PM] Grahame Grieve says: unknown birth name
[4:30:03 PM] Grahame Grieve says: unknown home phone number
[4:30:07 PM] Lloyd McKenzie says: Ahh. Those are recent, but true.
[4:30:19 PM] Grahame Grieve says: some of the things in PQ
[4:30:26 PM] Grahame Grieve says: TRC. But still with units
[4:30:35 PM] Lloyd McKenzie says: Can't have TRC with units
[4:31:03 PM] Grahame Grieve says: yeah, that's what the theorists would say, but I know what anyone in my company would do.
[4:31:06 PM] Lloyd McKenzie says: Or more specifically, "I can't think of any reason why units would be needed or relevant with TRC"
[4:31:48 PM] Grahame Grieve says: actually, not strictly true. TRC mg/mL would be interpreted rather differently to TRC pg/L
[4:31:52 PM] Lloyd McKenzie says: The uses are really narrowing down the "type" of data
[4:31:57 PM] Grahame Grieve says: though I agree there's no rational grounds for doing so
[4:32:15 PM] Lloyd McKenzie says: No rational grounds and misleading convention
[4:32:37 PM] Grahame Grieve says: well, I can tell you that working lab guys won't let go of units on TRC
[4:32:39 PM] Lloyd McKenzie says: If you want to express units, then do an IVL with an upper bound
[4:32:51 PM] Grahame Grieve says: yeah, that's why TRC shouldn't exist at all
[4:33:04 PM] Grahame Grieve says: it's wrong. Wrong wrong wrong, unless it's used in a CO
[4:33:08 PM] Lloyd McKenzie says: It should, because you don't want to talk about units
[4:33:12 PM] Grahame Grieve says: like for urinalusis
[4:33:27 PM] Lloyd McKenzie says: TRC is about something like peanut oil
[4:33:36 PM] Lloyd McKenzie says: "may contain a trace"
[4:33:46 PM] Lloyd McKenzie says: Don't know how much, don't care. Know it's not a lot
[4:33:58 PM] Grahame Grieve says: yeah, but what does it mean - how much is a trace? you shouldn't be able to say something like that without putting an upper bound on what a trace is
[4:34:02 PM] Lloyd McKenzie says: Lab would probably never use it.
[4:34:08 PM] Grahame Grieve says: never use what?
[4:34:12 PM] Lloyd McKenzie says: TRC
[4:34:19 PM] Grahame Grieve says: So who would?
[4:34:24 PM] Lloyd McKenzie says: Pharmacy
[4:34:29 PM] Grahame Grieve says: what's the use case?
[4:34:35 PM] Lloyd McKenzie says: Talking about drug ingredients
[4:34:55 PM] Lloyd McKenzie says: Specifically, non-active ingredients
[4:35:09 PM] Grahame Grieve says: what - we are telling you to add a trace?
[4:35:42 PM] Lloyd McKenzie says: When we want to say it contains red die #4, we don't know how much, we don't care. It's certainly not a significant constituent from a mass perspective.
[4:36:08 PM] Lloyd McKenzie says: Not generally for ordering, but for drug masterfiles and talking about contraindications, allergic reactions, etc.
[4:36:55 PM] Grahame Grieve says: well, I still reject the notion personally. No one should be allowed to be so careless. They should always give an upper bound
[4:37:01 PM] Grahame Grieve says: but I know I won't win that one
[4:38:36 PM] Lloyd McKenzie says: Upper bound doesn't tell you anything.
[4:38:50 PM] Lloyd McKenzie says: (in this use case)
[4:38:58 PM] Grahame Grieve says: maybe I just feel strongly about this because of experience. my daughter is allergic to cashew nuts. I know the amount that I need to be concerned about - it's not super small, like the kids that die from allergic shock, but it's not that high either, 1 whole cashew nut will be touch and go for her. So I'd like to know what "traces of nuts" really means
[4:40:34 PM] Grahame Grieve says: > It's certainly not a significant constituent from a mass perspective.
[4:40:42 PM] Grahame Grieve says: I'd like to know what you think is signficant
[4:41:50 PM] Lloyd McKenzie says: It means that if I'm weighing ingredients and mixing them, is going to be less than 1% of overall mass. I.e. Not sufficient to be considered a base or something.
[4:42:04 PM] Lloyd McKenzie says: Anyhow - this is a tangent.
[4:42:10 PM] Grahame Grieve says: oh - 1%? so we do have an upper bound?
[4:42:20 PM] Grahame Grieve says: but yes, it's a tangent.
[4:42:34 PM] Lloyd McKenzie says: That's the one in my head, but it may be different for others.
[4:42:49 PM] Lloyd McKenzie says: You've given one use-case for NULL with properties
[4:43:01 PM] Lloyd McKenzie says: Where the properties are really refining the "type" of thing we're talking about
[4:43:07 PM] Lloyd McKenzie says: Almost like structural attributes
[4:43:51 PM] Lloyd McKenzie says: Could do the same thing for telecom on type - telephone or e-mail
[4:44:07 PM] Lloyd McKenzie says: You said that already, never mind
[4:44:40 PM] Lloyd McKenzie says: Could you do it for ED - Non-available JPEG?
[4:45:00 PM] Grahame Grieve says: I guess that's another use case.
[4:45:05 PM] Grahame Grieve says: hadn't thought of that one
[4:45:38 PM] Grahame Grieve says: probably the new II properties are the same
[4:45:55 PM] Lloyd McKenzie says: Except they're not supposed to be constrained . . .
[4:46:12 PM] Lloyd McKenzie says: (I had thought of it and shivers started running up and down my spine)
[4:46:34 PM] Grahame Grieve says: like nullFlavor : SET<CS>? I thought I'd get a response for that one
[4:47:08 PM] Lloyd McKenzie says: That was more like a strong urge to retch ;)
[4:47:22 PM] Grahame Grieve says: The verified identifier is MSK, but I can tell what I'm using...
[4:47:46 PM] Lloyd McKenzie says: Ah. That new one. I thought you were talking "type"
[4:48:02 PM] Lloyd McKenzie says: Possible, I suppose
[4:48:09 PM] Grahame Grieve says: ouch, really, but there's a number of categorising properties like that where you need to be able to populate them when the thing is null
[4:48:29 PM] Grahame Grieve says: II.type might be gone. I've been hounding Tom to give me a convincing use case. He's running out of time.
[4:48:33 PM] Lloyd McKenzie says: Right. So we might want to declare that some properties are "categorizers"
[4:48:46 PM] Lloyd McKenzie says: :) :)
[4:49:19 PM] Lloyd McKenzie says: And categorizors can be present on null attributes just as structural attributes can be present on null associations.
[4:50:04 PM] Lloyd McKenzie says: The tricky bit is that structural attributes *must* be present on associations. However, mandatory properties such as mime type don't have to be present on attributes.
[4:51:06 PM] Grahame Grieve says: candidates are: II.scope AD.use EN.use TEL.use ED.lang ED.mediatype
[4:51:25 PM] Grahame Grieve says: though what were we going to do with CD?
[4:51:41 PM] Lloyd McKenzie says: Don't think there are categorizers for CD
[4:51:49 PM] Grahame Grieve says: so, there's no OTH on CD?
[4:52:10 PM] Lloyd McKenzie says: There's two separate use-cases for data on "null" things:
[4:52:14 PM] Lloyd McKenzie says: - categorizors
[4:52:25 PM] Lloyd McKenzie says: - supplemental (non-computable) information
[4:52:37 PM] Grahame Grieve says: you mean updateMode etc?
[4:52:42 PM] Lloyd McKenzie says: The latter is things like valueSet id and originalText on CD
[4:52:48 PM] Lloyd McKenzie says: Or just an extension on an II
[4:52:55 PM] Grahame Grieve says: yuck
[4:52:57 PM] Lloyd McKenzie says: You've got data, just not enough
[4:53:13 PM] Lloyd McKenzie says: It exists. Someone gives me a california drivers license number
[4:53:24 PM] Lloyd McKenzie says: I stick it in as a search criteria and it brings back possible matches
[4:53:53 PM] Lloyd McKenzie says: Not enough for a definitive match, but good enough to give me possibles
[4:54:39 PM] Grahame Grieve says: oh right, this is that use case were root is null and extension is populated.
[4:55:09 PM] Grahame Grieve says: but if that is a valid concept, why should it be "not enough"? either it is or it isn't.
[4:55:25 PM] Grahame Grieve says: If we allow that use case, then there's no reason to call the II null;
[4:57:22 PM] Lloyd McKenzie says: The II is no more null than the CD with originalText is
[4:57:47 PM] Grahame Grieve says: well, I was getting to that ;)
[4:58:06 PM] Lloyd McKenzie says: However, it's not a 'complete' II, in the sense that if II is mandatory, I'm going to be mad if you omit the root
[4:59:01 PM] Lloyd McKenzie says: I think every type should have an "isComplete" property. I some cases, it might take an argument of a particular spec (e.g. CD where it depends on the value-set asserted by the spec)
[4:59:09 PM] Grahame Grieve says: oh. we're talking about conformance again?
[4:59:32 PM] Grahame Grieve says: why are you going to be mad if omit the root? not for that use case, for others, of course
[4:59:33 PM] Lloyd McKenzie says: Conformance comes in where you add mandatory
[4:59:51 PM] Grahame Grieve says: but you're back to null and conformance
[5:00:28 PM] Lloyd McKenzie says: Mandatory is related to conformance and to null
[5:01:09 PM] Grahame Grieve says: we want to break the second part
[5:01:40 PM] Grahame Grieve says: null is related to the data itself (the instance, the object, the view), and not to anyone's expectations of it (the constraints, models, templates)
[5:02:03 PM] Lloyd McKenzie says: I think we want to say "mandatory = not null and isComplete"
[5:02:15 PM] Lloyd McKenzie says: I.e. Mandatory is about two things
[5:02:21 PM] Grahame Grieve says: I think you're getting ahead of yourself.
[5:02:38 PM] Lloyd McKenzie says: Though I guess the not null pieces is redundant because you couldn't be complete if you were null . . .
[5:03:17 PM] Grahame Grieve says: isComplete has to be a set - complete with regard to more than one specification
[5:03:43 PM] Lloyd McKenzie says: It does for CD for sure, not sure about others
[5:03:53 PM] Grahame Grieve says: or, we actually require validator to figure it out directly based on what is present, not based on a short cut called nullFlavor, or a new shortcut called isComplete
[5:04:13 PM] Lloyd McKenzie says: Or have it take an argument, such as a valueset id
[5:04:42 PM] Grahame Grieve says: well, it needs a series of arguments, one of which is domain/valueSet/codeSystem what ever
[5:04:52 PM] Lloyd McKenzie says: The datatypes need to be clear what the rules are to constitute a "complete" attribute value
[5:06:15 PM] Grahame Grieve says: why is it the datatypes problem?
[5:06:25 PM] Grahame Grieve says: it's the model's problem.
[5:07:07 PM] Lloyd McKenzie says: Don't think so. The rules for what constitutes a "complete" II, CD, TEL, etc. should be consistent.
[5:07:46 PM] Grahame Grieve says: no. you have a case where complete means it has an extension and no root
[5:07:49 PM] Lloyd McKenzie says: If I say "You must give me an X", I need to be confident in what I'm getting and shouldn't have to say any more than "You must give me"
[5:07:57 PM] Grahame Grieve says: but then most of your cases need a root
[5:07:58 PM] Lloyd McKenzie says: I don't think that's complete.
[5:08:15 PM] Grahame Grieve says: it's complete for a purpose.
[5:08:17 PM] Lloyd McKenzie says: I think that's useful, but wouldn't fit the criteria of mandatory.
[5:08:32 PM] Lloyd McKenzie says: It's got potentially useful information, but it's not complete
[5:08:54 PM] Grahame Grieve says: well, it doesn't meet the invariants so it's far less than complete at the moment
[5:09:05 PM] Grahame Grieve says: I guess you want a default conformance profile for a datatype
[5:09:07 PM] Lloyd McKenzie says: II is complete if it has root and extension (and maybe scope-not sure if that's required or not)
[5:09:23 PM] Lloyd McKenzie says: Sort of.
[5:09:46 PM] Grahame Grieve says: separation see. Datatypes is about what's possible. Conformance is about what's allowed where
[5:09:58 PM] Lloyd McKenzie says: Datatypes is about both
[5:10:24 PM] Lloyd McKenzie says: Datatypes are about constraints
[5:10:33 PM] Lloyd McKenzie says: Conformance is about enforcement of constraints
[5:11:03 PM] Lloyd McKenzie says: We have a high level concept at the attribute and association level called mandatory that says "must be present".
[5:11:11 PM] Grahame Grieve says: yes and no. Datatypes are about base level - things that are always true
[5:11:16 PM] Lloyd McKenzie says: For it to be meaningful, we need to know what "present" means
[5:11:50 PM] Grahame Grieve says: yes, I agree. and the more usecases we run into, the more we back off the innate requirements of the datatype - like making ii.root optional
[5:12:05 PM] Grahame Grieve says: so we put more into conformance.
[5:12:41 PM] Lloyd McKenzie says: But we still need a common definition of "present"
[5:13:10 PM] Grahame Grieve says: meets the default conformance requirements unless other requirements are specified
[5:14:56 PM] Lloyd McKenzie says: So a datatype flavor might assert additional constraints
[5:15:23 PM] Lloyd McKenzie says: And the datatype would need to meet those to be considered "present"
[5:15:33 PM] Grahame Grieve says: yes
[5:15:43 PM] Lloyd McKenzie says: Or it would be considered "OTH".
[5:16:01 PM] Lloyd McKenzie says: And 5 templates in play might each declare a different flavor
[5:16:14 PM] Grahame Grieve says: huh> where did OTH get into the picture?
[5:18:03 PM] Lloyd McKenzie says: Maybe it doesn't. My brain is too fuzzy to tell right now
[5:18:18 PM] Grahame Grieve says: ok. pick it up later.
[5:18:34 PM] Grahame Grieve says: I'll knock this into a r2 proposal. That one will be gunf
[5:18:37 PM] Grahame Grieve says: I'll knock this into a r2 proposal. That one will be fun
[5:18:44 PM] Lloyd McKenzie says: Indeed :)
follow up
[10:24:43 AM] Grahame Grieve says: hi everyone [10:24:46 AM] Grahame Grieve says: sorry for the long delay [10:24:55 AM] Grahame Grieve says: I have been on leave, and then sick [10:25:11 AM] Grahame Grieve says: I have been doing a lot of thinking about nulls and conformance. [10:26:00 AM] Lee Coller says: Hope your doing better [10:26:22 AM] Grahame Grieve says: wasn't too sick, though still too sick for work. [10:27:19 AM] Grahame Grieve says: The definition of other:
The actual value is not an element in the value domain of a variable. (e.g., concept not provided by required code system)
or
a CD value without a value for the code property, or with a value that is not from the cited coding system is an exceptional value (NULL of flavor other)
[10:27:28 AM] Grahame Grieve says: the question is, what is the value domain of the variable. [10:27:32 AM] Grahame Grieve says: we have a real confusion here. [10:28:02 AM] Grahame Grieve says: is the value set of the variable from the reference model (RIM + Datatypes) or from the constraint model(s) that apply to the variable. [10:28:33 AM] Grahame Grieve says: The definitions are not clear about this, but our usage is clear: it is the constraint model. [10:30:06 AM] Grahame Grieve says: I think this is wrong for a couple of reasons. The first is that the information may be constant, but different constraint models may be applied to it at different times. It's wrong that the information itself should change as the constraint models are changed (not it's representation in a wire form - that may change or not, a matter of convenience) [10:31:07 AM] Grahame Grieve says: the second is that more than one constraint model may apply. If the constraint models are a linear series (i.e. DIM/RMIM/HMD/MT) then it doesn't matter, they must be internally consistent, so that conformance to the must constrained model is conformance to all of them. [10:33:04 AM] Grahame Grieve says: but when it comes to templates, this is not necessarily true. Templates can apply multiple different constraints to the information. It's possible that there may be a subset of multiple models that is valid, or maybe there is nothing, it doesn't matter for this discussion. The point is, we link conformance to the constraint model: 1 flag. But then we allow multiple constraint models to which information must conform to. [10:34:47 AM] Grahame Grieve says: so I have proposed to unlink conformance from nullness, by defining the value domain of a variable to be the value domain as defined in the reference model itself. There should be a flag - a similar thing to updateMode - that indicates conformance to a particular model. (And the ITS might choose to do something clever here). [10:35:15 AM] Grahame Grieve says: but this is a sea change in V3 itself. it's too late to perform such major surgery. [10:37:10 AM] Grahame Grieve says: so we have to accept that things are broken. Although it could be an issue with simple types such as INT (model a says 1..3 and model b says 3..5, and the value is 4), it generally only becomes an issue for translations that have conformance constraints. In practice, therefore, it is mainly an issue for CD and CE. [10:37:35 AM] Grahame Grieve says: I propose to document this problem as an issue for discussion in the abstract datatypes. [10:41:41 AM] Lee Coller says: I imagine templates will want to be involved as well [10:43:10 AM] Grahame Grieve says: well, speaking as the templates author...... [10:43:30 AM] Grahame Grieve says: I did consider adding a comment to this effect in there, but I just don't know what to say [10:45:48 AM] Lee Coller says: So, template X says my code should be from value set A, and translated into terminology B. Template Y says my code should be from value set C, and translated into terminology D. [10:46:36 AM] Lee Coller says: Hopefully A and C intersect (hopefully they are the same). I think this implies there are different rules for conformance on translations than on the base term. For translations, at least one tranlsation conforming should satisfy the constriants. [10:52:13 AM] Grahame Grieve says: I don't agree. A and C don't need to intersect. the model is likely to say valueSet and the template is likely to specify only a template [10:52:18 AM] Lloyd McKenzie says: Templates should *never ever ever* say what the root concept needs to be. The template can say you need valueset A or B or C or D, but can't tell you where the value needs to appear [10:53:11 AM] Grahame Grieve says: so you propose to treat applied models differently for conformance. perhaps. I could discuss this in the templates spec. [10:53:29 AM] Grahame Grieve says: but this solution is specific to translations, doesn't deal with the general problem [10:53:41 AM] Lee Coller says: true [10:53:46 AM] Lloyd McKenzie says: I think models are different from a conformance perspective. [10:53:59 AM] Lloyd McKenzie says: When you create a system, you're not trying to conform with the RIM. [10:54:30 AM] Grahame Grieve says: who says? [10:54:32 AM] Lloyd McKenzie says: You're trying to conform to a specific message type - the one for the interaction [10:54:51 AM] Grahame Grieve says: I don't think this is true. You might be trying to conform to the clinical statement pattern [10:55:10 AM] Lloyd McKenzie says: That's what the methodology says [10:55:11 AM] Grahame Grieve says: or the RIM itself, as Gunther does. The point of the V3 methodology is that this is not only valid, but "right" [10:55:19 AM] Grahame Grieve says: where does it say this? [10:55:35 AM] Lloyd McKenzie says: The basis for conformance is application roles and the interactions they contain [10:55:48 AM] Lloyd McKenzie says: Patterns are used for designing messages. [10:56:11 AM] Lloyd McKenzie says: People may choose to implement a pattern directly, but that's outside the bounds of HL7 conformance [10:56:34 AM] Lloyd McKenzie says: Paterns aren't "normative" the way interactions are and application roles were supposed to be. [10:57:01 AM] Lloyd McKenzie says: If you want to just implement the RIM or clinical statement, have fun, but you can't say "I'm HL7 v3 conformant". [10:57:20 AM] Lloyd McKenzie says: You can say you're RIM-conformant, or clinical statement conformant. [10:57:34 AM] Lloyd McKenzie says: But a true conformance statement requires you to list interactions. [10:57:50 AM] Lloyd McKenzie says: The rules may be different for CDA - there anything that conforms to the CDA R-MIM is conformant [10:58:07 AM] Grahame Grieve says: oh, so you're talking about conformance, which is a narrower subject [10:58:34 AM] Lloyd McKenzie says: What I'm saying is that if the MT says 2..5 and you've got a 6, you can't send it. [10:58:46 AM] Lloyd McKenzie says: Templates are a different beast [11:01:17 AM] Grahame Grieve says: what do you mean, you can't send it? you send it with a nullFlavor of OTH. [11:01:23 AM] Grahame Grieve says: and why are templates different? [11:02:09 AM] Lloyd McKenzie says: The difference is that the MT for the interaction defines the outer bounds for what's allowed. Templates may add extra constraints, but so long as it's valid against the base specification, it's not null. [11:02:33 AM] Lloyd McKenzie says: It might however be invalid against a template constraint, which means if you've declared that template, you're non-compliant [11:02:45 AM] Lloyd McKenzie says: Failure to comply with a template doesn't make something null. [11:03:06 AM] Lloyd McKenzie says: However, if something isn't expressable with the MT, then it has to be null. [11:05:19 AM] Lee Coller says: Lloyd -- so youre saying a message can assert conformance to a template or templates, but a realm can't require conformance to a set of templates (in other words, the template designation is part of the instance, and can never by constrained in the MT) [11:15:52 AM] Grahame Grieve says: so we are saying that templates are a blunter instrument than a proper model [11:28:21 AM] Lee Coller says: That will be news to CCD [11:29:09 AM] Grahame Grieve says: you have anything specific in mind? [11:30:55 AM] Lee Coller says: I think the way they are trying to justify it now is that documents can constrain templateID, but message instances can't. I know not everyone will be happy with that. People want to constrain templateID (other than Lloyd that is). [11:32:19 AM] Lee Coller says: Do we have any real world examples where it is really an issue? [11:32:56 AM] Lee Coller says: If not, we could try and constrain the problematic models away [11:32:58 AM] Grahame Grieve says: well, there's orthogonal issues here. CCD is a template profile, so I don't see that there is any issues per se there. [11:34:12 AM] Grahame Grieve says: I'm interested in the detailed technical conformance issue here: do we know of any consequences of saying that nullFlavors cannot apply with regards to a template constraint:? [11:38:12 AM] Grahame Grieve says: a related issue: the nullFlavor UNC. For R2, I propose to handle this differently: as part of the codingRationale, a code NC "not coded" with definition: The source system did not attempt to encode the provided originalText in the defined valueSet or codingSystem [11:39:20 AM] Lee Coller says: I don't think its needed, but It's not worth having a batlle over it. [11:42:00 AM] Grahame Grieve says: I would like to change the definitions of two nullFlavors for R2. [11:42:18 AM] Grahame Grieve says: OTH: From
The actual value is not an element in the value domain of a variable. (e.g., concept not provided by required code system).
To
The actual value is not an element in the constrained value domain of a variable. (e.g., concept not provided by required code system).
[11:42:26 AM] Grahame Grieve says: addition of the word constrained [11:43:08 AM] Grahame Grieve says: second, to change NI from No information whatsoever can be inferred from this exceptional value. This is the most general exceptional value. It is also the default exceptional value.
To The value is exceptional (missing, incomplete, improper). No information as to the reason for being an exceptional value is provided. This is the most general exceptional value. It is also the default exceptional value. [11:43:47 AM] Lee Coller says: More properly we should say Value Set rather than domain. [11:44:27 AM] Lee Coller says: I like the NI wording. [11:44:46 AM] Grahame Grieve says: no. can't be valueSet- that is specific to CD. and value domain is defined in the spec, so it is the correct wording [12:11:22 PM] Lloyd McKenzie says: If a message definition asserts conformance to a template, the rules in the template are part of the rules of the message [12:12:03 PM] Grahame Grieve says: but they have to understood differently, since the nullFlavor is not in regard to the constraints in the template [12:13:26 PM] Lloyd McKenzie says: In terms of messages constraining templates, I'm comfortable with message types saying that certain templates must be declared, but I'm strongly opposed to saying that the set of templates that may be declared is limited. Any template you don't recognize, you ignore. And can safely do so, because templates gave no semantics. [12:13:51 PM] Grahame Grieve says: ? seems rather orthogonal to our discussion to this point [12:14:44 PM] Lloyd McKenzie says: I don't like "unencoded" as a coding rationale. It should behave as "null", the same as any other. [12:14:54 PM] Lloyd McKenzie says: If an element is mandatory, you can't send it as "un-encoded" [12:15:55 PM] Grahame Grieve says: that's what OTH is for. It's still OTH. unencoded is just extra information about the rationale underlying the concept [12:15:57 PM] Lloyd McKenzie says: I'm happy with NI definition as well [12:16:07 PM] Lloyd McKenzie says: It's *not* OTH. [12:16:17 PM] Lloyd McKenzie says: OTH says "this value can't be expressed" [12:16:24 PM] Grahame Grieve says: no. it does not [12:16:28 PM] Lloyd McKenzie says: UNC says "I haven't bother to try - it might be expressible, it might not" [12:16:44 PM] Grahame Grieve says: "The actual value is not an element in the constrained value domain of a variable"\ [12:16:45 PM] Lloyd McKenzie says: That's what it needs to say, and how it's always been interpreted [12:16:50 PM] Grahame Grieve says: nothing about cannot. [12:17:12 PM] Grahame Grieve says: is not, not can not. So I don't see an issue [12:17:17 PM] Lloyd McKenzie says: And if the actual value is "female", and the code system allows "F", then the actual value *is* an element in the constrained value domain of the variable [12:17:27 PM] Lloyd McKenzie says: I just might not have bothered to code it. [12:20:03 PM] Grahame Grieve says: well, that raises the question about what the "actual" value is. The value in the instance, or the value in concept. The spec itself uses both meanings in other places [12:20:17 PM] Lloyd McKenzie says: From earlier, if a message type asserts a template as part of the message type definition, then the constraints of the templates are enforced as if they had been declared as part of the message type itself. Referencing a template at design time is just a short-cut for re-use of design constraints. [12:20:29 PM] Lloyd McKenzie says: The actual value is the concept. [12:21:06 PM] Grahame Grieve says: but the current solution for UNC is that it is a specialisation of OTH, which is also incorrect according to your definition [12:21:34 PM] Grahame Grieve says: > if a message type asserts a template as part of the message type definition, then the constraints of the templates are enforced as if they had been declared as part of the message type itself [12:21:49 PM] Lloyd McKenzie says: Agree it shouldn't be a specialization of other [12:22:30 PM] Grahame Grieve says: cannot be - it's technical. templates are evaluated differently, whether they are nominated in the messasge type or not [12:23:37 PM] Lloyd McKenzie says: Can be. Just might not work with your existing code :) [12:23:53 PM] Grahame Grieve says: it's not my current code. I don't know how to make it work. [12:24:44 PM] Grahame Grieve says: when a template is applied, I have to find out whether the template is "defined" in the message type? (note that there is no proper to do this right now) [12:24:50 PM] Lloyd McKenzie says: If a message type contains template references, then for every node in the message type, you know what constraints apply. You simply take that set of constraints as the "Design set", and determine rules about Other, etc. based on that context. [12:25:09 PM] Lloyd McKenzie says: You don't need to. You've got the message type definition. [12:25:27 PM] Lloyd McKenzie says: Thus you know the "message" constraints before you even check what templates are declared. [12:25:37 PM] Lloyd McKenzie says: It's not even necessary that a design template *be* declared. [12:26:16 PM] Grahame Grieve says: I can see what you are saying, but it's ludicrously complicated. [12:26:43 PM] Grahame Grieve says: some fantasy land where implementation challenges are a good thing, not a bad thing. [12:26:44 PM] Lloyd McKenzie says: Cool, that must make it right ;) [12:27:03 PM] Lloyd McKenzie says: I don't think it makes things any more difficult to implement [12:27:13 PM] Grahame Grieve says: that's cause you don't implement this stuff. [12:27:54 PM] Grahame Grieve says: templates are hard enough without doubling the challenge by introducing a secondary template mode, one where thet template is part of the innate constraints on the instance, and one where it isn [12:28:12 PM] Grahame Grieve says: 't [12:28:48 PM] Grahame Grieve says: anyhow, this is no longer a datatypes discussion, it's a templates discussion [12:28:52 PM] Lloyd McKenzie says: I won't push for it now, because I don't need it. I'm happy to reference my constraints long-hand. However, it seems to me to be a perfectly reasonable thing to do. Referencing a template as a constraint is just another form of re-use. [12:28:57 PM] Lloyd McKenzie says: I'll agree with that :) [12:29:17 PM] Lloyd McKenzie says: The datatypes thing is "OTH" needs to be based on design-time constraints, not RIM. [12:29:27 PM] Grahame Grieve says: back to OTH and UNC. [12:29:38 PM] Grahame Grieve says: the revised definition for OTH makes this explicit. [12:29:46 PM] Grahame Grieve says: so I presume you are agreeing with this. [12:31:02 PM] Grahame Grieve says: UNC is therefore the a sibling of OTH, and we need a generalisation which means that " The value as represented is not an element in the constrained value domain of a variable" [12:31:20 PM] Lloyd McKenzie says: I'm happy with that. [12:33:35 PM] Lee Coller says: I still don't understand why I need UNC. If I'm too lazy to code, I just make code NI and populate original text (and whatever else). It's obvious that I didn't code it, and because I'm not OTH, I'm not asserting that there is no appropriate value. [12:33:51 PM] Grahame Grieve says: we could also define a nullFlavor EXPR that is a specialization of this common ancestor of OTH and UNC, that is the appropriate nullFlavor to use for when a ExPR<T> is provided [12:34:10 PM] Grahame Grieve says: Lee, this doesn't exclude UNK and it's derivatives, or NA [12:34:11 PM] Lloyd McKenzie says: The difference is what you're telling the receiver. UNC is saying something specific. [12:34:43 PM] Lloyd McKenzie says: Hmm, if an EXPR is provided, the value isn't exceptional. Derived isn't exceptional. [12:35:04 PM] Lee Coller says: UNC isn't exceptional either [12:35:25 PM] Lloyd McKenzie says: UNC is providing a non-valid value [12:35:39 PM] Grahame Grieve says: if an expr is provided, the actual value *may* not be null, but the value is represented in the instance is null - of some form [12:36:14 PM] Grahame Grieve says: we resolved this when we discussed EXPR. [12:36:23 PM] Lloyd McKenzie says: Don't agree. As a recipient, I interpret the value as the result of the expression. [12:36:37 PM] Lloyd McKenzie says: If something's mandatory, that shouldn't prevent me from using EXPR [12:36:56 PM] Grahame Grieve says: well, it prevents you from using QS, which seems like a much bigger deal to me. [12:36:59 PM] Lloyd McKenzie says: For example, dose might be mandatory in a prescription, but I can say 2mg/kg, so long as I send the patient's weight. [12:37:28 PM] Lloyd McKenzie says: True. [12:37:42 PM] Lloyd McKenzie says: (Technically, QS is a lazy EXPR<T> :)) [12:37:47 PM] Grahame Grieve says: agree [12:38:14 PM] Grahame Grieve says: the result of the expression is an imaginary value. You cannot meaningfully apply conformance requirements to imaginary values [12:38:54 PM] Grahame Grieve says: the value may be null when evaluated. Actually we started from one of your use cases, the one where you want to provide the value and the expression [12:39:48 PM] Lloyd McKenzie says: Expressions need to be evaluated by the recipient don't they? They're not imaginary then . . . [12:39:53 PM] Grahame Grieve says: anyhow, there is a series of validity rules in the datatypes which cannot be satisfied for T unless EXPR<T> is null if no proper value is contained in the instance [12:40:18 PM] Grahame Grieve says: agree that they are not imaginary in the end. but they are prospective at conformance time. [12:40:44 PM] Lloyd McKenzie says: Don't like that. I'd like to be able to make expressions mandatory - saying "this must evaluate to not null" [12:41:27 PM] Grahame Grieve says: We can only solve your conformance problem by separating nullFlavor and conformance. You know I have argued for this, but you have opposed this [12:41:42 PM] Grahame Grieve says: and you cannot make an expression mandatory [12:42:36 PM] Lloyd McKenzie says: Don't understand separating anything from conformance. Conformance is what this whole thing is about. [12:43:05 PM] Lloyd McKenzie says: Don't understand why you can't make expression mandatory. Expression is equivalent to putting a formula in Excel. Anyone looking at the spreadsheet can always see the value. [12:43:14 PM] Grahame Grieve says: so you don't understand. never mind. [12:43:49 PM] Grahame Grieve says: but with regard to expression mandatoriness, let's go back to your example. dosage depending on body weight, what does it mean to make this mandatory? [12:44:11 PM] Lloyd McKenzie says: It means that when you evaluate the dosage, you need to get a non-null value. [12:44:14 PM] Grahame Grieve says: are you mandating some fantasy world where the patients body weight is known at conformance time? [12:44:20 PM] Lloyd McKenzie says: (Which in this case, means weight must be non-null) [12:44:36 PM] Lloyd McKenzie says: In the instance. Expressions are always based on data in the instance. [12:45:21 PM] Grahame Grieve says: not true. decidedly not true. in fact, if expressions are always based on data in the instance, I will remove EXPR as meaningless [12:45:57 PM] Grahame Grieve says: since the sender could just execute the expression and stick the value in. the point of EXPR is to allow input from elsewhere. [12:46:25 PM] Grahame Grieve says: particularly of the doe definition is in an SPL document where there is no patient in scxope [12:46:27 PM] Lloyd McKenzie says: k. I'll accept that. And agree that expressions would need to behave as null then. [12:47:57 PM] Grahame Grieve says: in this sense, the EXPR is the actual value. It can be evaluated to derive another actual value. [12:48:34 PM] Grahame Grieve says: so it is appropriate to have a nullFlavor for expr, since we are saying, this value is null because an expression is provided. [12:49:09 PM] Grahame Grieve says: this is really close to the EXPR use case, and in some ways, CD.originalText is actually a shortcut for an EXPR<CD>, though this wouldn't be valid as we limited EXPR to QTY. [12:49:10 PM] Lloyd McKenzie says: Yes, I can accept that. And agree a special flavor would be good. [12:49:32 PM] Grahame Grieve says: sorry, "this is really close to the UNC use case..." [12:49:41 PM] Grahame Grieve says: but I do not propose conflating these concepts [12:54:31 PM] Grahame Grieve says: `````````````` [12:54:38 PM] Grahame Grieve says: sorry. that was the baby. [12:54:57 PM] Lloyd McKenzie says: No worries. My puppy does the same on occasion :) [12:56:43 PM] Grahame Grieve says: also in relation to issue 103, I propose adding a paragraph or two to nullFlavor explaining the similarities and differences between null (improper value; hl7 null) and null (empty value; sql null, null pointer, OCL null), and how nullflavors propagate on operations. [1:00:12 PM] Grahame Grieve says: finally, we can now specify that the root code in a CD is the code that is specified by the constraining model, if there is one, or any code if there isn't. I need to wordsmith around the concept of constraining model, but we have established that there can only be zero or one of these. I will add a section called conformance discussing this [1:01:06 PM] Lloyd McKenzie says: I *really* prefer to not constrain the root code. [1:01:19 PM] Grahame Grieve says: why [1:01:59 PM] Lloyd McKenzie says: Because I shouldn't have to move data around in a datatype based on who I'm sending the data to. [1:02:19 PM] Lloyd McKenzie says: A CD instance that contains the desired data should be good enough. [1:02:48 PM] Grahame Grieve says: nice try. But when I argued that the data shouldn't change because of this, you disagreed. Trying to have your cake and eat it too [1:03:04 PM] Grahame Grieve says: whether if has nullFlavor OTH or not will change., so I don't see your issue [1:04:29 PM] Lloyd McKenzie says: Hmm. I still see OTH applying on a per-constraint model basis. How will CDA manage OTH, given that their vocabularies are totally unconstrained? [1:05:14 PM] Grahame Grieve says: therefore it doesn't matter - pick any one [1:06:55 PM] Lloyd McKenzie says: So you'll never see OTH in a CDA document, regardless of invoked templates? [1:07:05 PM] Grahame Grieve says: no, you will if the code is blank. [1:07:27 PM] Lloyd McKenzie says: But OTH according to what valueset? [1:08:11 PM] Grahame Grieve says: when you have a blank code with a nullFlavor of other, you must specify either a valueSet or a codingSystem that it is null with respect to [1:08:28 PM] Grahame Grieve says: doesn't matter what the design says, you must always specify it [1:08:44 PM] Lloyd McKenzie says: And if multiple templates apply, and it's null against multiple value-sets? [1:09:08 PM] Grahame Grieve says: templates cannot affect the nullFlavor. That's what we've been discussing. [1:09:49 PM] Lloyd McKenzie says: But for CDA, there are no value-sets [1:10:03 PM] Lloyd McKenzie says: The ActCode domain will never be bound in any realm. [1:10:09 PM] Lloyd McKenzie says: (At least not in any sane realm :)) [1:10:22 PM] Lloyd McKenzie says: Thus no value-set will ever apply, except as asserted by templates. [1:10:30 PM] Grahame Grieve says: when you evaluate a template, and it says that a coded attribute is mandatory with respect to a particular codingSystem or valueSet, this has nothing to do with nullFlavor, you have to go hunting. [1:10:42 PM] Grahame Grieve says: I'm not sure what your CDA issue is [1:11:29 PM] Lloyd McKenzie says: I'm saying because CDA doesn't constrain attributes bound to bindable domains or to value-sets, it'll never be able to use OTH. [1:12:04 PM] Grahame Grieve says: not so. the definition of OTH is " The actual value is not an element in the constrained value domain of a variable" [1:12:28 PM] Lloyd McKenzie says: But the value in CDA is "anything under the sun", because it's not constrained. [1:12:45 PM] Lloyd McKenzie says: Thus no actual value can exist that isn't an element in the constrained value set [1:12:50 PM] Grahame Grieve says: whether or not constraints are applied to a CD externally in it's context of use, the inherent constraints in CD itself still apply [1:13:23 PM] Grahame Grieve says: which say that if you cannot find a code in the codingSystem or valueSet, then the nullFlavor is OTH, and a codingSystem or valueSet must be provided. [1:13:45 PM] Lloyd McKenzie says: But the code systems and value sets are only determined by templates [1:14:25 PM] Grahame Grieve says: so you pick any one, and the nullFlavor is controlled by the value itself according to CD constraints, and the templates do not affect the nullFlavor [1:14:43 PM] Grahame Grieve says: not sure what your issue is. maybe we need to be specific. [1:15:25 PM] Lloyd McKenzie says: If I have a CDA document that declares two templates, with two value-sets and the value doesn't exist in either of them, which do I declare as OTH? [1:15:34 PM] Lloyd McKenzie says: I'd want to declare *both* value sets [1:16:34 PM] Grahame Grieve says: you pick either as a root, and they are both OTH. They are not OTH because of the constraints in the template, but because of those in the CD type itself. [1:17:07 PM] Lloyd McKenzie says: The CD says "you must comply with the value-set in effect". That's what makes it OTH [1:17:16 PM] Grahame Grieve says: no. [1:17:22 PM] Lloyd McKenzie says: And to determine the valueset in effect, you need to look at the templates [1:17:26 PM] Grahame Grieve says: no [1:18:26 PM] Grahame Grieve says: The CD makes additional rules. it can be OTH irrespective of the value set in effect, if there is no code in the specified codingSystem or valueSet. Irrespective of what the specified valueSet is [1:19:09 PM] Grahame Grieve says: according the rules we have discussed, if the templates made those valuesets mandatory, the instance could not be created, since the OTH-ness does not apply to the template validation. [1:19:24 PM] Lloyd McKenzie says: Ok, so you could declare it's OTH against a value-set that isn't in the CDA spec or any other declared template and be compliant? [1:19:49 PM] Lloyd McKenzie says: I'm not saying it's mandatory. I'm saying I want to mention both value-sets and say it's OTH to each. [1:19:56 PM] Grahame Grieve says: yes. [1:20:06 PM] Grahame Grieve says: unless CDA said it was mandatory- then you have to find something that isn't OTH as the root. Or if the templates said it was mandatory, then you are stuffed. [1:20:33 PM] Grahame Grieve says: I know you aren't saying it's mandatory. I'm just extending the use case to be clear [1:20:42 PM] Lloyd McKenzie says: Understand the mandatory bit. How do I declare two valuesets and say it's OTH to both? [1:20:57 PM] Grahame Grieve says: one is root, other is translation. both have nullFlavors of their own [1:21:15 PM] Grahame Grieve says: translations always have their own nullFlavors [1:21:40 PM] Lloyd McKenzie says: That, I didn't know. [1:22:09 PM] Grahame Grieve says: ok. well, it's inherent in the concept of translations. always, translations have their own nullFlavors. [1:22:46 PM] Grahame Grieve says: though their own nullFlavors do not contribute to any assessment of conformance, and only arise from the inherent definitions and meanings of the types in the datatypes specification [1:23:00 PM] Lloyd McKenzie says: So I can declare all the value-sets I'm supporting, and indicate which ones are OTH and which ones aren't. [1:23:17 PM] Grahame Grieve says: yes. [1:23:18 PM] Lloyd McKenzie says: However, I have to re-order what's root and what's translation based on who I'm sending to? [1:23:22 PM] Lloyd McKenzie says: Why? [1:23:32 PM] Grahame Grieve says: potentially. and you have to change the nullFlavor. [1:23:40 PM] Lloyd McKenzie says: I don't have to change the null flavor [1:23:49 PM] Lloyd McKenzie says: I've got all the value-sets I support declared [1:23:56 PM] Lloyd McKenzie says: For all potential recipients [1:24:05 PM] Grahame Grieve says: this is about conformance. [1:24:34 PM] Lloyd McKenzie says: Yes - and why do I have to put a particular code in the root if it's already in a translation if I want to conform. [1:24:50 PM] Grahame Grieve says: in order to conform. [1:25:05 PM] Lloyd McKenzie says: What I'm saying is that's not a reasonable expectation to conform. [1:25:10 PM] Lloyd McKenzie says: I'm communicating the right data [1:25:17 PM] Lloyd McKenzie says: I'm communicating it in an interoperable way [1:25:24 PM] Lloyd McKenzie says: The receiver can find whatever they're looking for [1:25:36 PM] Grahame Grieve says: but not in a conformant way - unless you want to break these things apart [1:25:35 PM] Lloyd McKenzie says: Why should I have to re-jig my data for no clear benefit? [1:26:07 PM] Lloyd McKenzie says: Conformance is just about following the rules. The objective of the rules is interoperability. I'm questioning this particular rule. [1:28:11 PM] Grahame Grieve says: we discussed this a little while ago. I agree in principle, but life is complicated. [1:28:31 PM] Grahame Grieve says: in order to properly consider this, we need to consider a mandatory CD. [1:28:39 PM] Grahame Grieve says: not mandatory, who cares. [1:29:22 PM] Lloyd McKenzie says: And your challenge is "if I've got 10 translations, and half of them are null, is the attribute null?" [1:29:57 PM] Grahame Grieve says: the real question is, does a mandatory CD require the root attribute to be nonNull, or do I just need to find some nonNull CD in the complex that meets the requirements of the main model [1:30:32 PM] Lloyd McKenzie says: My answer would be the latter, but I can see where that would be challenging. [1:30:55 PM] Lloyd McKenzie says: Ok, I can live with it. [1:31:09 PM] Grahame Grieve says: yes. that's the problem. It's too challenging. So we agree that it's easier to say the root must be nonNull, and therefore you may have to reorganise the data you send. [1:31:36 PM] Lloyd McKenzie says: k. At least I'm convinced there's a good reason now :) [1:31:40 PM] Grahame Grieve says: cool [1:55:09 PM] Grahame Grieve says: I call #103 closed. I will write it up and apply changes.
Further Discussion
LeeColler Wow! There's a lot of stuff covered here. Much of it concerns me.
First of all, I think we need to break it into separate topics, some of which would be out of scope for a data types discussion. Having just read through all of us I see the following topics:
- When is a data type instance null?
- Can a null data type instance have non-null (or even meaningful) properties?
- Should categorization attributes (i.e., use) be allowed on null instances?
- How do I require codes from multiple code systems?
- How do I make it easy to evaluate
- How do I show translation paths
- Should we change the CD (and thus CE/CV) defintions to support this?
- How does "Mandatory" (and presumably Required) relate to Null/NullFlavor
- How does CWE/CNE relate to this?
- Is TRC a null flavor
- Is there a conept of a data type instance being complete?
I probably missed a few.
When is a data type instance null?
As has been pointed out, our current null rules don't match with the normally understood concept of null in the computer science world. I think this is a problem, and it makes HL7 difficult to understand, and its difficult for people to get their hands around things. I think we need to keep this simple. x.isNull implies that for all properties of x (other than those relating specifically to null) x.<property>.isNull (or better yet x.<property> is nonsensical). I'm willing to allow for type to be non-null as well -- I don't think that is a big deal.
I understand the arguments around Mandatory, but I think we need to keep the definition of Mandatory the way it is and not make it more complicated. Currently it means that the attribute must be present and non-null. If a particular realm or use case has further requirements on that, they can specify them via a template or whatever, but that shouldn't be mixed in with mandatory. There are various levels of specifying conformance, going from simple (dealing with presence/absence of attributes and null/non-null), to extensibility (CNE vs. CWE) to templates. Let's keep the simple things simple and save the complexities for the complex things.
also to deal with: this comment on ANY.nonNull: When a property, RIM attribute, or message field is called mandatory this means that any non-NULL value of the type to which the property belongs has a non-NULL value for that property, in other words, a field may not be NULL, providing that its container (object, segment, etc.) is to have a non-NULL value.
