This wiki has undergone a migration to Confluence found Here

Difference between revisions of "Normative XML ITS Schema"

From HL7Wiki
Jump to navigation Jump to search
Line 2: Line 2:
  
 
During the discussion of the proposed [[UML ITS]] it was suggested that in order to ease the implementation burden a set of normative schema/schematron could be created as part of (a new release of) the XML ITS. The advantages/disadvantages of this approaches have yet to be discussed.
 
During the discussion of the proposed [[UML ITS]] it was suggested that in order to ease the implementation burden a set of normative schema/schematron could be created as part of (a new release of) the XML ITS. The advantages/disadvantages of this approaches have yet to be discussed.
 +
 +
==Summary==
 +
This Summary was prepared from the discussions below and on the
 +
INM telecon by Grahame Grieve
 +
 +
* We acknowledge that the current situation is frustrating for the implementers as there is no final statement of what is the acceptable wire format, and it is hard to determine what is actually correct.
 +
* we believe that in spite of it's shortcomings, W3C XML schema is the most appropriate form to make this statement. It stands as human and computer usable documentation
 +
* we do not currently give any formal status to the schemas that we publish. We do not properly document why do not do this.
 +
* we would like to give a formal status to the current schemas, so that implementors know how much to rely on them, and what not to rely on them for
 +
* we do not want to make the schemas normative in the sense that each schema is subject to ballot and comments on models can be made against the schemas
 +
* but we would like to give the schemas a formal status and appropriate (and documented) procedures to provide assurance that they have real meaning.
 +
* we are still searching for the appropriate term to give implementation artifacts derived from normative artifacts. (some discussion below)
 +
* we also acknowledge that schema can never provide complete validation, and that we need to clearly document the limitations of this
 +
 +
But there is two real problems with giving the schemas a more formal basis:
 +
 +
* the set of normative instances described by the XML ITS (often called "the wire format") is not properly able to be described by schema. This is  discussed further below. This is the big problem in giving the schemas a more formal basis (whatever we call it)
 +
* we also know that there is various differences in the schema validation tools
 +
 +
The UML ITS may be the the only real way to solve the problem.
 +
The UML ITS has been designed from scratch to deliver schemas that
 +
describe the wire format, and to deliver schemas that are interpreted
 +
as uniformly as possible.
 +
 +
This may become a solution for this problem for the XML ITS if the UML
 +
ITS becomes R2 of the XML ITS. At the present time, in some configurations
 +
the UML ITS is very similar to the UML ITS, and this is still something
 +
to consider.
 +
 +
==Naming Problem==
 +
 +
We do not know what name to give to our desired status for the schemas. Some possibilities discussed:
 +
* conformant
 +
* official
 +
* normative
 +
* published
 +
 +
An alternative approach is the leave them as normative and just clearly document their status
 +
and how we manage them.
 +
 +
Action: Contact HQ for whether ANSI has some appropriate terms. (done, awaiting comment)
 +
 +
==Differences between the XML ITS wire format and the schemas==
 +
 +
Schema do not properly describe the wire format that the XML ITS describes.
 +
In some areas, instances that are conformant will fail schema validation
 +
and in others, instances that are not conformant will pass schema validation.
 +
 +
Obviously, there is many things that the schema do not validate. Essentially,
 +
the schema validate the XML and the format of the data itself. The schema
 +
do not - and never will - validate the semantic correctness of the instance.
 +
But in the things the schema does, we need to make the schema do them correctly
 +
before we can give the schema an enhanced status.
 +
 +
'''Co-occurance constraints''': The schema do not know the co-occurance constraints.
 +
This could be fixed in the XML ITS.
 +
 +
'''xsi attributes''': Schema based processing engines - which we would need to make work - this
 +
is a big part of making the schemas normative - will consume and produce xsi: attributes such
 +
as type, nil, schemalocation etc. For the moment:
 +
* we have banned schemaLocation
 +
* we make no comment on xsi:nil
 +
* we make very confusing comment on xsi:type
 +
 +
We appear to reuse the xsi:type mechanism for the HL7 type representation. This is not
 +
clearly documented, but it is highly implied. The biggest problem with using schema is
 +
that the behaviour of xsi:type follows the schema type rules, and we have been unable
 +
to match the behaviour of the schema type rules and the HL7 type rules, particularly
 +
in the datatypes area.
 +
 +
todo: complete this section
 +
  
 
==Discusion==
 
==Discusion==
Line 14: Line 86:
 
==Discussion during the 2006113 INM TelCon==
 
==Discussion during the 2006113 INM TelCon==
 
*Doug: support making them normative, irritating that there is no support. Tony agrees.
 
*Doug: support making them normative, irritating that there is no support. Tony agrees.
*Grahame: within UML ITS. Implementers should have norm artefacts. Trouble is that schema describe same wire format as XML ITS. XML ITS is not written for schema. Can't make schema normative, then turn around and state schema are not correct. Schema can't be enough, but have to be correct. Lots of stuf out of schema specification like xsi:nill. In UML ITS describe actually what goes on the wire. Again not all you need to know. We design wire-format differently in order to make schema normative. Interetsed in whether "normative" is correct word. Normative means schema are subject to ballot. We don't want that. What we want is for implementers to be able to "hang their hat on" some schema set.
+
*Grahame: within UML ITS. Implementers should have norm artefacts. Trouble is that schema doesn't describe same wire format as XML ITS. XML ITS is not written for schema. Can't make schema normative, then turn around and state schema are not correct. Schema can't be enough, but have to be correct. Lots of stuf out of schema specification like xsi:nill. In UML ITS describe actually what goes on the wire. Again not all you need to know. We design wire-format differently in order to make schema normative. Interetsed in whether "normative" is correct word. Normative means schema are subject to ballot. We don't want that. What we want is for implementers to be able to "hang their hat on" some schema set.
 
*Doug: is there analysis of what schema can't describe? Grahame: not written up. Doug: schema not ridgid enough? Grahame: having sche carries implications for wire format. Sch describe wire format. Cant ahve areas where areas are wrong. Can have areas where they are incomplete. reasonable for implementers to say that instance validates against schema. Schema type handing system is different in concept to v3 typing system. we dont describe differences. If you use schema based toolkit you trip over differences like xsi:nil.  
 
*Doug: is there analysis of what schema can't describe? Grahame: not written up. Doug: schema not ridgid enough? Grahame: having sche carries implications for wire format. Sch describe wire format. Cant ahve areas where areas are wrong. Can have areas where they are incomplete. reasonable for implementers to say that instance validates against schema. Schema type handing system is different in concept to v3 typing system. we dont describe differences. If you use schema based toolkit you trip over differences like xsi:nil.  
 
*Skirmantis: schema should be designed for extensability. Current schema dont allow for diff namespaces. Grahame: need to do in UML ITS. Skirmantas: Schema should allow attributes from other namespaces. Grahame: can extend attributes, can't extend datatypes. Schema dont have to describe them. Skirmantas: if schema are official/normative, should have extenibility build in.
 
*Skirmantis: schema should be designed for extensability. Current schema dont allow for diff namespaces. Grahame: need to do in UML ITS. Skirmantas: Schema should allow attributes from other namespaces. Grahame: can extend attributes, can't extend datatypes. Schema dont have to describe them. Skirmantas: if schema are official/normative, should have extenibility build in.
 
*What does it mean to make it normative. Where should it be used? Grahame: can use in implementation, dont have to. Describes wire format. Need to be clear about it and avoid interpretation that any instance that validates against schema is a valid v3 instance.  
 
*What does it mean to make it normative. Where should it be used? Grahame: can use in implementation, dont have to. Describes wire format. Need to be clear about it and avoid interpretation that any instance that validates against schema is a valid v3 instance.  
 
*"Normative" is the wrong word. Call them "Official"? "HL7-Sanctioned"? Stand behind them. Won't get tripped on if you implement these. Call them "Supported"?. "Official Reference Schema"? If you fix schema then you change wire format. Can't change them that freely anymore once balloted. Currently schema can be changed at will, as long as we have same wire format. How much a price are we willing to pay? Need to stay just short of being normative. Don't want ballot comments on individual schema nor schema construct principles. Call them "Conformant?"
 
*"Normative" is the wrong word. Call them "Official"? "HL7-Sanctioned"? Stand behind them. Won't get tripped on if you implement these. Call them "Supported"?. "Official Reference Schema"? If you fix schema then you change wire format. Can't change them that freely anymore once balloted. Currently schema can be changed at will, as long as we have same wire format. How much a price are we willing to pay? Need to stay just short of being normative. Don't want ballot comments on individual schema nor schema construct principles. Call them "Conformant?"

Revision as of 22:54, 14 November 2006

During the discussion of the proposed UML ITS it was suggested that in order to ease the implementation burden a set of normative schema/schematron could be created as part of (a new release of) the XML ITS. The advantages/disadvantages of this approaches have yet to be discussed.

Summary

This Summary was prepared from the discussions below and on the INM telecon by Grahame Grieve

  • We acknowledge that the current situation is frustrating for the implementers as there is no final statement of what is the acceptable wire format, and it is hard to determine what is actually correct.
  • we believe that in spite of it's shortcomings, W3C XML schema is the most appropriate form to make this statement. It stands as human and computer usable documentation
  • we do not currently give any formal status to the schemas that we publish. We do not properly document why do not do this.
  • we would like to give a formal status to the current schemas, so that implementors know how much to rely on them, and what not to rely on them for
  • we do not want to make the schemas normative in the sense that each schema is subject to ballot and comments on models can be made against the schemas
  • but we would like to give the schemas a formal status and appropriate (and documented) procedures to provide assurance that they have real meaning.
  • we are still searching for the appropriate term to give implementation artifacts derived from normative artifacts. (some discussion below)
  • we also acknowledge that schema can never provide complete validation, and that we need to clearly document the limitations of this

But there is two real problems with giving the schemas a more formal basis:

  • the set of normative instances described by the XML ITS (often called "the wire format") is not properly able to be described by schema. This is discussed further below. This is the big problem in giving the schemas a more formal basis (whatever we call it)
  • we also know that there is various differences in the schema validation tools

The UML ITS may be the the only real way to solve the problem. The UML ITS has been designed from scratch to deliver schemas that describe the wire format, and to deliver schemas that are interpreted as uniformly as possible.

This may become a solution for this problem for the XML ITS if the UML ITS becomes R2 of the XML ITS. At the present time, in some configurations the UML ITS is very similar to the UML ITS, and this is still something to consider.

Naming Problem

We do not know what name to give to our desired status for the schemas. Some possibilities discussed:

  • conformant
  • official
  • normative
  • published

An alternative approach is the leave them as normative and just clearly document their status and how we manage them.

Action: Contact HQ for whether ANSI has some appropriate terms. (done, awaiting comment)

Differences between the XML ITS wire format and the schemas

Schema do not properly describe the wire format that the XML ITS describes. In some areas, instances that are conformant will fail schema validation and in others, instances that are not conformant will pass schema validation.

Obviously, there is many things that the schema do not validate. Essentially, the schema validate the XML and the format of the data itself. The schema do not - and never will - validate the semantic correctness of the instance. But in the things the schema does, we need to make the schema do them correctly before we can give the schema an enhanced status.

Co-occurance constraints: The schema do not know the co-occurance constraints. This could be fixed in the XML ITS.

xsi attributes: Schema based processing engines - which we would need to make work - this is a big part of making the schemas normative - will consume and produce xsi: attributes such as type, nil, schemalocation etc. For the moment:

  • we have banned schemaLocation
  • we make no comment on xsi:nil
  • we make very confusing comment on xsi:type

We appear to reuse the xsi:type mechanism for the HL7 type representation. This is not clearly documented, but it is highly implied. The biggest problem with using schema is that the behaviour of xsi:type follows the schema type rules, and we have been unable to match the behaviour of the schema type rules and the HL7 type rules, particularly in the datatypes area.

todo: complete this section


Discusion

  • I guess the major question is the meaning of the ITS being normative. If it means instances must conform to the schema to be conformant, that's fine. (Though I believe there may be one or two places where the schema may still complain about instances that are technically conformant.) On the other hand, if it means that conforming with the schema is sufficient, that would be a problem. It would also be a problem if it actually required implementers to use the schemas in any way (either in the construction or validation of instances).
  • The question that I have is if we make the ITS schemas normative, why stop there? Why should those schemas be normative while others are not?
    • The issue is that implementers currently have nothing to implement against. We describe what XML instances are valid (that's what the XML ITS does) and give them schema. But we (as HL7) give no guarantees that those schema are correct. From an implementer's standpoint that's no good.
    • An implementer may use any schema they want - but having normative schema would allow them to test their instances against those normative schema. So this is not about the structure of the normative schema, but the ability to use them as a "benchmark" in testing and development.
    • The concept of a normative implementable model came up during discussions of the new UML ITS, see UML ITS and UML ITS Technical. It is a key element of that ITS. Therefore it nees to be examined if the concept can be retrofitted into the XML ITS.
    • While the idea of a Reference schema is good, the idea of a Normative one is not so good. It prevents people from coming up with a simpler to understand variant of the machine generated schema and it prevents people for extending or restricting the schema for specific purposes.
  • (png) If we agree that "Normative" is not the word to use, why not call them "Ready to Use"? Or is that too simple and to the point :-) What do other SDOs that create schemas call theirs?

Discussion during the 2006113 INM TelCon

  • Doug: support making them normative, irritating that there is no support. Tony agrees.
  • Grahame: within UML ITS. Implementers should have norm artefacts. Trouble is that schema doesn't describe same wire format as XML ITS. XML ITS is not written for schema. Can't make schema normative, then turn around and state schema are not correct. Schema can't be enough, but have to be correct. Lots of stuf out of schema specification like xsi:nill. In UML ITS describe actually what goes on the wire. Again not all you need to know. We design wire-format differently in order to make schema normative. Interetsed in whether "normative" is correct word. Normative means schema are subject to ballot. We don't want that. What we want is for implementers to be able to "hang their hat on" some schema set.
  • Doug: is there analysis of what schema can't describe? Grahame: not written up. Doug: schema not ridgid enough? Grahame: having sche carries implications for wire format. Sch describe wire format. Cant ahve areas where areas are wrong. Can have areas where they are incomplete. reasonable for implementers to say that instance validates against schema. Schema type handing system is different in concept to v3 typing system. we dont describe differences. If you use schema based toolkit you trip over differences like xsi:nil.
  • Skirmantis: schema should be designed for extensability. Current schema dont allow for diff namespaces. Grahame: need to do in UML ITS. Skirmantas: Schema should allow attributes from other namespaces. Grahame: can extend attributes, can't extend datatypes. Schema dont have to describe them. Skirmantas: if schema are official/normative, should have extenibility build in.
  • What does it mean to make it normative. Where should it be used? Grahame: can use in implementation, dont have to. Describes wire format. Need to be clear about it and avoid interpretation that any instance that validates against schema is a valid v3 instance.
  • "Normative" is the wrong word. Call them "Official"? "HL7-Sanctioned"? Stand behind them. Won't get tripped on if you implement these. Call them "Supported"?. "Official Reference Schema"? If you fix schema then you change wire format. Can't change them that freely anymore once balloted. Currently schema can be changed at will, as long as we have same wire format. How much a price are we willing to pay? Need to stay just short of being normative. Don't want ballot comments on individual schema nor schema construct principles. Call them "Conformant?"