This wiki has undergone a migration to Confluence found Here
<meta name="googlebot" content="noindex">

Difference between revisions of "FHIR QA Guidelines"

From HL7Wiki
Jump to navigation Jump to search
 
(6 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{FHIR Discussion Page}}
+
Content on this page has been migrated to https://confluence.hl7.org/display/FMG/FHIR+QA+Guidelines<nowiki/>{{FHIR Discussion Page}}
 
[[Category:Active FHIR Discussion]]
 
[[Category:Active FHIR Discussion]]
  
Line 8: Line 8:
  
 
=Precepts=
 
=Precepts=
* Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis
+
 
** Bold content below are those QA steps that validate this
+
*Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis
* Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels)
+
**Bold content below are those QA steps that validate this
 +
*Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels)
  
 
=QA Steps=
 
=QA Steps=
  
 
==Automated==
 
==Automated==
These processes are currently handled by the build process (though someone needs to view the build log and ensure it is clean)
+
The following are a subset of the checks currently handled by the build process:
  
Need to confirm that the build process actually does all of these
+
*All XML and JSON examples are valid according to the validator
* '''All XML examples & fragments are schema-valid and schematron valid'''
+
*Fragments labelled by a type are parsed by the parser without errors
* '''All JSON is valid''' - <!--need to figure out how this works-->
+
*All resource definitions and profiles are valid against their schemas & schematrons + additional rules
* '''All resource definitions and profiles are valid against their schemas & schematrons + additional rules'''
+
*All links resolve in the HTML
* '''All links resolve in the HTML'''
+
*All coded datatypes have bindings - currently raised as a warning
* '''All coded datatypes have bindings'''
+
*Fixed values only exist for simple types - raised as a warning
* '''Fixed values only exist for simple types'''
+
*All FHIRPath constraints are valid
* All OCL constraints compile
+
*Coding.system values if from hl7.org/fhir are valid
* Definitions, etc. only end with periods when they ought to
+
*At least one example must have a value for each search parameter
* '''UML views of everything (including data types) agrees with definitions'''
 
* '''sid (system identifier) values are legal. All non-sid/guid/oid ids are flagged as warnings unless part of a pre-defined example space'''
 
* '''Examples cover all query parameters'''
 
  
 
==Automatable==
 
==Automatable==
 
These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process
 
These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process
 +
 
*Ensure all RIM mappings are "legal"
 
*Ensure all RIM mappings are "legal"
 
*Ensure RIM mappings don't collide/overlap
 
*Ensure RIM mappings don't collide/overlap
 
*Ensure examples cover all data elements
 
*Ensure examples cover all data elements
 +
*Definitions, etc. only end with periods when they ought to
  
 
==Manual==
 
==Manual==
 
(Some of these can be focused only on those resources & sections that have changed from prior release)
 
(Some of these can be focused only on those resources & sections that have changed from prior release)
* Ensure examples demonstrate an appropriate breadth of use of the resource
+
 
* Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes)  
+
*Ensure examples demonstrate an appropriate breadth of use of the resource
** Ensure all content that's supposed to make it into the book form actually does
+
*Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes)  
** Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review
+
**Ensure all content that's supposed to make it into the book form actually does
* Content validation
+
**Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review
** (goal) Ensure the build runs successfully with no warnings
+
*Content validation
** Test that the xpath assertions for Schematrons are valid using Saxon SA
+
**(goal) Ensure the build runs successfully with no warnings
** Ensure a wiki page with the default content exists for each page ('''which wiki page?''')
+
**Test that the xpath assertions for Schematrons are valid using Saxon SA
** Formal process
+
**Ensure a wiki page with the default content exists for each page ('''which wiki page?''')
*** Do we have a PSS and resource request in place for all resources?
+
**Formal process
*** (goal) Do we have mappings for the "source" specifications used to determine/validate 80%?
+
***Do we have a PSS and resource request in place for all resources?
** Technical review
+
***(goal) Do we have mappings for the "source" specifications used to determine/validate 80%?
*** Place both forms into MS Word and run grammar & spelling checks (U.S. English) ('''what is meant by "both forms?"''')
+
**Technical review
*** Ensure style guide is followed for use of formatting
+
***Place both forms into MS Word and run grammar & spelling checks (U.S. English) ('''what is meant by "both forms?"''')
**** <b>To be defined.  Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections</b>
+
***Ensure style guide is followed for use of formatting
**** <b>Can we steal from w3c or someone?</b>
+
****<b>To be defined.  Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections</b>
** Text content review
+
****<b>Can we re-use from w3c or someone?</b>
*** Ensure all definitions for code sets are mutually exclusive (and comprehensive)
+
**Text content review
*** Ensure statuses on resources & profiles are accurate for ballot
+
***Ensure all definitions for code sets are mutually exclusive (and comprehensive)
*** Ensure definitions are non-tautological and clear
+
***Ensure statuses on resources & profiles are accurate for ballot
*** Ensure definition, rationale & notes are properly split
+
***Ensure definitions are non-tautological and clear
*** Ensure definitions include examples when appropriate
+
***Ensure definition, rationale & notes are properly split
*** Ensure text is clear and reads well, with references to other topics when appropriate
+
***Ensure definitions include examples when appropriate
** Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability
+
***Ensure text is clear and reads well, with references to other topics when appropriate
*** Check where Conformance = Required, minimum cardinality = 1
+
**Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability
** Are mappings valid against source specification
+
***Check where Conformance = Required, minimum cardinality = 1
 +
**Are mappings valid against source specification
 
**Is content "complete"
 
**Is content "complete"
 
***Are there any known issues declared that would prevent implementers from using the spec "as is"?
 
***Are there any known issues declared that would prevent implementers from using the spec "as is"?
 
**Are there any dependencies on content that doesn't exist?  (DSTU resources must not have dependencies on content that is not also DSTU)
 
**Are there any dependencies on content that doesn't exist?  (DSTU resources must not have dependencies on content that is not also DSTU)

Latest revision as of 15:47, 1 May 2020

Content on this page has been migrated to https://confluence.hl7.org/display/FMG/FHIR+QA+Guidelines

Go to DSTU 2 QA guidelines

This page identifies the guidelines used as part of the QA process. Not all of these will necessarily be evaluated as part of the QA process. As well, we need to define who is responsible for verifying these. Ideally, most QA would be required to be performed by authoring committees with a smaller set of criteria as well as spot-checks of other criteria performed by FMG. We may also want to differentiate what gets done for different levels of ballot. As well, QA can be focused on areas that have changed.


Precepts

  • Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis
    • Bold content below are those QA steps that validate this
  • Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels)

QA Steps

Automated

The following are a subset of the checks currently handled by the build process:

  • All XML and JSON examples are valid according to the validator
  • Fragments labelled by a type are parsed by the parser without errors
  • All resource definitions and profiles are valid against their schemas & schematrons + additional rules
  • All links resolve in the HTML
  • All coded datatypes have bindings - currently raised as a warning
  • Fixed values only exist for simple types - raised as a warning
  • All FHIRPath constraints are valid
  • Coding.system values if from hl7.org/fhir are valid
  • At least one example must have a value for each search parameter

Automatable

These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process

  • Ensure all RIM mappings are "legal"
  • Ensure RIM mappings don't collide/overlap
  • Ensure examples cover all data elements
  • Definitions, etc. only end with periods when they ought to

Manual

(Some of these can be focused only on those resources & sections that have changed from prior release)

  • Ensure examples demonstrate an appropriate breadth of use of the resource
  • Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes)
    • Ensure all content that's supposed to make it into the book form actually does
    • Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review
  • Content validation
    • (goal) Ensure the build runs successfully with no warnings
    • Test that the xpath assertions for Schematrons are valid using Saxon SA
    • Ensure a wiki page with the default content exists for each page (which wiki page?)
    • Formal process
      • Do we have a PSS and resource request in place for all resources?
      • (goal) Do we have mappings for the "source" specifications used to determine/validate 80%?
    • Technical review
      • Place both forms into MS Word and run grammar & spelling checks (U.S. English) (what is meant by "both forms?")
      • Ensure style guide is followed for use of formatting
        • To be defined. Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections
        • Can we re-use from w3c or someone?
    • Text content review
      • Ensure all definitions for code sets are mutually exclusive (and comprehensive)
      • Ensure statuses on resources & profiles are accurate for ballot
      • Ensure definitions are non-tautological and clear
      • Ensure definition, rationale & notes are properly split
      • Ensure definitions include examples when appropriate
      • Ensure text is clear and reads well, with references to other topics when appropriate
    • Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability
      • Check where Conformance = Required, minimum cardinality = 1
    • Are mappings valid against source specification
    • Is content "complete"
      • Are there any known issues declared that would prevent implementers from using the spec "as is"?
    • Are there any dependencies on content that doesn't exist? (DSTU resources must not have dependencies on content that is not also DSTU)