Difference between revisions of "FHIR QA Guidelines"
(→Manual) |
|||
(7 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | {{FHIR Discussion Page}} | + | Content on this page has been migrated to https://confluence.hl7.org/display/FMG/FHIR+QA+Guidelines<nowiki/>{{FHIR Discussion Page}} |
[[Category:Active FHIR Discussion]] | [[Category:Active FHIR Discussion]] | ||
Line 8: | Line 8: | ||
=Precepts= | =Precepts= | ||
− | * Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis | + | |
− | ** Bold content below are those QA steps that validate this | + | *Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis |
− | * Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels) | + | **Bold content below are those QA steps that validate this |
+ | *Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels) | ||
=QA Steps= | =QA Steps= | ||
==Automated== | ==Automated== | ||
− | + | The following are a subset of the checks currently handled by the build process: | |
− | + | *All XML and JSON examples are valid according to the validator | |
− | * | + | *Fragments labelled by a type are parsed by the parser without errors |
− | * | + | *All resource definitions and profiles are valid against their schemas & schematrons + additional rules |
− | * | + | *All links resolve in the HTML |
− | * | + | *All coded datatypes have bindings - currently raised as a warning |
− | * | + | *Fixed values only exist for simple types - raised as a warning |
− | * | + | *All FHIRPath constraints are valid |
− | * All | + | *Coding.system values if from hl7.org/fhir are valid |
− | * | + | *At least one example must have a value for each search parameter |
− | |||
− | |||
− | * | ||
==Automatable== | ==Automatable== | ||
These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process | These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process | ||
+ | |||
*Ensure all RIM mappings are "legal" | *Ensure all RIM mappings are "legal" | ||
*Ensure RIM mappings don't collide/overlap | *Ensure RIM mappings don't collide/overlap | ||
*Ensure examples cover all data elements | *Ensure examples cover all data elements | ||
+ | *Definitions, etc. only end with periods when they ought to | ||
==Manual== | ==Manual== | ||
(Some of these can be focused only on those resources & sections that have changed from prior release) | (Some of these can be focused only on those resources & sections that have changed from prior release) | ||
− | * Ensure examples demonstrate an appropriate breadth of use of the resource | + | |
− | * Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes) | + | *Ensure examples demonstrate an appropriate breadth of use of the resource |
− | ** Ensure all content that's supposed to make it into the book form actually does | + | *Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes) |
− | ** Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review | + | **Ensure all content that's supposed to make it into the book form actually does |
− | * Content validation | + | **Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review |
− | ** (goal) Ensure the build runs successfully with no warnings | + | *Content validation |
− | ** Test that the xpath assertions for Schematrons are valid using Saxon SA | + | **(goal) Ensure the build runs successfully with no warnings |
− | ** Ensure a wiki page with the default content exists for each page ('''which wiki page?''') | + | **Test that the xpath assertions for Schematrons are valid using Saxon SA |
− | ** Formal process | + | **Ensure a wiki page with the default content exists for each page ('''which wiki page?''') |
− | *** Do we have a PSS and resource request in place for all resources? | + | **Formal process |
− | *** (goal) Do we have mappings for the "source" specifications used to determine/validate 80%? | + | ***Do we have a PSS and resource request in place for all resources? |
− | ** Technical review | + | ***(goal) Do we have mappings for the "source" specifications used to determine/validate 80%? |
− | *** Place both forms into MS Word and run grammar & spelling checks (U.S. English) | + | **Technical review |
− | *** Ensure style guide is followed for use of formatting | + | ***Place both forms into MS Word and run grammar & spelling checks (U.S. English) ('''what is meant by "both forms?"''') |
− | **** <b>To be defined. Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections</b> | + | ***Ensure style guide is followed for use of formatting |
− | **** <b>Can we | + | ****<b>To be defined. Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections</b> |
− | ** Text content review | + | ****<b>Can we re-use from w3c or someone?</b> |
− | *** Ensure all definitions for code sets are mutually exclusive (and comprehensive) | + | **Text content review |
− | *** Ensure statuses on resources & profiles are accurate for ballot | + | ***Ensure all definitions for code sets are mutually exclusive (and comprehensive) |
− | *** Ensure definitions are non-tautological and clear | + | ***Ensure statuses on resources & profiles are accurate for ballot |
− | *** Ensure definition, rationale & notes are properly split | + | ***Ensure definitions are non-tautological and clear |
− | *** Ensure definitions include examples when appropriate | + | ***Ensure definition, rationale & notes are properly split |
− | *** Ensure text is clear and reads well, with references to other topics when appropriate | + | ***Ensure definitions include examples when appropriate |
− | ** Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability | + | ***Ensure text is clear and reads well, with references to other topics when appropriate |
− | *** Check where Conformance = Required, minimum cardinality = 1 | + | **Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability |
− | ** Are mappings valid against source specification | + | ***Check where Conformance = Required, minimum cardinality = 1 |
+ | **Are mappings valid against source specification | ||
**Is content "complete" | **Is content "complete" | ||
***Are there any known issues declared that would prevent implementers from using the spec "as is"? | ***Are there any known issues declared that would prevent implementers from using the spec "as is"? | ||
**Are there any dependencies on content that doesn't exist? (DSTU resources must not have dependencies on content that is not also DSTU) | **Are there any dependencies on content that doesn't exist? (DSTU resources must not have dependencies on content that is not also DSTU) |
Latest revision as of 15:47, 1 May 2020
Content on this page has been migrated to https://confluence.hl7.org/display/FMG/FHIR+QA+Guidelines
This page identifies the guidelines used as part of the QA process. Not all of these will necessarily be evaluated as part of the QA process. As well, we need to define who is responsible for verifying these. Ideally, most QA would be required to be performed by authoring committees with a smaller set of criteria as well as spot-checks of other criteria performed by FMG. We may also want to differentiate what gets done for different levels of ballot. As well, QA can be focused on areas that have changed.
Precepts
- Content should only be included in balloted DSTU which we believe is at least ready to be implemented on a trial basis
- Bold content below are those QA steps that validate this
- Content that is not ready for DSTU may be included as draft as noted by FMM level? (needs clarification, especially with regard to the FMM levels)
QA Steps
Automated
The following are a subset of the checks currently handled by the build process:
- All XML and JSON examples are valid according to the validator
- Fragments labelled by a type are parsed by the parser without errors
- All resource definitions and profiles are valid against their schemas & schematrons + additional rules
- All links resolve in the HTML
- All coded datatypes have bindings - currently raised as a warning
- Fixed values only exist for simple types - raised as a warning
- All FHIRPath constraints are valid
- Coding.system values if from hl7.org/fhir are valid
- At least one example must have a value for each search parameter
Automatable
These are processes that must be reviewed manually at the moment bug could be handled in an automated fashion with appropriate enhancements to the build process
- Ensure all RIM mappings are "legal"
- Ensure RIM mappings don't collide/overlap
- Ensure examples cover all data elements
- Definitions, etc. only end with periods when they ought to
Manual
(Some of these can be focused only on those resources & sections that have changed from prior release)
- Ensure examples demonstrate an appropriate breadth of use of the resource
- Tooling validation (validates that the build tooling is working correctly - only needs checking when build process changes)
- Ensure all content that's supposed to make it into the book form actually does
- Ensure all content from the website that doesn't appear in the book form appears in a secondary form for review
- Content validation
- (goal) Ensure the build runs successfully with no warnings
- Test that the xpath assertions for Schematrons are valid using Saxon SA
- Ensure a wiki page with the default content exists for each page (which wiki page?)
- Formal process
- Do we have a PSS and resource request in place for all resources?
- (goal) Do we have mappings for the "source" specifications used to determine/validate 80%?
- Technical review
- Place both forms into MS Word and run grammar & spelling checks (U.S. English) (what is meant by "both forms?")
- Ensure style guide is followed for use of formatting
- To be defined. Includes: when to use bold, italics, capitalization, hyperlinks, color, ordered vs. unordered lists, sections
- Can we re-use from w3c or someone?
- Text content review
- Ensure all definitions for code sets are mutually exclusive (and comprehensive)
- Ensure statuses on resources & profiles are accurate for ballot
- Ensure definitions are non-tautological and clear
- Ensure definition, rationale & notes are properly split
- Ensure definitions include examples when appropriate
- Ensure text is clear and reads well, with references to other topics when appropriate
- Ensure that constraints (cardinality, vocabulary, invariants, etc.) do not constrain extensibility more than necessary to allow safe base interoperability
- Check where Conformance = Required, minimum cardinality = 1
- Are mappings valid against source specification
- Is content "complete"
- Are there any known issues declared that would prevent implementers from using the spec "as is"?
- Are there any dependencies on content that doesn't exist? (DSTU resources must not have dependencies on content that is not also DSTU)