This wiki has undergone a migration to Confluence found Here
<meta name="googlebot" content="noindex">

Difference between revisions of "Proposed new FHIR IG build Process"

From HL7Wiki
Jump to navigation Jump to search
Line 3: Line 3:
  
 
= Summary =
 
= Summary =
 +
 +
Building an IG is a 3 part process:
 +
* Defining the resources that underpin the Implementation Guide resources (Implementation Guide, Conformance Resources, Knowledge statements, examples)
 +
* Building the publishing process that will publish it (linked to where it will be published)
 +
* Developing the structure and the narrative content that turns the resources into a useful implementation guide
 +
 +
The FHIR team provide a IG Publishing tool that takes the implementation guide resources and converts them to a set of 3 different types of files:
 +
* generated resources ready for inclusion into the published guide (xml, json, ttl formats)
 +
* a set of fragments ready to include in generated html files
 +
* 4 standard zip files: definitions.zip, xml.zip, json.zip, and ttl.zip - these are used by implementers for various purposes, so should be included in the final published version
 +
 +
= Using the IG Publisher =
  
 
The publishing process is a 2 step process:
 
The publishing process is a 2 step process:
 
# run the FHIR IG Publisher to validate all the resources & generate fragments
 
# run the FHIR IG Publisher to validate all the resources & generate fragments
 
# run a web site build tool to generate the final IG
 
# run a web site build tool to generate the final IG
 +
 +
Supported tools for the second part of the process:
 +
* Jekyll
 +
 +
To add tools to this list, discuss your tool of choice with Grahame Grieve
  
 
= FHIR IG Publisher =
 
= FHIR IG Publisher =
  
This is a java jar that can be run from the command line, or hosted in a process. Whether hosted in a process, or run from the command line, it works the following way:
+
This is a java jar called org.hl7.fhir.igpublisher.jar. You can get it from
 +
the downloads for the version of FHIR you are using, or - if you build locally - from
 +
your own publish directory. The jar includes everything from the spec
 +
that is required to generate the implementation guide. Make sure you use the
 +
correct version of the IG publisher for your guide.
  
* it takes 4 parameters:
+
The jar can be run from the command line, or hosted in a process. This documentation
** the name of the control file to load (see below for documentation) (-ig)
+
concerns the command line version. If you're hosting it, look at the class org.hl7.fhir.igtools.Publisher for guidance.
** URL of terminology server to use (-tx - defaults to http://fhir3.healthintersections.com.au)
+
 
** path to the version of the specification to generate from (the validation zip has to be from be from this version) (-spec)
+
Run the IG Publisher like this:
** folder to output to (-out)
+
 
 +
  java -jar org.hl7.fhir.igpublisher.jar -ig [source] -tool [tool]
 +
    -out [folder] -spec [path] -tx [url] -watch");
 +
 
 +
parameters:
 +
* -ig: a path or a url where the implementation guide control file is found. see below for Documentation of that format
 +
* -tool: The tool that will be used for the second step. Choices, see above. Implications, see below
 +
* -out: a local folder where the output from the IG publisher will be generated
 +
* -spec: the location of the FHIR specification relative to the guide (can be an absolute URL, or relative if the guide will be published on the FHIR site)
 +
* -tx: (optional) Address to use for terminology server (default is http://fhir3.healthintersections.com.au)
 +
* -watch (optional): if this is present, the publisher will not terminate; instead, it will stay running, an watch for changes to the IG or its contents and re-run when it sees changes
  
 
once running, it:
 
once running, it:
Line 23: Line 54:
 
* reads the IG, and loads all the resources in the IG
 
* reads the IG, and loads all the resources in the IG
 
* processes code systems, value sets, structure definitions, structure maps
 
* processes code systems, value sets, structure definitions, structure maps
* validates the all resources in the IG
+
* validates the all resources in the IG and produces an HTML file with any errors encountered
** produce an HTML file with any errors encountered
 
 
* for each resource in the IG, generate a set of files - renderings of the artifact for possible inclusion in the published IG
 
* for each resource in the IG, generate a set of files - renderings of the artifact for possible inclusion in the published IG
 
On the command line, there is a 5th parameter (-reside), instructing the validator to stay loaded, and rerun the process anytime any of the source files are changed
 
When run in process, the IG Publisher can be passed a resolver that is used to find content that is required when encountered (this is an alternative to specifying folders for the dependent implementation guides)
 
 
= IG Publisher Dependencies =
 
 
The IG publisher makes use of the following resources:
 
 
* NamingSystem
 
* CodeSystem
 
* ValueSet
 
* DataElement
 
* StructureDefinition
 
* ConceptMap
 
* StructureMap
 
 
When any of these resources change, the IG Publisher must be updated. for other resources the IG Publisher uses whatever definitions of them are in use in validation pack that it is passed
 
  
 
= Control file =
 
= Control file =
  
When the IG publisher is executed, it is pointed at a json file that contains publishing information about the Implementation Guide. Customarily, this is a json file in the same directory as the implementation guide resource, with the same [name].json. with this structure:
+
When the IG publisher is executed, it is pointed at a json file that contains publishing information about the Implementation Guide.  
 +
This is a json file in the same directory as the implementation guide resource, with the same [name].json, with this structure:
  
 
   {
 
   {
    // this must be in the same directory
+
     "source": "daf.xml",
     "source" : "payerextract.xml",
+
     "dependencies": {
      
+
      "_documentation": "not done yet"
     // the common base canonical URL for all conformance artifacts defined by the IG
+
     },
     // the IG will ensure that all URLs are [canonicalBase]/[type]/id
+
     "canonicalBase": "http://hl7.org/fhir",
     // and generate redirects accordingly
+
     "defaults": {
    "canonicalBase" : "...",  
+
      "_documentation": "this object contains the default publishing policy for different types. Anything not mentioned defaults to true",
   
+
      "Any": {
    // yet to be done / documented
+
         "_documentation": "fragment options - see below
    "dependencies" [
+
      },
          
+
      "[Type]": {
    ],
+
        "_documentation": "fragment options - see below
   
+
      }
    // a list of paths for each resource in the IG. This tells the
+
     },
    // IG publisher where links to a logical item should actually
+
     "resources": {
     // point to for cross-references within the IG
+
      "[Type]/[id]": {
     // this is an array of object
+
        "source" : "[optional source file]",
    "paths" : [
+
"base": "[destination page for things referring to this resource",
      {
+
"defns": "[destination page, for structure definitions]",
        "url" : "...", // either a simple [id], [type]/[id] or the full canonical URL
+
        "_documentation": "fragment options - see below
        "base" : "[filename]", // the file name for which references should point in final IG
+
      }
        "defns" : "[filename]", // the name of the file that will contain the formal definitions (structure definitions only)
+
     }
      }
 
     ]
 
 
   }
 
   }
  
This file must be maintained by the editor of the implementation guide
+
This file must be maintained by the editor of the implementation guide. The IG resource must use sourceReference references by id. The
 +
ids are resolved to a local file by this control file. if there is no "source" property, the files will be assumed to be in the same
 +
directory as the ig, with the name [type]-[id].xml/json, or just [id].xml/json
  
= Output =
+
== Fragment Options ==
  
The output consists of a set of files in /output:
+
When deciding whether to produce a particular kind of fragment, the IG Publisher will
* validation.html - the result of validating the resources
+
look for a property of type boolean with the name given below. It will look in the following
* instructions.txt - how to use the generated fragments in the second part of the build (including what must be placed in header of the html files that use the generated fragments (.js and .css includes)
+
places, in order:
  
In /output/content - anything in here goes in the final generated IG
+
* on the resource entry for the resource in question
* images and stylesheets - css file for all the generated content (copy from spec) + supporting images etc
+
* on the defaults entry for the resource type in question
* generated resources after post-processing, ready for distribution, in xml, json, turtle
+
* on the defaults entry for "Any"
* generated schema files (.shex, .schema.json, .sch)
 
* IG validation packs (validation.xml.zip and validation.json.zip) for the FHIR validator
 
* redirection files
 
  
In /output/_includes
+
if it doesn't find anything, it will produce the fragment.  
* variables.json : an array of "name" : "value" pairs that provide useful information about fhir (e.g. version) so they can be built into the generated pages
 
* different generated views of the resources
 
** each file has name [id]-[type].html where [id] is the id of the resource, and [type] is the type of generated html fragment. Todo: enumerate the types
 
  
In /output/_data
+
== Fragment Codes ==
* fhir.json: a set of data related to the build process for inclusion into an IG using Jekyll
 
  
in /output/pages
+
* xml: XML version of the resource (all resource types)
* a full html page, one for each generated fragment in /output/fragments
+
* json: JSON version of the resource (all resource types)
** these are used  for quality checking if there's problems with the fragment styling downstream
+
* ttl: Turtle version of the resource (all resource types)
 +
* xml-html: html representation of XML version of the resource (all resource types)
 +
* json-html: html representation of JSON version of the resource (all resource types)
 +
* ttl-html: html representation of Turtle version of the resource (all resource types)
 +
* html: narrative of resource as html
 +
* summary: An html summary for the resource (all conformance resources)
 +
* content: An HTML representation of the content in resource (code system, concept map, structure map)
 +
* xref: A list of all the places where the resource is used (all conformance resources)
 +
* cld: An HTML representation of the content in resource (value set)
 +
* expansion: The expansion of the value set (Value set)
 +
* shex: ShEx statement for the structure (Structure Definition)
 +
* sch: schematron statement for the structure (Structure Definition)
 +
* json-schema: JSON Schema statement for the structure (Structure Definition)
 +
* header: Description of the identification of the structure (Structure Definition)
 +
* diff: Logical Table of the diff (Structure Definition)
 +
* snapshot: Logical Table of the snapshot (Structure Definition)
 +
* template-xml: XML template for the snapshot (Structure Definition)
 +
* template-json: JSON template for the snapshot (Structure Definition)
 +
* template-ttl: Turtle template for the snapshot (Structure Definition)
 +
* uml: UML diagram for the structure (Structure Definition)
 +
* tx: Terminology Notes for the structure (Structure Definition)
 +
* inv: invariant summary for the structure (Structure Definition)
 +
* dict: Detailed Element Definitions (Structure Definition)
 +
* maps: Presentation of the mappings (Structure Definition)
  
== Output Fragments ==
+
= Build =
 
 
These fragments are produced for each resource:
 
* [id]-html.xhtml: the narrative of the resource, as html
 
* [id]-gen-html.xhtml: the generated narrative of the resource, as html
 
* [id]-xml-html.xhtml: the xhtml rendering of the xml format of the resource, with hyperlinks
 
* [id]-json-html.xhtml: the xhtml rendering of the json format of the resource, with hyperlinks
 
* [id]-ttl-html.xhtml: the xhtml rendering of the turtle format of the resource, with hyperlinks
 
 
 
For specific types of resource:
 
* CodeSystem
 
** [id]-cs-summary: narrative summary of the resource
 
** [id]-cs-content: presentation of the content of the code system
 
** [id]-cs-xref: list of all value sets where the code system is used
 
* ValueSet:
 
** [id]-vs-summary: narrative summary of the resource
 
** [id]-vs-cld: presentation of the content of the code system
 
** [id]-vs-xref: list of all places where the value set is used
 
** [id]-expansion: HTML representation of the expansion of the value set (or an error)
 
* ConceptMap:
 
** [id]-cm-summary: narrative summary of the resource
 
** [id]-cm-content: presentation of the content of the code system
 
** [id]-cm-xref: list of all places where the concept map is used
 
* StructureDefinition:
 
** [id]-sd-summary: narrative summary of the resource
 
** [id]-shex: shex as html
 
** [id]-sch: schematron as html
 
** [id]-json-schema: json schema as html
 
** [id]-diff: Logical table of Differential
 
** [id]-snapshot: Logical table of Snapshot
 
** [id]-xml: Xml template
 
** [id]-jso: Json template
 
** [id]-ttl: Turtle template
 
** [id]-uml: UML as SVG
 
** [id]-tx: Terminology Binding Summary
 
** [id]-inv: invariant summary
 
** [id]-dict: Formal definitions
 
** [id]-maps: mappings
 
** [id]-sd-xref: list of all places where the structure is used
 
  
= Build =
+
== Jekyll ==
  
The build program (todo: investigate and choose the appropriate tool) builds from a series of html or markdown pages, and may be integrated as part of a server (e.g. GitHub). The html source files and build tool injects the files that the IG publisher generated into the final output.
+
If you specify Jekyll as the tool, the the following outputs are created
  
todo: fill out the details of this
+
X (X is the nominated output directory) - untouched
 +
X\generation - the validation file, and the all the generated qa pages (one for each fragment, so you can see what the fragments are supposed to look like)
 +
X\html - the content for the final IG. This is the Jekyll root directory
 +
X\html\_includes - all the fragments, so you can {%include%} them
 +
X\html\_data - a data file fhir.json with additional properties so you can access them when templating

Revision as of 11:43, 2 June 2016

This is some notes about of the proposed new FHIR build process for publishing implementation guides

Summary

Building an IG is a 3 part process:

  • Defining the resources that underpin the Implementation Guide resources (Implementation Guide, Conformance Resources, Knowledge statements, examples)
  • Building the publishing process that will publish it (linked to where it will be published)
  • Developing the structure and the narrative content that turns the resources into a useful implementation guide

The FHIR team provide a IG Publishing tool that takes the implementation guide resources and converts them to a set of 3 different types of files:

  • generated resources ready for inclusion into the published guide (xml, json, ttl formats)
  • a set of fragments ready to include in generated html files
  • 4 standard zip files: definitions.zip, xml.zip, json.zip, and ttl.zip - these are used by implementers for various purposes, so should be included in the final published version

Using the IG Publisher

The publishing process is a 2 step process:

  1. run the FHIR IG Publisher to validate all the resources & generate fragments
  2. run a web site build tool to generate the final IG

Supported tools for the second part of the process:

  • Jekyll

To add tools to this list, discuss your tool of choice with Grahame Grieve

FHIR IG Publisher

This is a java jar called org.hl7.fhir.igpublisher.jar. You can get it from the downloads for the version of FHIR you are using, or - if you build locally - from your own publish directory. The jar includes everything from the spec that is required to generate the implementation guide. Make sure you use the correct version of the IG publisher for your guide.

The jar can be run from the command line, or hosted in a process. This documentation concerns the command line version. If you're hosting it, look at the class org.hl7.fhir.igtools.Publisher for guidance.

Run the IG Publisher like this:

 java -jar org.hl7.fhir.igpublisher.jar -ig [source] -tool [tool]
    -out [folder] -spec [path] -tx [url] -watch");

parameters:

  • -ig: a path or a url where the implementation guide control file is found. see below for Documentation of that format
  • -tool: The tool that will be used for the second step. Choices, see above. Implications, see below
  • -out: a local folder where the output from the IG publisher will be generated
  • -spec: the location of the FHIR specification relative to the guide (can be an absolute URL, or relative if the guide will be published on the FHIR site)
  • -tx: (optional) Address to use for terminology server (default is http://fhir3.healthintersections.com.au)
  • -watch (optional): if this is present, the publisher will not terminate; instead, it will stay running, an watch for changes to the IG or its contents and re-run when it sees changes

once running, it:

  • loads all resources in the validation.zip
  • reads the control file
  • reads the IG, and loads all the resources in the IG
  • processes code systems, value sets, structure definitions, structure maps
  • validates the all resources in the IG and produces an HTML file with any errors encountered
  • for each resource in the IG, generate a set of files - renderings of the artifact for possible inclusion in the published IG

Control file

When the IG publisher is executed, it is pointed at a json file that contains publishing information about the Implementation Guide. This is a json file in the same directory as the implementation guide resource, with the same [name].json, with this structure:

 {
   "source": "daf.xml",
   "dependencies": {
     "_documentation": "not done yet"
   },
   "canonicalBase": "http://hl7.org/fhir",
   "defaults": {
     "_documentation": "this object contains the default publishing policy for different types. Anything not mentioned defaults to true",
     "Any": {
       "_documentation": "fragment options - see below
     },
     "[Type]": {
       "_documentation": "fragment options - see below
     }
   },
   "resources": {
     "[Type]/[id]": {
       "source" : "[optional source file]",

"base": "[destination page for things referring to this resource", "defns": "[destination page, for structure definitions]",

       "_documentation": "fragment options - see below
     }
   }
 }

This file must be maintained by the editor of the implementation guide. The IG resource must use sourceReference references by id. The ids are resolved to a local file by this control file. if there is no "source" property, the files will be assumed to be in the same directory as the ig, with the name [type]-[id].xml/json, or just [id].xml/json

Fragment Options

When deciding whether to produce a particular kind of fragment, the IG Publisher will look for a property of type boolean with the name given below. It will look in the following places, in order:

  • on the resource entry for the resource in question
  • on the defaults entry for the resource type in question
  • on the defaults entry for "Any"

if it doesn't find anything, it will produce the fragment.

Fragment Codes

  • xml: XML version of the resource (all resource types)
  • json: JSON version of the resource (all resource types)
  • ttl: Turtle version of the resource (all resource types)
  • xml-html: html representation of XML version of the resource (all resource types)
  • json-html: html representation of JSON version of the resource (all resource types)
  • ttl-html: html representation of Turtle version of the resource (all resource types)
  • html: narrative of resource as html
  • summary: An html summary for the resource (all conformance resources)
  • content: An HTML representation of the content in resource (code system, concept map, structure map)
  • xref: A list of all the places where the resource is used (all conformance resources)
  • cld: An HTML representation of the content in resource (value set)
  • expansion: The expansion of the value set (Value set)
  • shex: ShEx statement for the structure (Structure Definition)
  • sch: schematron statement for the structure (Structure Definition)
  • json-schema: JSON Schema statement for the structure (Structure Definition)
  • header: Description of the identification of the structure (Structure Definition)
  • diff: Logical Table of the diff (Structure Definition)
  • snapshot: Logical Table of the snapshot (Structure Definition)
  • template-xml: XML template for the snapshot (Structure Definition)
  • template-json: JSON template for the snapshot (Structure Definition)
  • template-ttl: Turtle template for the snapshot (Structure Definition)
  • uml: UML diagram for the structure (Structure Definition)
  • tx: Terminology Notes for the structure (Structure Definition)
  • inv: invariant summary for the structure (Structure Definition)
  • dict: Detailed Element Definitions (Structure Definition)
  • maps: Presentation of the mappings (Structure Definition)

Build

Jekyll

If you specify Jekyll as the tool, the the following outputs are created

X (X is the nominated output directory) - untouched X\generation - the validation file, and the all the generated qa pages (one for each fragment, so you can see what the fragments are supposed to look like) X\html - the content for the final IG. This is the Jekyll root directory X\html\_includes - all the fragments, so you can {%include%} them X\html\_data - a data file fhir.json with additional properties so you can access them when templating