This wiki has undergone a migration to Confluence found Here
<meta name="googlebot" content="noindex">

V3 Publishing Process Steps

From HL7Wiki
Revision as of 13:24, 21 July 2010 by Gwbeeler (talk | contribs)
Jump to navigation Jump to search

V3 Publishing Steps - an initial pass as a hierarchical outline:


Primarytargets and documentation on this page

This page documents a number of named ANT "targets" (names like 00.10...REVIEW properties) that invoke specific tasks, and documents. The combined publishing tools include ten "primary" targets, and over 25 "supplemental" targets. These are distinguished as follows:
  • Primary targets are shown on this page in bold face; they correspond to "batch" files of the same name.
  • Supplemental targets are shown on this page indented and in normal font; they correspond to "batch" files named with the prefix "supp-" followed by the target name.

01...Content Submission

This section encompasses those steps undertaken by a Publishing Facilitator (or other person preparing part or all of a publication package) in order to assemble and "submit to publishing" their specific portion of the content. Note that in this context, "submit to publishing" means either to submit to HQ for publishing or to "submit" to oneself for local publishing.

The steps include:

1.1 Create a QA report and a manifest of all of the files needed to "publish" the specification.
1.2 Once the manifest is built, use it to build a submission package and reduce domain files to only those required for publication
1.3 Version the resulting file set in SVN and notify HQ of its availability (if appropriate)
1.4 (By HQ) Download the appropriate files and acknowledge receipt.
1.5 Revert to previous content.

Note: The two ZIP archives for each specification are distinguished by name and an appended date string (like uvct-Unused_20100917). There is also a "re-extraction" process that will allow one to "automatically" re-extract the file content from the current or an earlier date.

01.10...Review Quality and Prepare Manifests

The initial step in the publishing process is to identify those files that are required for publication. For a "domain" the entire required content can be determined from the Publication data base for the domain. For a specification submitted as "free-hand" documentation, the content must be determined by "inspection" of the source files. While automated processing of the "Pub XML" can identify all required content, this capability has not yet been extended to the other publications.

The challenge are the artifacts, image files, model files, etc. that must be collected to publish the document. The automated process tracks the references from the primary document, attempts to determine that the targets of those references exist, and then produces a file manifest that can be used to assemble a ZIP file of the content for submission.

The first step will be to extract the primary document (in the case of a domain it is extracted from the publication database), and analyze this document for its references. Transforms to accomplish this are already part of the desktop publishing process. The manifest from this review can be used in subsequent steps to isolate the submitted content and "hide" the unused content in ZIP archives.

  • 01.10...Review Quality and Prepare Manifests Extracts the content from all PubDbs found in the databases subdirectory of all domains; uses the Generator to build a QA preview in Generator/OutputFiles/Reports, and defines the resulting manifest for all domains in Generator/TemporaryFiles/specificManifests.

01.20...Package Active and Support Content for Submission

Once the manifest is built, it can be used to build a submission package and to reduce the domain files to only those required for publication. Specific steps that are followed include

01.21 ZIP the content for submission
01.22 Delete the "submission content" from the directories
01.23 ZIP the remaining "unused content" into a second ZIP archive
01.24 Delete the "unused content" from the directories (leaving the ZIP behind)
01.25 Extract only the "submission content" to repopulate the directories
Defined Processes (Eclipse Target or BAT file):
  • 01.20...Package Active-Support Content for Submission Processes the domains listed as 'active' and 'support' for submission. For each domain, this includes all five of the steps listed above.
  • 01.29...Extract and Package - Combines 01.10 and 01.20 - Extracts the content from all PubDbs; rebuilds the manifests and then processes the domains listed as 'active' and 'support' for submission. For each domain, this includes all five of the steps listed above.

01.30...Version and Notify

When the submission is packaged, HQ needs to be notified, if this is a submission or a ballot or a Normative Edition.

01.40...Download and Acknowledge

The recipient will extract the submission and assure that it is properly placed in the SVN system for the publication being prepared. The final step, in the case of a submission to HQ, will be the return of an acknowledgment to the submitter.

01.50...Revert to Prior Content

The dual archives for "submission" and "unused" content permit one to roll back the input file structures to what they were before the domain was packaged. This is the process step that supports such reversion.

Defined Processes (Eclipse Target or BAT file):
  • 01.50...Revert Active and Support Content - This process starts with a prompt to enter the date string on which the archive(s) for this domain were created. these strings are part of the archive file name and are formatted like 20100929. Once entered, the process will process all domains on the "active" and "support" lists looking for archives with that date string. If it finds the archives, it will delete all other files from that domain, including sub-directories, and then will extract both the "submission" and "unused" archives, thereby restoring the content to what it had been before packaging.

Jump to top of page

Jump to Publishing Process "Root" Page


Primarytargets and documentation on this page

This page documents a number of named ANT "targets" (names like 00.10...REVIEW properties) that invoke specific tasks, and documents. The combined publishing tools include ten "primary" targets, and over 25 "supplemental" targets. These are distinguished as follows:
  • Primary targets are shown on this page in bold face; they correspond to "batch" files of the same name.
  • Supplemental targets are shown on this page indented and in normal font; they correspond to "batch" files named with the prefix "supp-" followed by the target name.

02...Prepare Source Files For Publication

The various content files repaired by the Work Groups must be preprocessed in order to make them useful in the publication. For example, the domain content must be extracted( as an XML file) from the publication database; the Visio "vsd" files from HL7's RMIM Designer must be processed to create appropriately sized graphics, and a hyper-linked HTML "map" to drive graphic navigation; etc. This section documents the processes used in preparing these files.

02.00...Manual Preparation of Freehand MIF Files

The link above leads to instructions for editors of freehand MIF files, that is, documents such as those in the Help and Infrastructure sections that are hand edited by someone familiar with XML, XHTML and using a program such as XMLSpy. These documents include a "template" that can be downloaded and processed by selecting the miftemplate in the non.domains.general subject list in the defined-environment.properties file. The template provides both an example and details of the implementation of certain MIF mark-up entries.

02.10...PubDb Extract

This is the process that exports "domain xml" from the HL7 Publication data base (PubDb). The process is a part of the RoseTree tools.

Defined Processes (Eclipse Target or BAT file):
  • 02.10...PUBDB Extract - Extracts the content from all PubDbs found in the databases sub-directory of all domains regardless of the date of extraction of any pre-exisitng Pub XML file. IT places the result in generator/InputFiles/DynamicModelFiles. It also copies (and over-writes) the extracted content for the domains in domains.active and domains.support back to the input/domains/uvnn directories.
  • 02.15...PUBDB Update - Extracts the content from all PubDbs found in the databases sub-directory of all domains if the data base has been edited since the prior Pub XML file was extracted. It places the result in generator/InputFiles/DynamicModelFiles. It also copies (and over-writes) the extracted content for the domains in domains.active and domains.support back to the input/domains/uvnn directories.

02.20...Extract Visio Graphics

This process provides for the extraction of "png" files for the static model designs and for "thumb-nail" graphics of same.

Defined Processes (Eclipse Target or BAT file):
  • 02.20...EXTRACT Visio Graphics and Maps Actually combines steps 02.20 and 02.30. It copies the RMIM Designer Visio (vsd) files to output/processable/hmdvsd/vsd directory and the corresponding Visio xml files to generator/InputFiles/VisioModelXmlFiles. Then it processes the vsd files to extract the graphics, thumbnails and an html overlay map. These outputs are stored in output/processable/hmdvsd/Editable and output/processable/hmdvsd/Editable/images.
    Warning: The Visio to PNG conversion process does not compare the time-stamps on its source and output files. Therefore invoking this process a second time will re-generate the whole set of files.
    Also, the Visio will occasionally fail to close, or throw an error at the end of a larget set of conversions. If this happens, compare the last few files in output/processable/hmdvsd/editable with the last few in output/processable/hmdvsd/vsd if the former are from the same designs as the latter, the process completed normally, despite the error.

02.30...HTML Overlay Maps

This process builds an html "map" (overlay) that works with the extracted PNG files to provide clickable-graphics for navigating to details of the static model designs. (This is performed as part of process 02.20...EXTRACT Visio Graphics and Maps, above.

02.40...Position all files

The final processes of this section take the variety of source files received from a Work Group (or created by preceding extraction steps) and places them in the directories in which the V3 Generator and Publishing processes expect to find them.

Defined Processes (Eclipse Target or BAT file):
  • 02.40...POSITION Domain Source - XML and Visio - Copies PubXML from input to the V3 Generator/InputFiles/DynamicModelFiles and domain Visio files to output/processable/hmdvsd/vsd. This process is redundant if both steps 02.10 and 02.20 have just been run. It is used when the Generator source has been cleared, and needs to be re-freshed from input, even as the original extraction remains valid.
    • 02.41...Position Domain XML Source - Is the subset of 02.40 that copies PubXML to generator/InputFiles/DynamicModelFiles (There is no bat file for this.)
    • 02.42...Position Domain Visio - Is the subset of 02.40 that copies Visio vsd files to output/processable/hmdvsd/vsd and Visio xmlfiles to generator/InputFiles/DynamicModelFiles. (There is no bat file for this.)
  • Note: The domain-build processes will also move each domain's files to its output directories just before the domain is built.


Jump to top of page

Jump to Publishing Process "Root" Page

Primarytargets and documentation on this page

This page documents a number of named ANT "targets" (names like 00.10...REVIEW properties) that invoke specific tasks, and documents. The combined publishing tools include ten "primary" targets, and over 25 "supplemental" targets. These are distinguished as follows:
  • Primary targets are shown on this page in bold face; they correspond to "batch" files of the same name.
  • Supplemental targets are shown on this page indented and in normal font; they correspond to "batch" files named with the prefix "supp-" followed by the target name.

03...V3 PubProcess - Align, Check, Revise and Constrain Source Content

Version 3 publishing involves the aggregation of the variety of content that has been developed "in parallel" by a variety of work groups. As such, publication must include a series of processes, invoked iteratively, in which the content from multiple work groups is reviewed, corrected, aligned and perhaps constrained. This section documents these coordination and review processes. The details of the underlying design principles are documented as Design Principles for Alignment, Review and Constraint of V3 Publishing Content.

With the exception of the preparation of a quality assurance review and report, this section is not subject to automated support.

03.10...Process Initial QA

This step produces a complete quality assurance review of all domain material that has been provided to the V3 generator. The review result is found in directory output/welcome/issues/ballotqareview.htm. The report is self-documenting as to the process by which it is generated and the sorts of flaws it detects. More specific documentation on using this review to support refinement of your submitted content will be added to the Wiki ASAP.

Defined Processes (Eclipse Target or BAT file):
  • 03.10...Process Initial QA Performs a V3 Generator "Preview Reference Integrity" process that analyzes details from all of the domain PubXml content and the static model data from the Visio XML files to provide a comprehensive listing of inter-model references, and dependencies and then builds a detailed Quality Analysis Report from this data.

03.20...Document results and changes (if needed)

In this step, the facilitator should review the QA report and determine corrections for which he or she is responsible.

03.30...Correct content

This is the actual correction of the errant content in either a revised static model design or a PubDb.

03.40...Repost to SVN

This step simply posts the amended material to the SVN repository.

Jump to top of page

Jump to Publishing Process "Root" Page

    • Process Initial QA
    • Document results and changes (if needed)
    • Correct content
    • Repost to SVN


Primarytargets and documentation on this page

This page documents a number of named ANT "targets" (names like 00.10...REVIEW properties) that invoke specific tasks, and documents. The combined publishing tools include ten "primary" targets, and over 25 "supplemental" targets. These are distinguished as follows:
  • Primary targets are shown on this page in bold face; they correspond to "batch" files of the same name.
  • Supplemental targets are shown on this page indented and in normal font; they correspond to "batch" files named with the prefix "supp-" followed by the target name.

04...Generate MIF and Related Content

The HL7 V3 Generator is a tool whose primary purpose is to convert a set of domain and design content (frequently expressed in idiosyncratic xml formats, into a set of files expressed in the HL7 Model Interchange Format (MIF); and then to process the MIF files to create a supporting set of artifacts, such as static model schemas, table- and Excel- views of the static models, etc. This section covers the execution of the Generator processes.

04.10...Select configuration

This sub-step has no automated task. It involves establishing the "configuration" parameters for the V3 Generator in InputFiles/configuration.ext. This file is more or less self-documenting. For the purposes of the combined Pub-Gen tool suite, the default properties are just fine, except that "publicationId" should be set to something like "Ballot2010Sep".

04.20...Generate All MIF2

This process starts "from scratch" and builds MIF2 files for all Common source files (RIM.xml, Vocabulary.mif, cmetInfoExport,txt, etc.), static models (submitted in RoseTree-HMD or Visio-XML formats) and dynamic models (pubDb xml) that have been placed in the Generator.

Preparation for this process may include:

NOTE: This process is part of 04.30...Generate All MIF and Schemas (below). If your intent is to generate schemas, use that process instead.

Defined Processes (Eclipse Target or BAT file):
  • 04.20...Generate All MIF2 - Runs Generator up to Schema Generation point, (see 04.30 for full generator run.

04.30...Generate All MIF and Schemas

This process will first build MIF2 files if they have not already been generated, and then create static model and interaction schemas from them. This is equivalent to a "full run" of the generator.

Preparation for this process may include:

NOTE: It is not necessary to run 04.20...Generate All MIF2 before this is run.

Defined Processes (Eclipse Target or BAT file):
  • 04.30...Generate All MIF and Schemas - FULL Generator run - All MIFs and Schemas

04.40...Generate Tables, Excel

One of the functions of the Generator is to create HTML table-views and Excel-views of the static models. These are key documentation elements in publishing a domain. Depending upon the settings in the generator's "configuration.txt" file, the table-views and Excel-views may be created as part of the "FULL Generator run" (04.30...Generate All MIF and Schemas). If this has not been done, this process will create them. (Note: If these views have already been created, this process will not recreate them unless a source file has been changed.)

Defined Processes (Eclipse Target or BAT file):
  • 04.40...Generate Tables and Excel - Process Excel and Table Views from Generator"

04.50...Post-process Excel

The generator produces comma-separated-variable files that can be read by Excel. In order to convert these into viewable content, they must be post-processed in Excel. The Excel spreadsheet named "HL7_RMIM_HMD_master-vx.yy.xls" in the directory input/support/xls contains the macros for making this conversion.

The basic steps are:

  1. Open the Excel file "HL7_RMIM_HMD_master-v2.26.xls" with Excel
  2. Enable the macros that this sheet contains.
  3. The sheet opens to an instruction page.
  4. Using Excel menus to find the macros, execute the macro named batchFormatHMD
  5. The macro presents a file browser:
    1. Navigate to the OutputFiles/ExcelFiles directory of the generator
    2. Select one or more "*.csv" files to be converted (using "click", "Shift-click" or "Ctrl-click" as needed.
    3. Click "Open" to start the process
  6. Each selected file will appear as plain csv text and then be magically converted to a beautiful table that will then be saved before the process begins for the next file.
  7. When done, close Excel

WARNING: In my experience, it is best to not try to do anything else on your computer while the Excel file conversion process is running. It interferes with your ability to use the clip-board (apparently the automated process uses the clip-board heavily) and your attempts may cause the conversion process to jut stop - not fail or blow up, simply quit. When left alone, it will chug along at about 1 sheet per second until it is done.

04.60...Process QA on MIF

This is a place-holder for a future process definition.

04.70...Version MIF

At some point, we will begin versioning the generated MIF files in SVN.

04.80 or 05.10...Distribute Generated Content

Distribute generated files to Publishing Process directory structure.

Jump to top of page

Defined Processes (Eclipse Target or BAT file):
  • 04.80...Distribute Generated Content - Causes Domain content for 'active' and 'support' domains to be copies to output directories. This process is not required as a stand-alone, since the act of building a domain will do the same staps anyway.

04.9x...All Generator Processes

Execute all generator processes in a single invocation. (Actually activates the Generator two or three times to complete.) Depending on which option it will also completely clear the Generator and re-load its source before starting.

Defined Processes (Eclipse Target or BAT file):
  • 04.98...All Generator Targets - All Generator Targets is equivalent to running targets 04.20, 04.30, 04.40 and 04.80 in sequence. Although ot takes a bit more time, the following target is preferred because it assures a clean generation.
  • 04.99...Clear-Reload-Run All Generator Targets Resets the Generator environment (removing output and temporary files - see 00.30...RESET Generator Environment); clears and reloads the input files for dynamic models and static models (see - 00.40...CLEAR Selected Generator InputFiles); then runs all Generator Targets, which is equivalent to running targets 04.20, 04.30, 04.40 and 04.80 in sequence

Jump to top of page

Jump to Publishing Process "Root" Page

    • Select configuration
    • Generate Core MIF
    • Generate dynamic and static MIF
    • Process QA on MIF
    • Generate Schemas, Tables, Excel
    • Post-process Excel
    • Version MIF

04.80 or 05.10...Distribute Generated Content

Distribute generated files to Publishing Process directory structure.

Jump to top of page

    • Distribute generated content files

Primarytargets and documentation on this page


This page documents a number of named ANT "targets" (names like 00.10...REVIEW properties) that invoke specific tasks, and documents. The combined publishing tools include ten "primary" targets, and over 25 "supplemental" targets. These are distinguished as follows:
  • Primary targets are shown on this page in bold face; they correspond to "batch" files of the same name.
  • Supplemental targets are shown on this page indented and in normal font; they correspond to "batch" files named with the prefix "supp-" followed by the target name.

05...Publish Specific Content

"Specific content" refers to material that is the publishing responsibility of a single facilitator or Work Group. It might be many things - a domain, a CDA Implementation Guide or an "infrastructure" specification. This section documents the processes needed to publish this material. As a general rule, the publishing of this content is driven from sets of content lists with names such as non.domains.general. See the V3_PubProcess_-_Property_Files for a discussion of how these lists are defined and the actual property files for a list of all content identifiers.

04.80 or 05.10...Distribute Generated Content

Distribute generated files to Publishing Process directory structure.

Jump to top of page

Defined Processes (Eclipse Target or BAT file:
  • 05.10...Distribute Generated Content - Distributes generated files to publishing process directory structure. Causes Domain content copy for 'active' and 'support' domains to output directories. (SOMEWHAT REDUNDANT TO -buildDomain)"

05.20...Select content to publish

Selection of content to publish is done through defining the domain and non-domain content lists that are applicable to your publishing environment. These values are in file publish-environment.properties in the root directory of these tools. These files include the "default" lists which are all possible choices for a particular list. (See also V3_PubProcess_-_Property_Files)

05.30...Domain Pub Process

Defined Processes (Eclipse Target or BAT file:
  • 05.30...Domain Pub Process - Builds domains in combined domains.active and domains.support lists

05.40...Run All Non-Domain Lists

Publishes content included in the various non-domain content lists.

Defined Processes (Eclipse Target or BAT file:
  • 05.40...Run All Non-Domain From Lists - "Builds content from all of the the non.domains.** lists (see sub-targets for more specifics).
  • 05.42...Run General Non-Domain List - Builds content from the non.domains.general list
  • 05.44...Run DAMS List - Builds content from the non.domains.dams list
  • 05.46...Run Special Models List - Builds content from the non.domains.special.models list
  • 05.48...Run Special Other List - Builds content from the non.domains.special.other list

05.50...Review Domain QA from Pub Process

Builds content from the non.domains.special.models list

05.99...Process all specific content

Combines the effect or running all of the 05.xx processes in one pass.

Defined Processes (Eclipse Target or BAT file:
  • 05.99...Run ALL Specific Content Lists Builds content from all content lists - both domains.** and non.domains.**

Jump to top of page

Jump to Publishing Process "Root" Page

    • Select content to publish
    • Run Pub process
    • Review QA from Pub Process

06...Publish Common Content and TOC

"Common" content is all of the material that is part of a ballot or publication but is not the domains and specifications being voted upon. Examples include the Version 3 Guide, the Facilitator's Handbook, Introduction and Welcome sections,the table of contents used to navigate the ballot, etc.

06.10...Update common content as needed

06.20...Select common content

06.30...Process TOC

06.40...Publish common documents

Jump to top of page

Jump to Publishing Process "Root" Page

    • Update common content as needed
    • Select common content
    • Process TOC
    • Publish common documents

V3 PubProcess - Package Publication and Distrobute

    • Generate manifests from MIF
    • Review and annotate QA
    • Package
    • Distribute

GWBeeler 13:24, 21 July 2010 (UTC)

Retrieved from "https://wiki.hl7.org/w/index.php?title=V3_Publishing_Process_Steps&oldid=39345"