FHIR Ballot Process Review

Instructions

3/9/2015 - Wiki based review of the draft FHIR Ballot Process

• This wiki based review of the draft FHIR Ballot Process is sponsored by the TSC as well as the following groups: FMG, FGB, EST and Publishing.
• The comments submitted during this review will be reviewed and acted on by FMG. The TSC will have final approval of the proposed process.
• The wiki review is one of the steps identified in the TSC's process for adoption of new process document found here: http://www.hl7.org/documentcenter/public/procedures/IntroducingNewProcessesToHL7.zip
• The comment period is open from 3/9/2014 through EOD 3/20/2015.
• The draft FHIR Ballot Process is here: FHIR Ballot Process
• During the comment period, please submit your comments below including at least your name, item, existing wording, proposed wording, and comment. Thank you.

• This review officially closed on March 20th, though feedback can still be directed to the FHIR Management Group

Comments

Enter your comments below this line.

• Ioana Singureanu: We need to create a very specific type of tracker (regarding metadata) that ensures we have all the metadata required by ANSI/balloting. The current FHIR "Issues" tracker is not quite sufficient because it does not support the information in the ballot spreadsheet.

Draft Response: Agree. The tracker will need to be adjusted to provide specific locations to capture all relevant elements from the spreadsheet

• Ioana Singureanu: We could create one tracker per product, one per release, one per ballot. What are the the pros and cons of each?

Draft response: We would probably create a single tracker for all FHIR-related issues, including HL7-published implementation guides. This will make it easy to move comments from implementation guides to FHIR core and vice versa, as well as to measure statistics on feedback on resources across ballot cycles. Finally, it'll make it easy for balloters to reference comments submitted in the past but not yet resolved. It would be up to other product families (v2, CDA, etc.) to determine whether they were best served by a single tracker per family or multiple trackers.

• Ioana Singureanu: We could also use a tracker for "to do" items instead of a "todo.txt" file: http://gforge.hl7.org/gf/project/fhir/scmsvn/?action=browse&path=%2Ftrunk%2Ftodo.txt&view=log

Draft response: All changes to the spec and the publishing tools are now tracked using a single tracker. todo.txt is ancient. We'll migrate anything there that's still relevant into the tracker

• Brett Marquard: Is it possible to add custom metadata field to group comments? For example, the 'comment grouping' column in the existing spreadsheet is very helpful for categorizing comments and planned discussion dates.

Draft response: Yes. This aspect from the spreadsheets will be migrated to the tracker. Work groups can populate it as they see fit.

• Brett Marquard: Is it possible to export proposed dispositions for block vote?

Draft response: Possibly. We have a status we use now for block votes. If/when gForge supports exporting the original comment, then it would be possible to generate a spreadsheet export containing just block vote items. It may be possible via other means as well, though more complex. If in the future we're no longer dependent on gForge, then this would obviously be a desirable behavior.

• Brett Marquard: Is it possible to record votes in gForge? Meeting meetings are an important backup.

Draft response: Yes. We'll add explicit elements for vote date, vote work group(s), affirmative, negative and abstentions

• Brett Marquard: Will there be a way to assign vote counts to multiple comments at once?

Draft response: Yes. It should be possible to use the "change multiple items" at the bottom of the browse screen, though you can only change a maximum of 25 rows at a time.

• Brett Marquard: Is there a way to state comments are related to a specific project and resource?

Draft response: Linking to a single resource is already possible. We will add the ability to distinguish between FHIR core and other balloted implementation guides (with the ability to move comments back and forth between them)

• Paul Knapp: This should be tested for accuracy of import and export of ballot spreadsheets.

Draft response: Agree. Testing for imports has already occurred and will be repeated. Testing for exports will likewise be performed.

• Paul Knapp: This should be load tested and perhaps the import process can serve as that load test to identify if we may have load issues during actual use at the May WGM.

Draft response: There were no issues loading over 400 comments in less than half an hour during initial load, which is one of the factors that suggests that the issues during the May WGM were not load driven. We've probably been hammering on the change tracker at least has heavily in the last couple of weeks, but have had no similar issues.

• Grahame: agree with Ioana. We need to create the right fields and enforce their use correctly. I believe that we need: ballot id, commenters id (can be hidden), disposition, (agreed change = disposition comment), committee that made the disposition, date vote, vote details (text field for mover/seconder/counts). Also we need an outcome for withdrawn

Draft response: We will work to align the contents of the tracker more closely with the spreadsheet. Unfortunately, gForge provides no ability to hide content, so every field will be visible and there's a limit of 2 controls

per "row", so the user interface will look rather ugly - at least as long as we stay on gForge.

• Grahame - observationally, committees struggle with the difference between the task disposition and the ballot disposition. I'd rather collapse these to a single list

Draft response: Will collapse these if possible, possibly moving some of the existing "status" characteristics to a "change needed" element

• Grahame - the "reason" field is useless, we should get rid of it

Draft response: Agree. We will migrate content from this element into "Agreed change" which will be renamed "Disposition" and then remove the "reason" element.

• Freida Hall: re: ballot comments are submitted through the standard ballot procedure (e.g. excel spreadsheets) submitted using the HL7 Ballot Desktop. (Free text comments are accepted as well, though not preferred.) – Suggestion: would discourage "Free text" comments by adding 'by prior arrangement with ? Co-Chair, else you could end up with unexpected magnitude of free text that may be challenging to manage. Unless this is referring to the existing "Free text" on the Ballot Desktop, which is at the ballot level, not the line items comment level.

Draft response: This is referring to the existing ballot-level free text which gets turned into a ballot line item for this process. Ideally, we should get rid of the free text option altogether as a means of comment submission, but this process is trying to operate without any changes to the ballot management website.

• Freida Hall: re: The following columns have been removed: Ballot WG, Artifact ID, Chapter, Pubs, Disposition External Organization, Referred To, Received From and Notes – Since this project was generalized to HL7 Ballots, please don't remove these columns from existing ballot comment spreadsheet unless validate 'non-use' further; some are being used in other WGs.

Draft response: This draft process involves using a ballot comment spreadsheet that is tuned to FHIR. Inclusion of non-FHIR-relevant columns is confusing to balloters and frequently results in cells being populated incorrectly. E.g. no section numbers or in the wrong column, etc. An alternative would be to have a single common spreadsheet, but hide irrelevant columns, though there's still a risk of balloters accidentally exposing them. (And if we need to distribute different sheets for each ballot, the advantages of hiding vs. removing are unclear. Recommendation would be to consider having "tuned" spreadsheets for different product families to reduce confusion amongst balloters, though a high degree of consistency would still be expected between them.

• Freida Hall: re: Additional columns have been added to capture ballot URL, Tracker Item #, Summary, Disposition Date, Mover/Seconder – What is 'Summary' – summary of discussion about the item prior to or discussion after motion? Disposition Date, Mover/Seconder would be useful in all reconciliation spreadsheets.

Draft response: Summary is a summary of the ballot comment. In gForge, when you look at a list of comments, you see the "summary" of the comment, but not the comment detail. Think of it as a "title" for the ballot line item. Agree that some of these new items are not FHIR-specific

• Freida Hall: re: There are multiple comments about the changes to the spreadsheet but don't see sample spreadsheet to view the changes. - Where is example of the revised spreadsheet?

Draft response: Will send it by email

• Freida Hall: re: Reconciliation Process Reconciliation occurs following the HL7 GOM, bylaws, and processes, and also subject to the various committee DMPs – please add 1) HL7 Essential Requirements; this was in the GOM but moved to separate document and synchs with ANSI Essential Requirements (Normative Ballots) 2) Co-Chair Handbook, which includes 20 pages of instructions re: balloting.

Draft response: Adherence with HL7 ER is expected. Adherence to the Co-Chair Handbook will be partial as part of this process involves use of the gForge Tracker rather than spreadsheet to manage the disposition process, though still reverts to a spreadsheet (for now) to capture the final dispositions in the ballot desktop.

• Freida Hall: re: The only difference between the FHIR ballot reconciliation process and other ballots is that FHIR uses gForge as the authoritative mechanism to track the status of the ballot line items instead of using the ballot spreadsheets – Please clarify, is this only during the reconciliation process, then status is 'frozen' once Final Reconciliation spreadsheet is extracted/posted to the HL7 Ballot Desktop?

Draft response: gForge does not provide a mechanism to "freeze" trackers, but does allow us to detect if they have been adjusted after the freeze date (reconciliation posting date) as well as what changes were made and who made them. Some types of changes such as "change was applied" or "change was published" or even clarifications as to specific wording adopted that was left unspecified in the original disposition are acceptable. Other changes would not be. We will craft a report that will allow us to detect any inappropriate changes and revert them. As Grahame discussed below, a bigger issue is if a WG decides to re-address an issue based on new feedback or experience in attempting to apply a change. If this results in an effective change from non-persuasive to persuasive, there isn't an issue. However, in some cases, a change that is found persuasive is later effectively changed to non-persuasive, possibly as a result of subsequent ballot comment or challenges in actually applying it. If this happens after the voter has withdrawn their negative based on the reconciliation package, we don't have good process for handling that through the ballot desktop - or at all. If the change is reversed in a subsequent ballot, it's entirely possible that the change won't even be tied to the original comment. Suggest that PIC discuss how this is best handled. (And it may be appropriate to check w/ ANSI too.) Once there's policy, we can attempt to use the tracker mechanism to enforce it.

• Freida Hall – What are voter's 'appeal' rights for items FHIR considers "non-valid"? Or is this equivalent to existing "Not Persuasive", which has already been defined in the ballot comment spreadsheet? Suggest considering re-use of "Not Persuasive" or "Not Related" if same concept.

Draft response: Examples include lines with a vote but not comment, comments that are incomplete (e.g. a sentence fragment), etc. If we don't get a clarifying response from the balloter, we will mark them as Not persuasive or Not related and scheduled for block vote

• Freida Hall – is distribution of consolidated spreadsheet once at the beginning of reconciliation, or periodically as updated as backup in case of gForge issue, e.g. comment submitter has not obtained login, connectivity problem, etc. (suggest the latter).

Draft response: There are no plans to distribute a consolidated spreadsheet other than at the very beginning of reconciliation and at the very end. Work groups could copy and paste specific items to email or (if gForge updates their tool) use an excel report

• Freida Hall – re: It must be possible to seamlessly move ballot items between work groups in an efficient manner with no risk of items becoming unassigned or not properly tracked – Excellent! It would be great if this could work more efficiently for all ballots. What about tracking deferred items "Considered for Future Use" – do they roll over for development for next ballot cycle?

Draft response: Yes, or some future version

• Freida Hall – does every commenter/participate need a gForge log in? Will there be a recorded webinar for novices?

Draft response: Balloters can still submit comments via spreadsheet without any requirement to use the tracker or have a gForge account. A webinar will be made available for co-chairs to handle reconciliation. There are no plans to do a webinar for commenters for this ballot. (The time-frame doesn't really allow for one to be completed and published in sufficient time for it to be referenced with ballot instructions.) It's possible that one could be considered for future ballots.

• Grahame's response to Frieda:
  • yes, it would be lovely to get rid of free comments. I've seen several ballots miss them, and they tend to be unfocused and general and hard to resolve. It would be good to replace 'see ballot from so-and-so' with a structured field rather than than text (and maybe enforce that the ballot is actually submitted - we had one ballot where a pile of people referenced a ballot that was never submitted. Fortunately it was draft for comment)
  • Agree that which columns are appropriate in the spreadsheet depends on the nature of the document being balloted.
  • question of frozen-ness is interesting. In at last some cases, they get re-opened later because of editorial issues arising. This already happens in practice, but has no traceability. We need to sort out a policy here. I'd rather that the task itself is updated in an ongoing fashion, so that the ballot can see what's happened
  • non-valid - we always consult the balloter and ask them to clarify. Things we call non-valid really obviously are, but they could always appeal to TSC. Shouldn't happen though
  • HL7 needs to provide gForge training generally as well - don't know what is in place with regard to this