RnP Wednesday, February 17, 2016

The call today was cancelled. However, I (David Tao) have posted some thoughts for the project team's consideration.

The first part of this page, "OBSERVATIONS FROM DATA," is a result of my "homework" from previous RnP calls. I wrote notes regarding possible principles to guide recommendations, as well as some observations on the data themselves. These are based on Bob's preliminary short survey results Powerpoint deck.

OBSERVATIONS FROM DATA

Proposed Principles and Key Questions for Evaluating Data

• Is there a “positive minus negative” (agree minus disagree) that should be computed? A weighted average score (like Keith’s Data Element Relevance?) I FAVOR THIS. Boils it down to a single number.
• Segregate hospital responses from ambulatory. Discharge Summary only applies to hospital. Referral Note only applies to ambulatory. CCD may apply to both. Note there is a total difference between (A) “responses from hospitals” vs (B) “responses from EPs about documents received from hospitals?” In (A) the hospital is the RECEIVER. In (B) the hospital is the SENDER.
• Should there be breakout teams to analyze hospital vs ambulatory?
• Look for those with high degree of consensus across different populations, as “always includes”
• Always include MU-required data if relevant. OK to use nulls if not relevant. Remember, for actual usage, meeting the needs of the receiver is more important than meeting the letter of certification.
• What is the threshold of “Prefer” to make it a slam dunk?
• What do we do with the “experience” stats (which are different than the “preference” stats). Should we weight more heavily those with experience? Should those with no experience be excluded?
• Can we assume statistical validity based on the 500 or so responses for ambulatory? Are 33 for hospital sufficient?
• Should “neutrals” be counted with prefer, disagree, or not at all
• For those with significant negatives or “lack of interest” make them optional. Suggest supporting queries (FHIR or XD* or Direct messaging) to get additional data.
• Watch for specialty-specific preferences
• Question: if we can't arrive at strong conclusions because the data are mixed, should we consider permitting access to raw uninterpreted data so that developers can draw their own conclusions? Perhaps they would like to "slice and dice" the data to address their particular mix of customers.

Key findings from SHORT SURVEY

Numbers refer to the pages in the slide deck that Bob presented at the HL7 WGM.

• 10 – biggest responses are “organization and structure makes it difficult to use” and “Needs summary.” The last, along with “information that I need is missing” may go together. TOO MUCH INFORMATION is 50%, but what is it?
• 11 -- ToC From Hospital: seeming strong preference for traditional Discharge Summary. Should we just recommend that be the document sent, instead of CCD? It is required in certification anyway. If that were the recommendation, what would be the ramifications (what would be lost that is in CCD but not in DS)? Also high percentage wanted “limited information from current hospitalization” so how does that compare to “traditional DS?” Can we use HL7 DS document type as accurate representation of traditional DS?
• 13 -- ToC from Hospital: 46% say more than half of the documents are “missing important information)
• 16 – Toc from Hospital: Items in red are all not required for MU. Also Encounters was low (47%). Suggest that there is not much demand that they be added, unless relevant in special circumstances. Of those with high demand that are NOT currently required, Plan of Treatment was one of the top. Also Chief Complain/RfV (narrative). Surprisingly “Hospital Course” was high at 79% but not as high as I would expect. Also, what is “Hospital Consultation?” I notice that Goals and Health Concerns are not listed here, but were listed in long survey.
• 14 – Ambulatory ToC sounds like a combination of “All” information from current visit and limited info from “all” visits (e.g., new or changed info only). But not as many (32% vs 46%) say important info is missing, compared to info from hospital.
• 17 – Ambulatory ToC. High demand for Assessment and PoT (compatible with new rulings from PoT). Same conclusion that red items are not required in MU and not important to most respondents, so suggest not adding them except under special circumstances. Advance Directives also not so important. I notice that Goals and Health Concerns are not listed here, but were listed in long survey.
• 18, 19, 20 – I suggest normalizing and doing the “weighted average” to see where it lands. That will put the preference on a continuum scale ranging from “last only” to “all” and take into account those that are on opposite ends of scale (e.g., Procedures, 50% say current only, 27% say all visits).
• 19 – not sure I (or the respondents) understand the question. Take Results for example (and how does that different from “Hospital Studies/Results?). 34% say “last only.” Does that mean “The last result from EACH prior hospital stay” literally? Irregardless of when that hospital stay was? Can we use the answers to this question if it is unclear?
• 21 – Medications for hospital. It sounds like ALL of them are at least “useful,” though Discharge Meds is the winner. Why do they want admission meds too? What this doesn’t answer is whether they would want them all in the same document, or whether one of them (like DC meds) “trumps” the others.
• 22 – similar question to 21. Do any of these answers “trump” other answers? 53% say all previously discontinued meds are “necessary or useful” but what would they think if there was a long list of discontinued meds?
• 23 – what do the numbers 1-5 in the bottom table mean?

Key findings from LONG SURVEY

These are from Keith's slide deck summarizing the long survey. However, we need to discuss further as to how representative they are and how they jib with the short survey results.

• “Presentation, organization, and structure are more the issue, rather than any specific section”

…Except for Labs, which are “repetitive, unorganized, not useful.”

• Redundant and repeated data (see Labs comment)
• Some unusual spread of comments, e.g., target for document size: more said 5 or 10 pages, a couple said 1 page. Also on slide 1, some said “insufficient detail” but others said “too much completeness, no clinical summary” and “minimal highlighting of key findings.”
• Data Element Relevance Chart (slide 6) important. All “Historical” items got lower scores than all “Active” items.
• Of the non-MU2-required data, the highest scores were GOALS, HEALTH CONCERNS, ASSESSMENTS, CURRENT FUNCTIONAL STATUS. The first three of these are required for MU3.

PROJECT SUGGESTIONS

I think it would be helpful to have:

• A Project Plan to get from where we are to the conclusion (a paper ready for ballot, which I think we're targeted for the May ballot). We'd need to work backward from the deadlines (whatever they are) to make sure we have enough time and resources to do that.
• Longer call time (60 minutes) at least for the next few weeks. 30 minutes was fine for quick status updates, but not for the substantive work of coming up with recommendations.
• Discussion on on how RnP findings might be factored into the C-CDA Companion Guide project that Jean Duteau is leading. It seems very relevant and pertinent to the Companion Guide, although the RnP is based on experience in MU Stage 2 based on C-CDA 1.1 not 2.1. Nevertheless, the principles and survey results should be applicable to both Stage 2 or Stage 3.