201801 Bulk Data
Track Name
Bulk Data Access
Submitting WG/Project/Implementer Group
FHIR-I
Justification
This track is created at the request of the ONC. ONC provided this justification:
- Ecosystem outcome expected to enable many specific use case/business needs: Providers and organizations accountable for managing the health of populations can efficiently access to large volumes of informationon a specified group of individuals without having to access one record at a time. This population-level access would enable these stakeholders to: assess the value of the care provided, conduct population analyses, identify at-risk populations, and track progress on quality improvement.
- Technical Expectations: There would be a standardized method built into the FHIR standard to support access to and transfer of a large amount of data on a specified group of patients and that such method could be reused for any number of specific business purposes.
- Policy Expectations: All existing legal requirements for accessing identifiable patient information via other bulk methods (e.g., ETL) used today would continue to apply (e.g., through HIPAA BAAs/contracts, Data Use Agreements, etc).
Proposed Track Lead
Dan Gottleib with support from Grahame Grieve (Connectathon_Track_Lead_Responsibilities)
Expected participants
- Grahame Grieve
- Cerner
- Epic
- CARIN health alliance
Roles
Data Provider: provides data in the manner specified by the bulk data API Data Consumer: consumes data in the manner specified by the bulk data API and displays/processes the data
Scenarios
The bulk data track is divided into 3 scenarios:
- asynchronous access
- nd-json support
- smart back-end services
Participants are encouraged to support either of the following initiation points:
- GET [base]/Patient/[id]/$everything
- GET [base]/Patient/$everything
However for the purposes of this connectathon track, data providers can nominate any other URL to initiate the process, as long as it returns a FHIR Bundle
Scenario Step 1: Asynchronous Access
Action
Data Consumer makes a request of the nominated URL, with the Preference Header set to 'respond-async', and a format of xml or json as negotiated with the server. Data provider sees this preference header, and returns a 202 Accepted header, and a Content-Location at which the client can use to access the response. Subsquently, the client repeatedly queries this content location using GET [content-location]. The response can be one of 3 outcomes:
- a 202 Accepted that indicates that processing is still happening. This may have an “X-Progress header” that provides some indication of progress to the user (displayed as is to the user – no format restrictions but should be <100 characters in length).
- a 5xx Error that indicates that preparing the response has failed. The body is an OperationOutcome describing the error
- a 200 OK with the response for the original request. This response has one or more Link: headers (see rfc 5988) that list the files that are available for download as a result of servicing the request. The response can also carry a X-Available-Until header to indicate when the response will no longer be available
The client repeats this request periodically until it gets either a 200 or a 5xx
Precondition: Use the nominated URL for the particular Data Provider Success Criteria: client is able to retrieve the bundle
Note: Data providers and Consumers can skip this step and only implement step #2
Scenario Step 2: ND-json support
Action
Data Consumer specifies the content-type in the request as 'application/fhir+ndjson'.
Instead of a single bundle (per step 1), the Data provider returns a set of links in the link header(s), one for each type of resource in the return bundle.
Each link in the return header contains a set of resource of the same type, in order to make processing the content easy using existing tools. Since data consumers may need to specify import scripts etc and/or route the content for processing before loading, the consumer needs to know the resource type for each link. For now, in order to support this, each link shall end with the string [type].ndjson, so that the type can be determined without loading the content.
Criteria
- Precondition: specify the correct content-type
- Success Criteria: Client can retrieve the links. In addition, at least one link SHALL contain the bundle itself, and the bundle itself shall have the id 'start-here'. This identifies the bundle that contains the actual result set, and may be used by the data consumer to navigate through the set of data that is returned
Scenario Step 2: ND-json support
Action
Data Consumer specifies the content-type in the request as 'application/fhir+ndjson'.
Instead of a single bundle (per step 1), the Data provider returns a set of links in the link header(s), one for each type of resource in the return bundle.
Each link in the return header contains a set of resource of the same type, in order to make processing the content easy using existing tools. Since data consumers may need to specify import scripts etc and/or route the content for processing before loading, the consumer needs to know the resource type for each link. For now, in order to support this, each link shall end with the string [type].ndjson, so that the type can be determined without loading the content.
Criteria
- Precondition: specify the correct content-type
- Success Criteria: Client can retrieve the links. In addition, at least one link SHALL contain the bundle itself, and the bundle itself shall have the id 'start-here'. This identifies the bundle that contains the actual result set, and may be used by the data consumer to navigate through the set of data that is returned
Scenario Step 3: Smart Backend Services Support
Action
Data Consumer uses the Smart backend services spec to create a session prior to the calls in steps #1/#2
See [[1]]. For the purpose of this connectathon, servers need only support the scope system/*.read
Criteria
- Precondition: Application is registered with the server
- Success Criteria: Client can contain an access token that grants them the right to make the calls in step #1 and Step #2
TestScript(s)
This is an API extension, and will require extensions to the test script resource in order to be tested
Security and Privacy Considerations
- Obviously, access to APIs like this in production require both authentication and consent
- Step 3 tests out application authentication
- for now, it is assumed that consent is managed elsewhere, though extensions may be added to the stream for this (see [[2]])
- Audit: for now, it is assumed that applications will audit the initial FHIR retrieval, and a smart on fhir login, but there are no rules about that
