CTS2/doc/Value Set REST API and Implementation Examples

Value Sets in CTS2 REST

CTS2 and some standard value set calls

CTS2 is a comprehensive specification that encompasses many aspects of terminology service. It is designed to serve users in a variety of narrow and fairly simple use cases. The CTS2 REST API for value sets offers immediate, human readable XML or JSON to any user with a browser. For example a user may want to see all value sets on a given service. These would be retrieved with a REST call like the following:

 https://informatics.mayo.edu/cts2/services/mat/valuesets

This returns a list of value sets in xml format, each of which has a linked reference to an individual value set like the following:

 https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.114222.4.11.836

While the resulting XML provides some meta data around the value set, most users would prefer to see the list of values defined by this entry. Adding resolution to the end of the URL provides a list:

 https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.114222.4.11.836/resolution

While functionally human readable the resulting XML documents are not really end user friendly. Fortunately the platform independent natures of REST, XML and JSON allow programmatic access giving developers any number of opportunities to fine tune the end user experience.

Measurable Use and eMeasure specific calls

Users who are specifically interested in how and where value sets are used in eMeasures may want to take advantage of REST calls to CTS2 that will search on NQF numbers and eMeasure id's. For instance an NQF number query would return valuesets used in the eMeasure designated by that NQF number:

http://bmidev3:1984/valuesets?filtercomponent=nqfnumber&matchvalue=0052

A similar call can be done using the eMeasure id:

http://bmidev3:1984/valuesets?filtercomponent=emeasureid&matchvalue=30

Further filtering is available on text and NQF or eMeasure id's. The following is a more complex, combined query:

 http://bmidev3:1984/valuesets?filtercomponent1=nqfnumber&matchvalue1=0052&filtercomponent2=resourceSynopsis&matchvalue2=office

Note how the text matches are paired with the NQF or Measure id by adding a digit to the end of filtercomponent and mathchvalue

CTS2 REST in Java

Java developers have number of options for REST connection and processing including popular third party libraries. Java's built in http connection library offers perhaps the most transparent look at how the connection works:

Retrieving all value sets on the service

   public  void getValueSets(){
       String uri =
               "http://informatics.mayo.edu/cts2/rest/valuesets";
       URL url;
       try {
           url = new URL(uri);
       HttpURLConnection connection =
               (HttpURLConnection) url.openConnection();
           if (connection.getResponseCode() != 200) {
               throw new RuntimeException("Failed : The HTTP error code is : "
                       + connection.getResponseCode());
           }
           BufferedReader br = new BufferedReader(new InputStreamReader(
                   (connection.getInputStream())));
           String output;
           System.out.println("Output from Server .... \n");
           while ((output = br.readLine()) != null) {
               System.out.println(output);
           }
       connection.disconnect();
       } catch (MalformedURLException e) {
           e.printStackTrace();
       } catch (ProtocolException e) {
           e.printStackTrace();
       } catch (IOException e) {
           e.printStackTrace();
       }
   }

This does little more than print out the XML page retrieved from the service

Searching for and retrieving a value set from the service

  public  void getValueSet(){
       String uri =
               "http://informatics.mayo.edu/cts2/rest/valuesets?matchvalue=Sequence&format=json";
       URL url;
       HttpURLConnection connection = null;
       try {
           url = new URL(uri);
           connection = (HttpURLConnection) url.openConnection();
           connection.setRequestProperty("Accept", "text/json");
           if (connection.getResponseCode() != 200) {
               throw new RuntimeException("Failed : The HTTP error code is : "
                       + connection.getResponseCode());
           }
           BufferedReader br = new BufferedReader(new InputStreamReader(
                   (connection.getInputStream())));
           String output;
           System.out.println("\nOutput from CTS2 Service .... \n");
           while ((output = br.readLine()) != null) {
               System.out.println(output);
           }
       } catch (MalformedURLException e) {
           e.printStackTrace();
       } catch (ProtocolException e) {
           e.printStackTrace();
       } catch (IOException e) {
           e.printStackTrace();
       } finally {
           if(connection != null){
               connection.disconnect();
           }
       }
   }

In the example above we return a specific value set with the page formatted in JSON. In both of the above cases we still have a less than ideal end user product. In some cases we may wish to parse the XML or JSON, in others we might want to deserialize the XML into Java objects. The CTS2 Framework offers some solutions to the parsing problems in it's framework project.

Processing XML and JSON into CTS2 Java objects

The CTS2 Framework project and it's required dependency are available publicly on GitHUb. Download the CTS2 Framework Corewhich is to be used with the CTS2 Framework Model.

Alternatively, developers can incorporate dependencies into a maven project with the following pom entries:

       <dependency>
           <groupId>edu.mayo.cts2.framework</groupId>
           <artifactId>core</artifactId>
           <version>0.8.0</version>
       </dependency>

And using this repository:

   <repositories>
       <repository>
           <id>edu.informatics.maven.release</id>
           <name>Informatics Maven Release Repository</name>
           <url>http://informatics.mayo.edu/maven/content/repositories/releases</url>
       </repository>
   </repositories>

The first example is compact to show the simple functionality of the framework client deserializer which has prepackaged solutions to both REST client and XML un-marshalling:

   public void unMarshallandPrintValueSets() throws Exception {
       System.out.println("\nUnmarshalling and printing using CTS2 REST client\n");
       Cts2Marshaller marshaller = new DelegatingMarshaller();
       Cts2RestClient client = new Cts2RestClient(marshaller);
       ValueSetCatalogEntryDirectory result =
               client.getCts2Resource("http://informatics.mayo.edu/cts2/rest/valuesets", ValueSetCatalogEntryDirectory.class);
       System.out.println(result);
   }

Switching to similar functionality but consuming JSON we'll cherry pick portions of the CTS2 value set related objects to return to the user. The following snippet is similar to simple examples above only incorporating a call to the server via REST to format the results in JSON

    public void unMarshallandPrintFromJson() throws IOException {
        System.out.println("\nUnmarshalling and printing from json\n");
        String uri =
                "http://informatics.mayo.edu/cts2/rest/valuesets?matchvalue=Sequence&format=json";
        URL url;
        HttpURLConnection connection;
        url = new URL(uri);
        connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Accept", "text/json");
        if (connection.getResponseCode() != 200) {
            throw new RuntimeException("Failed : The HTTP error code is : "
                    + connection.getResponseCode());
        }
        BufferedReader br = new BufferedReader(new InputStreamReader(
                (connection.getInputStream())));
        String output;
        StringBuilder builder = new StringBuilder();
        while ((output = br.readLine()) != null) {
            builder.append(output);
        }

Continuing in the same method we call the CTS2 Framework method incorporating client and un-marshalling functionality for JSON:

        JsonConverter converter = new JsonConverter();
        ValueSetCatalogEntryDirectory valuesetcat = converter.fromJson(builder.toString(), ValueSetCatalogEntryDirectory.class);

Looping through the entry directory we break down the constituent metadata and return it to the user:

        for(ValueSetCatalogEntrySummary sum : entries) {
            System.out.println("Value Set Name: " + sum.getValueSetName());
            System.out.println("Value Set Definition: " + sum.getCurrentDefinition());
            System.out.println("Value Set About: "  + sum.getAbout());
            System.out.println("Value Set Formal Name: " + sum.getFormalName());
            System.out.println("Value Set Resource Synopsis: " + sum.getResourceSynopsis());
            System.out.println("Value Set Href: " + sum.getHref());
            System.out.println("Value Set Resource Name: " + sum.getResourceName());

Notice the call to sum.getHref(). A notable feature of CTS2 is the inherent ability to jump to other services to retrieve information. In this case we'll retrieve the contents of the linked reference, resolve it, un-marshall it to an iterator and loop through the resulting value set members.

            StringBuilder vsBuffer = getRestFromHref(sum.getHref());

Jumping out of our method for a moment to show what this call does -- we make the REST call to another service:

   private StringBuilder getRestFromHref(String uri) throws IOException {
       URL url;
       uri = uri.concat("/resolution");
       if(!uri.endsWith("?format=json"))
           uri = uri.concat("?format=json");
       HttpURLConnection connection;
       url = new URL(uri);
       connection = (HttpURLConnection) url.openConnection();
       connection.setRequestProperty("Accept", "text/json");
       if (connection.getResponseCode() != 200) {
           throw new RuntimeException("Failed : The HTTP error code is : "
                   + connection.getResponseCode());
       }
       BufferedReader br = new BufferedReader(new InputStreamReader(
               (connection.getInputStream())));
       String output;
       StringBuilder buffer = new StringBuilder();
       while ((output = br.readLine()) != null) {
           buffer.append(output);
       }

Return to the method to convert the resulting JSON to a CTS2 model object

            IteratableResolvedValueSet iterableVS = getEntryMsg(vsBuffer);

Which is done in the now familiar manner here:

   private IteratableResolvedValueSet getEntryMsg(StringBuilder buffer){
       JsonConverter converter = new JsonConverter();
       return  converter.fromJson(buffer.toString(), IteratableResolvedValueSet.class);
   }

Finally we iterate through the value set enumeration to display the results as namespace (Terminology source), name (unique identifier), and designation (brief, semantically significant, description). The added bonus is a link to the referenced entity where a user can obtain more information as needed.

            Iterator<? extends EntitySynopsis > iterator = iterableVS.iterateEntry();
            while(iterator.hasNext()) {
                EntitySynopsis entry = iterator.next();
                System.out.println("Value designation" + entry.getDesignation());
                System.out.println("Entity href" + entry.getHref());
                System.out.println("Entity unique Identifier: " + entry.getName());
                System.out.println("Entity namespace: " + entry.getNamespace());
            }

Authorizing a CTS2 REST call to a service with proprietary or restricted content

This is particularly important to value set consumers who want to use value set content produced by the curated NQF eMeasures. Since much of this content is annotated with content from the National Library of Medicine's United Medical Language System users must first obtain a license and apply for a username and password for the UTS security system. This can be accomplished here:

UTS License Application Page

Switching to the HttpsURLConnection SSL version of the Java implementation we require a few extra steps to insure not only that passwords are properly encoded, but also that security certificates are handled without returning errors. Important Note: The security certificate is bypassed.

Starting with a specific import list to avoid confusion over similarly named classes:

import sun.misc.BASE64Encoder;
import javax.net.ssl.*;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.*;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

To accept all certificates we declare an inner class and turn off verification:

   private static class DefaultTrustManager implements X509TrustManager {
       @Override
       public void checkClientTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {}
       @Override
       public void checkServerTrusted(X509Certificate[] arg0, String arg1) throws CertificateException {}
       @Override
       public X509Certificate[] getAcceptedIssuers() {
           return null;
       }
   }

We start with creating a Secure Socket Layer Context and initialize it:

   public void getValueSet() throws KeyManagementException, NoSuchAlgorithmException {
       String uri =
               "https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.113883.3.526.02.99/resolution?format=json";
       URL url;
       HttpsURLConnection connection = null;
       try {
           url = new URL(uri);
           SSLContext ctx = SSLContext.getInstance("TLS");
           ctx.init(new KeyManager[0], new TrustManager[] {new DefaultTrustManager()}, new SecureRandom());
           SSLContext.setDefault(ctx);

Initialize our connection:

           connection = (HttpsURLConnection) url.openConnection();
           connection.setRequestProperty("Accept", "text/json");

Create the user password pair, encode it and convert the encoding to a string.

           String user_pwd = ("your_user_name:your_pwd");
           BASE64Encoder enc = new sun.misc.BASE64Encoder();
           String encodedAuthorization = enc.encode( user_pwd.getBytes() );

Create a request property named Authorization and denote it as "Basic";

           connection.setRequestProperty("Authorization","Basic " +  encodedAuthorization);

Enforce the acceptance of the cerificate.

           connection.setHostnameVerifier(new HostnameVerifier() {
               @Override
               public boolean verify(String arg0, SSLSession arg1) {
                   return true;
               }
           });

And continue to process the output as usual:

           System.out.println(connection.getResponseCode());
           if (connection.getResponseCode() != 200) {
               throw new RuntimeException("Failed : The HTTP error code is : "
                       + connection.getResponseCode());
           }
           BufferedReader br = new BufferedReader(new InputStreamReader(
                   (connection.getInputStream())));
           String output;
           System.out.println("\nOutput from CTS2 Service .... \n");
           while ((output = br.readLine()) != null) {
               System.out.println(output);
           }
       } catch (MalformedURLException e) {
           e.printStackTrace();
       } catch (ProtocolException e) {
           e.printStackTrace();
       } catch (IOException e) {
           e.printStackTrace();
       } finally {
           if(connection != null){
               connection.disconnect();
           }
       }
   }

CTS2 REST in Python

Python has high use in the scientific and academic community. It has third party and built in classes that allow users to handle REST calls and parse the results if necessary.

The first three examples use the python-rest-client library:

Print a list of CTS2 formatted value sets in XML

from restful_lib import Connection
conn = Connection("http://informatics.mayo.edu/cts2/rest")
reply = conn.request_get("/valuesets")
if reply['headers']['status'] == '200':
    print reply['body']

Print a list of CTS2 formatted values sets in JSON

from restful_lib import Connection
conn = Connection("http://informatics.mayo.edu/cts2/rest")
reply = conn.request_get("/valuesets?format=json",headers={'Accept':'application/json;q=1.0'})
if reply['headers']['status'] == '200':
    print reply['body']
    print eval(reply['body'])

Note that there are two approaches to using JSON - the first print statement prints the JSON verbatim. The second takes advantage that the JSON used in CTS2 is fully compatible with the Python dict syntax.

Find all value sets matching specific criteria

from restful_lib import Connection
conn = Connection("http://informatics.mayo.edu/cts2/rest")
print conn.request_get("/valuesets?matchvalue='Sequence'")['body']

Python CTS2 Client with XML parsing using DOM

Native Python libraries urllib2 and minidom offer tools for a REST client and a dom library to parse the results.

import urllib2
from xml.dom.minidom import parseString

Urllib2 opens and downloads the pertinent XML.

file = urllib2.urlopen('http://informatics.mayo.edu/cts2/rest/valuesets')
data = file.read()
file.close()

Using minidom we parse the XML in to a dom document

dom = parseString(data)

Choose the element in the XML to parse and iterate through it

print "Listing Value Sets from this resource: "
entries = dom.getElementsByTagName('entry')
for node in entries:
   print node.getAttribute('resourceName')

From a single value set, list the individual values.

print "Resolving a single large value set:"
firstVs = dom.getElementsByTagName("entry")[0]
valueSet = firstVs.getAttribute("href")
valueSet += "/resolution"
file1 = urllib2.urlopen(valueSet)
data1 = file1.read()
file1.close()
dom1 = parseString(data1)
entries = dom1.getElementsByTagName('entry')
for node in entries:
       print "\nValue Set Entry: "
       vsentryName = node.getElementsByTagName('core:name')[0]
       name = vsentryName.firstChild
       text = name.nodeValue
       vsentryNSpace = node.getElementsByTagName('core:namespace')[0]
       namespace = vsentryNSpace.firstChild
       nstext = namespace.nodeValue
       vsentryDesignation = node.getElementsByTagName('core:designation')[0]
       designation = vsentryDesignation.firstChild
       desigtext = designation.nodeValue
       print text
       print nstext
       print desigtext

CTS2 REST in Python using Authentication

This example adds the base64 library and does no parsing.

import urllib2, base64

request = urllib2.Request("https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.113883.3.526.02.99/resolution")
base64string = base64.encodestring('%s:%s' % ("username", "password")).replace('\n', )
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
data = result.read()
print data

REST from the Command Line and XML Transforms

Linux and Unix operating systems have a "curl" utility which can allow a user to simply download an XML file from a service (perhaps as a regular cron job) providing the user with a set of XML files comprising value sets that could be pulled into a web service and transformed via an xslt file or dropped into an eXist data base and accessed directly from an application.

Using curl and XML transforms with CTS2 REST and XML

curl http://informatics.mayo.edu/cts2/rest/valuesets > valuesets.xml

The first example resolves to a document with ValueSetCatalogEntryDirectory as root. A relatively simple XML transform example adapted from the W3C school site ...

 <?xml version="1.0" encoding="ISO-8859-1"?>
 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:core="http://schema.omg.org/spec/CTS2/1.0/Core"
    xmlns:valueset="http://schema.omg.org/spec/CTS2/1.0/ValueSet"
    >
    <xsl:template match="/">
        <html>
            <body>
                <h2><xsl:value-of select="valueset:ValueSetCatalogEntryDirectory/@complete"/></h2>
                <table border="1">
                    <tr bgcolor="#9acd32">
                        <th>Description</th>
                        <th>Link</th>
                    </tr>
                    <xsl:for-each select="valueset:ValueSetCatalogEntryDirectory/valueset:entry">
                        <tr>
                            <td><xsl:value-of select="@formalName"/></td>
                            <td><a href="{@href}/resolution"><xsl:value-of select="@resourceName"/></a></td> 
                        </tr>
                    </xsl:for-each>
                </table>
            </body>
        </html>
    </xsl:template>
 </xsl:stylesheet>

... provides an immediate end user interface

Authentication for Restricted CTS2 Content

Authentication in curl is also possible but don't forget to escape any special characters such as &*#$ with a "/" or authentication will fail:

curl -u username:password https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.113883.3.526.02.99/resolution  > matvaluesets.xml

This returns an XML document with IterableResolvedValueSets as root. A transform for this would look something like this:

 <?xml version="1.0" encoding="UTF-8"?>
 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:core="http://schema.omg.org/spec/CTS2/1.0/Core"
    xmlns:valuesetdef="http://schema.omg.org/spec/CTS2/1.0/ValueSetDefinition">
    <xsl:template match="/">
        <html>
            <body>
                <h2>ResolvedValueSet</h2>
                <table border="1">
                    <tr bgcolor="#9acd32">
                        <th>Source</th>
                        <th>Unique Identifier</th>
                        <th>Description</th>
                    </tr>
                    <xsl:for-each select="valuesetdef:IteratableResolvedValueSet/valuesetdef:entry">
                        <tr>
                            <td><xsl:value-of select="core:namespace"/></td>
                            <td><xsl:value-of select="core:name"/></td> 
                            <td><xsl:value-of select="core:designation"/></td> 
                        </tr>
                    </xsl:for-each>
                </table>
            </body>
        </html>
    </xsl:template>
 </xsl:stylesheet>
 

... resulting in a transform like this one

Rest in Scala

Scala is an increasingly popular functional programming language that compiles to Java byte code

A very simple REST client in Scala

This client imports the java HttpUrlConnection package, but uses scala.io to handle input streams

import java.net.{HttpURLConnection, URL}
import io.Source

object CTS2RestClient extends App{
  val connection =
    new URL("http://informatics.mayo.edu/cts2/rest/valuesets").openConnection().asInstanceOf[HttpURLConnection]
  val inputStream = connection.getInputStream
  val src = Source.fromInputStream(inputStream)
  src.getLines().foreach(println)
}

A Scala REST client with authentication and certificate handling

This client uses a fair number of Java libraries to take care of certificate handling and authentication. They are largely the same as those used in the Java examples above.

 import javax.net.ssl._
 import java.net.URL
 import io.Source
 import sun.misc.BASE64Encoder
 import java.security.cert.X509Certificate

As seen in the Java examples, authentication is handled in the header. Note: The security certificate is bypassed.

 object CTS2RestClientAuthentication extends App{
  val connection =
    new URL
        ("https://informatics.mayo.edu/cts2/services/mat/valueset/2.16.840.1.113883.3.526.02.99
        /resolution").openConnection().asInstanceOf[HttpsURLConnection]
  val encoder = new BASE64Encoder()
  val credentials: String = encoder.encode(("username" + ":" + "password").getBytes)
  connection.setRequestProperty("Authorization", "Basic "+credentials)
  val hv = new HostnameVerifier() {
    def verify(urlHostName: String, session: SSLSession) = true
  }
  connection.setHostnameVerifier(hv)
  val trustAllCerts = Array[TrustManager](new X509TrustManager() {
    def getAcceptedIssuers: Array[X509Certificate] = null
    def checkClientTrusted(certs: Array[X509Certificate], authType: String){}
    def checkServerTrusted(certs: Array[X509Certificate], authType: String){}
  })

  val sc = SSLContext.getInstance("SSL")
  sc.init(null, trustAllCerts, new java.security.SecureRandom())
  connection.setSSLSocketFactory(sc.getSocketFactory())
  val inputStream = connection.getInputStream
  val src = Source.fromInputStream(inputStream)
  src.getLines().foreach(println)
 }

---

Navigation – Common Terminology Services (CTS2)

Home
About CTS2
Purpose
CTS2 History
Business Case
How it works
Federation
Functionality
Implementing CTS2
Architecture
Development
Resources
Purpose
FAQ
Business Case
Glossary of Terms
Specification
REST
SOAP
HL7 SFM
Development
CTS2 Development Framework
Implementations
Github Page
Community
Who is Using CTS2?
Get Help
Links