Published online August 21, 2002, 10.1148/rg.e5
(Radiographics. 2002;22:e5.)
© RSNA, 2002
Collaboration System for Radiology Workstations1
John Eng, MD,
Jeffrey P. Leal, BA,
Wei Shu, BS and
Guo Liang Yang, MS
1 From the Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Central Radiology Viewing Area, Room 117, 600 N Wolfe St, Baltimore, MD 21287 (J.E., J.P.L., W.S.) and the Biomedical Imaging Laboratory, Laboratories for Information Technology, Singapore (G.L.Y.). Presented as a scientific exhibit at the 2001 RSNA scientific assembly. Received April 28, 2002; revision requested June 13; revision received and accepted July 15. Supported in part by Intel Corporation, Johns Hopkins Singapore Pte Ltd, and Kent Ridge Digital Labs. Equipment support from Intel Corporation. Software support in part from Emageon. Address correspondence to: J.E. (e-mail: jeng@jhmi.edu).
 |
Abstract
|
|---|
Consultation between radiologists and referring physicians is part of routine medical practice. Nevertheless, a typical picture archiving and communication system contains no provision that will allow this critical interaction to occur on-line. The authors describe an image viewing system designed for real-time interactive consultation over the Internet. The system has two main components: an image viewer and a collaboration server. The image viewer connects to the collaboration server over an Internet-compatible network. Once the image viewer is connected, its display can be synchronized with that of another connected image viewer, so that radiologists can point out image findings and diagnoses in real time to remotely located physicians. The image viewer can retrieve images from any DICOM-compatible archive. In addition to standard image manipulation functions, the image viewer contains a new user interface for image annotation. Developed specifically for medical imaging, this user interface is activated by mouse actions instead of conventional on-screen controls, greatly improving the ease with which annotations can be created. The collaboration system is based on a simple yet flexible programming interface that can be readily generalized to other types of collaborative applications. The system was developed with the Java programming language because of Java's integrated support of Internet-compatible networking capabilities.
© RSNA, 2002
Index Terms: Computers • Picture archiving and communication system (PACS) • Radiology reporting systems
|
Introduction
|
|---|
The objective of a picture archiving and communication system (PACS) has been summarized in phrases such as "any image, anywhere, anytime." Implementations to achieve this objective commonly involve many image acquisition modalities and image viewing workstations connected over a high-speed data network to a centralized archive for storing medical images (1). Such an implementation theoretically allows a user to access any medical image in the archive at any time from any location at which a workstation can be installed. The potential benefits of PACS-based image distribution and image management are well known. These benefits include elimination of lost films, reduction of film handling and storage costs, simultaneous access to images by multiple care providers, and workflow optimization (2).
Despite the numerous potential benefits of PACS, there may be some fundamental aspects of radiology practice that are not optimally supported by a typical PACS. One example is the practice of consultation between radiologist and referring physician (Fig 1). Suppose a physician wants to review some medical images with a radiologist or another physician. Most PACS do not support this interaction directly, so it must occur over the telephone or in person. Can the PACS concept be adapted to support consultation as well? Prototype systems have been demonstrated (3), but readily deployable configurations or products do not, to our knowledge, currently exist.
View larger version (34K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1. Schematic shows connectivity of the radiologist's and referring physician's workstations to a central image archive. The red double-headed arrow represents connectivity required for collaboration between radiologists and referring physicians that is not supported in a typical PACS.
|
|
This article describes a prototype system developed to explore this question. The objective was to develop a simple yet effective software collaboration system that allows consultation functions to be added to PACS software. After describing this system, we introduce gesture-based image annotation as an example of interactive functions whose development can be stimulated by the collaboration system.
|
System Design and Implementation
|
|---|
Collaboration System
A basic image viewing application, ImageViewer, was developed. In the prototype system, ImageViewer represents a generic diagnostic image viewing workstation supporting DICOM. ImageViewer performs basic image viewing functions one would expect from a DICOM workstation: image query and retrieve from a DICOM image archive, window and level manipulation, interactive scrolling through multisection image sets, and display magnification.
In addition to supporting basic image viewing functions, ImageViewer allows the user to control another user's ImageViewer or to be controlled by the other user's ImageViewer. This interactive control is mediated through an intervening server application, the collaboration server (Fig 2). The system of collaboration is analogous to an Internet "chat room," with the collaboration server acting as the chat room. For two ImageViewer users to collaborate, they must each log their ImageViewer into the collaboration server. When the first user logs into the collaboration server, the server creates a session containing data describing the current context of the first user's ImageViewer, including the study being viewed, the image number of the study being viewed, window and level settings, and so on. When the second (or subsequent) user logs into the same session on the collaboration server, his or her ImageViewer receives data representing the context of the session. On the basis of this context, the second user's ImageViewer automatically retrieves the appropriate image from the DICOM archive and sets the display parameters to match those of the first user. While the ImageViewers are logged into the collaboration server, all display changes occurring in each ImageViewer are sent to the collaboration server for distribution to all of the other ImageViewers logged into the same session. Such display changes include adjustments to the window and level, change of the displayed image in a sequence of images, and loading of a new study.
View larger version (21K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2. Schematic shows the relationship between the two major components of the system described in the text: ImageViewer and collaboration server. The components are connected via a standard Internet-compatible network.
|
|
Display changes are communicated via control messages that are passed through a standard Internet "socket" connection between the ImageViewer(s) and the collaboration server. A standard Internet connection is the only networking requirement for the ImageViewer(s) and collaboration server. The control messages are small, perhaps only a few hundred bytes, so that real-time responsiveness of the collaboration functions is rapid.
When more than two users are participating in a single collaboration session, it may be desirable to allow only one user to control the session, as in the case of a teacher discussing an image with a group of students. To support such sessions, the collaboration server implements a "teaching mode," in which only one ImageViewer is allowed to control the session. The teaching mode can be activated at any time during the session when one of the users in the session chooses to be the session "master."
All software was developed in the Java programming language (Java 2 platform; Sun Microsystems, Palo Alto, Calif) by using an integrated software development package (JBuilder 4; Borland, Scotts Valley, Calif). DICOM compatibility was implemented through a DICOM programming toolkit for Java (JDDK; Emageon, Birmingham, Ala). The software was developed for the Windows operating system (Microsoft, Redmond, Wash), although we have also successfully tested the software in a Linux environment (Linux version 7; Red Hat, Durham, NC). The Java language was chosen for this project primarily because of its integrated, high-level support of standard Internet-compatible networking functions.
A Collaboration Example: Interactive Gesture-based Annotation
With the prototype collaboration system, we had the opportunity to create new collaborative user interfaces. In our collaboration prototype, the primary example of user interface development was in the area of image annotation.
Marking an image and pointing to structures on an image are sometimes unconscious activities that are nevertheless essential to effective face-to-face collaboration between radiologists and other physicians, so these activities must be emulated in the electronic collaboration system. To allow these important interactions to occur electronically, ImageViewer implements an annotation mode that can be synchronized with other ImageViewers through the collaboration server (as is done for all ImageViewer functions). The annotation mode is an attempt to emulate the colored wax crayons commonly used by radiologists to annotate film (4). The annotation functions of typical PACS workstations are clumsy because they are patterned after computer drawing programs, where all functions must be chosen from a menu, screen palette, or "toolbar." For example, to draw an arrow, the user must typically choose the arrow function from a screen palette or toolbar before drawing the arrow. If the arrow is to be labeled, the user must choose the text function from the screen palette or toolbar before entering the text.
In ImageViewer, image annotation is accomplished without any menus, palettes, or toolbars, more closely emulating the radiologist's wax crayon rather than a computer drawing program. This is accomplished though the use of a gesture recognition algorithm analogous to those used in handheld electronic organizers (Fig 3). To draw an arrow in ImageViewer, the user simply drags the mouse along an approximately straight line from the point of interest. The gesture recognition algorithm recognizes the mouse movement as a straight line and draws an arrow that best fits the corresponding path, with an arrowhead at the beginning of the line. To draw a circle or oval, the user drags the mouse along a round path around the area of interest. The gesture recognition algorithm recognizes this mouse movement as round and draws a circle or oval that best fits the path of the mouse. ImageViewer also recognizes rectangles. To add text to the image, the user clicks the mouse once at the point of interest (without dragging), which produces a text cursor. The user can immediately begin typing one or more lines of text. With gesture recognition, all of these annotation functions can be activated and used in any order (Figs 4, 5) without making explicit selections from a menu, palette, or toolbar.
View larger version (26K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 3. Diagram of gestures (mouse movements) recognized by ImageViewer's annotation function. The annotation shape is determined by gesture recognition rather than selection from a menu, palette, or toolbar. A gesture is defined as dragging the mouse along one of the types of paths shown. The starting position for each gesture is indicated by a black dot.
|
|
View larger version (86K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4. One image from a contrast-enhanced helical CT scan of the chest in a patient with pulmonary embolism, as displayed by ImageViewer. Three types of annotation are demonstrated in red: oval, arrows, and text labels. The crosshair cursors indicate the two physicians logged into the collaboration session. In this example, the white cursor with the magenta label is controlled by the local user (Dr Smith), and the cyan cursor tracks the real-time cursor movements of Dr Jones, who is located remotely. The physicians can use the cursors to point out features of the image to each other remotely.
|
|
View larger version (68K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5. Screen capture of two ImageViewer applications connected to a collaboration server. This demonstration loads a few sample images with which the pointing and annotation features can be used. After logging each ImageViewer into the collaboration server, the two image displays will be synchronized. The collaboration server window shows a log of collaboration messages that are being received and forwarded between the two running ImageViewer programs. System requirements: This demo requires Netscape Navigator 4 (or higher) or Microsoft Internet Explorer 5 (or higher) running on the Windows operating system (as of this writing, the demo is not compatible with Web browsers for the Macintosh operating system). This demo also requires the Sun Microsystems Java 2 plug-in. If the plug-in is not present, you will be asked to download it from the Sun Microsystems Web site. To run demo (1022 kBytes): Go to the start page and follow the instructions. You will be asked to grant security permission to allow the demo to run. The demo will not alter your computer in any way.
|
|
In addition to making digital "marks" on the images, a method must also be provided for convenient collaborative pointing out of structures on the images. One method might be to use the screen cursor as the pointer, with a method of passing control of this pointer among the ImageViewers logged into a collaboration session. Although logical, we found this paradigm to be clumsy because too many steps were required to pass the pointer from one user to another. Instead, in the present implementation of collaborative pointing, each user continues to be able to control and point with his or her own cursor, with the locations of all other cursors shown at the same time on the image (Figs 4, 5). Each cursor has a unique color and is labeled with the name of the corresponding user to minimize confusion.
Programming Interface
The collaboration system is defined by a simple programming interface for exchanging control messages in the form of Java objects between two or more client applications participating in a collaboration. A Java object is essentially any data structure formatted in Java. The exchange of the Java message objects is mediated by the collaboration server. The collaboration server places no requirements on the format or content of the Java message objects; the format and content are defined by the programmer.
In the collaboration system, the programming interface is implemented as a programming library. Because of the simplicity of the programming interface, it is straightforward for a programmer to add collaboration functions to an existing application by using the programming library. The programmer attaches the library to the application and is required to write two additional subroutines: one subroutine to initialize the state of all collaboration functions on initial connection to the collaboration server, and one subroutine to act on incoming Java message objects. The programming library provides Java subroutines that the programmer can call to log into the collaboration server and to send message objects to the server for distribution to others participating in the collaboration session. The programming library is flexible because both the message objects and the subroutines for handling them are defined solely by the application programmer. In fact, the ollaboration server is capable of supporting different types of collaboration applications simultaneously, as long as each type of application satisfies the programming interface (Fig 6).
View larger version (42K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 6. The same collaboration server used in the demonstration in Figure 5 can be used to support diverse collaborative applications. Here, the collaboration server is used to support a simple real-time interactive "chat" application in order to demonstrate the flexibility of the collaboration server and its programming interface. Because of the functional leverage of the collaboration server, the chat application was written with less than three pages of Java program code. System requirements: This demo requires Netscape Navigator 4 (or higher) or Microsoft Internet Explorer 5 (or higher) running under the Windows operating system (as of this writing, the demo is not compatible with Web browsers for the Macintosh operating system). This demo also requires the Sun Microsystems Java 2 plug-in. If the plug-in is not present, you will be asked to download it from the Sun Microsystems Web site. To run demo (45kB): Go to the start page and follow the instructions. You will be asked to grant security permission to allow the demo to run. The demo will not alter your computer in any way.
|
|
|
Discussion and Summary
|
|---|
In developing the prototype described in this article, we have demonstrated that a collaboration system based on a simple programming interface is sufficient to support common diagnostic image workstation functions. We have also demonstrated that such a system can stimulate the development of new collaborative functions such as gesture-based image annotation, a function which also has application in imaging workstations outside of a collaborative environment.
It is possible to use general-purpose teleconferencing software such as NetMeeting (Microsoft) to support collaboration functions like the ones we have implemented in our system. However, in preliminary testing, we found that the performance of such teleconferencing software was inadequate for a diagnostic image viewing application because of the large image data sets that must be manipulated.
In this project, we have attempted to use only established communications standards such as DICOM and Internet-compatible networking to optimize interoperability with existing and future systems and networks. The one link in our system that does not use an established protocol is the connection between the ImageViewer(s) and the collaboration server. While this connection is a standard Internet socket connection, the mechanism of messaging is specific to programs written in the Java language. A number of industry standards for intra- and inter-application messaging do exist, such as the common object request broker architecture (CORBA) (5), the clinical context management specification (CCOW) (6), and the simple object access protocol (SOAP) (7). Because these standards are designed to cover a potentially large variety of applications, they are considerably more complex than the simple collaboration needs of our application. Therefore, we chose instead to develop a simplified system of messaging objects implemented in Java. Despite the simplicity of the system, it has considerable potential flexibility because the format, content, and handling of the message objects is entirely defined by the application developer.
The current collaboration prototype has a number of technical limitations. First, it does not change the speed or networking requirements for image distribution. The loading time of images from archive to workstation over the network is not made faster by the system, nor was the system designed to improve the speed of this process. Common strategies for reducing the waiting time for image loading, such as prefetching, are not obviated. Second, incorporation of the collaboration system into existing PACS applications still requires additional programming, although this programming is simplified by the straightforward programming interface. Third, the prototype system uses only a simple user login and unencrypted password. Data transfer is also not encrypted. Stronger methods of user authentication and authorization would be required for a production-level implementation. However, since standard Internet-compatible networking is used, a number of virtual private network solutions are readily available to provide both access control and data security. Security methods are also built into the Java programming platform.
Image annotations are currently not stored permanently in the image archive. The recently approved DICOM supplement for presentation-state storage (8) provides the needed mechanism for storage of image annotations in the archive. Image annotations provide an opportunity to add value to the interpretation of medical images (4) and should be stored in the archive as a part of the medical imaging record. Each session of annotations and gestures could also be stored for educational or legal purposes.
Formal testing of the collaboration system in a production environment was beyond the scope of the present project. Ideally, testing would include diverse environments, such as the operating room and the intensive care unit, where immediate consultation between radiologist and referring physician is often desired. Such evaluation would be required before general implementation is feasible and represents important future work.
A real-time collaboration system such as the one described here assumes the simultaneous availability of the users who are collaborating. This assumption also holds true with regard to the usual interactions between radiologists and consulting physicians. Increasing economic pressures and complexity of medical care are certain to make the simultaneous availability of any two physicians more difficult in the future. Therefore, we anticipate that collaboration functions will evolve from the real-time methods presented here to "off-line" methods analogous to electronic mail. Support of off-line collaboration is another avenue for future work and may result in another generation of user interface developments for diagnostic imaging workstations.
|
Acknowledgments
|
|---|
The authors thank John Carrino, MD, for his helpful comments during preparation of the manuscript.
|
References
|
|---|
- Carrino JA, Unkel PJ, Miller ID, Bowser CL, Freckleton MW, Johnson TG. Large-scale PACS implementation. J Digit Imaging 1998; 11(3 suppl 1):3-7.
- Siegel E, Reiner B. Work flow redesign: the key to success when using PACS. AJR Am J Roentgenol 2002; 178:563-566.
- Chang PJ. Multimodality diagnostic workstation that features "intelligent" modality-specific display and navigation tools, real-time interactive consultation, and World Wide Web browser capability (abstr). Radiology 1995; 197(P):535.
- Simon M, Hall FM. Red arrows in radiology. AJR Am J Roentgenol 1992; 159:897.[Medline]
- Object Management Group. Common object request broker architecture basics Needham, Mass: Object Management Group, 2001; Available from: http://www.omg.org/gettingstarted/corbafaq.htm. Accessed 19 July 2002.
- Health Level Seven. Clinical context management specification (CCOW) Version 1.1. Ann Arbor, Mich: Health Level Seven, 2000.
- Box D, Ehnebuske D, Kakivaya G, et al. Simple object access protocol (SOAP) Version 1.1. Boston, Mass: World Wide Web Consortium, 2000; Available from: http://www.w3.org/TR/SOAP/. Accessed 19 July 2002.
- DICOM Standards Committee. Digital Imaging and Communications in Medicine (DICOM) Supplement 33: grayscale softcopy presentation state storage. Rosslyn, Va: National Electrical Manufacturers Association, 1999.
Top
Abstract
Introduction
