The wish for an own platform for ePathology, eHistology, eHematology, eBiology!

Many Universities and other Institutions require their own platform for virtual specimens, for example, digital pathology or digital histology, digital hematology and digital biology slides. In many cases there is already a neat collection of specimen in existence that is used for teaching and often already used in courses. The first questions “How to scan?” and “How to publish on the internet?” already confront the scientist or IT professional in charge with their technical and logistic limitations. An initial thorough planning is important, taking into consideration not only the technology and quality but also costs, availability and devices that are to be used. In the following, we shine a light on the most common problems and outline some approaches to them.

Challenges of digital microscopy – Microscope slide scanners

Digitalization of the specimens

The specimen collection (in this context: microscopic slides) forms the basis for a system for virtual microscopy, and is available for learning purposes at all universities and most institutes. It is important to pay attention to quality when selecting the slides, as much effort is put into scanning the slides and processing them, for example, the effortful writing and placing of annotations. In return for the efforts, a virtual specimen is created that, once perfectly selected, never again has to be edited or replaced.

At first, it needs to be made clear what the outcome is, when speaking about the digitalization of a slide. The result of scanning a slide is a digital image similar to those taken with today’s digital cameras or smart phones. The difference between a microscopical slide scan and a photograph is the resolution: today’s digital cameras achieve a resolution of up to 36 mega pixel (MP), while a microscopically scanned slide using a 40x objective (400x magnification) easily reaches 10 giga pixel (GP).

Nowadays most pictures are saved and transferred in a compressed format. The most popular format is the JPEG standard. Unfortunately, the normal JPEG standard only supports image sizes up to 64.000×64.000 pixels, so that other formats(JPEG2000, TIFF, etc.) have to be used.

The microscopic scans are typically saved as so-called image pyramids. In this process not only the high resolution image is saved but also smaller versions of the picture (similar to a pyramid) in order to allow a later display without recalculations. The following figure shows an example of an image pyramid (in most cases “TIF-pyramid”), in which every smaller plane is always a quarter (sides: always half) of the prior plane. Most manufacturers of microscope scanners, with some exceptions, follow this algorithm.

Virtual microscopy – An image pyramid digitalized with a microscope slide scanner

In order to digitalize slides, it is currently not possible to just put them onto a flatbed scanner and scan them. Instead these options exist:

1. 1. Buying a microscope slide scanner

      Microscope slide scanners are currently offered commercially by a couple of companies. The prices are in some cases high (approx. € 100.000 and more), for example, by Hamamatsu, Zeiss, 3DHistech, Leica, Aperio. Theses scanners are delivered with so-called slide loaders that make it possible to load hundreds of slides at a time and then process them one after another. Scanning a slide with a 20x magnification takes approximately 2 – 5 minutes, depending on the scanner. The different slides scanners have different scanning qualities and should always be tested extensively, and in consideration of the requirements, before a purchase is made. There is a manufacturer that offers a very inexpensive system but with a rather low resolution (Histology Scanner).

       

       

       

      However,there is another possibility, the manual whole-slide scanning. With manual scanning, you use your own microscope, add a low-cost camera and a stitching software to it and scan by moving the microscope table. A leading product is PathoZoom Scan for under U$10,000.

      Most microscope slide scanner systems are sold with closed software that can primarily read the proprietary image formats. Therefore most microscope scanners are incompatible regarding their image formats. So, even though in principle all scanner software is based on the above mentioned image pyramid, one often has to export images in to an open format in order to exchange them between different systems. Except for the TIF format there is currently no common standard format. This is often to the disadvantage of scientists, who want to exchange virtual specimen (e.g. for ePathology). For this reason it should be made sure, before purchasing, that images can be exported into more common formats (JPEG2000, TIFF) in order to be able to extend and edit one’s virtual slide box with different machines and software in the future. In any case, before purchasing you should test the equipment with your own specimens for quality and see if it meets your requirements. In addition to that it is not improbable that, with the advancement of the technology and production in larger numbers, prices for microscope slide scanners will fall drastically.
      

      Image formats of different manufacturers

      Manufacturer	Image format
      Hamamatsu	.vms, .vmu, .ndpi
      Trestle	.tif
      Leica	.scn
      Aperio	.svs
      3DHistech	.mrxs
      Zeiss	.czi

   2. Scanning services

      There are some service providers that scan object slides as a service. University employees should use their contact network to find a University that scans object slides for a fair price. Apart from that some commercial businesses and the scanner manufacturers themselves offer scanning. In any case one should inquire beforehand what image format will be supplied and if it is possible to integrate this format into the own system.

   3. Photographing specimens section by section through a microscope

      This option is only suitable for specimens, of which only a small area needs to be captured. In this procedure pictures are taken with a suitable digital camera that is mounted on the microscope and then moved horizontally producing overlapping photos. If needed, the field of view can then be moved vertically in a zigzag pattern taking more photos. Concerning this, the following figure:

Figure 4: This image shows the movement pattern to capture a specimen with multiple photos, similar to a microscope slide scanner

1. After taking the pictures it is possible to tediously join them with graphics software (remember the position well!), for example with Adobe Photoshop or with the free software Gimp or by trying to use Microsoft ICE, a free software that automatically joins matching images to one big image. Usually Microsoft ICE functions well, but small differences in the photos can already cause some pictures not to be inserted appropriately, resulting in black rectangles that disrupt the overall picture. Furthermore, when designing this experimental software, a broad functionality was not kept in mind (for example, a white in place of a black background). Nevertheless, there are some enthusiasts who created stunning specimens using this method. However, the required working time per specimen is sometimes very long (2-4 hours per specimen). An alternative to Microsoft ICE is Hugin.

Presenting specimens via a website

The purpose of virtual microscopy is the presentation of the specimens in a digitized form also referred to as digital pathology, digital hematology and so on. This is of course possible with the installed programs of the manufacturer, but the purpose – especially in teaching – of virtual microscopy is the presentation of the images to a large audience. Therefore virtual microscopy should meet the following requirements:

1. Fast presentation of the specimens
2. Location-independent presentation of the specimens
3. Time independence (under the maxim “365/24/7”) without downtime
4. Presentation on all devices
5. Presentation with annotations on top of the specimen

When taking a look at these requirements, it soon becomes obvious that the only practical solution, achieving all requirements, is web-based and therefore a website. Especially a web server (better several servers), that assures an uninterrupted service, is required. In the future, teaching should therefore become a cloud service.

But how to create a website for virtual microscopy? First of all one has to understand the technology. Because of the limited bandwidth of an internet connection, it is not possible to download an image in gigabyte-size from the server. Therefore the method of tiling large images is used nowadays. In doing so every zoom level is segmented into small images of 256×256 pixels. These are approximately 8-15 kB in size, depending on the content.

On the server side there are two possible ways of storing pictures: 1. as original files, thus one file per specimen, just like they are created by the scanner. The advantage being that you don’t have to deal with thousands of small images (tiles). But at the same time it brings the disadvantage that the server has to execute processor intensive operations in order to extract the image tiles from the original file and to send them to the user’s browser.
2. as tiled images, which means that the original file is cut into thousands of 256×256 pixel images. This method has the great advantage that the images are loaded and presented a lot faster than out of the original files. On the other side there is the disadvantage that transferring these numerous files on the server (via FTP) takes a lot of time.

In addition there is the following problem: the software of many microscope scanner manufacturers is not able to segment the original files into tiles. The image formats are mostly not open and can therefore not be converted freely.

The users need a convenient and fast interface, whereas the administrator of the virtual microscopy website should have access to a comfortable administration panel where he can add and edit his annotations. The administrator also needs to be able to do his tasks location- and time-independently and from all kind devices. Changes should be visible to the user immediately and without tedious uploads and tying in of the new content.

A lot of microscope scanner manufacturers make use of Flash-plugins, Silverlight-plugins or Java in their software solutions. These solutions ignore today’s user’s habits.. Especially modern-day students relocate more and more of their activity to Apple devices such as MacBooks and Ipads. MacBooks don’t support the Windows Silverlight format and iPads still ignore Flash-plugins.

The solution of the future is HTML5 in combination with Javascript. There are currently two technologies that support most scanner formats and that can display them in a browser: Smart Zoom and Zoomify. Smart Zoom is installed on the devices of more than 200.000 people worldwide and has been used in several apps. The developers of Smart Zoom, themselves physicians and scientists, have set themselves the goal to develop a comprehensive solution for the teaching of virtual microscopy.

Mobile Devices

As already mentioned above, the maxim “mobile first” is today the guiding principle among developers – and rightly so! Today fewer TV sets, PCs and games consoles are being sold, because consumers shift their digital life more and more to mobile platforms. Manufacturers that ignore mobile devices, will soon be ignored by customers. Software providers, that to this day are not working on solutions for virtual microscopy for Android, Apple IOS as well as Windows Mobile, will find even less acceptance in the near future. The fact that the installation of apps can already be a threshold for users, will also become crucial in the future of mobile microscopy. The future of virtual microscopy will be a website through which the software is started – and that without installation!

Annotations as an essential component

An important element of virtual microscopy are the annotations, that by all means should be part of virtual microscopy. They contain the essential information and constitute the “teaching” by which the student can learn more about the specimen and the exploration of the specimen.

The following figure shows an example of various annotations. They can, for example, be in in the form of pins or even whole areas (e.g. polygons, planes, etc.).

Annotations are an important part in systems for virtual microscopy. Here are some examples with pins and yellow polygons that contain the teaching contents.

Quiz functionality

A quiz feature is often desired by students. It is quite easy to develop, but the development must be monitored by a lecturer that is experienced in didactics. A quiz feature that is developed only by computer scientists will often fail to meet all the requirements of teaching.

Availability and Service

Imagine the following situation: As a teacher you have professionally prepared a website for ePathology or eBiology, you have given access to your students so that they can study for their exams, and two days before the exam the server or the service fails. You can imagine the flood of mails you will receive from your students! Since your students are in principle your customers, it is important to offer them a service that assures continuous availability and assistance with technical problems (“I cannot log in to the website”).

In order to provide permanent availability with a downtime of less than 5 minutes per year, you need at least to mirrored servers. The second server takes over in case the first server fails (this is a so-called failover system). To prevent that a lot of users strain your service with simultaneous requests, a distribution of the server load over several servers is highly recommended, for example, through a Round-robin system. Thus, what you need is a DNS server that distributes the requests of the users according to the load.

Many major corporations relocate more and more of their resources into the cloud services of Amazon (Amazon Web Services), Microsoft (Windows Azure) or similar providers, because these services have a higher availability and in addition offer a fast connection in all regions of the world. The Smart Zoom technology operates on such a modern cloud-based solution, so that teachers and lecturers don’t have to worry about long downtimes of their virtual microscopy platform, but are able to focus their full energy on their core competence – the wonderful teaching of microscopy.