Automatic segmentation of subfigure image panels for multimodal biomedical document retrieval.
Cheng B, Antani SK, Stanley J, Demner-Fushman D, Thoma GR
Proceeding of: Document Recognition and Retrieval XVIII - DRR 2011, 18th Document Recognition and Retrieval Conference, part of the IS&T-SPIE Electronic Imaging Symposium, San Jose, CA, USA, January 24-29, 2011, Proceedings.
Abstract:
Biomedical images are often referenced for clinical decision support (CDS), educational purposes, and research. They appear in specialized databases or in biomedical publications and are not meaningfully retrievable using primarily text-based retrieval systems. The task of automatically finding the images in an article that are most useful for the purpose of determining relevance to a clinical situation is quite challenging. This task can be done by automatically annotating images extracted from scientific publications with respect to their usefulness for CDS. As an important step toward achieving this goal, we proposed figure image analysis for content based image retrieval (CBIR) techniques. Extracted image features from the entire image and relevant local image regions can then be associated with identified biomedical concepts extracted from the meta-text in figure captions and discussion in the full text for improved hybrid (text and image) retrieval of biomedical articles. A challenge toward this goal is separating individual panels from a multi-panel figure that is often found as a single image in the biomedical article. In a previous study, the feasibility of automatically classifying images by usefulness (utility) in finding evidence was explored using supervised machine learning and achieved 84.3% accuracy using image captions for modality and 76.6% accuracy combining captions and image data for utility from articles over 2 years from a clinical journal. However, the figures images in this study had to be manually segmented into individual panels. In this work we present methods that add make robust our previous efforts reported here that, though successful, were limited in their scope and were unable to meet the challenges of segmenting figure illustrations, graphs, and charts. For the latter, we present a novel particle swarm optimization (PSO) clustering algorithm to locate related figure components. Results from preliminary evaluation are very promising with the area under the ROC curve at 94.9% for regular (non-illustration) figure images and 92.1% accuracy for illustration images. More intensive tests are in progress to evaluate impact of automatic figure panel segmentation and use of ROI in image annotation and retrieval.
Cheng B, Antani SK, Stanley J, Demner-Fushman D, Thoma GR. Automatic segmentation of subfigure image panels for multimodal biomedical document retrieval.
Proceeding of: Document Recognition and Retrieval XVIII - DRR 2011, 18th Document Recognition and Retrieval Conference, part of the IS&T-SPIE Electronic Imaging Symposium, San Jose, CA, USA, January 24-29, 2011, Proceedings.