1.IntroductionHistopathology investigates tissue specimens traditionally employing light microscopy. However, recent technological advancements have empowered pathologists to transit to digital pathology to use high-resolution (HR) scanning and storage of glass slides. The generation of whole slide images (WSIs) in digital pathology accelerates the research and clinical utility of computerized techniques. Among other advantages, digital pathology enables the inspection and comparison of tissue samples with annotated digital images. Available metadata may include clinical data, such as pathology reports and molecular data. One of the main objectives in digital pathology is investigating image analysis applications. Computer-assisted diagnosis (CAD) of histopathology images is practical due to repetitive tissue patterns that automated recognition can exploit.1 Furthermore, as a relatively new technology, WSI allows assessing various techniques to analyze HR images. However, a series of problems, such as high demand for storage, labelling (annotating) image regions, and scanning quality are still hindering the practical usage of digital pathology.2 Overcoming these challenges may improve the diagnosis by correlating comparable patients and retrieving similar images through “image search.” During the past few years, microscopic images were in the focus of many researchers in deep learning.3,4 Overall, proposed approaches mainly concentrate on the classification of WSI patches resulting from sliding a window.5 Generally, CAD algorithms require HR images. Diagnosis in histopathology relies on the information of the tissue samples mounted and fixed on glass slides visually inspected at several magnification levels. It is common for the pathologist to repeatedly zoom in and out on specific tissue regions to locate and categorize abnormalities. Consequently, we digitize glass slides at high magnifications to generate HR images for computational pathology. The gigapixel WSIs are extremely large files [usually in scanscope virtual slide (SVS) or BigTiff format] that inevitably lead to high storage needs and transfer bottlenecks. Each scan is generally an image of gigabyte size, and for each patient, multiple glass slides are usually available. Every day, numerous patients are biopsied. All these factors result in the creation of massive histopathology archives that may not be easily accessed or analyzed for research and clinical purposes. Hospitals and clinics may decide to erase older scans to free up some storage space, which is a rather undesirable solution leading to loss of valuable information in evidently diagnosed cases. Instead of removing older slides, a more desirable solution is keeping a low-resolution (LR) version of slides and upsampling it to HRsupon access, with no information loss. In contrast, digital scanners are expensive devices that may not be available to all pathologists worldwide. However, the availability of lower-cost devices may be more feasible but would negatively impact the scanning resolution. Having tools to virtually increase the WSI resolution at any magnification of scanned slides by affordable devices facilitates the adoption of digital pathology, hence enabling computational pathology. Finally, the scans and the scanners are far from ideal. Digitized slides may be blurry in some of the regions of a scan. This issue is not rare since the optical devices of the scanner need to focus on the biopsy. The digital scanners manage the proper focusing by measuring the depth of the tissue surface. However, focusing is error-prone due to issues like specks of dirt or stains on a slide and physical limitations of optics. All in all, some notion of upsampling of the scanned images may help solve these problems. This study mainly aims to explore available methods capable of upsampling WSIs. The overall approach would help with storing smaller images while capable of restoring HR upon request. The LR images are magnifiable by new types of decompression methods. A promising approach to address the discussed issues is super-resolution (SR). In this technique, the goal is to retrieve pixel-level information of an image based on perceptual information. Beyond images, some researchers work on video resolution enhancement in computer vision using SR.6 Generally, SR is implemented using classical methods like kernel-based techniques.7 However, recent deep-learning approaches outperform traditional methods by a such large margin that it is quite difficult to find reports that compare the two. The SR technique has intertwined with deep neural networks during the past few years.8 Image SR is a class of image-processing techniques aiming to construct a HR image based on a LR image. Deep learning has recently successfully addressed several image enhancement problems in computer vision. The single image SR (SISR) network uses a single LR image to reconstruct its HR mapping. The essence of the problem has been approached in multiple real-world applications, including but not limited to medical imaging9–12 and security.13–15 In addition, image SR improves the performance of other tasks.16–18 Image SR is a considerably challenging issue which also is an ill-posed problem due to its intrinsic problem definition. A single LR image can act as the respective LR image of multiple HR images. Conventional techniques emerged a few decades ago.19 Among the classical methods, researchers introduced methods based on predictions,20 edges,21 statistics,22 patches,23 and sparse representation.24 Deep learning has largely impacted image SR. In this area, state-of-art results have been achieved recently. Specific benchmarks for this task are also designed but these are commonly general-purpose images.25–28 Deep networks to address this problem stem from the very early convolutional neural networks (CNNs), such as SRCNN,29 to more recent ideas like generative adversarial networks (GANs), which have been employed in SRGAN.30 Deep SR networks differ in aspects such as the architecture of the network31–33 loss function34,35 and learning strategies36,37. Many researchers have also applied SR in medical imaging. There are publicly available datasets to train networks for SR tasks. However, the introduced SR datasets rarely focus on the domain of histopathology. Subsequently, there are only a few SR studies in histopathology. Medical imaging contains many image modalities for diagnostic and treatment-panning purposes. Researchers initially investigated classical SR methods on radiological modalities like magnetic resonance imaging. Radiology images are generally small (mainly in megapixel range) and hence easier to process.38 However, by the emergence of deep learning and better results, more complex structures and data like pathology images are being processed. In this study, the main focus is on histopathology images. Among the earliest SR in histopathology studies was a paper published by Mukherjee et al. in 2018. The authors introduced a deep framework for reconstructing HR images in the pathology. Their results showed promising outcomes, which they suggested as a comparative counter-part to the expensive scanners.39 Later, they investigated a recurrent network to enhance the quality by a multiresolution approach. Lately, Bin Li et al. proposed a framework to benefit from the hierarchical structure of the WSIs and achieved good results. Their approach showed that downsampling could act as a training data solution for these deep networks. The studies used tissue microarray datasets and a two-site whole tissue section dataset.40 This study is structured as follows. First, the general SR problem, which corresponds to the upsampling method, is described. This section briefly elaborates on the commonly used traditional upsampling methods. Next, in Sec. 3, deep SR networks are discussed, and common aspects like architecture, loss function, and assessment metrics are described. Then the experiments and training phase discussions are presented. Next, in Sec. 4, the materials used for the experimentation are described. Finally, in Sec. 5, the results and conclusion summarize the outcomes. 2.Problem FormulationThe main goal of SR is to find the most relevant HR image that corresponds to a LR image . We assume that the LR image is an image of pixels (where and are the number of pixels in horizontal and vertical axis, respectively); therefore, LR image consists of pixels and is the number of pixels of the HR image . The integer parameter is the factor that shows the increase of the image size. Now, the degradation is defined as Here is the degradation function and corresponds to the related parameters (e.g., kernel size, noise, and scaling factor). The degradation function is assumed to be unknown in many problems called blind SR; however, we can consider it known if the degradation is digitally applied. The approximate HR image , which is called the SR image of the LR image, is then constructed according to Here and correspond to the SR function and the parameters of the approximation, respectively. The degradation model could be modeled by a downsampling function applied to the HR image where denotes the downsampling operation and the set are the parameters of it. Many downsampling methods are introduced in the literature.41,42 Among these methods, bicubic downsampling is commonly used in image SR applications, although other methods are used as well. The bicubic upsampling method has been covered in Sec. 2.1. Generally, it is possible to model the downsampling as43Here denotes a kernel of convolution with the HR image to apply a filter (e.g., blurring) on the image. is the Gaussian noise added to the model with an average of zero and the standard deviation of . The model described by Eq. (4) has been proven to have more relativity with real-world problems.44 Finally, the image SR objective is formulated as Here, is the loss function which is measuring the difference between the generated SR image and the ground truth HR image. The regularization term is formulated by , and is the trade-off parameter. The losses are usually a combination of multiple functions. 2.1.InterpolationInterpolation of the data is a part of either upsampling or downsampling. Here, we briefly explore the upsampling methods while the downsampling can be analogously described. In image resampling, the main aim is to predict pixel values based on the available data. This task has been conventionally investigated to introduce relatively fast and easy methods. Some of these methods are the nearest neighbor, bilinear, and bicubic interpolations. They aim to construct a smoother image. Sample images for these methods are shown in Fig. 1.45 The nearest neighbor methods assign the closest available value of known pixels to the unknown ones. The bilinear approach estimates the value of unknown pixels by constructing a bilinear in and axis directions, while the bicubic implements the same idea but in a second-order function. 3.SR Deep NetworksDuring the past few years, SISR, like other areas, has significantly improved. These improvements were not possible without deep neural networks. Wang et al. categorized the deep neural networks for SR into four main types, which have been summarized in following section. 43 The categories with samples of the successful networks are discussed in Sec. 3.1. Then the loss function and evaluation metrics are given in Secs. 3.2 and 3.3, respectively. 3.1.ArchitecturesOne of the first networks for image SR is SRCNN, which Dong et al. introduced in 2014. The network they introduced enhanced the classical mapping methods between the LR and HR images. Their success gained attention, so many other networks were proposed based on their initial proposal. A pre-upsampling module follows their proposed CNN network to enhance the image initially. A pre-upsampling strategy enlarges the LR image first and feed it into the network. In contrast, the post-upsampling approach processes the LR at its original size through the layers of the network and then upsample the image. Despite their success, these structures faced issues like noise amplification. Later, researchers shed light on the considerably lengthy operation time of these networks due to the higher dimensional computation framework comparing other networks.29 Post-upsampling networks then solve the computational expense problem. Researchers benefitted from similar ideas for shorter training and testing times. Residual channel attention network (RCAN) and SRGAN were among the networks that benefited from this framework. These networks were successful enough to establish state-of-the-art results. Although RCAN presented higher-accuracy values in many studies, SRGAN presented more realistic images by hallucinating the textures.30,46 The benefits of the postupsampling networks were promising; however, there were also shortcomings. The single-step upsampling module made the training an arduous task when a higher magnification like 4 or 8 was required. To help this issue, researchers that experienced the high-quality results of progressive networks such as StyleGAN implemented similar approaches in the areas of SISR.47 One of these networks that were upsampling iteratively is ProSR which achieved relatively high performance.37 Finally, iterative up-and-down sampling SR was investigated as well since even the progressive upsampling encountered some problems. For instance, the learning strategies required to train them were rather complicated. In contrast, iterative upsampling applies back-projection refinement. To name a successful example, feedback network for image SR (SRFBN) fits this category which benefits from feedback block and offers better representations.48 3.2.Loss FunctionsIn deep learning, loss functions are crucial in guiding the model to optimize with desired weights. In image SR, the optimal outcome is a network that enables a high-quality reconstruction. Therefore, the loss functions are designed to help the training lead the network to reconstruct an image close to the HR image. Here, we discuss three of the loss functionals, which are used in many studies. 3.2.1.Reconstruction lossReconstruction loss or pixel loss indicates the similarity of the generated SR image and the desired HR image. This is computed by L1-norm or L2-norm of the image differences as where is the volume which is the multiplication of height and weight and number of channels. -norm showed better performance from sharpness in addition to easier convergence. The sharper images emerge since the -norm penalizes the significant deviation exceedingly while minor errors slightly. It is important to note that for image comparison corresponds to the pixel equality where it cannot backpropagate meaningful information during learning. Thus, the use of norm alone if not facilitated by other methods is usually not practiced.3.2.2.Perceptual lossA perceptual loss, also called content loss, evaluates the perceptual similarity of the generated image with the HR image. This is done by comparing the semantic content of the images using a pretrained network (e.g., VGG16). Researchers computed the distance of the ’th layer of the network based on where denotes the output of the ’th layer of the pre-trained deep network. In other words, the perceptual loss guides the network based on the hierarchical image features of a network that has been trained for a task (e.g., ImageNet classification).493.2.3.Adversarial lossThe application of GANs has reached the SR as well. In general, a GAN-based training consists of two main networks, namely the “generator” and the “discriminator.” These two networks compete against each other. Consequently, the adversarial loss for the generator and discriminator networks, respectively, is defined as denotes the discriminator for a binary decision whether the image is real data or generated. 3.3.Reconstruction Quality MeasurementHuman operators assess image quality most reliably. Although the reliability is ensured by human inspection, the mainstream methods are supposed to computational methods for higher efficiency. Here, we briefly describe two of the most common metrics in this area. 3.3.1.Peak signal-to-noise ratioThe peak signal-to-noise ratio (PSNR), as one of the most commonly used metrics, measures the maximum pixel value rate to the mean squared error of the images. The PSNR of the two images and is then computed by where is the maximum pixel-value which usually is equal to 255. Despite the common use of this metric, it does not reflect the human perception.3.3.2.Structural similarityThe human visual system is primarily concerned with recognizing image structures. Hence, a structural similarity metric is proposed to extract structural information from images. This metric is a combination of three parts, including luminance, contrast, and structures. It is given as where and are the mean and variance of the SR image, and are the mean and variance of the HR image, and is the covariance between SR and HR image. Finally, and are the relaxation constants.3.4.ExperimentsThe reconstruction path uses LR patches, and a deep network trained for SISR generates the patches for synthetic WSI. Six networks are trained to find a better network that can adequately enhance the images. First, we benefit from the deep back-projection networks (DBPNs). The network benefits from iterative up- and downsampling layers called stages. The stages include an error feedback mechanism for projection errors. The network can learn multiple content information by its up-and-downsampling layers.50 Residual dense networks (RDNs) are then trained to exploit the hierarchical features from all the convolutional layers.51 The third is a very deep residual channel attention network (RCAN) that benefits from residual in residual blocks in a post-upsampling structure.46 Forth is an SRFBN that focuses more on feedback mechanisms based on the fact that the human visual system follows a similar method.48 Fifth is enhanced deep residual networks for SISR (EDSR), which is proposed in addition to the multi-scale deep SR system.36 Finally, a network we use to do the SR is among the enhanced SR GANs (ESRGAN). This network architecture has achieved some state-of-the-art results in addition to providing a high perceptual index.52 3.5.TrainingThe training patches of the extracted SR dataset are used to train the network. The relativistic discriminator is used for adversarial training to achieve the best results. The discriminator classifies the images as fake or real and relatively compares the extent of fake and real images that enhance the learning procedure. The networks are all trained for at least 100,000 iterations. The optimal weights selection is on the basis of the validation accuracy and loss. The training were done on a Tesla V100 32 GB GPU. 4.MaterialsWSIs are the scanned histopathology slides. The cancer genome atlas (TCGA) is the largest publicly available scanned slides and reports dataset. This dataset (available at Ref. 53) allows researchers to experiment and propose new methods and compare their results easily. The generated data by TCGA is now over 2.5 petabytes and spans genomic, epigenomic, transcriptomic, and proteomic data.54 TCGA repository (i.e., genomic data commons) contains 11,007 cases and 30,072 SVS files of the slides. The WSIs of this repository span 26 organs (primary sites) with 32 cancer subtypes. The subtypes are abbreviated by a few letters in the repository. Complete subtype names and the distribution of the number of the patients in each category is explained in Table 1. The demographic information attached to each scan includes “morphology,” “primary diagnosis,” “tissue or organ of origin,” “patient age at the time of diagnosis,” “tumor stage,” “age,” “gender,” “race,” and “ethnicity” and some other information like the patients current status (e.g., dead or alive). Table 1The TCGA codes (in alphabetical order) of all 32 primary diagnoses and corresponding number of evidently diagnosed patients in the dataset.
4.1.Creation of a DatasetThis section describes the creation of a dataset of histopathological slides. The dataset includes patches, and the labels of the patches are the information attached to the WSIs. The scanned slides of the TCGA repository include many frozen section WSIs. A frozen section (cryosection) is a laboratory technique that helps to get to microscopic analysis of a specimen rapidly. The fast diagnosis is beneficial for the management of the patient during an operation. However, due to its procedure, frozen sections are not usually of high quality. These lower-quality slides were dropped to avoid confusion for deep network training sessions. Frozen section scans and diagnostic slides do not belong to the same domain from a machine-vision perspective. As well, frozen sections are prepared using a different, more rapid process. The image SR problem seeks to translate one instance within the same domain to another instance. Therefore, for a proper application, the images in each must be from the same domain. Hence, we only used diagnostic TCGA slides. To assign labels to the images, the information provided by the repository are used. The beneficial information is determined to be the primary diagnosis and the section site (/tissue/organ of origin). These two labels are presented under the “diagnoses/treatments” of the slides. Although other information are also available for further diagnosis, we have decided to use the aforementioned labels. Finally, some of the slides did not include the required information to create the image labels. These slides were removed. In addition, the classes with a number of WSIs were also eliminated. 4.2.Dataset for Histopathology SRTo create a dataset for histopathology SR, some considerations may be necessary to eliminate the appearance of undesired data. We have first removed the background patches to minimize the artifact since these do not contain valuable tissue information for training. Second, the slides are cropped into the same size tiles. The tiles (or patches) are , , , , and in three channels of RGB at , , , , and magnification levels. One sample per cancer subtype is provided in Fig. 2. 4.3.Diagnostic DatasetThe breast cancer histopathological image classification (BreaKHis) dataset was utilized to evaluate the quality and diagnostic accuracy of the SR images compared to original magnifications.55 The dataset consists of 9109 microscopic pictures of breast tumor tissues obtained from 82 individuals at four magnification levels. The primary categories of the BreaKHis dataset are benign and malignant tumors. There are a total of 1820 pictures at magnification, of which 588 are malignant and 1232 are benign. The images are in size, 3-channel RGB with 8-bit depth per channel, and saved in PNG format. 5.Results and DiscussionsThe results are assessed from multiple perspectives. First, we provide the quantitative comparison of trained networks results for image generation based on PSNR and structural similarity index measure (SSIM) in Table 2. These values are expected to quantify similarity at some level within the same tissue type. For instance, although the fatty content may contribute to variance in these values, the anatomic structure of tissue in various organs is not same. Among others, differences in these values may stem from various cell density and size in different tissue types. In this table, we first compare different sites and subtypes against each other. The comparison shows the details and complexity of the images in each category. The minimum and maximum reported values of the bicubic upsampling of the LR images is shown in italic and bold italic, respectively. It is evident that brain/glioblastoma multiforme (GBM) has the lowest SSIM, which could be interpreted as the highest structural loss within a traditional downsample upsample procedure. In contrast, the endocrine/thyroid carcinoma (THCA) offers the highest SSIM value. This could be interpreted as the maximum complexity of sharp structures and minimum, respectively. In contrast, the lowest and highest PSNR values were observed in testis/testicular germ cell tumors (TGCTs) and liver cholangiocarcinoma (CHOL). These values show the extent of details lost in the interpolation. Table 2Accuracy of image reconstruction using six networks trained on TCGA dataset; the PSNR/SSIM numbers are shown where the bold numbers are best reconstructions across all networks; and italic and bold italic are the worst and best bicubic reconstruction among data categories, respectively.
Another comparison is also provided in Table 2. In this table, the highest values with respect to the subtype are reported in bold. Two networks, DBPN and RCAN, mostly achieved the best results. The DBPN network is superior to others in 15 subtypes, while RCAN is also superior in 15 subtypes. If one network outperforms the other for one measure (SSIM or PSNR), both are mentioned in bold. The qualitative comparison of the generated images shown in Figs. 3, 4, and 5. A randomly selected patch per site is shown in which another randomly selected window has been taken into focus. The window has been compared from eight approaches, including the HR, LR, and the six networks generation. The LR image shows the bicubic interpolation of the downsampled image. Network inputs were the LR image, while the ideal outcome was a HR image. The PSNR and SSIM of the images are also given below. Taking a look at images, the ESRGAN-generated images look most like the HR image. As shown in the figures, the networks can map the LR images to a reasonable HR image that seems to include a lot of details. This has been achieved due to the usage of GANs. Overall, ESRGAN provided the sharpest images in reconstructing the HR image, while RCAN and DBPN provided the closest estimation of unknown pixel values. Although ESRGAN images were sharp enough to clearly make the cells distinguishable, it could be observed that due to the generative nature of the processing, the formation of cells is slightly manipulated based on the training of the network. In contrast, RCAN and DBPN could produce more accurate cellular shapes. However, the edges were not as sharp as ESRGAN. In addition, to evaluate the applicability of the SR to histopathology images, we performed two breast cancer-focused human assessment studies. A comprehensive evaluation by three board-certified pathologists was performed to determine the quality of these images. The SR images in this part are generated by the ESRGAN network. First, a diagnostic visual inspection has been conducted where pathologists were asked to classify images in eight categories. The categories (types) included breast tumors, benign and malignant, where each had four subtypes. The questions included four distinct histological categories of benign breast tumors: adenosis, fibroadenoma, phyllodes tumor, and tubular adenoma; and four malignant breast tumors: carcinoma, lobular carcinoma, mucinous carcinoma, and papillary carcinoma. Each pathologist evaluated 64 images, of which half were SR and the other half were HR (in random order). Second, we assessed the image quality preference of the pathologists by presenting them with 50 pairs of images, each consisting of the original image and the SR image. To prevent observer bias toward any of the image categories, the images were arranged at random order (e.g., HR and SR). The outcomes are shown in Table 3. Table 3Pathologist assessment of fake (SR) versus real (HR) images; type defines benign versus malignant; and subtype is the eight categories of BreakHis dataset.
The diagnostic findings reveal that there is no significant difference between the usage of generated SR images versus original HR images. This is demonstrated by the higher accuracy or kappa score of the evaluation diagnosis in six instances with HR pictures versus six cases involving SR images. As shown in Table 3, assessing SR images, the third pathologist received higher diagnostic score values. The second pathologist obtained higher diagnostic scores for HR pictures, but the first pathologist had slightly better findings (e.g., 3.1% higher malignancy detection accuracy) for type and subtype identification in SR and HR visuals, respectively. Although this may be subject to higher observer variability, the image quality preference findings show that SR image are generally preferred by specialists. According to the data, just one pathologist (i.e., number two) favored the HR images, and even in this instance, the expert prioritized or was unable to identify substantial differences in 46% of the images. One pathologist regarded 64% of the images to be fairly comparable, although in 34% of the instances all observers preferred the SR images. Ultimately, the second pathologist deemed 68% of SR photos to be preferable, whereas the rest of the images were found without discernible difference. In summary, the gigapixel nature of WSI in digital pathology is absolute necessary to generate HR images for diagnostic purposes. However, the large size of WSIs also creates obstacles for practical utilities of digital pathology, most notable extreme demands for high-performance storage. This study, based on the largest publicly available dataset, demonstrated that the deep GANs are indeed capable of predicting HR images from their LR versions. Some generative models may be more suitable for computerized processing (e.g., RCAN and DBPN), and some other based on adversarial training for visual inspection (e.g., ESRGAN). Our findings indicate that actual/real (HR) and synthetic/fake (SR) images are identified by pathologists at equivalent accuracy levels. As well, the pathologists have even visually preferred synthetic images to the real images in many cases. AcknowledgmentsSupport for this study was provided by a grant from Ontario Research Funds - Research Excellence (ORF-RE), Gigapixel image identification, 2018–2023, Government of Ontario, Canada. In addition, Mayo Clinic (Rochester, USA) provided sponsorship for the ORF-RE project, 2021–2023. Tizhoosh also acknowledges a start-up grant from Mayo Clinic, Rochester, USA. ReferencesL. H. Eadie, P. Taylor and A. P. Gibson,
“A systematic review of computer-assisted diagnosis in diagnostic cancer imaging,”
Eur. J. Radiol., 81
(1), e70
–e76 https://doi.org/10.1016/j.ejrad.2011.01.098 EJRADR 0720-048X
(2012).
Google Scholar
M. K. K. Niazi, A. V. Parwani and M. N. Gurcan,
“Digital pathology and artificial intelligence,”
Lancet Oncol., 20
(5), e253
–e261 https://doi.org/10.1016/S1470-2045(19)30154-8 LOANBN 1470-2045
(2019).
Google Scholar
Y. Fu et al.,
“Deep learning in medical image registration: a review,”
Phys. Med. Biol., 65
(20), 20TR01 https://doi.org/10.1088/1361-6560/ab843e PHMBA7 0031-9155
(2020).
Google Scholar
M. I. Razzak, S. Naz and A. Zaib,
“Deep learning for medical image processing: overview, challenges and the future,”
in Classification in BioApps,
323
–350
(2018). Google Scholar
C. L. Srinidhi, O. Ciga and A. L. Martel,
“Deep neural network models for computational histopathology: a survey,”
Med. Image Anal., 67 101813 https://doi.org/10.1016/j.media.2020.101813
(2020).
Google Scholar
X. Tao et al.,
“Detail-revealing deep video super-resolution,”
in Proc. IEEE Int. Conf. Comput. Vision,
4472
–4480
(2017). Google Scholar
J. Van Ouwerkerk,
“Image super-resolution survey,”
Image Vision Comput., 24
(10), 1039
–1052 https://doi.org/10.1016/j.imavis.2006.02.026 IVCODK 0262-8856
(2006).
Google Scholar
W. Yang et al.,
“Deep learning for single image super-resolution: a brief review,”
IEEE Trans. Multimedia, 21
(12), 3106
–3121 https://doi.org/10.1109/TMM.2019.2919431
(2019).
Google Scholar
K. Christensen-Jeffries et al.,
“Super-resolution ultrasound imaging,”
Ultrasound Med. Biol., 46
(4), 865
–891 https://doi.org/10.1016/j.ultrasmedbio.2019.11.013 USMBA3 0301-5629
(2020).
Google Scholar
H. Greenspan,
“Super-resolution in medical imaging,”
Comput. J., 52
(1), 43
–63 https://doi.org/10.1093/comjnl/bxm075
(2009).
Google Scholar
F. Rousseau et al.,
“On super-resolution for fetal brain MRI,”
Lect. Notes Cmput. Sci., 6362 355
–362 https://doi.org/10.1007/978-3-642-15745-5_44
(2010).
Google Scholar
E. Van Reeth et al.,
“Super-resolution in magnetic resonance imaging: a review,”
Concepts Magn. Reson. Part A, 40
(6), 306
–325 https://doi.org/10.1002/cmr.a.21249
(2012).
Google Scholar
L. Zhang et al.,
“A super-resolution reconstruction algorithm for surveillance images,”
Signal Process., 90
(3), 848
–859 https://doi.org/10.1016/j.sigpro.2009.09.002 SPRODR 0165-1684
(2010).
Google Scholar
H. Seibel, S. Goldenstein and A. Rocha,
“Eyes on the target: Super-resolution and license-plate recognition in low-quality surveillance videos,”
IEEE Access, 5 20020
–20035 https://doi.org/10.1109/ACCESS.2017.2737418
(2017).
Google Scholar
P. Shamsolmoali et al.,
“Deep convolution network for surveillance records super-resolution,”
Multimedia Tools Appl., 78
(17), 23815
–23829 https://doi.org/10.1007/s11042-018-5915-7
(2019).
Google Scholar
D. Dai et al.,
“Is image super-resolution helpful for other vision tasks?,”
in IEEE Winter Conf. Appl. of Comput. Vision (WACV),
1
–9
(2016). https://doi.org/10.1109/WACV.2016.7477613 Google Scholar
M. Haris, G. Shakhnarovich and N. Ukita,
“Task-driven super resolution: Object detection in low-resolution images,”
(2018). Google Scholar
M. S. Sajjadi, B. Scholkopf and M. Hirsch,
“Enhancenet: Single image super-resolution through automated texture synthesis,”
in Proc. IEEE Int. Conf. Comput. Vision,
4491
–4500
(2017). Google Scholar
R. Keys,
“Cubic convolution interpolation for digital image processing,”
IEEE Trans. Acoust. Speech Signal Process., 29
(6), 1153
–1160 https://doi.org/10.1109/TASSP.1981.1163711 IETABA 0096-3518
(1981).
Google Scholar
C. E. Duchon,
“Lanczos filtering in one and two dimensions,”
J. Appl. Meteorol. Climatol., 18
(8), 1016
–1022 https://doi.org/10.1175/1520-0450(1979)018<1016:LFIOAT>2.0.CO;2
(1979).
Google Scholar
K. I. Kim and Y. Kwon,
“Single-image super-resolution using sparse regression and natural image prior,”
IEEE Trans. Pattern Anal. Mach. Intell., 32
(6), 1127
–1133 https://doi.org/10.1109/TPAMI.2010.25 ITPIDJ 0162-8828
(2010).
Google Scholar
W. T. Freeman, T. R. Jones and E. C. Pasztor,
“Example-based super-resolution,”
IEEE Comput. Graphics Appl., 22
(2), 56
–65 https://doi.org/10.1109/38.988747 ICGADZ 0272-1716
(2002).
Google Scholar
J. Yang et al.,
“Image super-resolution as sparse representation of raw image patches,”
in IEEE Conf. Comput. Vision and Pattern Recognit.,
1
–8
(2008). https://doi.org/10.1109/CVPR.2008.4587647 Google Scholar
D. Martin et al.,
“A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics,”
in Proc. 8th Int. Conf. Comput. Vision,
416
–423
(2001). Google Scholar
E. Agustsson and R. Timofte,
“Ntire 2017 challenge on single image super-resolution: Dataset and study,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit. Workshops,
126
–135
(2017). https://doi.org/10.1109/CVPRW.2017.150 Google Scholar
C. Dong, C. C. Loy and X. Tang,
“Accelerating the super-resolution convolutional neural network,”
Lect. Notes Comput. Sci., 9906 391
–407 https://doi.org/10.1007/978-3-319-46475-6_25 LNCSD9 0302-9743
(2016).
Google Scholar
Y. Matsui, T. Yamasaki and K. Aizawa,
“Interactive manga retargeting,”
in ACM SIGGRAPH 2011 Posters, SIGGRAPH ’11,
(2011). Google Scholar
C. Dong et al.,
“Image super-resolution using deep convolutional networks,”
IEEE Trans. Pattern Anal. Mach. Intell., 38
(2), 295
–307 https://doi.org/10.1109/TPAMI.2015.2439281 ITPIDJ 0162-8828
(2015).
Google Scholar
C. Ledig et al.,
“Photo-realistic single image super-resolution using a generative adversarial network,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
4681
–4690
(2017). Google Scholar
J. Kim, J. Kwon Lee and K. Mu Lee,
“Accurate image super-resolution using very deep convolutional networks,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
1646
–1654
(2016). https://doi.org/10.1109/CVPR.2016.182 Google Scholar
W.-S. Lai et al.,
“Deep laplacian pyramid networks for fast and accurate super-resolution,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
624
–632
(2017). https://doi.org/10.1109/CVPR.2017.618 Google Scholar
N. Ahn, B. Kang and K.-A. Sohn,
“Fast, accurate, and lightweight super-resolution with cascading residual network,”
in Proc. Eur. Conf. Comput. Vision (ECCV),
252
–268
(2018). Google Scholar
J. Johnson, A. Alahi and L. Fei-Fei,
“Perceptual losses for real-time style transfer and super-resolution,”
Lect. Notes Comput. Sci., 9906 694
–711 https://doi.org/10.1007/978-3-319-46475-6_43 LNCSD9 0302-9743
(2016).
Google Scholar
A. Bulat and G. Tzimiropoulos,
“Super-FAN: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
109
–117
(2018). https://doi.org/10.1109/CVPR.2018.00019 Google Scholar
B. Lim et al.,
“Enhanced deep residual networks for single image super-resolution,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit. Workshops,
136
–144
(2017). https://doi.org/10.1109/CVPRW.2017.151 Google Scholar
Y. Wang et al.,
“A fully progressive approach to single-image super-resolution,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit. Workshops,
864
–873
(2018). Google Scholar
Q. Lyu et al.,
“Super-resolution MRI through deep learning,”
(2018). Google Scholar
L. Mukherjee et al.,
“Convolutional neural networks for whole slide image superresolution,”
Biomed. Opt. Express, 9
(11), 5368
–5386 https://doi.org/10.1364/BOE.9.005368 BOEICL 2156-7085
(2018).
Google Scholar
L. Mukherjee et al.,
“Super-resolution recurrent convolutional neural networks for learning with multi-resolution whole slide images,”
J. Biomed. Opt., 24
(12), 126003 https://doi.org/10.1117/1.JBO.24.12.126003 JBOPFO 1083-3668
(2019).
Google Scholar
W. Lin and L. Dong,
“Adaptive downsampling to improve image compression at low bit rates,”
IEEE Trans. Image Process., 15
(9), 2513
–2521 https://doi.org/10.1109/TIP.2006.877415 IIPRE4 1057-7149
(2006).
Google Scholar
M. Irani and S. Peleg,
“Improving resolution by image registration,”
CVGIP, Graphics Models Image Process., 53
(3), 231
–239 https://doi.org/10.1016/1049-9652(91)90045-L CGMPE5 1049-9652
(1991).
Google Scholar
Z. Wang, J. Chen and S. C. Hoi,
“Deep learning for image super-resolution: a survey,”
IEEE Trans. Pattern Anal. Mach. Intell., 43 3365
–3387 https://doi.org/10.1109/TPAMI.2020.2982166 ITPIDJ 0162-8828
(2020).
Google Scholar
K. Zhang, W. Zuo and L. Zhang,
“Learning a single convolutional super-resolution network for multiple degradations,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
3262
–3271
(2018). Google Scholar
R. E. Carlson and F. N. Fritsch,
“Monotone piecewise bicubic interpolation,”
SIAM J. Numer. Anal., 22
(2), 386
–400 https://doi.org/10.1137/0722023 SJNAEQ 0036-1429
(1985).
Google Scholar
Y. Zhang et al.,
“Image super-resolution using very deep residual channel attention networks,”
in Proc. Eur. Conf. Comput. Vision (ECCV),
286
–301
(2018). Google Scholar
T. Karras, S. Laine and T. Aila,
“A style-based generator architecture for generative adversarial networks,”
in IEEE Conf. Comput. Vision and Pattern Recognit. (CVPR),
(2019). https://doi.org/10.1109/CVPR.2019.00453 Google Scholar
Z. Li et al.,
“Feedback network for image super-resolution,”
in Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recognit.,
3867
–3876
(2019). Google Scholar
A. Krizhevsky, I. Sutskever and G. E. Hinton,
“Imagenet classification with deep convolutional neural networks,”
in Adv. Neural Inf. Process. Syst. 25,
1097
–1105
(2012). Google Scholar
M. Haris, G. Shakhnarovich and N. Ukita,
“Deep back-projection networks for super-resolution,”
in Proc. IEEE Conf. Comput. Vision and Pattern Recognit.,
1664
–1673
(2018). Google Scholar
Y. Zhang et al.,
“Residual dense network for image super-resolution,”
in IEEE Conf. Comput. Vision and Pattern Recognit. (CVPR),
(2018). https://doi.org/10.1109/CVPR.2018.00262 Google Scholar
X. Wang et al.,
“ESRGAN: enhanced super-resolution generative adversarial networks,”
Lect. Notes Comput. Sci., 11133 63
–79 https://doi.org/10.1007/978-3-030-11021-5_5 LNCSD9 0302-9743
(2018).
Google Scholar
J. N. Weinstein et al.,
“The cancer genome atlas pan-cancer analysis project,”
Nat. Genet., 45
(10), 1113 https://doi.org/10.1038/ng.2764 NGENEC 1061-4036
(2013).
Google Scholar
F. A. Spanhol et al.,
“A dataset for breast cancer histopathological image classification,”
IEEE Trans. Biomed. Eng., 63
(7), 1455
–1462 https://doi.org/10.1109/TBME.2015.2496264 IEBEAX 0018-9294
(2015).
Google Scholar
|
Histopathology
Lawrencium
Education and training
Super resolution
Diagnostics
Pathology
Deep learning