3.1.1.

Variation in scoring of visual inspection results

Variation in scoring among the experts and the cell line-dependent labeled dataset is described in this section.

The interexpert scoring variation is shown in Fig. 4(a) and Table 2. The $F$-measures were distributed within the range of 0.651 and 0.897, depending on the experts and the cell quality class, the average of which was $\sim 0.8$. The large variation was mainly caused by the widespread distribution of the Mod class. For example, expert A evaluated none as good for all 259 Mod images, whereas expert B evaluated 80 images as good. The Mod class evaluation indicated that the difference in an individual expert’s scoring classification skill was significant for the Mod class compared to the other classes (good and poor).

Fig. 4

Results of visual inspection of hiPSCs. (a) Interobserver difference for quality classification, (b) the distribution of the features of each cell line and the quality class, and (c) two-dimensional distribution of the features. In (b), each feature was significantly different among the quality classes ($p<0.01$, with Tukey HSD test).

Table 2

Visual inspection performance for cell quality evaluation of the F-measures varied in the range of 0.651 to 0.897 depending on the quality class and the expert. Classification performance for Mod class was worse than the other classes.

The cell line dependency of the scoring accuracy is shown in Table 3. The expert-averaged $F$-measures were distributed widely in the range of 0.630 to 0.826 and 0.664 to 0.892 for the Mod class and the good class, respectively, whereas the poor class was within the range of 0.861 and 0.906. The experts appeared to be able to evaluate the poor class more accurately than the others, regardless of the cell line.

Table 3

Cell line dependency of visual inspection performance. The F-measure difference of the poor class was smaller than that of the other classes.

3.1.2.

Effectiveness of the biological features for classification

The distributions of the features with respect to each cell quality class [Fig. 4(b)] were investigated in the masked dataset, and the three hypotheses were significantly different. First, the number of nucleoli was significantly increased in accordance with improvement in the cell quality class ($p<0.01$). Second, the crack area rate was significantly higher in the Mod class ($p<0.01$). Third, the differentiating cellular nuclei area rate was significantly higher in the poor class ($p<0.01$). These observations coincided with the result of discussion with the expert, and these features were effective for quality evaluation in the visual inspection.

3.1.3.

Classification based on features

Classification based on biological features was investigated. The features of each cell quality class were located in the specific region of the feature space [Fig. 4(c)], in which two of the three appropriate features were selected according to the respective cell quality class. In this feature space, we reclassified each labeled dataset image using a machine learning technique. As shown in Fig. 4(c), the data points distributed on a nonlinear decision hyperplane, which separated each quality class, and the distribution appears to be complicated. Moreover, because the sample size of the labeled dataset is relatively small compared to the size generally used in machine learning applications, over fitting of the training set was a concern. To resolve these issues, we employed a nonlinear support vector machine (SVM) with a radial basis function kernel. This approach has a high generalization ability even for small and complicated datasets, because it employs margin maximization function and a kernel method.^19,20

To obtain good classification performance, a grid-search approach was employed to optimize the SVM model for the soft margin constant $C$ and the kernel function parameter gamma ($\gamma$). A three-fold cross validation was applied.^21–23 As a result of this grid-search optimization, the $F$-measures of the classification performance were 0.843, 0.683, and 0.831 for the poor, Mod, and good classes, respectively (Table 4), with $C=7.94$, $\gamma =0.05$. Because these numbers are within the range of the numbers generated by visual inspection by the experts, classification with these three features is considered to be sufficiently effective.

Table 4

Classification performance using the masked dataset. The performance of the classification using the masked data was equivalent to the average performance of the experts.

3.2.1.

Image analysis framework

As explained in the previous section, the three cell quality classes can be distinguished by measuring the three features that experts use in visual inspection. In this section, the image analysis method for iPSC quality evaluation for replacement of the visual inspection process by culture experts is shown, and the flow is described in Fig. 5. The three feature detectors and the cell quality classifier, the inputs of which are the outputs of the detectors, compose the image analysis method, in which the feature detectors and the classifier are applied to each of the regions of interest (150 pixels, $50\mu \mathrm{m}$) for a phase contrast image.

Fig. 5

Overview of the proposed method for quality control in the cultivation of hiPSCs.

3.2.2.

Feature detectors and cell quality classifiers

The details of the feature detectors and the cell quality classifiers are described in this section. For machine learning of the nucleolus detectors, the nucleoli dataset was used as training data. The crack detector and the differentiating cellular nuclei detector were tuned with the masked dataset. The cell quality classifier was developed with the labeled dataset.

The nucleoli detector detects significant, prominent nucleoli in undifferentiated hPSCs. Nucleoli observed in undifferentiated cells are nearly oval-shaped, 3- to $6\text{-}\mu \mathrm{m}$ in diameter, and appear black under phase contrast microscopy. To obtain the robustness of the shape variation of the nucleoli, the nucleolus detector was implemented with the nonlinear SVM classifier,^24,25 which classifies a black blob-like region in an image with features such as size and roundness, where the gauss kernel was applied, and then the parameters were optimized with the grid-search technique.

The crack detector detects crack regions that occur between cells during the differentiation process. In many cases, dead cells and stacked regions appear with high luminance intensity in the cell images, appear similar to cracks, and are difficult to distinguish from cracks. On the other hand, because cracks are linear-shaped and different from the shapes of dead cells and stacked regions, a crack region can be detected by extracting a linear-shaped area with high luminance intensity. In this study, the Gabor filter method,²⁶ which is frequently used for detecting linear-shaped structures such as blood vessels, was introduced into the crack detector.

In the differentiating cellular nuclei detection process, the nuclear region, which becomes darker after the chromatin structure has changed due to differentiation, is detected. Differentiating cellular nuclei are mostly oval and $\sim 10\text{-}\mu \mathrm{m}$ in diameter. In this study, the detector detected a differentiating cellular nuclei region by selecting round, black blobs of the indicated sizes after extracting the blobs by the GrabCut method,²⁷ which assumes dark and highlighted areas as the source and sink, respectively.

Finally, we trained the cell quality classifier on the nonlinear SVM model with the labeled dataset, in which the SVM kernel was the Gaussian kernel, as shown in Sec. 3.1.3. The kernel parameters $C$ and $\gamma$ were also optimized by a grid search and we obtained $C=0.631$, $\gamma =0.126$. The labeled dataset was divided into two sets, in which 600 images were for training and the rest were for the cross-validation test.

3.3.1.

Correlation between the detectors and the visual inspection results

The correlation between the outputs of the detectors and the visual inspection results of the experts was investigated [Fig. 6(a)]. The correlation between the detectors was $R^2=0.83$, 0.82, and 0.72 for the nucleoli detector, the cracks, and the differentiating cellular nuclei, respectively. The performance of the nucleoli detector on the nucleoli dataset was $F\text{-}\text{measure}=0.985$ and 0.982 for training and testing, respectively.

Fig. 6

Performance of the developed feature detectors. (a) Correlation between the detectors and the visual inspection results with respect to the cell lines and (b) examples of the detection results. In (b), the red area indicates the area detected by the expert and the detectors. The differences in the manual and detector images are due to errors by the detectors.

3.3.2.

Performance of the cell quality classifier

The performance of the cell quality classifier was evaluated with 300 test images of the labeled dataset (Table 5). The $F$-measures calculated with data from the three cell lines were 0.820, 0.686, and 0.812 for poor, Mod, and good classes, respectively. These measures performed close to the result of expert B and the result when using the manually masked data mentioned in Sec. 3.1.3.

Table 5

Performance of the cell quality classifier. The performance of the cell quality classifier was close to that of the experts. In addition, the performance for Edom was lower than that for the other cell lines because of the cell line dependency.

3.3.3.

Cell line dependencies of the cell quality classifier

The difference in discrimination performance for each cell line was investigated and compared with the visual inspection results. Calculating the $F$-measures after reclassifying Sec. 3.3.2 result with respect to each cell line, the $F$-measures were 0.816, 0.845, and 0.792 for the three quality classes of MRC5, 0.787, 0.576, and 0.723 for Edom, and 0.854, 0.564, and 0.892 for 201B7, respectively. The performance of the cell quality classifier was close to the range of the results of the four experts. A difference in performance was noted among the cell lines, similar to the visual inspection results.