2.1

Selected methods

Problem of power line inspection applies to every country, researching cannot be done freely. Conducting research requires obtaining approval of the infrastructure owner for line inspection. Some researchers use satellite [2] or aerial [3, 5, 7] imagery taken from high altitude with use of gliders. These methods give results in the form of imagery from a bird’s eye view (angle of observation 90º). A range of information is not accessible when images are taken from high altitude but is available when images are taken from the level of transmission lines or below. Imagery taken from high altitude also gives limited possibilities to asses technical conditions of power lines.

Table 1.

Comparison of selected solutions.

Legend: CHM (canopy height model), SfM (structure from motion), DEM (digital elevation model), DSM (digital surface model), ELM (extreme learning machine).

Application of such viewing angle along with LIDAR gives accurate spatial model of the observed area. Using standard camera raise an issue of object height estimation made with stereo vision [2, 5, 7] or markers located in the observed area [8].

Imagery taken from bird’s eye view with angle of observation 30º are shown in [8]. Very interesting results are achieved using measurements from three independent sensors (LIDAR, multispectral camera, RGB camera) mounted on plane and UAV. In the absence of studies of short distance observations with zero angle of observations, in this paper it is proposed to start researching the possibilities of detecting vegetation using only RGB imagery and applications of machine learning methods.

2.2

Vegetation detection

Advances in automated data processing methods takes place in the field of deep learning. In the cited papers there are also systems using neural networks. In [1] manually segmented tree corn textures are used in support-vector machine (SVM) using radial basis function kernel (RBF). Objective of [6] is proposing a method of aiding specialist in detecting single trees in the rainforest imagery taken from RGB camera mounted on UAV. The characteristics that allows to separate tree crowns with human’s eye has been described. Based on this a tree crown lighting model with noise affecting it were described. To reduce leaf level noise standard gauss convolution is used. Images are segmented using watershed method. Convolutional neural network may be used for both segmentation and classification.

In this paper U-Net convolutional network is used. This kind of neural network was originally used for medical imagery segmentation. Such images hold many similarities to leaf collection type objects, which is a tree type object that may not keep proper shape in the vicinity of other vegetation. U-Net network effectiveness has been tested for many applications, such as satellite imagery [12], hand [17] or retina vessel segmentation [18].

U-Net architecture in its first half consists of a series of layers with decreasing width (Picture 1). This allows to determine the context in the image. Second half of the U-Net network consists of a series of layers with increasing width allowing positioning of information [13]. Such network architecture is able to achieve good segmentation accuracy even for a small amount of input images. In U-Net [13] network must be forced to learn to separate segmentation areas between different classes. A loss weight map computed using morphological operations is used.

Data for research was collected using 12 MP resolution camera (mounted on a micro UAV weighting under 300 gram) and 16 MP resolution sports camera. Data was collected at a height of 2 – 3 meters above the ground. Every taken image contained vegetation with parts of high voltage power lines infrastructure. Training data was manually labelled. According to [13] U-Net architecture can achieve good performance even with relatively small data sets (as small as dozens of samples). As a result of conducted tests it was shown that, using 50 training images is enough for all 15 testing images (not presented to the network previously) to contain correctly segmented vegetation. Sample testing images and segmentation results were shown on Picture 2 and Picture 3. Detected area does not cover vegetation entirely as shown by presented results. This issue will be alleviated significantly by analyzing next frame with partial shift. Each consecutive frame may yield varying perspective of the same vegetation, making previously hard to spot parts, more easily detectable.

One of the methods for measuring similarity between sample sets is Jaccard index, also known as intersection over union. Jaccard index can also be used to measure accuracy of image segmentation. In this scenario it gives ratio of detected area and ground truth overlap to their union. It can be expressed as:

In case of testing set average Jaccard index for image was 0.81. Trained network had issue detecting not only vegetation but also part of the poles themselves. This had the biggest impact on segmentation error and can be alleviated by better preparing training images or preprocessing during classification [14, 15, 16].

Figure 1.

U-Net architecture [13].

Figure 2.

a – input image, b – detected vegetation area, c – comparison of input image and its segmentation.

Figure 3.

a – input image, b – detected vegetation area, c – comparison of input image and its segmentation.

Lighting conditions are important when collecting data. Data was collected during both cloudy and sunny days to diversify features of the learning set. Data was collected in summer-autumn months (July-September) that provided a diverse color of leaves.