2.1.

Confocal Microscopy Imaging

Confocal laser scanning microscopy is a type of fluorescence microscopy where the specimen is chemically labeled with fluorescent markers. During imaging, a focused laser beam deflected by a set of xy -scanning mirrors excites the fluorescently stained specimen (i.e., the dissected brain made transparent by histological means). The emitted fluorescence is then spatially filtered by inserting a so-called “confocal pinhole” into the microscope’s optical path. This pinhole ensures that only light from the focal plane reaches the detector, thus enabling the formation of an image that can be considered an optical section through the specimen. By moving the position of the specimen along the optical axis of the microscope, a 3-D image is generated.^{1 2}

The staining of the bee brains used for the present work followed an adapted immunohistological protocol. Dissected and fixed brains were incubated with two primary antibodies (nc46, SYNORF1) that label synapse proteins.^{15 16} Because cell bodies in insects reside separately from fibers and tracts, these antibodies bind well in regions of high synaptic densities, i.e., neuropil, but not in somata regions. A Cy3-conjugated secondary antibody [mouse antirabbit whole IgG (H+L)] sensitive to the primary antibody was subsequently used to render labeled regions fluorescent. After dehydration and clearing with methyl salicylate, which is a standard procedure to increase specimen transparency, the specimens were mounted in double-sided custom slides.

The brains were imaged with a confocal laser scanning microscope (Leica TCS 4D). The chromophor was excited with an ArKr laser (568-nm wavelength), and the fluorescence was detected using a long-pass filter (590-nm cutoff wavelength). The intensity of the fluorescence was quantized with a resolution of 8 bits. Due to the size of the dissected and embedded brain (about 2.5×1.6-mm laterally and about 0.8-mm axially), it cannot be imaged in a single scan. Therefore, we used multiple image-stack acquisition (3D-MISA) as described by Zuschratter et al.¹⁷ The entire brain was scanned in 2×3 partially overlapping single scans, each using 512×512 pixels laterally and between 80 and 120 sections axially. The stacks were combined into a single 3-D image using custom software or a script running in Amira (Mercury Computer Systems, Inc., Boston, MA). Coordinates of the stack origins in the xy plane were recorded by the microscope and provided the relative positions needed for alignment of adjacent stacks. Because of the refractive index mismatch between the media in the optical path, images exhibit a shortening of distances in axial direction that was accounted for by a linear scaling factor of 1.6.¹⁸ Examples of the final microscopy images from four different subjects are shown in Fig. 1.

Figure 1

Examples of CM images of bee brains. The bee brains are labeled fluorescently (see text for details). Shown are the central axial slices of the 3-D image stacks acquired from four subjects.

2.2.

Magnetic Resonance Microscopy

The MRM acquisition was performed on a Bruker AMX 500 spectrometer with 11.75 T and maximum gradient strength of 0.66 T/m. The solenoid coil had a length of 5.5 mm and an inner diameter of 4 mm. A constant coil and sample temperature of 5 °C was maintained during the experiment. The cooling unit utilizes a heater and temperature sensor that already exist on the AMX 500 spectrometer. Liquid nitrogen is evaporated at a constant rate and heated subsequently to keep the sample at 5 °C, which is monitored and controlled by the temperature sensor.

A 3-D spin-echo image with 256×256×30 voxels, a field of view of 4.0×4.0×0.9 mm, and a nominal resolution of 15.6×15.6×30.0 μm was acquired. The echo time was 14.8 ms; the repetition time was 1.5 s. For the excitation, a frequency-selective pulse with a Gaussian lineshape was used, whereas for the refocusing, a nonfrequency selective block pulse was chosen. To remove DC offsets, the mean value over the last 30 points of each spin-echo acquisition was calculated and subsequently subtracted from every point of this acquisition. Zero filling by a factor of 2 prior to image reconstruction resulted in an image resolution of 7.8×7.8×15.0 μm. The total acquisition time (averaging over eight repetitions) was 25.7 h. Three orthogonal slices from the resulting MRM image are shown in Fig. 2.

Figure 2

An MRM image of the cranium of a honey bee. No contrast agent was used during image acquisition. Top left: axial slice. Top right: sagittal slice. Bottom left: horizontal slice. The arrows denote the three coordinate directions of the 3-D representation of this image.

The preparation of the bee head reflects the tradeoff between noninvasiveness and optimization for MRM imaging contrast. To minimize the radio frequency coil’s size and thus maximize its sensitivity, the head of a honey bee (Apis melifera carnica) foraging worker was separated from the thorax, and the mandibles were removed by a horizontal cut approximately 1 mm ventrally from the brain boundary. The tentorium, a sustaining sclerotized inner head structure ventral from the brain, was left intact, thus preventing a deformation of the head capsule. To enhance imaging contrast and to stabilize the brain’s structure, the bee head was fixed overnight in a solution of 4% formalin and 0.1% Triton X before removing the mandibles. After washing the specimen for 2 h with iso-osmotic sucrose solution, the bee head was kept in 30% ethanol, which was replaced again by iso-osmotic sucrose solution prior to the MRM acquisition. To remove gas, which is dissolved in the iso-osmotic sucrose solution and fills the tracheal air sacs surrounding the brain, the bee head was evacuated for 30 min at the beginning of the fixation, and again for 10 min just prior to the MRM acquisition. For the measurement, the head was placed in a glass tube filled with iso-osmotic sucrose solution to prevent a magnetic susceptibility difference at an air-specimen border. Further details of the MRM imaging process can be found in Haddad et al.¹⁴ and Schaupp et al.¹⁹

2.3.

Image Preprocessing

To make the registration of MRM and CM images an easier problem, we apply a series of image operators to the MRM image. Figure 3 shows the individual steps and three orthogonal slices from the resulting intermediate MRM image after each of them.

Figure 3

Sequence of image processing operators applied to the MRM image in Fig. 2. (a) Crop field of view to approximately match microscopy image. (b) Median filter, 3×3 neighborhood. (c) Histogram equalization of image intensities. (d) Inversion of image intensities. (e) Gamma correction with γ=0.2.

The first processing step is a cropping operation that reduces the field of view of the MRM image to exclude the majority of nonbrain parts of the bee cranium. Since the MRM image exhibits severe salt-and-pepper noise, a median filter with a 3×3 neighborhood is applied next. This nonlinear filter reduces the noise while preserving edges. The next operator is a histogram equalization of the image intensities, which enhances image contrast. This is followed by an inversion of image intensities so that the brain is represented by bright voxels, whereas the background is turned dark. The final processing step is a gamma correction with γ=0.2, which improves the separation of brain and image background.

2.4.

Image Unwarping

Unwarping the CM images is achieved by computing an anatomically correct mathematical transformation between each microscopy image and the MRM image. This procedure is commonly referred to as image-to-image registration.²⁰

For the purpose of the present work, registration is a two-stage process. In the first stage, the global relative pose of both images is estimated and modeled by an affine transformation with 9 degrees of freedom (three translations, three rotations, and three anisotropic scale factors). Using a multiresolution search strategy,²¹ we find the transformation parameters that maximize the normalized mutual information (NMI) image similarity measure²² between the two images.

The second stage estimates and models the residual nonrigid component of the transformation using an algorithm first described by Rueckert et al.²³ The transformation model is a free-form deformation (FFD), which is defined on a regular, uniform grid of control points with cubic B-spline interpolation between them. As in the affine stage, the transformation parameters are determined that optimize NMI between the two images. An efficient implementation of this algorithm with support for parallel computation keeps the computational cost within reasonable bounds.²⁴

To enforce smoothness of the nonrigid coordinate transformation and to prevent unrealistic deformations and folding, we regularize the optimization function of the nonrigid registration stage. A constraint term E _constraint is added to the NMI image similarity E _NMI with a user-defined weight w between zero and one. The complete optimization function is thus

The rigid and nonrigid registration methods outlined before are only two of many successful such algorithms (for an overview of the field, see Refs. 26 and 27). We have chosen these particular methods mostly because of our own positive experience. For example, we have found the NMI image similarity measure to be relatively insensitive to noise and image intensity inhomogeneity, which is quite strong in our CM images. Likewise, the coordinate transformation model of the nonrigid registration can be substituted with a different one, but we have found the implicit smoothness constraint of the B-spline quite helpful when registering such different imaging modalities as MRM and CM.

3.1.

Image Data and Methods

We evaluate the proposed unwarping technique by registering CM images from 20 individuals to an MRM image from a different individual. For each subject, the CM image is independently registered to the reference MRM image, first with a 9 degrees of freedom affine transformation, followed by a nonrigid FFD transformation. Figure 4 illustrates the successive stages of alignment: rigid correction of pose and orientation differences, anisotropic scale correction, and finally, nonrigid deformation.

Figure 4

Example of axial slices at two locations from the MRM image with corresponding slices from one individual CM image. The CM image is shown after alignment, scaling, and nonrigid deformation. The two slices are approximately 156 μm apart. The affine scaling factors for this subject were 1.14 in the x direction and 0.87 in the z direction (out of plane). There was no substantial scaling in the y direction.

For the nonrigid registration, the control point spacing of the FFD was initialized as 480 μm, which in a multiresolution strategy was refined twice to reach a final resolution of 120 μm. The relatively coarse control point spacing reflects the relatively coarser resolution of the MRM image. It is also motivated by the fact that we are attempting to model deformations of the specimen preparation process, which are likely to be smooth and coarse in scale, rather than fine-scale interindividual shape differences. The regularization constraint weight was w=0.01, which we have found in previous work to produce good results.^{28 29}

3.2.

Overlap Consistency of Segmented Brains

We verify that the computed coordinate transformations between the MRM image and the CM images are meaningful by computing the overlap of manual segmentations of the brains in all images. From previous work, we have access to manual segmentations of each of the microscopy images used in the present study.^{29 30} In each image, 22 anatomical structures were labeled by an expert. These segmentations are binarized to obtain a binary segmentation of the entire brain versus the image background. Similarly, we obtain a manual binary segmentation of the brain in the MRM image.

For each CM image, the binary brain segmentation is deformed according to the transformation computed between the respective CM image and the MRM image. The relative number of voxels for which both segmentations agree is a measure of how well the brains from the CM are mapped into the coordinate system of the MRM image. The results for all 20 subjects are shown in Fig. 5 (left box). The average percentage of matching voxels over all subjects is 90% with a standard deviation of 1%.

Figure 5

Percentage of matching voxels. Left box: matching voxels between manual segmentation of brain in MRM image and coregistered CM images. Right box: matching voxels between a segmented brain in one individual deformed CM image and the consensus segmentation of the remaining deformed CM images.

As a second experiment, we perform a leave-one-out comparison between the deformed segmented brains from the individual CM images. For that, an individual image is selected, and the remaining 19 brain segmentations are combined into an estimate of the correct segmentation.²⁹ Then the overlap between the selected brain and the combination of the remaining segmentations is computed. This is repeated in a round-robin manner, selecting each individual as the test case once. The results of the round-robin consistency test are also shown in Fig. 5 (right box). The average percentage of matching voxels here was 92% with a standard deviation of 2%.

3.3.

Affine Transformation Components

The scaling factors from the affine registration are plotted by dimension in Fig. 6. It is immediately clear from this plot that the majority of the CM images are smaller in the x direction, i.e., they have a smaller lateral extent (scaling factor 0.92±0.05, mean±standard deviation). There is also a clear increase along the z direction, i.e., in brain thickness (1.05±0.06). The scaling factors in the y direction are tightly clustered around 1.0, i.e., on average the brains in the CM images have the same height as the one in the MRM image. The product of the three scaling factors for each subject is the total scaling factor for that subject. From Fig. 6 it appears that overall, the brain volume is smaller in the CM images than it is in the MRM image (scaling factor 0.97±0.05). In general, this suggests that the expansion in thickness does not entirely compensate for the lateral contraction. This is not necessarily surprising, since the bee brain is not a homogeneous block of matter, but instead an arrangement of multiple large structures with relatively small connections between them.

Figure 6

Box plot of affine scaling factors by coordinate direction. The box on the far right corresponds to the total scale factor, which is the product of the individual scale factors for each subject. The scale factors are relative to the MRM image coordinates, i.e., factors smaller than one correspond to the microscopy image being smaller than the MRM image.

3.4.

Nonrigid Transformation Components

In addition to the affine transformation components, there is substantial residual nonrigid motion. By computing the difference between affine and total motion for each image voxel, the pure nonrigid component is easily determined. To suppress the essentially random motion in the image background, we only include voxels inside the brain (as determined by manual segmentation). Averaged over all 20 subjects, the magnitude of the nonrigid motion in the brain is 94 μm. The range of the per-subject averages over all voxels is from 70 to 125 μm.

All transformations between the MRM image and the CM images are defined in the coordinate system of the MRM image. We can, therefore, easily compute the deformation field that is the mean over all microscopy images. The resulting vector field is visualized in Fig. 7. Most notable in the horizontal slice is that the shrinking in the lateral direction is accompanied by an expansion of similar magnitude along the body axis. It appears that compared to the MRM image, the brains in the microscopy images on average lost part of their width while gaining thickness. This is in good agreement with the findings of the affine analysis in the previous section. Note, however, that the scaling is not global; the expansion of the brain in the z direction, for example, is concentrated around the center of the brain, whereas the peripheral regions move inward but approximately maintain their thickness [Fig. 7(b)].

Figure 7

Mean deformation vector field (left) and dominant deformation components (right) in MRM image coordinates. The deformation field is only shown in the brain because the deformation field in the image background outside the brain is meaningless, since it is a result of B-spline continuity and the smoothness constraint of the registration algorithm. (a) Axial slice. (b) Horizontal slice. Each arrow in the left images points in the direction of deformation; the length of an arrow represents the local deformation magnitude.

3.5.

Computational Requirements

Although it is not one of the core points of interest for the present work, it is important to note that the methods proposed and evaluated here are computationally efficient. Memory requirements are dominated mostly by the space allocated for the image data, which is on the order of 25 to 30 MB per CM image and about 3 MB for the cropped and processed MRM image. Since we use a multiresolution pyramid of smoothed and downsampled images, and due to some storage space for the FFD control points, the total peak memory allocation is approximately 55 MB.

The image operators applied to the MRM image are easily implemented and almost instantaneous. The most expensive operator is the median filter, which takes about 2 s using a not particularly well-optimized implementation on a modern PC with a 3-GHz Intel Pentium 4 CPU (image size 290×169×60 voxels). Note that the entire preprocessing needs to be applied to the MRM image only once, independent of the number of CM images to be unwarped.

Affine registration needs to be repeated for each CM image, and on average took about 25 min of computation time per subject. Nonrigid registration on average consumed about 15 min per subject. Nonrigid registration is less computationally expensive, despite the substantially larger number of degrees of freedom in the nonrigid transformation, because the B-spline interpolation functions have compact support and, therefore, each control point in the FFD only influences a small local neighborhood. This locality can be exploited for local recomputation,²⁴ whereas each parameter in the affine transformation has a global effect.