2.1

Hardware Setup

The hardware components of the system are displayed in fig. 1 and 2. The main difference to a conventional X-ray CT setup is the seven degrees of freedom robotic arm Panda from the manufacturer FRANKA EMIKA.³ It has a maximum reach of 855 mm and a repeatability of 0.1 mm when repeatedly moved from a specific starting pose to a goal pose. It has two fingers that can move on a fixed axis and grasp objects. The maximum allowed payload is 3 kg. The robotic arm and the depth cameras are connected directly to a computer while the detector is accessible through a network interface. The robotic arm can be turned off in case of emergency from outside of the safety hutch with a power switch (see fig. 2).

Figure 1.

Hardware setup. The robotic arm is mounted on a table with the source and the detector inside a safety hutch. The source to robot distance is 40 cm and robot to detector distance is 176 cm. The sample holder consists of two physical parts. The first part is grasped by the robotic arm and houses the calibration structure. The second part is where the sample is mounted and slides into the base part.

Figure 2.

Hardware schematics. Two depth cameras monitor the movement of the robot and send a stop signal to the robot controller when the executed trajectory interferes with obstacles. The robotic arm can be stopped by a manual power switch that was routed to the operator table outside of the hutch.

Two Intel Realsense D435 depth camera capture the movements of the robot and provide 3D information about the surroundings as a point cloud. The cameras are connected directly to the workstation and they are used for the collision detection mechanism.

The robotic arm is mounted on a table inside a safety hutch for X-ray CT which houses the X-ray source and the detector (see fig. 1). The detector has a maximum resolution of 2880x2880 and is connected to a different workstation on the network which provides a network interface for triggering image capturing.

2.2

Sample Holder

The sample holder is a critical component of the system as it allows the robotic arm to grasp samples of arbitrary shape and is a fundamental part of the calibration process where the position and orientation of the sample is identified. The 3D models of the sample holder and the rail component are visualized in figure 1. The sample holder consists of two parts. The bottom part is where the robot’s fingers can grasp the holder steadily. The upper part fulfills the actual purpose of placing a geometric structure around the sample on a cylinder.

The cylinder is 5.6 cm tall and 3.5 cm in diameter inside. The sample holder was designed with a 3D modeling software and printed using a 3D printer with accuracy of 0.08 to 0.2 mm on all three axes. The printing accuracy is important as the local coordinates of the spheres in the 3D model are used as reference points in the calibration algorithm.

The geometric structure embedded in the sample holder is a helix which is made up of 50 embedded aluminium spheres of 0.678 mm diameter. These spheres were fixed by hand on notches that were included in the design process of the holder. The spheres appear as circles on the detector images that will be segmented during calibration.

The helix can be parametrized by the following 3D parametric curve:

τ runs between the local w coordinates of the first sphere and the last sphere of the helix: w_min < τ < w_max where w_min, w_max ∈ ℝ.

The parameters r (radius), ρ (frequency) and ϕ (phase shift) parametrize the helix. They can be determined by fitting the sphere coordinates from the 3D model of the sample holder to eq. (1) with a least-squares term. The source code of this process can be found in the file helix_fitter.py in our repository.⁴

The helix can be discretized by choosing a fixed number H ∈ ℕ of points {τ _i}_i=1,…,H ∈ [w_min, w_max] for the free parameter τ :

2.3

Calibration

The calibration procedure tackles the issue that the robotic arm does not sufficiently accurately place the sample at the desired position due to inaccurate path planning and inaccurate electrical motors at its joints. With the calibration procedure we are able to identify the actual positions and orientations of the sample for the reconstruction step. For the calibration a sample holder with an embedded geometric structure that can be detected on the detector images is necessary. A suitable sample holder was introduced in section 2.2.

The calibration is implemented in multiple steps (see fig. 3). The first step is the post-processing of the detector image. Its contrast is enhanced and a median filter with kernel size 5 is applied to reduce noise and improve the segmentation results. The calibration circles on the image are detected in the next step with the circle Hough transform algorithm.⁵ The result is a set of 2D circle center coordinates _j = (d_x,j d_y,j)^⊤ on the detector.

Figure 3.

Calibration procedure. In a) the flat-field corrected detector image is displayed. This image is contrastenhanced and subsequently a circle detection algorithm is executed. The resulting image where the detected circle centers are marked with red crosses is displayed in b). Given the geometry of the sample holder and the robot’s sensor readings when acquiring the image, an initial guess of the helix location (blue crosses) is projected onto the image plane (c). The parameters that define the rotation and translation of the helix are optimized in a least-squares problem in the 2D image domain. The resulting parameters are used to project the helix again to the image domain to display the final outcome of the calibration (d).

Eq. (1) and the current position of the robotic arm are now used to project a set of helix points h_i (eq. (2)) onto the detector image for comparison with the segmented points _j and determining the geometry of the sample.

For this projection the intrinsic camera matrix K and the external parameters R and t are needed. K is fixed for the current X-ray CT setup and R, t are determined by the robotic arm’s current position.

There are three critical coordinate systems in our setup. The first is fixed to the X-ray source with x, y and z-axis. The second is fixed to the center of the sample holder with u,v and w-axis and moves with the robotic arm as it is attached to the arm’s fingers. The third is fixed to the detector with dx and dy axis.

The rotation R of the sample holder relative to the source can be parametrized w.l.o.g. by consecutive rotations about the z, y and x-axis:

t is the offset of the source center to the sample holder’s center:

K is fixed for the current setup and can be calculated with the parameters sdd (source to detector distance), d_x,p, d_y,p (principal points on d_x and d_y-axis) and d_w, d_h (detector pixel width and height):

We introduce the short notation ζ = (α, β, γ, x, y, z) for the free parameters. The camera projection matrix P can now be calculated:

The projection matrix is now used to project a set of H ∈ ℕ fixed points h_i ∈ ℝ⁴ on the discretized helix from eq. (2) onto the detector:

are the homogeneous detector pixel coordinates and m_i are the projected analytical helix points on the detector. These points resemble the expected position of the helix structure and they will be used for constructing an error term in the 2D detector image domain.

An appropriate cost function for comparing the error between the current and expected position of a measured circle center _j and a projected point on the helix m_i is the reprojection error:

Eq. (9) will only measure the error for a specific pair of points. In our case there are c (detected) circles on the current image, s spheres glued onto the holder and H projected points on the helix from eq. (1). It is important to note that c ≤ s because the segmentation algorithm might fail to detect all circles.

We now compare each of the c detected circle centers _j to all H sampled and projected points m_i and choose the pair with the smallest distance.

We can formulate this algorithm as a least-squares problem:

The optimization problem is nonlinear due to the sine and cosine terms in the rotation parametrization. In our implementation we use the Levenberg Marquardt algorithm.

The resulting parameters α, β, γ, x, y, z can be used for the reconstruction as the geometry of the given acquisition.

2.4

Reconstruction

For tomographic reconstruction, the sinogram contained 1000 equidistant X-ray projections along a circular trajectory sized 720×720 pixels with a spacing of 600μm. The reconstruction volume was sized 720×720×720 with isotropic voxel spacing of 100μm. Using our C++ reconstruction framework elsa,⁶ reconstruction was performed using an iterative conjugate gradient solver run for 50 iterations on a Tikhonov regularized weighted least squares problem, with the Josephs method for X-ray transform discretization and parallel beam geometry. Further iterations showed no improvement on the cost function.

2.5

Software Stack

The central part of our software stack is the Robot Operating System (ROS).⁷ Robot manipulation is accomplished with the MoveIt! framework^{8, 9} and the franka_ros configuration package.¹⁰ For image processing tasks and the circle segmentation we use OpenCV, ¹¹ for multithreading on the CPU OpenMP¹² and for the tomographic reconstruction elsa.⁶ The scientific calculations in section 2.3 are implemented with scipy.¹³

2.6

CT measurements

We conducted two experiments: a walnut was measured with the robotic arm and with a conventional rotational stage.

For each CT measurement, 1000 images were acquired with a source voltage of 30 kV, source power of 1445μA, and exposure time of 1s.

In fig. 4 a) the reconstruction of the walnut with the rotational stage is compared to the robotic arm as the sample holder. The two volumes were registered manually as we found automatic registration of the two discretized volumes to be unreliable. The slices were chosen manually for illustration purposes. The center slices of the volume from the top and the front view were extracted and cropped to the region of interest.

Figure 4.

Experimental results. A walnut was measured and reconstructed in order to compare the conventional rotational stage (reference) with the robotic sample holder (robot). The reconstruction volumes were registered and aligned but small differences are still visible. The detector images were binned with 4∗4 and the reconstruction volume has dimensions 720³. The front slice is from the perspective of the x-ray source. The top slice is from the bird’s eye view. Our observation is that the reconstruction quality is identical despite the fact that the volumes are not aligned perfectly and hence the contrast does not match.

Our observation is that there is no qualitative difference between the results of the two different sample holders.

2.7

Future work

In future work the system can be improved in several ways.

The sample holder could be more flexible. Its size currently limits the size of the sample but this can be tackled in another design iteration by embedding the geometrical calibration structure into the base of the holder when it is positioned upwards of the base and compressed in its height. The cylindrical envelope surrounding the sample could be removed and as a consequence, the sample also doesn’t strictly need to be inserted from the top.

Moreover, experiments with non-standard trajectories are subject of future work.

Finally, the accuracy of the calibration algorithm could be improved by improving the circle detection algorithm that is run on the acquired images. Currently, we are using the circle Hough transform algorithm which could be replaced by a more precise algorithm with sub-pixel segmentation accuracy.