This PDF file contains the front matter associated with SPIE Proceedings Volume 12620, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Analytical techniques such as macro X-ray fluorescence can capture dense multi-dimensional data that provides unique quantitative information at each point over the surface of a work of art. The processed outputs from scanning Macro-XRF are the elemental distribution maps that provide measurements of the abundance of the constituent elements at each point. The ability to visualize and correlate this data with other imaging modalities is an important step in understanding the material composition of a work of art. However, this data is often in a form which is hard to use directly or integrate with the results from other complementary analytical techniques. For example with the multi-dimensional data produced by reflectance imaging spectroscopy, or even with other standard scientific imaging modalities. As a result, Macro-XRF data is often handled and processed separately and the results only made accessible in the form of exported image renderings. In this paper, therefore, we will examine how Macro-XRF data can be processed into a practical and re-usable form that is compatible with other imaging modalities and how this data can be made more easily available through an integrated platform for multi-modal imaging. Open source software will be presented which implements this architecture, showing how quantitative Macro-XRF data can be visualized interactively through a web browser, integrated with other imaging modalities and how the underlying quantitative data can be made accessible and re-used using open formats and standard protocols.

Implicit information exploration techniques are of great importance for the restoration and conservation of cultural relics. At present, the hyperspectral image analysis technique is one of the main methods to extract hidden information, which mainly contains two analysis methods such as principal component analysis (PCA) and minimum noise fraction rotation (MNF), both of which have achieved certain information extraction effects. In recent years, with the development of artificial intelligence, deep learning, and other technologies, nonlinear methods such as neural networks are expected to further improve the effect of implicit information mining. Therefore, this paper is oriented to the problem of extracting hidden information from pottery artifacts and tries to study and explore the hidden information mining method based on deep neural networks, expecting to obtain more stable and richer hidden information. In this paper, an auto-encoder-based implied information mining method is proposed first, and the auto-encoder (AE) framework achieves good performance in feature learning by automatically learning low-dimensional embedding and reconstructing data. However, during the experiments, it is found that some important detailed information (e.g., implicit information) is often lost in the reconstruction process because the traditional autoencoder network only focuses more on the pixel-level reconstruction loss and ignores the overall distribution. Therefore, this paper further proposes a multi-scale convolutional autoencoder network (MSCAE). It constructs a multi-scale convolutional module based on the traditional AE and designs a cyclic consistency loss in addition to the reconstruction loss, to reduce the loss of detailed information in the reconstruction process and improve the implicit information mining effect. In the experiments, we find that the proposed method can achieve effective implied information mining by extracting implied information from cocoon-shaped pots, and its visual effect has been improved compared with the traditional AE network.

We present an automated method for registration and mosaicking of multimodal technical images of artworks based on mutual information. We focus on the registration of element distribution maps resulting from macro X-ray fluorescence (MA-XRF) scanning, which can be considered as a layered stack and treated as the moving image. The target fixed image is the visible image of the same artwork. In consecutive stages, a unique, optimised transformation that provides the highest average mutual information across all images in the stack is identified with consensus. This transformation can be applied to the moving image to obtain the best alignment between the moving and fixed images when overlapped.

In the investigation, protection and restoration of murals, the exact location and size of fragmentation disease can be labeled to facilitate the subsequent protection and cultural heritage of murals. However, manual labeling is time-consuming and laborious, and the results will be various due to the different experience of experts, which is not conducive to the promotion of intelligent cultural relic protection and restoration. The intelligent labeling of mural diseases through artificial intelligence can greatly improve the efficiency of mural restoration and solve these deficiencies. Therefore, an intelligent labeling method for mural fragments based on gradient-trainable Gabor and U-Net is proposed. In this paper, the disease labeling problem is transformed into the image segmentation problem for disease regions. However, due to the rich texture of the mural image and the complex edges of the fragmentation regions, a lot of detail is lost for disease labeling directly using the U-Net network. Different from previous studies, this method uses gradient-trained Gabor in the encoders to extract texture features of fragmentation disease and obtain more texture information of the disease region. In particular, res2convolution is embedded into the skip connections to narrow the semantic gap between encoder and decoder and better inject the texture information of fragmentation disease into the deep network. Finally, we proved that the method proposed in this paper can realize the intelligent labeling of fragmentation diseases accurately and efficiently through the murals of Han Tomb at Xi 'an Jiaotong University.

Line-field confocal optical coherence tomography (LC-OCT) is an alternative to conventional OCT that combines OCT and confocal microscopy. This technique gives access to three-dimensional (3D) images with a micrometer resolution in the three spatial directions and enhances signal from the deepest layers within the material. After an experimental determination of the device characteristics, the technique is used for the investigation of16th to 18th century fragments of gilt leathers wall-hangings. In these objects, the various layers within the varnish can be identified and the effect of a restoration treatment can be observed to validate the varnish removal process.

Combining machine learning and physical optical models with advanced portable equipment system, we demonstrate the applicability of a new illumination-induced multispectral imaging system to the examination and detection of forgery in antique polychromic objects.

The moir´e method is a well-known tool in NDT; it is based on the principle of superposition of two sets of lines or gratings, which creates a moir´e pattern that can reveal surface deformations caused by underlying defects or damage. In this paper, we propose a simple artwork diagnostic using the moir´e method and a smartphone. The technique is based on coupling the acquisition of fringe patterns by the smartphone camera to an effective fringe generator. The fringe generator consists of a diffractive optical element (DOE) illuminated by a laser diode; this optical device proved to be very effective thanks to its ability to produce in a simple way grid patterns of different spatial frequencies. The smartphone camera is used to capture the grid patterns and to store them in the cloud. We demonstrate the proposed approach by giving some preliminary experimental results.

The colour of the ground layers of a painting has an influence on its visual appearance. In addition to the commonly used white ground layers, other colour ground layers have been used, for example, the grey ground layer used in Peter Paul Rubens’s painting Portrait of Clara Serena Rubens helps the colour transition of the skin tones. Understanding the effects caused by the colours of the ground layers is of significance for both technical art history and conservation. Optical non-destructive testing (NDT) techniques are useful tools for the investigation of paintings, for example, optical coherence tomography (OCT) can be used to study the surface and subsurface layers non-destructively. In this work, the interaction of light with paint and ground layers is modelled to supplement OCT measurements of paintings with ground layers. A previously described near infrared light range OCT system provides high spatial and depth resolution measurements. A four-flux model has been developed for analysing the light interaction in the paint and ground layers. This model considers forwards propagating collimated light, backwards-propagating collimated light, forwards-propagating diffuse light and backwards-propagating diffuse light. The model uses the optical material properties, including refractive index (RI), absorption and layer thickness, as input. This paper describes the construction of the model and an evaluation of its performance by comparison with OCT data.

The assessment of the structural condition of cultural heritage objects is important for conservation interventions and their long-term preservation. This investigation concerns The Night Watch (1642), a large-format 17th-century canvas painting by Rembrandt van Rijn that is on display in the Rijksmuseum, Amsterdam. This painting, which has a complex treatment history, has various damaged areas and has undergone three wax-resin relinings. In 1975 the canvas was slashed twelve times with a serrated dinner knife, including several long slashes in the area of Captain Frans Banninck Cocq’s breeches. In 2021, prior to a proposed new structural intervention involving retensioning of the canvas, it was important to evaluate the structural condition of the repaired slashes and of another repair, specifically an old canvas insert in the drum. For this, an in-situ inspection was carried out in the Rijksmuseum as a part of Operation Nightwatch. 3D shearography instrument with thermal loading was used to inspect these two areas of interest on the reverse of The Night Watch. The results showed that the out-of-plane strain in the breeches does not show any large deviations, which alleviated conservators’ concerns about the adhesion of the lining canvas and stability of previous repairs in this region. The patch in the drum showed higher out-of-plane strain variations. This was explained by the lower quality of the patched canvas compared to the repaired slashes in the breeches of Banninck Cocq. Overall, 3D shearography provided valuable inspection results for assurances regarding the structural integrity of the 1975 repairs and the wax-resin lining in The Night Watch, reducing the risks and providing the confidence to proceed with the planned retensioning of the canvas.

The surface is the most representative part of an artwork and it is also the part most exposed to alterations due to interaction with the surrounding environment. Non-destructive surface monitoring is of crucial importance in preserving and conserving cultural heritage and optical interferometric techniques allow to acquire the surface structure down to the submicrometric scale. In this work, we start from laser microprofilometry based on conoscopic holography sensors to unlock a new way of measuring the surface. In the last years, this technique has proven effective for surface diagnostic in heritage science providing high-quality surface dataset on diffusive, highly reflective, and polychrome artworks. However, an open problem in profilometry is the spatial referencing of surface topography at the micrometer scale, due to the lack of references in the height data with respect to the visually readable surface. We have recently developed a solution that exploits the raw intensity signal collected by the single-point sensor (i.e. the backscattered signal of the laser diode) and the interferometric height dataset, which are intrinsically registered. This method provides additional information about material texture, color variations or artist’s marks that enable spatial registration and data fusion tasks, otherwise difficult in traditional laser profilometry. In this paper we analyzed the feasibility and the performance of the whole process chain from the acquisition to the exploitation of the dual height-intensity datasets, focusing the attention on the raw intensity signal interpreted as a “raw reflectance signal”. We demonstrate the effectiveness of the proposed approach by presenting results on exemplary case studies.

In the engineering field, surface metrology is a valuable tool codified by international standards that enables the quantitative study of small-scale (down to micrometer) surface features, i.e., the surface topography. However, it is not recognized as a resource in heritage science. In literature we find a large use of qualitative inspection of surface morphology or of single-parameter roughness analysis, which confirms the need and potential of such diagnostics. Reasons of the gap are variegate; artworks are hand-made peculiar targets with heterogeneous surfaces, a multiscale approach is necessary, lack of guidelines and unclear meaning of surface roughness descriptors. We propose a critical-constructive discussion through Proof-of-Concept (POC) applications, on the use of surface metrology based on ISO descriptors. Exemplary case studies include: 1) In situ and in-process monitoring of painting microtexture in a Venetian masterpiece: wide and in-band roughness analysis is performed through the complementary use of amplitude, spatial, and hybrid parameters. 2) Multiscale roughness analysis for treatment monitoring in highly reflective metal artworks, requiring high micrometer accuracy in both depth (0.1 µm and lateral (5 µm) directions: surface analysis is performed on scale-limited components to discriminate different surface processes. Surface data are acquired using a prototype of a laser scanning profilometer based on conoscopic holography, with a versatile setup and a surface data pipeline tailored to artwork applications.

The use of neural encodings has the potential to replace the commonly used polynomial fitting in the analysis of artwork surface based on Reflectance Transformation Imaging (RTI), as it has proved to result in more compact encoding with better relight quality, but it is still not widely used due to the lack of efficient implementations available to practitioners. In this work, we describe an optimized system to encode/decode neural relightable images providing interactive visualization in a web interface allowing multi-layer visualization and annotation. To develop it, we performed several experiments testing different decoder architectures and input processing pipelines, evaluating the quality of the results on specific benchmarks to find the optimal tradeoff between relighting quality and efficiency. A specific decoder has been then implemented for the web and integrated into an advanced visualisation tool. The system has been tested for the analysis of a group of ancient Roman bronze coins that present scarce readability and varying levels of preservation and that have been acquired with a multispectral light dome. Their level of corrosion and degradation, which in some cases hinders the recognition of the images, numerals, or text represented on them, makes the system testing particularly challenging and complex. Testing on such a real case scenario, however, enables us to determine the actual improvement that this new RTI visualization tool can offer to numismatists in their ability to identify the coins.

Hyperspectral imaging has been increasingly used for non-destructive analysis of historical documents. Spectral reflectance data allow material identification and mapping using a library of reference spectra. Similarity metrics are crucial for quantifying the differences between reference and test spectra. Despite the apparent simplicity of the metrics, little work has been done on comparing their performance in the classification of historical inks. In this work, we propose three methods for selection of optimal spectral metrics, with an application to classification of historical inks. Hyperspectral images of laboratory and real historical samples are acquired in VNIR [400-1000 nm] and SWIR [900- 1700 nm] spectral ranges. Two spectral reflectance libraries are obtained (one for each range) including eight inks: iron gall, sepia, and carbon-based inks, and some mixtures. Six spectral similarity metrics are used: RMSE, SAM, SID, SIDSAM, NS3, and JMSAM. Firstly, metrics values in laboratory samples are studied to determine the classification confidence threshold of each metric. Then, the optimal metrics found for classification are selected using diverse approaches: (1) considering the confidence threshold; (2) evaluating classification performance metrics; (3) studying the probability of spectral discrimination and the power of spectral discrimination of each metric. Finally, inks of historical samples are classified by searching through the spectral libraries using optimal spectral metrics. Our method can correctly identify inks in both laboratory and historical samples in a simple and semi-supervised way.

Taj Mahal, made of exquisite white calcite, continues to deteriorate due to the emission of sulphur dioxide, methane etc. by industries and vehicular exhaust caused by the dense population in the region. Our previous collaborative works on samples with Pietra-Dura works already showed damages and irregularities including surface discolouration due to methane, water inclusions in the volume, and sub-surface cracks employing micro-Raman spectroscopy, broadband Terahertz Time Domain Imaging (THz-TDI) and THz Laser Feedback Interferometry (THz-LFI). Here, two types of samples having similar artwork, but one made of marble having high sulphur content have been investigated. Employing energy dispersive X-Ray analysis (EDAX), the sulphur content in the previous calcite sample is found to be nil while the new one has 16% by weight. While visually the samples are similar, under optical and scanning electron microscopy (SEM), calcite presents grainier structure with larger porosity while the other one appears denser with finer porosity. In ultra-low-frequency (ULF) Raman spectroscopy, the calcite sample (less than 0.15% Mg content) produces a significant line at 1100 cm-1 while the marble with sulphur shows a markedly different spectral response with the significant line at 1010 cm-1 . Using both THz-TDI and THz Continues Wave (THz CW) imaging, we concluded that calcite marble has significantly larger THz penetration even at 1 THz, while the marble with high sulphur content has very low THz penetration even below 0.5 THz and high THz absorption offering higher THz reflectivity. These observations pave the way to objectively detect the extent of environmental damage to marble structures across the globe.

Pulsed infrared thermography is applied to the study of a mold casting Chinese bronze lei 罍 dated to the late Shang dynasty (c.a.1250–1050 BC), currently housed in the Capital Normal University Museum. Many spacers and a defective area of this ancient bronze are partly covered with repair material. By analyzing thermographic images using a one-layer thermal diffusion model, it is found that the spacers were specifically made for this bronze. The thickness of the repairing material in the defective area is measured using thermal quadrupole modelling in multilayer materials. This is the first application of this method to the field of cultural heritage conservation. These results provide a deeper understanding of the manufacturing process of ancient Chinese bronzes from the viewpoint of archaeological research. They also help assess the repair status from the conservation viewpoint.

In recent years, the applications of hyperspectral imaging in the protection and analysis of cultural relics have received widespread attention. However, due to the limitation of imaging sensors, the spatial resolution of existing hyperspectral images is low, which hinders the development of hyperspectral digitization of cultural relics. Hyperspectral (HS) and RGB image fusion technology can generate hyperspectral images with high spatial resolution, which has gradually become a research hotspot. Inspired by the astounding performance of deep learning in various hyperspectral image processing tasks, this paper proposes a hyperspectral image fusion method based on dual-resolution fusion feature mutual guidance network (DRFFMG). Firstly, two feature extraction networks for HS and RGB images with different resolution pairs are designed to increase the richness of extracted features and reduce the loss of original hyperspectral information. Then, the spatial and spectral features extracted from the above feature extraction networks are fused, and a fusion feature mutual guidance module is designed to promote the mutual learning of different spatial features through information transmission, effectively reducing spatial distortion. Finally, the desired high spatial resolution HS image is restored from the fused features through an image reconstruction network. Experiments demonstrate that the proposed DRFFMG network can produce fusion images competitive with even better to state of the arts, and retain spectral information while improving spatial resolution.