Towards in-line real-time characterization of roll-to-roll produced ZTO/Ag/ITO thin films by hyperspectral imaging

Large area manufacturing processes of thin films such as large-area vacuum roll-to-roll coating of dielectric and gas permeation barrier layers in industry require a precise control of e.g. film thickness, homogeneity, chemical compositions, crystallinity and surface roughness. In order to determine these properties in real time, hyperspectral imaging is a novel, cost-efficient, and fast tool as in-line technology for large-area quality control. We demonstrate the application of hyperspectral imaging to characterize the thickness of thin films of the multilayer system ZTO/Ag/ITO produced by roll-to-roll magnetron sputtering on 220 mm wide polyethylene terephthalate substrate. X-ray reflectivity measurements are used to determine the thickness gradients of roll-to-roll produced foils with sub nanometer accuracy that serve as ground truth data to train a machine learning model for the interpretation of the hyperspectral imaging spectra. Based on the model, the sub-layer thicknesses on the complete substrate foil area were predicted which demonstrates the capabilities of this approach for large-scale in-line real-time quality control for industrial applications.

These authors contributed equally. * Authors to whom any correspondence should be addressed.
Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

Introduction
Coating is the process of applying a thin layer of material to a substrate. This can be realized by deposition of a liquid-solution, for example in a spin-coating process with solved metalorganics in solution, or vapor phases, for example in a sputtering process [1,2] or atomic layer deposition. Depending on the application requirements, substrate material, coating parameters and coating technology are selected. In large-area coating techniques, especially in industry applications, a fast, efficient and very accurate quality control is necessary but its implementation is challenging.
A widely used coating method is the vacuum roll-to-roll (R2R) magnetron sputtering process. It is used to coat plastic substrates with thin films with a thickness in the nanometer range. For example, it can be applied in manufacturing processes for packaging foils with a gas permeation barrier for protection against water vapor and atmospheric gases, to manufacture metallic foils for the production of conductive surfaces [3], lacquer coatings and laminations between two glasses for low E-coatings for heat reflection and energy saving, electrochromic smart windows [4], or for flexible perovskite solar cells [5,6]. Large scale industrial manufacturing of such functional coatings would benefit from an in-line quality control with real-time data analysis, requirements that can be addressed by application of hyperspectral imaging (HSI).
HSI is a method that collects a transmittance and/or reflectance spectrum from an illuminated object at each pixel in the camera image of a scene, hence it combines digital photography and spectroscopy. HSI is often performed in the visible and near-infrared spectral range (VNIR). HSI is routinely applied to many remote sensing usages, such as agriculture, waste sorting and recycling, surveillance, ocean monitoring, geology and astronomy [7][8][9][10]. It is a promising tool to access fast and accurate data during a manufacturing process in e.g. food, thin film and pharma industry for cost-efficient quality control [11][12][13][14][15][16][17]. Moreover, HSI has emerged in medical imaging, for example in detection of cancer tissue [18][19][20], Alzheimer disease biomarker [21] or ophthalmology [22,23] and further diagnostic approaches [24,25]. However, the commercial usage of HSI to characterize in-situ and in real-time large-area thin films during the production process still possess a major challenge. There is an inherent complexity in extracting the material's properties of interest such as e.g. the layer thickness, homogeneity, and crystallinity from the spectral intensity detected by each pixel.
There are two inherent challenges in the evaluation and use of HSI data: (i) the complex relationship between HSI spectra and sample properties requires physical modeling. Hence, evaluation of the measurements to obtain quantitative results by an expert is needed, i.e. large personnel effort has to be invested. (ii) Due to the large amount of data, especially for R2R and in-line applications, a high computational effort has to be made. This concerns particularly cases where physical models must be applied to analyze each pixel of an HSI image. Here, soft modeling approaches with model training and live applications on incoming raw data and fast conversion are the key solutions. Addressing and optimizing such approach can lead to revolutionizing HSI machine designs for quality management in very different areas in industry [26], and may replace or support costly and complex downstream analytics that often provide no in-line production control.
In this manuscript, we report an approach for large-area thin film quality control that has the potential for application in industrial manufacturing processes for automated inline thickness monitoring. We use x-ray reflectivity (XRR) to produce ground truth data of the layer thickness of a zinc tin oxide/silver/indium tin oxide (ZTO/Ag/ITO) layer system on polyethylene terephthalate (PET) manufactured by R2R magnetron sputtering in order to develop a prediction model for HSI spectra. We show how multimodal combination of XRR and HSI can be used to validate and refine a machine-learning model based on the partial least squares (PLSs) algorithm for prediction of film thicknesses.

Multilayer fabrication and sample preparation
ZTO/Ag/ITO thin films on PET with a nominal thickness of 70 nm, 5 nm, and 40 nm, respectively, have been produced at Fraunhofer FEP (Dresden) using a vacuum web coater [27]. Figures 1(A) and (B) show the foil and a single cut out, part (C) sketch the multilayer system. The individual layers are applied to an unheated 125 µm thick PET substrate (Melinex ST504, DuPont) by R2R magnetron sputtering of ceramic mixed targets with the nominal composition ZnO/SnO 2 (50.4/49.6 wt.%) and In 2 O 3 /SnO 2 (97/3 wt.%) in an Ar/O 2 mixture. The multilayers were produced over a width of 220 mm with web lengths up to 300 m. A part of the foil was extracted after the production process as shown in figure 1(A) and measured by HSI and XRR. For the XRR investigations, three pieces of the foil were cut out, laminated onto glass substrates (see colored areas in figure 1(A)) and each measured at five different positions by XRR. The XRR setup required a slightly different measurement geometry, i.e. rotation of the sample by 90 • , for the measurements at the sample positions backward and forward in winding direction (see SI for details (figure S1)). The HSI measurements were performed on the full substrate area at ∼200 mm width. In the following we

XRR measurements and data analysis
XRR is an established and widely used technique for investigation of structural properties such as layer thickness, roughness, and electron density. It provides electron density profiles ρ (z) perpendicular to the layers' surface/interface with high resolution on an angstrom length scale. In such experiments, the reflected intensity of the sample surface is measured as a function of the angle of incidence θ at specular condition, i.e. the angle of incidence is equal to the angle of reflection. Typically, the reflectivity data is presented as a function of the momentum transfer q z perpendicular to the surface given by q z = 4π λ sin θ with the wavelength λ of incident and reflected x-rays. The reflected intensity can be described by the first Born approximation [28] via Thus this method is highly sensitive to changes of the electron density perpendicular to the sample surface ( dρ(z) dz ) which allows to determine layer thicknesses of thin multilayer samples with highest precision. Consequently, XRR can be used to provide ground truth data for the training of machine learning models for the interpretation of HSI data. The HSI approach then can be applied to analyze lateral thickness gradients on a large scale.
The XRR data were measured at beamline BL9 of the synchrotron radiation source DELTA (Dortmund, Germany) [29,30] using a photon energy of 8.048 keV which corresponds to wavelength of 1.54 Å with a beam size set to 1 × 0.1 mm 2 (h × v). A Pilatus 100K area detector (Dectris, Switzerland) was utilized to measure both, the reflected intensity and the diffuse scattering for background subtraction simultaneously within a q z range between 0.014 to 0.28 Å −1 . The area probed by each XRR measurement is given by 1 mm perpendicular and on average about 8 mm parallel to the web winding direction which corresponds to approx. Six pixels and sixtyfour pixels of a hypercube in the HSI camera, respectively (see next section).
Each of the three laminated ZTO/Ag/ITO-covered PET samples (cut-outs) was investigated at five different positions on the samples, both perpendicular and parallel to the transport direction of the production process; please refer to the supplementary information (SI) figure S1 for more details. For data evaluation the XRR curves were normalized by the intensity of the incident radiation and then corrected by subtraction of the diffuse scattered intensity. Finally, the reflectivity data were evaluated with a combination of the Parratt algorithm [31] and the effective density model [32] using the program package LSFit [33]. To minimize the number of fitting parameters, we initially assigned the literature electron density values [34] of ITO, Ag and ZTO to 1.91 e − Å −3 , 2.771 e − Å −3 and 1.642 e − Å −3, respectively, as starting parameters. Latter refers to a 3:2 Zn:Sn-ratio. The electron density of ZTO was determined from a separate XRR analysis of a ZTO single layer sample to 1.62 ± 0.05 e − Å −3 and was fixed during the refinement in order to reduce the amount of variable fit parameters. The density of the PET substrate was set to the calculated value of 0.44 e − Å −3 and kept constant during the refinement while its roughness was fitted in the initial refinement loop and then fixed. We applied a three-layermodel to analyze the layer system on the PET substrate. A set of parameters was determined for each layer: the roughness (σ), the thickness (d) and the electron density (ρ). The experimentally determined reflectivity curves together with the fit results are shown in figure 2(A) exemplarily for the central part of the three different cut-outs that are representative for a potential thickness gradient perpendicular to the transport  direction. One clearly observes that the reflectivities differ particularly regarding the oscillation period between the center sample and the ones extracted from the rims (left, right) of the foil. This is even more evident if the corresponding density profiles obtained from the fitting scheme are compared (see figure 2(B)) which show a significant decrease of the layer thickness of all sublayers to the rim of the foil. In web winding direction the sample is homogeneous as the XRR curves and density profiles hardly change (see figures S2-S4 in the SI). The thickness gradients of the sublayers are compiled in figure 3 including all measurements. The open symbols show the thicknesses determined parallel to the web winding direction which supports the thickness variation in that direction being much smaller. In contrast to the ZTO and the ITO layers, the density of the Ag layer is reduced by about 24% compared to the nominal Ag density. This indicates that the Ag layer is not fully closed but may show pore-like structure. All extracted fit parameters are given in the SI (table S1). This data can be used as ground truth data to train and validate the machine learning models for interpretation of the HSI images and is applied in the following to the prediction of the layer thickness.

HSI
The HSI measurements were performed at the Fraunhofer IWS (Dresden, Germany) using a Headwall VNIR HSI camera (Headwall Photonics, USA) with 1004 × 1002 sensor pixels, together with a Schneider Xenoplan f /1.4 23 mm objective at an operating distance of 250 mm to the foil's surface resulting in 100 mm field of view. The wavelengths detected by the camera range from 400 nm to 1000 nm.
A two times spatial and a four times spectral binning was used resulting in an effective pixel size of the camera of around 170 µm and a spectral resolution of around 4 nm. The foil was illuminated using a diffuse illumination system (Fraunhofer IWS, Dresden, Germany) with halogen lamps. The measurements were performed with the HSI software Imanto ® pro (Fraunhofer IWS, Dresden, Germany) with a frame rate of 30 Hz and an exposure time of 25 ms. This exposure time at a given web speed of approximately 2 m min −1 results in a foil translation of less than 1 mm in web winding direction during one HSI measurement which determines the spatial averaging. This is in good agreement with the spatial resolution provided by the XRR measurements. Figure 4 shows a schematic drawing and a photo of the used HSI system.
To avoid irregularities in the lighting and to eliminate the influence of dark current, a white and a dark correction for each wavelength was carried out for the VNIR HSI measurement according to equation (1), where I c (λ) is the corrected image intensity, I o (λ) the original image intensity, I d (λ) is the dark current recorded with the light source switched off and the lens covered, and I w (λ) is the intensity of the white reference for the wavelength λ. For the white reference, a plate of optical polytetrafluorethylene was scanned under the same measuring conditions as the original image. Since the specimen under study is a transparent film, a highly absorbing material (MaxiBlack Foil, Acktar Ltd, Israel) was placed under the specimen for the measurement to minimize a possible background effect. The measurement of the foil was performed before cutting it for XRR sample preparation. Figure 5 shows the result of the HSI measurement of the complete sample in terms of the color-coded reflectivity at a wavelength of 700 nm. The areas probed by the XRR measurement and used for evaluating the ground truth data are indicated with blue stripes, each corresponding to around 100 spectra.
To obtain the training data for machine learning, the spectra in the marked stripes were selected and a mean spectrum was calculated. Because of the sample rotation in the XRR setup for selected sample positions in web winding direction, the orientation of the footprint area differs for these measurements as indicated in figure 5. However, inspection of the results presented in figure 3 shows that the layer thicknesses are not significantly affected by the footprint orientation so that we considered all data to train the model. Then, for each stripe, the most similar spectrum to the respective mean spectrum was selected. Figure 5 (right) shows the 15 spectra selected in this way. These 15 spectra and the corresponding layer thickness values from the XRR measurement could now be used in the next step for training machine learning regression models.

Training and optimization of the PLS regression model for HSI thickness prediction
In the next step, a regression model was trained with the selected HSI spectra and the thicknesses for the Ag, the ITO and the ZTO layer determined from the XRR measurements in order to subsequently predict the layer thickness for the complete sample. The PLSs algorithm was chosen as the regression algorithm [35]. PLS is an algorithm used for regression analysis and dimensionality reduction. It is a supervised learning technique that finds a linear relationship between a response variable and one or more predictor variables by constructing a set of orthogonal latent variables that explain the maximum possible variance in both the predictor and response variables. PLS is commonly used in fields such as chemistry and biology to analyze complex data sets and to build predictive models. Unlike other regression techniques, PLS can handle data sets with collinearity, missing values, and unbalanced design and it is particularly useful for modeling relationships between a response variable and a large number of predictor variables.
Before training the PLS model, optional preprocessing of the spectra by L1, L2 as spectrum division norm or standard normal variate normalization was performed. The type of preprocessing, as well as the number of latent variables (between 1 and 15) for the PLS were performed by automatic hyperparameter optimization using the random search algorithm [36]. For the hyperparameter optimization and the validation of the obtained models, a nested leave one out (LOO) cross validation was performed. To determine the quality and to compare the obtained regression models, the root mean square error (RMSE), the relative RMSE (rRMSE) and the coefficient of determination (R2) were analyzed. The RMSE is one of the most commonly used measures for evaluating the quality of predictions and shows how far predictions fall from measured true values using Euclidean distance. The determination was made for each of the outer cross-validation results. All model calculations were repeated three times and the mean and standard deviation for the metrics were determined. The calculations were performed with Python 3 and the library Scikitlearn [37].

Results and discussion
The results of the PLS regression model for the thickness of the three layers of the sample are shown in table 1. The results for the Ag and the ITO layer are comparable and provide an accurate prediction of the layer thickness confirmed by a low RMSE, rRMSE and a high R 2 . In contrast, the predictions of the ZTO layer thicknesses are worse, with an rRMSE more than double and an R 2 of only 0.515. This is also reflected in the regression plots of the three models (see figure 6). The ZTO model shows the poorest fit, while the other models exhibit a remarkable correlation between the ground truth and the predicted thickness values. In order to assess the quality of the underlying ground truth data, error bars of the measured layer thicknesses are shown.
The results for the ZTO prediction deviate from the desired fit much stronger than for the other layers.
We trace this back to an increased model error due to the relatively large error of the ground truth data regarding the thickness determination by fitting the XRR curves. This is accompanied by a smaller variation in the training data. Most of the 15 positions investigated show the same ground truth layer thickness of about 68 nm. This makes a reasonable prediction in case of the ZTO layer questionable. It should also be noted that the differences between the HSI spectra of the different areas are relatively small and the spectra also have a low signal-to-noise ratio, which also complicates the training of the regression models. In addition, the measurement of the ZTO layer is complicated by the strongly deviating reflectivity  of the overlying Ag layer. Furthermore, the thicknesses of the three layers are strongly correlated with each other (see figure 3) and only a small thickness range is covered for all three coating materials. It can therefore be assumed that the models obtained cannot be generalized to arbitrary layer thickness combinations of ZTO/Ag/ITO. As only a few data points and no independent test samples could be examined in the present experiment, it can be assumed that the PLS model is overfitted which prevents a final conclusion on the predictive accuracy of the model for completely new samples.
Nevertheless, the models obtained were used to predict the thicknesses for the entire foil sample. For this purpose, the PLS models were applied to all spectra of the HSI measurement and the predicted film thicknesses were color coded. The results are shown in figure 7 (bottom). The same irregularities can be observed in all three images. These are artifacts attributed to substrate imperfections like surface contaminations and dust before coating. In the top part of the figure we present cross-sections perpendicular to the web-winding direction at different y-positions of the predicted image in order to compare the predicted layer gradients for the position where the ground truth data were measured (111 mm) with the predictions at other parts of the foil. Again, the strong correlation of the sub-layer thicknesses can be recognized, with the layer thickness distribution for the ZTO layer differing slightly from the distribution of the Ag and ITO layers. However, the model is capable to provide reasonable thickness gradients for the complete sample. Despite the deficiencies on predicting the ZTO layer thickness with highest accuracy compared to the Ag and ITO layers, the detection of film thicknesses using HSI over a foil width of 220 mm for such a complex layer system is remarkable. For evaluation of the HSI data, analysis of one spectrum using the trained prediction model is in the range of some milliseconds. The prediction of the whole image takes below 5 s which provides reasonable reaction times in case of production failure. These results validate the capabilities of the presented approach for accurate on-line thickness control of multilayer coatings although a larger ground truth data base is required which needs to cover large thickness gradients for a generalization of such models to various layer systems.

Conclusion
In summary, we conclude that the presented method is well suited for the complete and fast thickness prediction of thin film samples via HSI using models that are trained using a relatively small amount of ground truth data obtained by XRR measurements. In future, a combination of HSI measurements and XRR as ground truth data source could provide a rapid thickness prediction for various materials by using libraries of pre-trained machine learning models. The present approach is especially suited to determine thickness variations of large-area samples which is of utmost relevance for industrial production processes. In fact, we have recently succeeded to model and predict film thicknesses in the sub-100 nm range for a web width of 300 mm and a web speed of 2 m min −1 . The concept could also be extended to other layer properties that are accessible by XRR such as density or roughness or different type of ground truth data e.g. the crystallinity and phase composition of thin films determined by x-ray diffraction.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).

Acknowledgments
The research has been carried out within the NanoQI project which has received funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreement No. 862055. We thank Stefan Jacobs (Bruker AXS) for discussions and training concerning XRR evaluation strategies. DELTA at TU Dortmund is acknowledged for providing synchrotron radiation at beamline BL9.

Contributions
S C, F G, P W, M T and C S designed the research. All ZTO/Ag/ITO samples were produced and prepared at Fraunhofer FEP with support by P S. P W and C M implemented the HSI camera and the local XRR system used for on-site thickness determination, respectively, and provided user support. F G and P W performed the HSI measurements. S C and P S performed and analyzed lab XRR measurements on ZTO and ITO reference single layers. S B, S D S, C A, E S, N T, M P and C S performed the XRR measurements at beamline BL9 of DELTA. S B analyzed the XRR measurements with help of S D S and M P. F G and P W analyzed the HSI spectra, trained, optimized and validated the machine learning models. S D S, F G and C S wrote the manuscript with contributions from all coauthors.