On the reproducibility of extrusion-based bioprinting: round robin study on standardization in the field

The outcome of three-dimensional (3D) bioprinting heavily depends, amongst others, on the interaction between the developed bioink, the printing process, and the printing equipment. However, if this interplay is ensured, bioprinting promises unmatched possibilities in the health care area. To pave the way for comparing newly developed biomaterials


Introduction
Three-dimensional (3D) bioprinting is attracting widespread interest due to the possibility to manufacture customized artificial tissues in regards to individual patient treatment, designing models for medical studies or organ-on-a-chip applications [1,2].For these products to be approved by authorities for medical application or clinical studies, high reproducibility and robust processes will inevitably be a challenge.The safety of the products should be guaranteed by standards and norms.In the field of bioprinting, there is still a need for universally applicable guidelines and standard operating procedures (SOPs).These should be included from the very beginning of the production of bioinks through to process analytical technology strategies and standardized analytical methodologies.In order to be able to compare, for example, bioinks or bioprinter setups in interlaboratory tests or prepare technology transfers, standardized methodologies are a prerequisite.Due to the missing guidelines, research groups develop own expertise and there is hardly any exchange of information between groups.This said, the use of interlaboratory data bases as advocated by the research community is still in its infancy.In this regard, there have been advances in the field of bioprinting, where the Kadi4Mat infrastructure was used for process design, documentation, data storage, and exchange [3].Looking at other related research areas such as tissue engineering, the International Organization for Standardization (ISO) has already constituted a technical committee (TC) for tissueengineering of medical products, i.e.ISO/TC 150/SC 7, aiming to implement relevant standards for testing and manufacturing methods.Therefore, it is only a matter of time before such subcommittees for bioprinting form.At the local level, for example, policy committees have already been formed as part of the Verein Deutscher Ingenieure (VDI), the Association of German Engineers.The VDI guidlines committee (Richtlinienausschuss 5708) aims for the definition of basic terminology, device requirements, and bioink testing methodologies.The development of robust protocols and standards is of crucial importance for a successful clinical translation of biomaterials and bioprinting as has been mentioned in literature [4].
With regard to future application in clinical studies, there are some process steps which would benefit from standardization.The first aspect is the production and characterization of newly developed bioinks which consist of biomaterials, cells, and other additives [5,6].It is, for example, not an easy undertaking to homogeneously introduce cells into the highly viscous solution to complete the bioink because air bubbles present a common challenge [7][8][9].One suggestion is to introduce the cells above the gelation temperature of the solution [10].The second aspect includes the printing process itself and the equipment of the bioprinters.There are different techniques for extrusion-based bioprinters, such as pneumatically operated or piston-driven methods [11,12].In addition, there are bioprinters where the cartridge or nozzle can be tempered including different principles, such as the use of a cooling agent or thermoelectric control.Another feature that not every bioprinter is equipped with is the automatic calibration of the coordinates of the nozzle tip which can have an influence on the printing result and thus on the reproducibility.
To ensure the quality and reproducibility of bioprinted structures, the performance of bioprinting evaluation is already being discussed [13,14].Filament collapse, fusion and grid tests are three commonly suggested evaluation methods.These evaluation methods are useful for both developers and users of bioprinting processes.The filament collapse test evaluates the printer's ability to deposit a filament on a platform with pillars at increasing intervals without the strand collapsing [15,16].This is important for the production of stable and reproducible complex structures.The yield stress of the material used can be derived here, which plays an important role in the design of bioinks as it defines the shear stress for initiating the material flow.The filament fusion test evaluates the printer's ability to deposit filaments meander-like at different distances, which can be used to determine the accuracy of the print test [17,18].The grid test evaluates the accuracy and precision of the printer by printing a grid pattern.The distance between the lines, pore diameter, pore geometry and the crossing point of struts are analyzed here for assessment [19].Special attention should be given to the methodologies for quantitative analysis as the printed objects are evaluated and the outcome is translated into numbers.Image analysis (IA) is a suitable tool as it is fast, non-invasive, can be automated and, thus increases objectivity.Currently, there are still publications where 'the printability was judged by eyes' [20], or the images were cropped, and subsequently analyzed manually by a person [21].This is a time-consuming procedure and dependent on the subjective impression of one observer only.Furthermore, acquired photographs were 'analysed in ImageJ where the width of the strand was measured at multiple locations and averaged' [22].Hence, the number and location of measurements are variable and depend on a subjective decision by the observer.There have been advances toward automation of IA, however, the study still includes several manual steps for detection and cropping of artifacts [23].In conclusion, the studies listed above show user-dependent methodologies based on subjective decisions which limit the overall comparison.So far, cell analytics in the field of bioprinting is also lacking standardization.Microscopy is a commonly used method for investigating cell viability and morphology, however, this method is user-dependent as well due to manual focusing and manual counting of observations [24,25].A more advanced alternative for cell analytics is offered by raman imaging, which has been used for the analysis of cells within 3D printed objects [26].Even though this topic is out of the scope of the present study, in which no cells were used, it should be standardized as well.Throughout this work, we use the term ink to refer to the applied polymeric solutions in our studies as all experiments were cell-free.
To date and to the best of our knowledge, no studies of the reproducibility and comparison of bioprinting experiments in multiple laboratories have been reported.In this round robin study, we tested how consistent printed-geometry results can be obtained with 12 laboratories cooperating.Therefore, an infrastructure for central distribution of SOPs with information regarding the preparation procedure for inks, and parameters for printing experiments was established.Additionally, the consumables and biomaterials were distributed by the organizing laboratory.A device for standardized image acquisition was developed and used within the round robin experiments.In total ten prototypes of this so called Bioprinting Fidelity Imager (BioFI) were fabricated and used by different laboratories to record images of the printed objects.For quantitative assessment of the recorded images automated IA was included in order to extract features from the images of the printed objects.Finally, based on the experimental results, several factors that can be improved to increase reproducibility have been identified in order to make bioprinting more reliable and reproducible for future medical applications.

Round robin workflow and design
The round robin test was designed and performed to provide a general overview of to which extent bioprinting experiments concur in different laboratories.A compact schematic of the workflow of the project is depicted in figure 1.As already suggested by the name of the project SOP_BioPrint, SOPs for the individual stages were developed in advance and distributed to all participating entities by the organizing laboratory.

Biomaterials, labware, and geometries
In order to achieve a high degree of standardization, the organizing laboratory provided the participating laboratories with the biomaterials, labware, and the files of the designed geometries including orientation to be printed.Three different polymeric solutions were used, two of which were provided by research institutions including preparation instructions.Alginate was provided by the Department for Functional Materials in Medicine and Dentistry from the University of Würzburg (Würzburg, Germany) with instructions for preparing a 4% (w/v) alginate ink in ultrapure water.The properties of this biomaterial in bioprinting is summarized by Karakaya et al [27].The biomaterials for the gelatin-based ink and guidelines on its preparation were provided by the Department of Functional Surfaces and Materials at Fraunhofer Institute for Interfacial Engineering and Biotechnology (Stuttgart, Germany).The methacrylation of gelatin and its subsequent use as an ink was performed according to Wenz et al [28].Briefly, the ink consisted of 12% (w/w) functionalized gelatin, i.e. a mixture of 7% (w/w), and 5% (w/w) gelatin with degrees of methacrylation of 0.62 mmol g −1 , 0.82 mmol g −1 , respectively, and 0.84% (w/w) of LAP photo initiator dissolved in Dulbecco's phosphate-buffered saline.The commercially available Cellink Bioink was used as a third ink and purchased from Cellink (Gothenburg, Sweden).This alginate nanocellulose Figure 1.Illustrative scheme summarizing the workflow in the round robin tests.Biomaterials, labware, and geometries as well as detailed standard operation procedures (SOPs) regarding the preparation and use of materials were distributed.The Round robin-3D printing was performed in 12 independent laboratories and comprised the 3D printing experiments and documentation.Subsequently, images of the printed geometries were acquired using the Bioprinting Fidelity Imager (BioFI) system.The Kadi4Mat platform was used for central data exchange and storage.The Round robin-image analysis was divided into two parts.A qualitative evaluation of the images was performed by the organizing laboratory and extraction of quantitative geometric features was done individually by three IA groups independently.Statistical analysis and data visualization in the presented study were performed by the organizing laboratory.bioink [29] was delivered as ready-to-use cartridges and procured from a single batch.Labware used in the printing experiment consisted of a single-use conic 25G (0.25 mm inner diameter) nozzle from Cellink and six-well glass bottom plates from iBL (Gerasdorf, Austria).An SOP specified the procurement details (part number, manufacturer, distributor) for well plates and nozzles.Additionally, the STL files, or in case of problems during slicing, the sketches of the objects were provided by the organizing laboratory.A larger sketch of the geometries is presented in the supplementary section.The first model was a pattern of three parallel 25 mm long lines with a distance of 2 mm between the lines.These were to be printed in parallel to the short edge of the well plate.The second object was a circle with a diameter of 12 mm.The first and second models both consisted of a single layer.The third model was designed as a rectangle with two layers.There were two meandering lines printed in two separate layers rotated by 90 • .To create a closed square, there was a start up line with the length l = 5 mm in each layer.The fourth and last model was a stack of several layers of the scaffold, resulting in a total height of 1.2 mm.The start up filaments of both structures had to be printed oriented to the top left corner of each well plate.

Round robin-3D printing
The experimental design in the Round robin-3D printing test is summarized schematically in figure 2. The twelve participating laboratories located in Germany are listed alphabetically below:  ).Line, circle, and rectangle (n = 3) geometries were printed using three different inks (n = 3).Additionally, a three-dimensional (3D) scaffold was printed using Cellink Bioink (n = 1).As the experiments were performed without cells, independently prepared inks from the same batch were considered as analogs of biological replicates.In the case of Cellink Bioink, the use of an independent cartridge was considered as a biological replicate.For each biological replicate (n = 3), the ink was used in order to print six samples as technical replicates (n = 6).The resulting data sets contained 180 images for each individual laboratory.Exemplary images of the printed geometries with each ink are provided in the supplementary section.
• Institute of Pharmacy and Food Chemistry, University of Würzburg, Würzburg.The biomaterials for gelatin-based ink and alginate were distributed as dry materials by the organizing laboratory.Both inks were prepared independently as three batches prior to the printing experiments in the laboratories of the participating entities.The commercially available Cellink Bioink was purchased as specified in the SOP.Three readyto-use cartridges containing the ink were used.As the 3D-printing experiments were performed without cells, independently prepared inks from the same batch were considered as analogs of biological replicates.Six samples of each structure, e.g.line, circles, and rectangle, were printed using each ink (i.e.technical replicates).Additionally, a 3D scaffold was printed using Cellink Bioink as six samples of the structure (i.e.technical replicates).This resulted in a number of 180 objects per laboratory and in a total number of 2160 for the entire Round robin-3D printing test.For the printing process, a window of operation regarding printing parameters was set by the organizing laboratory in order to print filaments in a similar metric range.These printing parameters were dependent on the used inks.They are listed in table 2. For the case of the mechanically driven printers, the printing parameter was the axial speed of the piston.Therefore, a target width of 1 mm should be met.Once a parameter was set, it should not be changed over the complete series of printing experiments.Moreover, a standardized record was filled by each participating laboratory for documentation [3].After the printing process, the well plates containing the six objects of a single structure were imaged externally.The BioFI system was specially developed for this purpose by the Laboratory for MEMS Applications, Department of Microsystems Engineering of the University of Freiburg (Freiburg, Germany) and Hahn-Schickard-Gesellschaft für Angewandte Forschung e.V. (Villingen-Schwenningen, Germany).Imaging equipment and method are specially adapted to polymeric solutions showing low contrast to the background.Simply speaking, the BioFI consisted of calibrated digital image sensor with fixed magnification optics and illumination, that enables acquisition of microscopy images in bright field and dark field mode.The whole equipment works as a stand alone device controlled by an embedded Linux system.The BioFI contained a bracket to hold the well plate in a proper position.Size scales were integrated below each single well for size determination.
Imaging settings including lighting and exposure time were standardized and could not be modified by the operator.Printing records and acquired images were uploaded in the central database [3].

Central data exchange and storage
The distribution of SOPs and storage of documentation of the printing experiments of the Round robin test was implemented in a research data management system named Kadi4Mat [30].Furthermore, all images were uploaded systematically so that, in combination with the relevant records, every step can be retrieved and analyzed at a later point in time.This systematic data storage was not only used for the purpose referred to, it simultaneously fulfilled the function of an electronic lab notebook and proved to be important to the development toward digital laboratory.The detailed use of the Kadi4Mat database used in this study is described by Schmieg et al [3].After completing the experimental part in the laboratories, the analysis of the images of printed structures was divided into two parts.Hereby, the database allowed the quantitative analysis of objects using automated IA with the target to extract geometric features of the printed structures and to store them systematically again.

Round robin-image analysis
A qualitative analysis was performed by the organizing laboratory in order to explore possible challenges which may complicate an automated evaluation and the extent of occurrence.To do so, two independent observers studied all images and classified them into categories.The categories used were offset position, orientation of structure, additional paths, non-continuous filaments, material excess, off focus, and weak contrast.In addition to the qualitative analysis, a quantitative assessment was carried out independently from the qualitative analysis.Three IA groups were included in the study for the quantitative assessment of the printed structures.Throughout this study, group identification numbers are used as IA Group 1, IA Group 2, and IA Group 3 regarding IA.The academic entities are given alphabetically in the list below: This study was designed as a randomized and doubleblinded multicenter study.The IA groups had access to the submitted images, and were able to freely develop a workflow to extract sets of parameters.It is noteworthy that the three groups were chosen to have different backgrounds and have not worked previously in the field of bioprinting or tissue engineering to avoid any bias.The backgrounds of the chosen groups were active vision where sensor data is processed and reacted to, application-oriented information including process automation, and development of process systems engineering concepts, among other things, also for biological processes.The extracted geometric features include but are not limited to determinations of line width and length, circle inner and outer radius, and circle gap size.To ensure a nonbiased analysis, the IA groups did not receive any information about the categorization performed in the qualitative analysis.All images were analyzed and the data sets were extracted using the three IA methodologies as presented below.
The first IA process consisted of four consecutive steps.The same general four-step process chain was used for all patterns, but the specific implementation of each step may differ between the lines, circle, and rectangle patterns.A manual preliminary step rotated all images by 0 • , 90 • , 180 • and 270 • in order to bring them into the expected orientation as specified in the STL template.Step (I) of the automated process chain estimated the location of the printed structure based on a search for geometric primitives.For the circle and rectangle structures, the inner contours of the circle and of the large square cavity were used, respectively.A fixed ROI (region of interest) was formed around the calculated coordinates.For the lines structure, the small crosses of the substrate were used instead as lateral delimiters of the ROI.Step (II) consisted in tracing the contours of the printed structures in the estimated ROI.For the lines structure and the circle structure, both contours of each line and of the circle were extracted.For the rectangle structure, only the inner contours of the four cavities were traced.For the circle and rectangle structures, the nominal position of the extracted contours was defined by the pattern position detected in the previous step.For the lines structure, the six strongest step edges in a vertical projection profile of the ROI were used as reference contour locations.
Step (III) analyzed the contour traces and removed parts that are classified as artifacts, e.g. from dust particles, rather than actual printed contours.Step (IV) calculated quantitative feature metrics.The printed line width and circle width were calculated as the distance between corresponding contour pairs, measured horizontally, and radially.For the rectangle structure, the areas of the square and rectangular cavities were calculated as numbers of pixels inside the cavity contours, if the contours were closed.Steps (I) and (II) were implemented in Visual C# 9.0 (Microsoft, Redmond, USA) using the Matrox Imaging Library MIL X 22H1 (Matrox Electronic Systems, Montreal, Canada).Steps (III) and (IV) were implemented in MATLAB® R2022a (TheMathWorks Inc.Natick, USA).
The second IA workflow started with determining the conversion factor from pixel to millimeter.Locations of the size scales which were integrated in the pictures were obtained by correlating binarized reference images and the input grayscale image.This factor was determined by summing up the rows or columns belonging to the 10 mm scale.Thereby, another reference image containing the printed geometry was drawn with the measurements of the construction sketches.The ROI for each geometry just as printing angle for lines and scaffolds were determined by rotating the reference and calculating the highest correlation to the input image.Contour detection was performed using a canny edge detector returning a binary edge image.Next up, small impurities were filtered by deleting all contours with a smaller area than a given threshold.To single out each strand of the line geometry individually, the image was subdivided.If necessary, the recognized edges were closed by active contours creating the segmentation outlines of the printed geometry, allowing images for each individual contour.A full segmentation as well as images of each scaffold hole were preserved by filling the contours with a flood fill procedure.These images enabled the calculation of different geometry characteristics such as area and line width.For the area calculation, all pixels of the filled segmentation were summed up.The first step of the line width computation was the partition of the segment outlines into two matching contours by the medial axis of the geometry.Circle geometries were divided to an inner and outer contour whereas line geometries were separated to left and right.Scaffolds were split to multiple parts, resulting in several upper and lower as well as right and left relating contours.Afterwards, the smallest distance from each pixel of one contour to all pixels of the matching contour was computed and stored as an array.As a last step, the mean and median line widths of the complete geometry were calculated using the mean and median values of the entire array.Additionally, the results were transferred into the metric system by multiplication with the conversion factor.The program was implemented in Python 3.8.13(Python Software Foundation, Delaware, USA) using the libraries OpenCV 4.6.0,NumPy 1.22.4,and Scikit-image 0.19.3 [31].
The third IA approach for evaluating print quality consisted of four main steps: (I) segmentation of the printed geometry, (II) edge detection, (III) matching of the target geometry, and (IV) the evaluation of its quality.For the segmentation step (I), the scaled geometry and blank images were denoised with guided filtering [32] under self-guidance, resulting in edgepreserving noise reduction.KAZE features [33] were then computed and the blank image was aligned with the image of the printed geometry.By subtracting the aligned blank image from the image of the geometry, the marks on the plate and the backlight were removed.For edge detection (II), the gradient of the segmented geometry image is calculated.Local minima were reduced in order to be able to detect edges with the watershed algorithm.For this purpose, the 0.9 quantile of the gradient distribution was calculated.This 0.9 quantile was then used as a threshold for an H-minima transformation, where all local minima with a depth below this threshold were removed.Subsequently, the edges were extracted as watershed ridge lines, and small artifacts were removed by area opening with a minimum area of 0.25 mm 2 .The next step was to match the given geometry templates to the edge image (III).For this purpose, points on the inner and outer edges of the geometries were sampled.On the edge image, edge pixels of parallel lines with distances between 0.25 mm and 2 mm were selected.This selection was done by finding the maximum response of a Frangi filter [34].Finally, the geometry template was aligned with the selected edge pixels by kernel correlation registration [35], and all edge pixels not connected to the region covered by the matched template were removed.Finally, for the quantitative evaluation of the printed structure (IV), the width of the detected structure was measured for the line and circle geometries.Missing segments were defined by zero width.For the rectangle and scaffold geometries, a flood fill was seeded to the expected center of the hole, and the centroid, as well as the covered area were measured.The program was implemented in MATLAB® R2022a used with the libraries Signal Processing Toolbox 9.0, Image Processing Toolbox 11.5, Statistics and Machine Learning Toolbox 12.3, Global Optimization Toolbox 4.7, Computer Vision Toolbox 10.2, and Parallel Computing Toolbox 7.6.
The data sets extracted by the IA groups were evaluated by the organizing laboratory.Outliers were detected and compared to the corresponding image.Only for the case of outliers arising from artifacts not recognized as such by the image processing workflows, the corresponding data was not used for further analysis.In terms of metrics regarding the printed structures, line and circle width were used in this study for the purpose of clarity.For the calculation of both parameters, filament width of each printed geometry were determined at several point along the complete structure.Data provided below is the mean and the corresponding standard deviation of the mean width of single printed geometries.Both parameters were used for the calculation of the percentage coefficient of variation (CV) as the ratio of mean value to the associated standard deviation.A schematic representation of the two parameters is shown in the supplementary section.

Data handling and visualization
Data processing and evaluation including the calculation of mean and standard deviation as well as CV of the data sets, and data visualization were performed with MATLAB® R2022a.

Round robin-3D printing
The experimental part of the round robin study was conducted in twelve independent research institutions.After completion of the printing experiments, each laboratory submitted the acquired images and reports on Kadi4Mat.Thereby, support from the organizing laboratory was provided during data submission if required, as the project participants used this platform for the first time.It is noteworthy to mention that the experiments were not performed simultaneously as only a limited amount of BioFi imaging systems were used.The experimental part took place in a time frame of five months.

Round robin-image analysis: qualitative
After having completed the practical part of the elaborate Round robin-3D printing test, a visual inspection of images of printed structures was performed to get a first impression of the data quality.Some deviations from the specifications were found and sorted into five categories, and a possible root cause of the deviation was defined.However, not every deviation can be clearly traced back to a single cause, which is why there are some overlapping causes.The results are shown in figure 3 by exemplary images.
Category C1 includes samples where the orientation was different from the original STL file or samples with an offset position in relation to the center of the plate.Images that qualified for this category showed one of both of the named conditions.As can be seen in figure 3(A), the line structures were deposited in a horizontal orientation, i.e. parallel to the long edge of the well plate, whereas the intended STL files contained structures aligned vertically.Category C2 contains printed structures showing additional paths other than the expected structure.Figure 3(B) shows the circle geometry with an excentric strand connecting the middle of the circle and the circumference.Both categories were probably produced by mistakes in the printing setup, i.e. software and printer.Category C3, and Category C4 are comprised of samples with non-continuous filaments, and material excess, respectively.Category C3 can be caused by a printing system that cannot keep the set pneumatic pressure stable or it can be the product of nozzle clogging.Corresponding to this category, figure 3(C) exemplifies an intermittent line structure.Category C4 can arise due some printer equipment not having the feature to set a defined temperature leading to a lower viscosity of the ink.Similarly, a further origin of issues concerning this category can be an excessively high pneumatic pressure applied by the printer equipment.Figure 3(D) displays an excessive amount of material deposited on the substrate to the extent that all three intended strands fuse together to a single outspread shape without any specific geometry.Both categories C3 and C4 might also be affected by the inappropriate setting of printing parameters, which could be originated both by the user and by the printing setup.Category C5 consists of images lacking high contrast between printed structures and structures out off focus.This can occur if the well plate is not properly placed in the bracket in the imaging system as seen in figures 3(E) and (F).Each image was classified into the categories of the presented challenges.The ratio of images found in each category to the total amount of images is presented in figure 4(A).
C1 with 73.1% represents rotation and/or shifts from the origin and C2 with 6.1% the amount of additional paths.C3 with 39% is the percentage of noncontinuous filaments, and C4 with 13.3% shows the occurrence of material excess.The fifth category-C5-with 14.6% includes images of limited image quality.
At first sight, the C1 category seems to be a big issue, however, this issue could be overcome by the development of algorithms that identify the ROI by matching the imaged structure to the expected structure and rotate the image as required.Similarly, the matching of the expected geometry with the binarized images enabled the selection of ROIs avoiding additional paths present in images categorized as C2.Remarkably, only a subset of images showed the geometry as it was designed regarding the intended orientation.C5 contained images of limited acquisition quality, which could also be analyzed by the developed workflows as denoizing steps.Sensitive thresholding was included.These steps could overcome artifacts such as dust or scratches on the substrate.The deposition of excessive amounts of material leading to fusing of the filaments or random geometries was categorized as C4.The images in this category did not resemble the expected geometries and could therefore not be analyzed.Regarding category C3, non-continuous filaments resembled the intended structure and could therefore be analyzed by the algorithms used.It is noteworthy that even though a sequence of dots can resemble a line, the intended structure was not complete.All in all, this evaluation of the images shows what kind of issues are to be expected in the future and helps with the development of analysis methods to distinguish between the various cases and the challenges to be reckoned with.
The subset of data of line geometries consisted of 648 images of line geometries, i.e. data sets of 54 images per laboratory.The share of images of printed line geometries showing broken filaments , thus classified as category C3, is presented in figure 4(B).An exemplary image is shown in figure 3(C).What can be clearly seen here is that there is a scatter in the laboratories between 2% and 65%.Which leads to the conclusion that even a simple structure like a single line or circle could not be printed by any laboratory with 100% reliability.This indicates that there are still significant challenges to be overcome regarding reliability and reproducibility of bioprinting, before medical applications with high regulatory requirements are intended.Although the entire process was standardized as much as possible, the results differed significantly between the laboratories.This large scatter might be a clear indicator of the results being nevertheless still highly dependent on the operator and printer.However, it was not feasible to determine which of both factor is the main cause of variability.After having conducted the visual qualitative analysis, the main deviations were identified and can be considered to be applied in the future development of analytics.Possible countermeasures for future round robins might be (a) further improvement of the SOPs, (b) training of the laboratory staff along the SOPs, and (c) refinement of the IA procedures towards potential outliers.

Round robin-image analysis: quantitative
Following printing and data storage, three groups specialized in IA developed an image processing workflow to extract the features of the objects without prior knowledge of the categorized deviations.In order to investigate whether the three methods developed by the IA groups might lead to different results, the geometric features of each group extracted per laboratory were compared.For simplification, exemplary results of one analysis parameter, i.e. the circle width (see supplementary section) printed with Cellink Bioink and with gelatin-based ink for three laboratories, are presented in figure 5.
In figure 5, the results as regards circle width and variation of the 'strand' when printing a circle and the associated standard deviations printed by three laboratories and extracted by all three IA groups are shown, Cellink Bioink in (A) and the gelatin-based ink in (B).For Cellink Bioink, the circle width determined by the different IA groups were: (a) Lab 1 1.4 mm, (b) Lab 2 0.6 mm, (c) Lab 3 1.1 mm.Comparing single laboratories, the results extracted by the IA groups are in the same range.The largest difference of extracted data was observed for Lab 3 ranging from a circle width of 0.95 ± 0.1 mm, to a circle width of 1.17 ± 0.1 mm.The extracted measurements regarding filament width of line and circle structures differed slightly when comparing the data extracted by the different IA workflows.The deviations in the metrics might originate from the different methodologies used in the preprocessing of the images, where background noise and artifacts are removed.Similarly, the edges of the printed structures were not detected in an equal manner by all IA groups.The extraction of data was thus affected.As mentioned above, the acquired images showed issues such as weak contrast between ink and background.These issues regarding image quality also influence the extraction capability of automated image processing.Future studies should include the assessment of performance of automated IA including a comparison of images by determination of the Jaccard index or the Sorensen-Dice coefficient.These parameters enable quantification of the similarity between the analyzed image and the ground-truth presented as the desired geometry.This might not be an easy task, as the designed geometry is transferred as STL file between locations.The geometry to be printed is sliced by each printer software in different manner.Especially for more complex structures, this could lead to differences in the printing path and therefore in the produced geometry.It must be noted, that the image processing groups were not included in the development of the imaging setup.This can be improved in future studies.Nevertheless, all three independently developed methods delivered results in similar ranges which in turn emphasizes the suitability and robustness of automated image processing as an evaluation tool.Here, the focus was initially placed on singlelayer objects, since the first layer is crucial because all other layers are applied on top of it.Automated image processing should also be applied to 3D structures in future studies.
In the following, for a quantitative analysis of printed structures and the assessment of the reproducibility of printing processes, only a subset of data is presented.Several hypotheses were tested and only data of qualified laboratories were selected.Any variations in mean and standard deviations of the geometric features account for the effects proceeding from the printing process or the used ink.The exemplary data is presented in three case studies comparing extrusion mechanism, coordinate calibration, and temperature control of the printing equipment used (chapters 3.4.1-3.4.3).Additionally, the selection of inks was also limited to Cellink Bioink and gelatin-based ink, since these differ most in terms of appearance.Furthermore, the gelatin-based ink shows a thermo-sensitive property due to the sol-gel transition of the protein solution [36].

Case study 1: pneumatic vs. mechanical extrusion
In the design of the round robin test and the conception of SOPs, the utmost possible standardization was used.The printers represented the parameter with the greatest leeway, since not all laboratories had the same models available.Thus, factors influencing the reproducibility of 3D bioprinting in different laboratories were examined by grouping the data from different laboratories that presented similarities in terms of the function of the used printing equipment.A first aspect to be examined was the extrusion mechanism of the used equipment.Therefore, data sets of laboratories using the same device model were compared with those of laboratories that used custom models.
The Cellink BioX printer is equipped with pneumatic powered printheads.Lab 3, Lab 4, Lab 8, Lab 11, and Lab 12 set the pneumatic pressure to 23 kPa, 25 kPa, 20 kPa, 23 kPa and 20 kPa, respectively.In contrast, all three custom models used piston-driven extrusion using the velocity of axial movement of the piston as a printing setting.In figure 6, the line and circle structures were selected and quantified in terms of filament width.
The widths of the lines and circles printed with Cellink Bioink are presented in figures 6(A) and (B), respectively.The printed lines by the laboratories using the pneumatically driven mechanism showed a width in the range of 0.96-1.36mm with standard deviations between 0.07 mm and 0.3 mm.The line widths extracted from geometries printed with the mechanical extrusion systems were in the range of 0.71-1.07mm, and the associated standard deviations showed values varying from 0.08 to 0.19 mm.The widths of printed circles by the group of laboratories using pneumatic extrusion had a minimal mean value of 0.74 mm achieved by Lab 8 with a standard deviation of 0.12 mm.The maximum mean circle width and standard deviation were produced by Lab 12 with values of 1.64 mm and 0.53 mm, respectively.The circle width extracted from the samples printed with the piston-driven devices presented mean values between 0.57 mm and 1.4 mm.The related standard deviations are within the range of 0.14-0.29 mm.The effect of the extrusion mechanism was compared by calculation of the CV, and the values are provided in table 3. The CVs of the pneumatically driven process are in a wide range between 7.2% and 42.0%.In contrast, the CVs of the mechanical piston printers are within a clearly narrower range from 7.2% to 23.7%.The filament width should be independent of the trajectory of the printhead, as long as the printing velocity is the same.Filament width of geometries printed with Cellink Bioink.In (A), the mean line width, and in (B), the mean circle width are presented.The laboratories are classified according to the extrusion mechanism of the used printers.On the left side of each diagram, data is shown from laboratories using pneumatically driven printers.On the right side, data is presented from laboratories using custom-made printers based on the movement of a piston.Therefore, low variability should be the case in the comparison between line and circle width.The higher variabilities of the widths of lines and circles printed with all five pneumatically driven systems was shown although the same experimental setup was used using pressures within the range of 20 kPa-25 kPa.All five bioprinters are the same model, windows of printing parameters were the same in all laboratories, and the labware used was provided centrally by the organizing laboratory.Additionally, the ink was acquired centrally from a single batch and distributed the participating laboratories.It is noteworthy that the biomaterial is delivered in a filled cartridge ready to use.The high inter-and intralaboratory variability of the printed structures might be related to the function of the device where the extrusion pressure is supplied by a compact, built-in compressor that might not be able to hold the set pressure over the whole processing time.Furthermore, the tubes connecting the air supply with the cartridge are loose in the housing and, depending on the tube length, might get squeezed depending on the position of the printhead in relation to the housing.Additionally, several laboratories reported occasional clogging of the nozzles when using this ink.Such deficiency can occur when the material is not homogeneously mixed, leading to an aggregation of the nanocellulose used as a thickener.Similar issues regarding heterogeneities of bioinks and the effect on printing have been shown by Chung et al [37].Regarding the mean values of the filament width produced in the laboratories, the difference can be explained by the round robin setup and the functionality of the printers.The organizing laboratory set a minimum and maximum target filament width with additional information as to which pressure values were necessary to produce these strand widths with a representative printer during the test design.As the mechanical extrusion printers used in the round robin test were custom-manufactured devices, the respective laboratories had to determine the printer parameters necessary to reach the target filament width individually with regard to the individual printer.The lower variability in the data produced by the single laboratories could be explained by the extrusion mechanism where the piston displacement pushes the ink out of the cartridge.In contrast to pneumatic extrusion, material heterogeneities do not affect the ink flow as this is defined by the chosen speed of the displacement of the piston.The presented data in this comparison demonstrated the influence of the extrusion mechanism leading to non-reproducible printing processes, and showed the advantages of piston-operated extrusions systems used in the field of bioprinting.Alternatively, further methods to increase the reproducibility of pneumatic extrusion have been presented by Armstrong et al [38] and by Wenger et al [39].Both studies involve the process monitoring and adaptation of the pneumatic pressure.The first method uses a laser scanner to measure the deposited filament width and corrects the printing parameter to reach a certain width.The latter monitors the flow rate and corrects the pneumatic pressure to overcome fluctuations.A further aspect of printing systems that can highly influence the printed geometry is the calibration of coordinates, i.e. the distance between the tip of the nozzle and the surface of the used substrate used.

Case study 2: coordinate calibration
In terms of the investigation of the effect of coordinate calibration on the reproducibility of bioprinting, the five laboratories operating with the same printer were grouped again.This model requires the operator to set the z-height manually.A second group of laboratories was considered for comparison, these used printing systems equipped with optical sensors for determination of the coordinates of the nozzle tip, i.e. the process was automated.In figure 7, the results of line width (A), and circle width (B) are shown, both structures were printed with Cellink Bioink.The CV was also calculated for each laboratory and is given in table 4.
The range of the mean line width printed after manual calibration of the coordinates was 0.92-1.36mm, and the line width range for the automated setting of coordinates was 0.94-1.19mm.In terms of standard deviation, the highest value was 0.3 mm, shown by the data from equipment where manual coordination calibration was performed, and was generally in the range of 0.07-0.3mm.For the printers equipped with automated coordinate calibration features, the standard deviation with a value of 0.26 mm was the highest.Deviations from the mean were in a lower range with 0.12-0.26mm compared to the group of printers with manual calibration procedures.Regarding the data from circle mean width, the range is even larger with a range of 0.74-1.64mm for the data produced with the manually calibrated printers.A smaller range was detected for automatic calibration with a range of 1.06-1.18mm.Similarly to the mean width of the printed circle, the standard deviation increases as well for the first group, i.e. printers required to be manually calibrated regarding the z-height.The values of the standard deviation were in the range of 0.12-0.53mm.The standard deviation of the circle width produced by the printers of the second group was in the range of 0.09-0.32mm.The CVs of the line width printed with Cellink Bioink using manually calibrated printers showed a maximum value of 28.3%.The maximum value of the CVs was lower in the group of automated calibration printers with 22.3%.The difference between CVs was larger when circle geometries were printed.The first group and second group showed CVs up to 42.0% and 27%, respectively.Overall, the results regarding filament width and CVs are an indication of the fact that the operator-dependent calibration step introduces variations into the printing process as the distance between surface of the substrate and nozzle tip cannot be manually set to a standard value.The effect of varying distance between the tip of the nozzle and the surface of the substrate on the filament width has been shown by Naghieh et al [40].Therefore, the automation of the coordinate calibration enhances the robustness of the printing process.In the first and second case studies, Cellink Bioink was used.This ink shows viscoelastic properties such as shear-thinning, as reported in literature [41,42].A more complex rheological behavior is presented by the gelatin-based ink which was used as a third case study.The protein solution undergoes gelation under physiological temperatures [36,43], and, therefore, the printing process using gelatin-based ink is challenged by the ability of the printer to heat and control the temperature at the cartridge.

Case study 3: temperature control
To examine the impact of the different printer configurations influencing the reproducibility of printed geometries with gelatin-based ink, the laboratories were grouped according to the types of temperature control of the respective printers.The extracted data is shown in figure 8, where the line width is shown again in (A), and the circle width in (B) from the different laboratories.The four bioprinters on the left side have no cartridge temperature control, the five in the middle have an electric heating, and the three bioprinters on the right side are equipped with a fluid circulation heating, where the fluid temperature is set externally.In Lab 3, Lab 9, Lab 12, Lab 1, Lab 6, Lab 10 a temperature of 23 The results of the mean line width of the group with no temperature control showed values in the range from 0.95-1.32mm and standard deviations in a range of 0.09-0.42mm.In the second group of printers, i.e. devices equipped with electric heating, the mean line widths were in the range of 0.91-1.15mm with standard deviations in the range of 0.05-0.23 mm.The mean widths of lines printed with equipment where the temperature is controlled by fluid circulation were in the range of 0.86-1.27mm with standard deviations between 0.07 mm and 0.59 mm.The results of mean circle width are in the range of 0.89-1.54mm with standard deviations between 0.08-0.59mm without heating, in Figure 7. Filament width of geometries printed with Cellink Bioink.In (A), the mean line width, and in (B), the mean circle width are presented.The laboratories are classified according to the mechanism used for coordinate calibration.On the left side of each diagram, data is shown from laboratories using a printer where the position of the tip of the nozzle is calibrated manually by the operator.On the right side, the process of coordinate calibration is performed automatically using an optical sensor.the range of 0.65-1.54mm with standard deviations between 0.07 and 0.47 mm for electric heating, and in the range of 0.74-1.09mm with standard deviations between 0.05-0.23 mm for fluid circulation.
The variability of the intralaboratory comparison is reflected in the respective coefficients of variation.The highest variability was observed in the samples produced by the group of laboratories using printers without the capability of heating the cartridge during the process, where the CVs show values in the range  [36], the heating of the cartridge containing bioinks is essential.Without the control of temperature, the gelatin-based ink undergoes a transition into a gel state where the viscosity increases during processing time.This process has been shown to be time-dependent and to be affected by the temperature difference between solution and environment [44,45].A further challenge regarding the use of gelatin-based inks is the possible clogging of nozzles which is mentioned in further bioprinting studies [46,47].The SOPs indicated that the printing parameters should be kept constant during a printing session, i.e. printing of six technical replicates.However, it was mentioned as well that it is possible to increase the pneumatic pressure as a measure to counteract the gelation of the gelatinbased ink.The low variability of the structures, both lines and circles, produced by Lab 7 without heating can be explained by the fact that the pressure was adjusted, i.e. increased, during the printing process.In contrast, Lab 5 increased the pneumatic pressure during printing as well, which did not improve the variability of printed geometries.A second possibility to counteract gelation of the gelatin-based ink was the possibility to increase the temperature of the cartridge holder.This measure was taken by Lab 8 and Lab 11.Both groups used printers equipped with electric heating.While the CV of printed lines and circles by Lab 8 stayed in the same range, the CVs of printed lines and circles by Lab 11 differed notably.This shows the important effect of the individual operators on the printing process and printed geometries.Laboratories using printers of the third group, i.e.where the temperature is regulated over fluid circulation, completed the experimental series without the adjustment of printing parameters, both pneumatic pressure and printing temperature.Over the external setting of fluid temperature, the cooling of the cartridge is possible in contrast to the electric elements that are only able to heat the cartridge.It is noteworthy that the lowest variability was shown by the results by Lab 3. Overall, it can be stated that individual operators, and the different equipment of the printer affect the results and, thus, limit the comparability of the provided data.In order to provide robust printing processes, there is still a need for process control and higher degrees of automation in the printing equipment.

Conclusion
The presented study highlights the successful collaboration of 15 groups nationwide with the shared goal of analyzing reproducibility and introducing standards to the bioprinting field in order to accelerate the transition from laboratory practice to production for clinical applications.For this round robin study, SOPs were written containing information about material preparation, use of labware, and experimental printing setup.Materials were acquired and distributed centrally by the organizing laboratory.Identical imaging conditions were provided by use of the BioFI prototype instrument and the data was centrally managed in the Kadi4Mat platform.During the evaluation process, a distinction was made between a qualitative and a quantitative IA.In the qualitative investigation, it was shown that several deviations in the printing and imaging processes occur.This study provides an up-to-date overview of possible deviations and helps to analyze where the process needs to be enhanced.An important outcome was that the individual operators still have a significant impact on the resulting structure.Similarly, the recognition of possible factors diminishing the reproducibility of the process after imaging can be differentiated and included in the development of automated IA.These issues were not considered by the three IA groups because the analysis was performed simultaneously and independently.
Three different methods obtained results in the similar range regarding geometric features of the printed samples.This proved that automated IA is a suitable tool for the assessment of printing process reproducibility and quantitative comparability in the bioprinting field is by far not achieved, yet, due to lack in standardization in terms of bioprinting equipment.Hereby, devices equipped with pistons for mechanical extrusion, automated calibration of coordinates, especially z-height, and temperature-controlled printheads proved to be advantageous.Although target line widths were used in this study as a method for device-independent transfer, the product equivalency between locations could not be shown.In the future development of bioprinters, the above-mentioned problems need to be addressed.Ultimately, different cell types must be included in the process and the effects of cellular material on the reproducibility need to be characterized.Thereby, a significant effect is expected.The production of bioprinted structures might face requirements imposed by regulatory agencies when trying to make the leap into clinical stages.These agencies require information on the range of operating conditions that will result in the products and materials meeting certain quality criteria.This preliminary round robin test identified significant present challenges to be overcome in order to provide robust bioprinting processes.Furthermore, a nationwide infrastructure and network is now established, which can be used for material evaluation and evaluation of standards in the field of bioprinting.

Figure 2 .
Figure 2. Scheme of the experimental design of the Round robin-3D printing.Twelve different academic laboratories throughout Germany participated in the Round robin-3D printing test (n = 12).Line, circle, and rectangle (n = 3) geometries were printed using three different inks (n = 3).Additionally, a three-dimensional (3D) scaffold was printed using Cellink Bioink (n = 1).As the experiments were performed without cells, independently prepared inks from the same batch were considered as analogs of biological replicates.In the case of Cellink Bioink, the use of an independent cartridge was considered as a biological replicate.For each biological replicate (n = 3), the ink was used in order to print six samples as technical replicates (n = 6).The resulting data sets contained 180 images for each individual laboratory.Exemplary images of the printed geometries with each ink are provided in the supplementary section.

Figure 3 .
Figure 3. Overview of deviations with images shown as examples that occurred during the printing experiments.These can be caused by the setup (printer and software), the operator, or problems during image acquisition.Not every deviation can be traced back to a single cause.Therefore, some deviations are assigned to several causes and are in-between.(A) Lines are rotated 90 • compared to the original STL file, categorized as C1.(B) Additional paths printed within the circle, categorized as C2.(C) Discontinuous lines and simultaneously off-centered positioning, categorized as C1 and C3.(D) Lines merged together due to excessive material corresponding to category C4. (E) Substrate not properly placed in the BioFI imaging system, categorized as C5.(F) Weak contrast between background and printed material as well as printed in an offset position, categorized as C5 and C1.Scale bars left and below printed geometries: 10 mm.

Figure 4 .
Figure 4. Deviations of the images from the specifications taken in the round robin test.The deviations presented in the images were divided into challenge categories.Single images could be qualify for more than one challenge The respective share of the total image data set is shown in (A) (n = 2160).In (B), the subset of images of printed line geometries that were classified as category 3 is shown by occurrence in the 12 laboratories (n = 648).

Figure 5 .
Figure 5. Filament width of circle geometries extracted by the three image analysis groups 1-3.In (A), the circle width printed with Cellink Bioink, and in (B), the circle width printed with gelatin-based ink are depicted with the respective standard deviations.The data of laboratories 1-3 are shown as exemplars.Each image analysis groups analyzed every single printed structure by all laboratories (n = 108), and the geometric parameters were determined where possible.

Figure 6 .
Figure 6.Filament width of geometries printed with Cellink Bioink.In (A), the mean line width, and in (B), the mean circle width are presented.The laboratories are classified according to the extrusion mechanism of the used printers.On the left side of each diagram, data is shown from laboratories using pneumatically driven printers.On the right side, data is presented from laboratories using custom-made printers based on the movement of a piston.

Figure 8 .
Figure 8. Filament width of geometries printed with gelatin-based ink.In (A), the mean line width, and in (B), the mean circle width are presented.The laboratories are classified according to the mechanism used for temperature control.On the left side of each diagram, data is shown from laboratories using equipment without heating elements.In the middle, data is shown from the devices equipped with electric heating in the cartridge holder.On the right side, the temperature control uses fluid circulation and an external setting of the fluid temperature.

Table 1 .
Extrusion principles and specifications of printing equipment used by the participating laboratories in the Round robin-3D printing test.Laboratory identification numbers were generated randomly.

Table 2 .
Materials employed in the Round robin-3D printing test with recommendations of printing parameters.

Table 3 .
Summary of the coefficients of variation (CV) for each laboratory regarding the examination of the effects of the extrusion mechanism on the filament width.
• C, 23.5 • C, 23 • C, 21 • C, 24 • C and 22.5 • C was used respectively.Two laboratories increased the temperature during the experiments.Lab 8 performed the experiments in a range of 23 • C to 26 • C and Lab 11 in a range of 21 • C to 25 • C. The respective CVs are given in table 5.

Table 4 .
Summary of the coefficients of variation (CV) for each laboratory regarding the examination of the effects of the method used for coordinate calibration on the filament width.

Table 5 .
Summary of the coefficients of variation (CV) for each laboratory regarding the examination of the effects of the mechanism for temperature control on the filament width.