Table of contents

Head and neck surgery is a fine surgical procedure with a complex anatomical space, difficult operation and high risk. Medical image computing (MIC) that enables accurate and reliable preoperative planning is often needed to reduce the operational difficulty of surgery and to improve patient survival. At present, artificial intelligence, especially deep learning, has become an intense focus of research in MIC. In this study, the application of deep learning-based MIC in head and neck surgery is reviewed. Relevant literature was retrieved on the Web of Science database from January 2015 to May 2022, and some papers were selected for review from mainstream journals and conferences, such as IEEE Transactions on Medical Imaging, Medical Image Analysis, Physics in Medicine and Biology, Medical Physics, MICCAI, etc. Among them, 65 references are on automatic segmentation, 15 references on automatic landmark detection, and eight references on automatic registration. In the elaboration of the review, first, an overview of deep learning in MIC is presented. Then, the application of deep learning methods is systematically summarized according to the clinical needs, and generalized into segmentation, landmark detection and registration of head and neck medical images. In segmentation, it is mainly focused on the automatic segmentation of high-risk organs, head and neck tumors, skull structure and teeth, including the analysis of their advantages, differences and shortcomings. In landmark detection, the focus is mainly on the introduction of landmark detection in cephalometric and craniomaxillofacial images, and the analysis of their advantages and disadvantages. In registration, deep learning networks for multimodal image registration of the head and neck are presented. Finally, their shortcomings and future development directions are systematically discussed. The study aims to serve as a reference and guidance for researchers, engineers or doctors engaged in medical image analysis of head and neck surgery.

Objective. Monte Carlo (MC) codes are increasingly used for accurate radiotherapy dose calculation. In proton therapy, the accuracy of the dose calculation algorithm is expected to have a more significant impact than in photon therapy due to the depth-dose characteristics of proton beams. However, MC simulations come at a considerable computational cost to achieve statistically sufficient accuracy. There have been efforts to improve computational efficiency while maintaining sufficient accuracy. Among those, parallelizing particle transportation using graphic processing units (GPU) achieved significant improvements. Contrary to the central processing unit, a GPU has limited memory capacity and is not expandable. It is therefore challenging to score quantities with large dimensions requiring extensive memory. The objective of this study is to develop an open-source GPU-based MC package capable of scoring those quantities. Approach. We employed a hash-table, one of the key-value pair data structures, to efficiently utilize the limited memory of the GPU and score the quantities requiring a large amount of memory. With the hash table, only voxels interacting with particles will occupy memory, and we can search the data efficiently to determine their address. The hash-table was integrated with a novel GPU-based MC code, moqui. Main results. The developed code was validated against an MC code widely used in proton therapy, TOPAS, with homogeneous and heterogeneous phantoms. We also compared the dose calculation results of clinical treatment plans. The developed code agreed with TOPAS within 2%, except for the fall-off and regions, and the gamma pass rates of the results were >99% for all cases with a 2 mm/2% criteria. Significance. We can score dose-influence matrix and dose-rate on a GPU for a 3-field H&N case with 10 GB of memory using moqui, which would require more than 100 GB of memory with the conventionally used array data structure.

Objective. To investigate synchrotron-based proton pencil beam scanning (PBS) beam delivery time (BDT) using novel continuous scanning mode. Approach. A BDT calculation model was developed for the Hitachi particle therapy system. The model was validated against the measured BDT of 36 representative clinical proton PBS plans with discrete spot scanning (DSS) in the current Hitachi proton therapy system. BDTs were calculated with the next generation using Mayo Clinic Florida system operating parameters for conventional DSS, and novel dose driven continuous scanning (DDCS). BDTs of DDCS with and without Break Spots were investigated. Main results. For DDCS without Break Spots, the use of Stop Ratio to control the transit dose largely reduced the beam intensity and consequently, severely prolonged the BDT. DDCS with Break Spots was able to maintain a sufficiently high beam intensity while controlling transit dose. In DDCS with Break Spots, tradeoffs were made between beam intensity and number of Break Spots. Therefore, BDT decreased with increased beam intensity but reached a plateau for beam intensity larger than 10 MU s⁻¹. Averaging over all clinical plans, BDT was reduced by 10% for DDCS with Break Spots compared to DSS. Significance. DDCS with Break Spots reduced BDT. DDCS has the potential to further reduce BDT under the ideal scenario which requests both stable beam intensity extraction and accurately modelling the transit dose. Further investigation is warranted.

Objective. To develop a multimodal model that combines multiphase contrast-enhanced computed tomography (CECT) imaging and clinical characteristics, including experts' experience, to preoperatively predict lymph node metastasis (LNM) in pancreatic cancer patients. Methods. We proposed a new classifier fusion strategy (CFS) based on a new evidential reasoning (ER) rule (CFS-nER) by combining nomogram weights into a previous ER rule-based CFS. Three kernelled support tensor machine-based classifiers with plain, arterial, and venous phases of CECT as the inputs, respectively, were constructed. They were then fused based on the CFS-nER to construct a fusion model of multiphase CECT. The clinical characteristics were analyzed by univariate and multivariable logistic regression to screen risk factors, which were used to construct correspondent risk factor-based classifiers. Finally, the fusion model of the three phases of CECT and each risk factor-based classifier were fused further to construct the multimodal model based on our CFS-nER, named MMM-nER. This study consisted of 186 patients diagnosed with pancreatic cancer from four clinical centers in China, 88 (47.31%) of whom had LNM. Results. The fusion model of the three phases of CECT performed better overall than single and two-phase fusion models; this implies that the three considered phases of CECT were supplementary and complemented one another. The MMM-nER further improved the predictive performance, which implies that our MMM-nER can complement the supplementary information between CECT and clinical characteristics. The MMM-nER had better predictive performance than based on previous classifier fusion strategies, which presents the advantage of our CFS-nER. Conclusion. We proposed a new CFS-nER, based on which the fusion model of the three phases of CECT and MMM-nER were constructed and performed better than all compared methods. MMM-nER achieved an encouraging performance, implying that it can assist clinicians in noninvasively and preoperatively evaluating the lymph node status of pancreatic cancer.

Objective. The purpose of this study is to develop a treatment planning process (TPP) for non-isocentric dynamic trajectory radiotherapy (DTRT) using dynamic gantry rotation, collimator rotation, table rotation, longitudinal, vertical and lateral table translations and intensity modulation and to validate the dosimetric accuracy. Approach. The TPP consists of two steps. First, a path describing the dynamic gantry rotation, collimator rotation and dynamic table rotation and translations is determined. Second, an optimization of the intensity modulation along the path is performed. We demonstrate the TPP for three use cases. First, a non-isocentric DTRT plan for a brain case is compared to an isocentric DTRT plan in terms of dosimetric plan quality and delivery time. Second, a non-isocentric DTRT plan for a craniospinal irradiation (CSI) case is compared to a multi-isocentric intensity modulated radiotherapy (IMRT) plan. Third, a non-isocentric DTRT plan for a bilateral breast case is compared to a multi-isocentric volumetric modulated arc therapy (VMAT) plan. The non-isocentric DTRT plans are delivered on a TrueBeam in developer mode and their dosimetric accuracy is validated using radiochromic films. Main results. The non-isocentric DTRT plan for the brain case is similar in dosimetric plan quality and delivery time to the isocentric DTRT plan but is expected to reduce the risk of collisions. The DTRT plan for the CSI case shows similar dosimetric plan quality while reducing the delivery time by 45% in comparison with the IMRT plan. The DTRT plan for the breast case showed better treatment plan quality in comparison with the VMAT plan. The gamma passing rates between the measured and calculated dose distributions are higher than 95% for all three plans. Significance. The versatile benefits of non-isocentric DTRT are demonstrated with three use cases, namely reduction of collision risk, reduced setup and delivery time and improved dosimetric plan quality.

Gradient and spin echo (GRASE) is widely employed in arterial spin labeling (ASL) as an efficient readout sequence. Hemodynamic parameter mappings of perfusion, such as cerebral blood flow (CBF) and arterial transit time (ATT), can be derived via multi-delay ASL acquisitions. Multi-delay ASL perfusion imaging inevitably suffers limited signal-to-noise ratio (SNR) since a motion-sensitized vessel suppressing module has to be employed to highlight perfusion signals. The present work reveals that in multi-delay ASL, manipulation of GRASE sequence on either planar imaging echo echo train for adjusted spatial resolutions or FSE echo train for modulated extent of T₂-blurring can significantly alter the mapping contrasts among tissues and among cerebral lobes under different pathways of blood circulation, and meanwhile regulates SNR. Four separate multi-delay ASL scans with different echo train designs in 3D whole brain covering GRASE were carried out for healthy subjects to evaluate the variations in regard to the parameter quantifications and SNR. Based on the quantification mappings, the GRASE acquisition with moderate spatial resolution (3.5 × 3.5 × 4 mm³) and segmented k_z scheme was recognized for the first time to be recommended for more unambiguous CBF and ATT contrasts between GM and WM in conjunction with more enhanced ATT contrast between anterior and posterior cerebral circulations, with reasonably good SNR. The technical proposal is of great value for the cutting-edge research of a variety of neurological diseases of global concerns.

Objective. We are developing a small-fish positron emission tomography (PET) scanner dedicated to small aquatic animals relevant for biomedical and biological research, e.g. zebrafish. We plan to use Monte Carlo simulations to optimize its configuration and the required water-filled imaging chambers. Our objectives were: (1) to create a digital 3D zebrafish phantom using conventional micro-CT, (2) include the phantom into a simulated PET environment based on the framework GATE, and (3) investigate the effects of the water environment on the reconstructed images. Approach. To create the phantom, we performed ex vivo measurements of zebrafish specimen using a tabletop micro-CT and compared three methods to fixate the specimen. From segmented micro-CT images we created digital emission and transmission phantoms which were incorporated in GATE via tessellated volumes. Two chamber sizes were considered. For reference, a simulation with the zebrafish in air was implemented. The simulated data were reconstructed using CASToR. For attenuation correction, we used the exact attenuation information or a uniform distribution (only water). Several realizations of each scenario were performed; the reconstructed images were quantitatively evaluated. Main results. Fixation in formalin led to the best soft-tissue contrast at the cost of some specimen deformation. After attenuation correction, no significant differences were found between the reconstructed images. The PET images reflected well the higher uptake simulated in the brain and heart, despite their small size and surrounding background activity; the swim bladder (no activity) was clearly identified. The simplified attenuation map, consisting only of water, slightly worsened the images. Significance. A conventional micro-CT can provide sufficient image quality to generate numerical phantoms of small fish without contrast media. Such phantoms are useful to evaluate in-silico small aquatic animal imaging concepts and develop imaging protocols. Our results support the feasibility of zebrafish PET with an aqueous environment.

Objective. Proton pencil beam scanning (PBS) treatment fields needs to be verified before treatment deliveries to ensure patient safety. In current practice, treatment beam quality assurance (QA) is measured at a few selected depths using film or a 2D detector array, which is insensitive and time-consuming. A QA device that can measure all key dosimetric characteristics of treatment beams spot-by-spot within a single beam delivery is highly desired. Approach. We developed a multi-layer strip ionization chamber (MLSIC) prototype device that comprises of two layers of strip ionization chambers (IC) plates for spot position measurement and 64 layers of plate IC for beam energy measurement. The 768-channel strip ion chamber signals are integrated and sampled at a speed of 3.125 kHz. It has a 25.6 cm × 25.6 cm maximum measurement field size and 2 mm spatial resolution for spot position measurement. The depth resolution and maximum depth were 2.91 mm and 18.6 cm for 1.6 mm thick IC plate, respectively. The relative weight of each spot was determined from total charge by all IC detector channels. Main results. The MLSIC is able to measure ionization currents spot-by-spot. The depth dose measurement has a good agreement with the ground truth measured using a water tank and commercial one-dimensional (1D) multi-layer plate chamber. It can verify the spot position, energy, and relative weight of clinical PBS beams and compared with the treatment plans. Significance. The MLSIC is a highly efficient QA device that can measure the key dosimetric characteristics of proton treatment beams spot-by-spot with a single beam delivery. It may improve the quality and efficiency of clinical proton treatments.

Objective. Recently, dental cone-beam computed tomography (CBCT) methods have been improved to significantly reduce radiation dose while maintaining image resolution with minimal equipment cost. In low-dose CBCT environments, metallic inserts such as implants, crowns, and dental fillings cause severe artifacts, which result in a significant loss of morphological structures of teeth in reconstructed images. Such metal artifacts prevent accurate 3D bone-teeth-jaw modeling for diagnosis and treatment planning. However, the performance of existing metal artifact reduction (MAR) methods in handling the loss of the morphological structures of teeth in reconstructed CT images remains relatively limited. In this study, we developed an innovative MAR method to achieve optimal restoration of anatomical details. Approach. The proposed MAR approach is based on a two-stage deep learning-based method. In the first stage, we employ a deep learning network that utilizes intra-oral scan data as side-inputs and performs multi-task learning of auxiliary tooth segmentation. The network is designed to improve the learning ability of capturing teeth-related features effectively while mitigating metal artifacts. In the second stage, a 3D bone-teeth-jaw model is constructed with weighted thresholding, where the weighting region is determined depending on the geometry of the intra-oral scan data. Main results. The results of numerical simulations and clinical experiments are presented to demonstrate the feasibility of the proposed approach. Significance. We propose for the first time a MAR method using radiation-free intra-oral scan data as supplemental information on the tooth morphological structures of teeth, which is designed to perform accurate 3D bone-teeth-jaw modeling in low-dose CBCT environments.

Objective. In this study, we propose the adaptive permissible region based random Kaczmarz method as an improved reconstruction method to recover small carotid atherosclerotic plaque targets in rodents with high resolution in fluorescence molecular tomography (FMT). Approach. We introduce the random Kaczmarz method as an advanced minimization method to solve the FMT inverse problem. To satisfy the special condition of this method, we proposed an adaptive permissible region strategy based on traditional permissible region methods to flexibly compress the dimension of the solution space. Main results. Monte Carlo simulations, phantom experiments, and in vivo experiments demonstrate that the proposed method can recover the small carotid atherosclerotic plaque targets with high resolution and accuracy, and can achieve lower root mean squared error and distance error (DE) than other traditional methods. For targets with 1.5 mm diameter and 0.5 mm separation, the DE indicators can be improved by up to 40%. Moreover, the proposed method can be utilized for in vivo locating atherosclerotic plaques with high accuracy and robustness. Significance. We applied the random Kaczmarz method to solve the inverse problem in FMT and improve the reconstruction result via this advanced minimization method. We verified that the FMT technology has a great potential to locate and quantify atherosclerotic plaques with higher accuracy, and can be expanded to more preclinical research.

Objective. Oxygen plays an important role in affecting the cellular radio-sensitivity to ionizing radiation. The objective of this study is to build a mechanistic model to compute oxygen enhancement ratio (OER) using a GPU-based Monte Carlo (MC) simulation package gMicroMC for microscopic radiation transport simulation and DNA damage calculation. Approach. We first simulated the water radiolysis process in the presence of DNA and oxygen for 1 ns and recorded the produced DNA damages. In this process, chemical reactions among oxygen, water radiolysis free radicals and DNA molecules were considered. We then applied a probabilistic approach to model the reactions between oxygen and indirect DNA damages for a maximal reaction time of t₀. Finally, we defined two parameters P₀ and P₁, representing probabilities for DNA damages without and with oxygen fixation effect not being restored in the repair process, to compute the final DNA double strand breaks (DSBs). As cell survival fraction is mainly determined by the number of DSBs, we assumed that the same numbers of DSBs resulted in the same cell survival rates, which enabled us to compute the OER as the ratio of doses producing the same number of DSBs without and with oxygen. We determined the three parameters (t₀, P₀ and P₁) by fitting the OERs obtained in our computation to a set of published experimental data under x-ray irradiation. We then validated the model by performing OER studies under proton irradiation and studied model sensitivity to parameter values. Main results. We obtained the model parameters as t₀ = 3.8 ms, P₀ = 0.08, and P₁ = 0.28 with a mean difference of 3.8% between the OERs computed by our model and that obtained from experimental measurements under x-ray irradiation. Applying the established model to proton irradiation, we obtained OERs as functions of oxygen concentration, LET, and dose values, which generally agreed with published experimental data. The parameter sensitivity analysis revealed that the absolute magnitude of the OER curve relied on the values of P₀ and P₁, while the curve was subject to a horizontal shift when adjusting t₀. Significance. This study developed a mechanistic model that fully relies on microscopic MC simulations to compute OER.

Objective. A semi-supervised learning method is an essential tool for applying medical image segmentation. However, the existing semi-supervised learning methods rely heavily on the limited labeled data. The generalization performance of image segmentation is improved to reduce the need for the number of labeled samples and the difficulty of parameter tuning by extending the consistency regularization. Approach. We propose a new regularization-driven Mean Teacher model based on semi-supervised learning for medical image segmentation in this work. We introduce a regularization-driven strategy with virtual adversarial training to improve segmentation performance and the robustness of the Mean Teacher model. We optimize the unsupervised loss function and the regularization term with an entropy minimum to smooth the decision boundary. Main results. We extensively evaluate the proposed method on the International Skin Imaging Cooperation 2017(ISIC2017) and COVID-19 CT segmentation datasets. Our proposed approach gains more accurate results on challenging 2D images for semi-supervised medical image segmentation. Compared with the state-of-the-art methods, the proposed approach has significantly improved and is superior to other semi-supervised segmentation methods. Significance. The proposed approach can be extended to other medical segmentation tasks and can reduce the burden of physicians to some extent.

Objective. This study aims to theoretically investigate the dynamics of ultrasound-induced interstitial fluid streaming and tissue recovery after ultrasound exposure for potentially accelerating nanoagent transport and controlling its distribution in tissue. Approach. Starting from fundamental equations, the dynamics of ultrasound-induced interstitial fluid streaming and tissue relaxation after an ultrasound exposure were modeled, derived and simulated. Also, both ultrasound-induced mechanical and thermal effects were considered in the models. Main results. The proposed new mechanism was named squeezing interstitial fluid via transfer of ultrasound momentum (SIF-TUM). It means that an ultrasound beam can squeeze the tissue in a small focal volume from all the directions, and generate a macroscopic streaming of interstitial fluid and a compression of tissue solid matrix. After the ultrasound is turned off, the solid matrix will recover and can generate a backflow. Rather than the ultrasound pressure itself or intensity, the streaming velocity is determined by the dot product of the ultrasound pressure gradient and its conjugate. Tissue and nanoagent properties also affect the streaming and recovery velocities. Significance. The mobility of therapeutic or diagnostic agents, such as drugs, drug carriers, or imaging contrast agents, in the interstitial space of many diseased tissues, such as tumors, is usually extremely low because of the inefficiency of the natural transport mechanisms. Therefore, the interstitial space is one of the major barriers hindering agent deliveries. The ability to externally accelerate agent transport and control its distribution is highly desirable. Potentially, SIF-TUM can be a powerful technology to accelerate agent transport in deep tissue and control the distribution if appropriate parameters are selected.

Objective. To measure the monoenergetic x-ray linear attenuation coefficient, μ, of fused deposition modeling (FDM) colored 3D printing materials (ABS, PLA_white, PLA_orange, PET and NYLON), used as adipose, glandular or skin tissue substitutes for manufacturing physical breast phantoms. Approach. Attenuation data (at 14, 18, 20, 24, 28, 30 and 36 keV) were acquired at Elettra synchrotron radiation facility, with step-wedge objects, using the Lambert–Beer law and a CCD imaging detector. Test objects were 3D printed using the Ultimaker 3 FDM printer. PMMA, Nylon-6 and high-density polyethylene step objects were also investigated for the validation of the proposed methodology. Printing uniformity was assessed via monoenergetic and polyenergetic imaging (32 kV, W/Rh). Main results. Maximum absolute deviation of μ for PMMA, Nylon-6 and HD-PE was 5.0%, with reference to literature data. For ABS and NYLON, μ differed by less than 6.1% and 7.1% from that of adipose tissue, respectively; for PET and PLA_orange the difference was less than 11.3% and 6.3% from glandular tissue, respectively. PLA_orange is a good substitute of skin (differences from −9.4% to +1.2%). Hence, ABS and NYLON filaments are suitable adipose tissue substitutes, while PET and PLA_orange mimick the glandular tissue. PLA_white could be printed at less than 100% infill density for matching the attenuation of glandular tissue, using the measured density calibration curve. The printing mesh was observed for sample thicknesses less than 60 mm, imaged in the direction normal to the printing layers. Printing dimensional repeatability and reproducibility was less 1%. Significance. For the first time an experimental determination was provided of the linear attenuation coefficient of common 3D printing filament materials with estimates of μ at all energies in the range 14–36 keV, for their use in mammography, breast tomosynthesis and breast computed tomography investigations.

Objective. This paper proposes an automatic breast tumor segmentation method for two-dimensional (2D) ultrasound images, which is significantly more accurate, robust, and adaptable than common deep learning models on small datasets. Approach. A generalized joint training and refined segmentation framework (JR) was established, involving a joint training module (J_module) and a refined segmentation module (R_module). In J_module, two segmentation networks are trained simultaneously, under the guidance of the proposed Jocor for Segmentation (JFS) algorithm. In R_module, the output of J_module is refined by the proposed area first (AF) algorithm, and marked watershed (MW) algorithm. The AF mainly reduces false positives, which arise easily from the inherent features of breast ultrasound images, in the light of the area, distance, average radical derivative (ARD) and radical gradient index (RGI) of candidate contours. Meanwhile, the MW avoids over-segmentation, and refines segmentation results. To verify its performance, the JR framework was evaluated on three breast ultrasound image datasets. Image dataset A contains 1036 images from local hospitals. Image datasets B and C are two public datasets, containing 562 images and 163 images, respectively. The evaluation was followed by related ablation experiments. Main results. The JR outperformed the other state-of-the-art (SOTA) methods on the three image datasets, especially on image dataset B. Compared with the SOTA methods, the JR improved true positive ratio (TPR) and Jaccard index (JI) by 1.5% and 3.2%, respectively, and reduces (false positive ratio) FPR by 3.7% on image dataset B. The results of the ablation experiments show that each component of the JR matters, and contributes to the segmentation accuracy, particularly in the reduction of false positives. Significance. This study successfully combines traditional segmentation methods with deep learning models. The proposed method can segment small-scale breast ultrasound image datasets efficiently and effectively, with excellent generalization performance.

Objective. To develop a convolutional neural network (CNN) noise reduction technique for ultra-high-resolution photon-counting detector computed tomography (UHR-PCD-CT) that can be efficiently implemented using only clinically available reconstructed images. The developed technique was demonstrated for skeletal survey, lung screening, and head angiography (CTA). Approach. There were 39 participants enrolled in this study, each received a UHR-PCD and an energy integrating detector (EID) CT scan. The developed CNN noise reduction technique uses image-based noise insertion and UHR-PCD-CT images to train a U-Net via supervised learning. For each application, 13 patient scans were reconstructed using filtered back projection (FBP) and iterative reconstruction (IR) and allocated into training, validation, and testing datasets (9:1:3). The subtraction of FBP and IR images resulted in approximately noise-only images. The 5-slice average of IR produced a thick reference image. The CNN training input consisted of thick reference images with reinsertion of spatially decoupled noise-only images. The training target consisted of the corresponding thick reference images without noise insertion. Performance was evaluated based on difference images, line profiles, noise measurements, nonlinear perturbation assessment, and radiologist visual assessment. UHR-PCD-CT images were compared with EID images (clinical standard). Main results. Up to 89% noise reduction was achieved using the proposed CNN. Nonlinear perturbation assessment indicated reasonable retention of 1 mm radius and 1000 HU contrast signals (>80% for skeletal survey and head CTA, >50% for lung screening). A contour plot indicated reduced retention for small-radius and low contrast perturbations. Radiologists preferred CNN over IR for UHR-PCD-CT noise reduction. Additionally, UHR-PCD-CT with CNN was preferred over standard resolution EID-CT images. Significance. CT images reconstructed with very sharp kernels and/or thin sections suffer from increased image noise. Deep learning noise reduction can be used to offset noise level and increase utility of UHR-PCD-CT images.