Information system of multi-stage analysis of the building of object models on a construction site

This study focuses on the development of a multi-stage analysis of building object models (BOM) on a construction site for modeling an “evolutionary” digital twin, by integrating building information modeling (BIM) technology and an artificial intelligence system. The concepts of photo modeling of the construction site using a group of moving cameras were outlined, as well as the possibility of integrating IoT technologies. The dynamic transition of real building structures into intermediate BIM representations of digital twins was investigated, with the prospect of enabling augmented reality technology. An artificial intelligence system combining Convolutional Neural Network (CNN) and Feed Forward Neural Network (FFNN) architectures has been developed as a comprehensive mechanism for the detection, categorization, and evaluation of BIM projects at all stages of their life cycle. The paper addresses the scaling prospects for the development of point cloud and mesh models, as well as the use of big data technology while optimizing the representation of the “evolutionary” BIM project of the digital twin of the construction site. The effectiveness of site conformance detection during the step-by-step construction of a BIM model, which shows consistency and provides a quantitative assessment of the processes occurring on the site, has been determined. The results of this research can be used to improve BIM modeling methods and concepts, in particular towards a multi-stage “evolutionary” representation of the digital twin of the construction site.


Introduction
Nowadays, BIM technologies have attained an extraordinary level of integration with many new technologies, such as artificial intelligence, the Internet of Things (IoT), Big Data, and Augmented Reality (AR), which has significantly increased the quality and speed of the entire set of operations and processes related to construction project modeling, implementation, and support.The generation of digital twins of the BOM and the construction site, in general, was an essential branch of the direction.
The purpose of this research is to provide scientific support for the concepts of multi-stage modelling of building site objects utilizing artificial intelligence based on BIM technology.The author's team created a concept that defines the BOM creation process in the BIM system as being separated into an arbitrary number of pieces on a suitable timetable.Such changes enable the formation of a model with distinct evolutionary development and growth dynamics.The implementation of a Structure from Motion (SfM) system, which will allow the creation 1254 (2023) 012075 IOP Publishing doi:10.1088/1755-1315/1254/1/012075 2 of digital models from photographs from the construction site and displays a three-dimensional depiction of the state of the BOM in real-time, is critical to the collection of training and learning data.This technology will enable neural networks to categorize not only one representation of a BOM picture, but three reciprocal representations at the same time, namely point clouds, mesh models, and the BOM itself.These will enable the YOLOv5 and FFNN artificial intelligence models to identify faults more precisely during the execution of the BOM on the construction site.
Deng et al conducted a rigorous theoretical investigation of the development of BIM technologies across time.According to the paper [1], BIM technologies have evolved in five stages, as shown below: Level 1. Static 3D BIM visualization tool; Level 2. BIM model analyses and simulations; Level 3. BIM and IoT method integration; Level 4. BIM and artificial intelligence methods integration; Level 5. Make a digital twin.
It is vital to highlight that in order to accomplish the fifth level, all preceding levels must be implemented in a single BIM project.
In scientific publications [2][3][4] the authors consider SfM systems when resolving problems by building three-dimensional models from a point cloud using photometry or videometry.Scientists list a wide variety of uses for SfM systems, such as simulating worker evacuation from a construction site to visualize the site's actual condition, reducing variance between planned and actual conditions, and creating an "as-built" model of an object based on an image of it for reconstruction.Also defined is the ability to interact with images taken by SfM in real-time and by BIM standards.Geometric information is extracted using three-dimensional grid models utilizing tools such as Meshroom (AliceVision 2020), Agisoft Photoscan, or COLMAP.
In the studies [5][6][7] scientists take intelligent technologies, including SfM, into consideration and find that when scaling projects, they frequently lead to Big Data, which in turn necessitates the use of advanced management, diagnostics, and forecasting from the executors and visualization of this data, in particular, using modern capabilities of The idea of big data analysis is also given a lot of attention since it makes it possible to address several issues linked to the productivity and digitization of different spheres of human activity.As a result, machine learning, intelligent data analysis techniques, and a variety of statistics have all been included in big data analytics.
In papers [8,9], researchers examine BIM building employing IoT analysis.In addition, the authors determine the considerable importance of high-tech cloud solutions in connecting to the IoT.Given the expansion of BIM technology, it is expedient and important to use IoT, which is defined as a combination of physical and virtual components such as sensors, mechanisms, cloud services, communications, and protocols, to create digital twins of BOM and construction sites.This serves as the foundation for IoT systems.
Sezen et al [10] conduct an experimental study where they investigate precision, recall and mean average precision (mAP) are used as evaluation metrics among YOLOv3, YOLOv4, YOLOv5, and Faster R-CNN algorithms.Thus, based on the results of model training and direct tests, it was concluded that model YOLOv5 is the fastest and most productive when compared to YOLOv4 and others.Dolhopolov et al [11] provided a practical example of how to employ multi-label classification with FFNN to address issues with a sizable number of input and output classes and the potential to scale into Big Data problems.The design of FFNN is impacted by the duties assigned to the performers, according to the findings of scientific research.To illustrate the parameter space, for instance, training on several parameters is necessary when doing a regression on multiple parameters.
The goal of this research is to provide an analytical foundation for an information system that will allow the concept of multi-stage modeling of building site objects using artificial intelligence based on BIM technology to be confirmed.Achieving this goal may result in the development of a quality standard for working with homogeneous BOMs, which will be decided by the intricate interaction of IoT, Big Data, BIM, YOLOv5, SfM, and FFNN.

Materials and method
Considering a complete examination of the scientific works of scientists of various profiles and proving the concept of multi-stage modeling of construction site objects using artificial intelligence based on BIM technology.Figure 1 shows the developed model of an information system that allows you to study a construction object or construction site in real-time and highlight aspects that allow you to model the representation of a construction object in a threedimensional projection using SfM and IoT cameras.As a result, point clouds, mesh models, and BOMs with varied levels of detail are modeled.In the architecture of YOLOv5, each model of a building object is described as a class, allowing artificial intelligence to identify many photographs based on the stages of execution of the object on the construction site.The categorization data is then sent to the FFNN, which is compared to the standardized data that the project customer expects.As a result, the system's last procedure is to give the user information about the compliance of a certain building object with the standards, which can be determined manually or based on the pre-formed Dataset BOM.The unique feature of this information system's operation is the ability to determine whether the derived models of the architectural item reflect its general trends.The process of obtaining information, known as photo modeling or videometry, is carried out via IoT devices, which can be used in two ways.Figure 2 shows static photo modeling, which is distinguished by the fact that cameras are installed just once and are utilized throughout the whole observation process, and dynamic photo modeling, which may not require the installation of as many cameras and is in motion at specific time intervals.
The advantages of the first method are complete autonomy and reliability of the data collection procedure.However, in the case of a big-scale construction project, a large amount of equipment may be necessary, which is why, in some circumstances, dynamic photo modeling is preferable, especially if building operations are slow.
Thus, within the framework of the study, we have the opportunity to extract information from panoramic photos or photo sequences and model on their basis Point Clouds, Mesh models, and  BOMs of varying degrees of detail based on the project requirements using photo modeling and SfM, which is based on the Agisoft Photoscan software.Geometric forms are replicated with varying degrees of detail during photo modeling (approximately one point for every 5 cm of real size).The higher the image quality, the more detail is captured and the more natural it appears.In practice, however, different sets of input photos are frequently required.Figure 3 shows the BOM construction process utilizing photo modeling and SfM.
Within the scope of the project, 600 visualizations of BOM construction fragments were performed at various time intervals and construction objects, forming the study's learning and training Dataset. Figure 4 shows one of the BOM construction fragments on a real-world example of a building object with serial №5.
A dataset of 600 complete images of the building object was created as part of the study's framework (i.e., 2400 images representing the building object, Point Clouds, BOM, and Mesh Model).The artificial intelligence system recognizes diverse construction sites and their resulting models before determining their resemblance and compliance with the defined standard.Following the development of the models, they were manually tagged for additional training.Given the YOLOv5 model's excellent performance as the fastest and most successful model in the YOLO family, it was chosen for the categorization of building items and their derived models.Experiment results showed that 100 iterations of training took no more than 8 minutes of real-time.
The evaluation metrics chosen were Average Precision -the average accuracy for a given where AP P C is Average Precision for Point Clouds model; x i is Average Precision for each iteration of the building object according to the Point Clouds model.The value of the average precision of the Mesh model is determined by equation: where AP M M is Average Precision for Mesh model; y i is Average Precision for each iteration of the building object according to the Mesh model.The value of the average precision of the BOM is determined by equation: where AP BOM is Average Precision for BOM; ω i is Average Precision for each iteration of the building object according to the BOM.The value of the average precision of the class is determined by equation: where AP is general Average Precision of the class; AP P C is Average Precision for Point Clouds model; AP M M is Average Precision for Mesh model; AP BOM is Average Precision for BOM.The value of the mean average precision (mAP) is determined by equation: where AP i is Average Precision of i class; mAP is mean Average Precision; k is number of defined classes to recognize.Figure 5 shows the visualization of mathematical processes depicted in (1), ( 2), ( 3), (4), and (5) during YOLOv5 work on creating object classification.
The YOLOv5 model is trained in the ratio of 80% of training images, and 20% of test images.Images were separated according to the stages of development of simple building things.Table 1 shows the training parameters for the YOLOv5 model.
Comparison according to the FFNN model should be performed when certain standards are defined.The artificial limits that the standard must take into account are shown in table 2.

Results
Metrics were computed for all classes defined in the Dataset.The maximum Average Precision for the general classification was found for specimen №5 and was 0.88.In general, the YOLOv5 model produced correct classification results for the stated classes and had no discrepancies.However, because some building objects were more difficult to distinguish, the mAP of the YOLOv5 model is 0.73. Figure 6 shows an example of the findings of Instance №5.BO5, BOPC5, BOM5, and BOMM5 are building object, building object (Point Cloud model), building object model (BOM), and building object (Mesh model) in this example, respectively.Table 3 shows the generalized YOLOv5 classification results by building objects.The indicators range from 0 to 1 and are pre-normalized.
Table 4 shows how the FFNN model works regarding the standards given by the customer based on the building objects.The indicators range from 0 to 1 and are pre-normalized.As a result, the acquired values demonstrate the conformity of classified architectural objects and their derivative models to the required requirements.

Conclusion
Thus, the concept of multi-stage modeling of building site objects utilizing artificial intelligence based on BIM technology was proved to be feasible.The classification by time intervals was handled by the information system.The YOLOv5 model displayed an incredibly quick learning process (92 minutes for 500 iterations) as well as a relatively serious mAP of 0.73.Simultaneously, the YOLOv5 model performed classification by the building object's class as well as by its derivative models Point Cloud, Mensh model, and BOM, where it also displayed the related findings.The possibility of establishing quality standards with the help of the FFNN model, which was performed for general indicators by class, was also evaluated, but it can be realized for each stage of the building object's development individually.As a result, the comparison of models and standards based on example No5 revealed a high degree of conformity, ranging from 0.92 to 1.00.The information system model was accurate.Simultaneously, two choices for setting up the building site for photo modeling, static and dynamic, were provided.
The research has implications for the implementation of novel Augmented Reality technologies

Figure 1 .
Figure 1.Multi-stage modeling of building site items in an information system model.

Figure 2 .
Figure 2. Possibilities for implementing photo modeling of the modeled building object.

Figure 3 .
Figure 3.The BOM building process employs photo modeling and SfM.

Figure 4 .
Figure 4. BOM construction using photo modeling and SfM for sample №5.

Figure 5 .
Figure 5. Visualization of mathematical processes about the derivatives of the building object during their classification.

Figure 6 .
Figure 6.BOM construction is based on photo modeling and SfM classification results based on building objects.The indicators are pro-normalized and range from 0 to 1.

Table 1 .
The training parameters to model YOLOv5.

Table 2 .
Limitation to the instance of class №5 relative to the FFNN model.