Student Attendance Manager Using Beacons and Deep Learning

An efficient method for attendance management system is always been a challenging task for any organization varying from schools and colleges. This paper discusses about the attendance management system (AMS) and it's challenges. The paper proposes a student attendance system for schools and colleges using Beacon technology and Deep Learning techniques. The aim of this device is too savvy smart attendance system which includes the removal of issues like intermediary participation (for example proxy attendance for a student by another student). This is accomplished by obtaining live feeds from a fisheye camera at the beginning of each hour, which would be processed by a Convolutional Neural Network in the back end to enable students to choose their heads in the picture provided to them in their interface when they are in beacon proximity. Teachers interface will receive this same picture at the end of each hour to solve any discrepancy after which the data will be stored in a database. Hence, providing a foolproof system and efficient attendance tool.


Introduction
Ever since the educational system has incorporated the roll call for checking the participation of students, it has been a critical undertaking to guarantee the nearness of the student and also the educator. Since the start, the dreary arrangement of calling out the student names one by one has been taken after. This framework does not just take up an imperative segment of the instructive time yet also has numerous disadvantages like human mistakes, proxy attendance, teachers not able to listen to the sound of the student due to noise and so forth. Numerous endeavors were made in the ongoing occasions to change this strategy however just none h-ave prevailing to make a cost-productive process. Nowadays, many techniques like RFID [13,14,20], biometric scanner [19], facial recognition [16] and iris recognition [21] are used in many institutions, but all these frameworks are not productive.
The main problem present in the existing attendance system [15] using biometric scanner or facial recognition is that it needs the students to make the attendance manually one by one. Attendance systems using RFID reader or identity card reader also needs manual submission of attendance. Moreover, anyone can bring the RFID or ID card of other students and give proxy attendance for their friends. In schools and colleges, one time attendance entry and exit is not sufficient, since for each class there will be a different subset of people present, so giving 2 attendance in every class manually like this, will take more time than simple roll call by teacher. Hence, the paper proposes an automatic attendance marking system without disturbing the class, no wastage time and also avoids the issue of proxy attendance.
In this paper, AMS is implemented using Beacon and deep learning (DL) techniques [17,18]. Beacon is used to provide the access range within the classroom for a student to enroll his attendance. Once the class is started, a session will be created and all the registered mobile phones in the range of Beacon will be added to the attendance list. This may lead to the proxy issue of just keeping the mobile phone without physical presence, so inorder to avoid this proxy issue, DL method is used to for headcount identification and head selection by the presentees.
A fisheye camera is mounted in the classroom, that captures the fish-eye view of the class. This image is sent to the back end of this system where a deep learning model is used to detect heads. If the count of the mobile numbers and heads are unmatched then a situation of proxy attendance had happened in the class. If the number of registered mobiles is less than the head count, then some students would be in the class without their mobile phone, he/she has to enroll his attendance manually. If the number of registered mobiles is greater than the head count, then some students are not physically present in the class (proxy) but their mobile phones are present. This system will place rectangular grids over each head and this image with rectangular grid would be sent to each student's mobile phone to select their heads. This system will find any discrepancy in a selection of heads when multiple students select a particular head, which would be solved by the teacher at the end of the hour with a simple roll call of these students hence saving valuable time in most of the cases.

Literature Survey
Dhiman Sarkar et. al., [3] design and implement the attendance system using multi-step authentication. In the traditional attendance system in Bangladesh the teachers either call the name or identification number of the students to which the students respond or pass the attendance sheet to the students to sign. With the increase in the number of students in the last two decades, the difficulties in system management has increased remarkably. Again, in case of passing the attendance sheet to the students, some students sign multiple times, and proxy attendance is taken. These two systems are very time-consuming. To overcome these inconveniences this paper represents a smart attendance system. In this paper, radio frequency identification, biometric fingerprint sensor, and password-based technologies are integrated to develop a cost-effective, reliable attendance management system. A desktop application is developed in C environment to monitor the attendance system. Priya Pasumarti et.al, [4] in their paper presents a face recognition system for attendance. Attendance for the students is an important task in class. When done manually it generally wastes a lot of productive time of the class. This proposed solution for the current problem is through automation of the attendance system using face recognition. Face is the primary identification for any human. This project describes the method of detecting and recognizing the face in real-time using Raspberry Pi. This project describes an efficient algorithm using an open-source image processing framework known as Open CV. Their approach has five modules -Face Detection, Face Pre-processing, Face Training, Face Recognition and Attendance Database. The face database is collected to recognize the faces of the students. The system is initially trained with the student's faces which are collectively known as the student database. B Prabhavathi et.al, [5] in their paper presents a spontaneous presence for students in the classroom. At first classroom image has been in use and after that image is kept in a data record. For the images that are stored in the database we apply system algorithm which includes steps such as histogram classification, noise removal, face detection, and face recognition methods. So, by using these steps we detect the faces and then compare it with the database. The attendance gets marked automatically if the system recognizes the faces.  [6] in light of a legitimate concern for ongoing achievements in the advancement of deep Convolutional Neural Networks (CNNs) for face detection and recognition tasks, another deep learning-based face detection attendance system is proposed in this paper. The whole procedure of building up a face detection model is portrayed in detail. This model is made out of a few basic advances created utilizing the present most progressive methods: CNN course for face detection and CNN for producing face embeddings. The essential objective of this examination was the commonsense work of these state-of-the-art deep learning approaches for face detection assignments. Due to the fact that CNN's accomplish the best outcomes for bigger datasets, which isn't the situation production environment, the principal challenge was applying these techniques on smaller datasets. The overall accuracy was 95.02 on a small dataset of the original face images of employees in the real-time environment The proposed face acknowledgment model could be coordinated in another framework with or without some minor shifts as a supporting or a fundamental segment for observing purposes.
Mi-Young Bae et.al., [1] discusses the use of beacon technology for attendance system. When a student enters a lecture hall which is within the beacon proximity, it automatically logs their attendance to the database, but this paper doesn't address the problem of fake attendance made the students by bringing their friends mobile phones for attendance purpose. Shota Noguchi et.al., [2] discusses the addendum of RFID technology over the usage of the Beacon for creating a foolproof system, where their ID card will be scanned and in order to prevent illegal attendance, the beacon present in the class would transmit a unique ID to the student's smartphone for registering their attendance.

Proposed System
The proposed system consists of web interface app to collate the information and communication between the teacher and the student. When during an hour, teacher will start the session in teacher companion app which initiates the beacon to take up list of registered mobile phones under its coverage. Then the camera is initiated to take the picture of the classroom and this is send to Pi board. At the background a deep learning technique named Mask R-CNN is used to identify the persons in the image and does the head count calculation. If the head count matches with the mobile count then attendance of students are marked. In a situation where, the headcount is more than the mobile count, then some students would have dead mobile phones or they would have not carried their phones for that session. These students can give their attendance using their classmate 's student companion app, which will be verified by the teacher. In an alternative situation, where the headcount is less than the mobile count, there is a chance for proxy attendance inside the class or the beacons covered the mobile phones of the students standing outside the class. In this case, the captured image is send back to the students via student companion app. Students have to select any head (not necessary their heads) to mark their attendance in their registered mobiles. If there exist any coverage problem of the Beacons, it can be rectified since the students in class selects the heads. If it is a proxy attendance issue, then a single head will be selected in more than one mobile phone. Teacher can roll call these collided heads and rectify this problem.
The architecture diagram is shown in Figure 1. Raspberry Pi board [22] is connected with the camera and the feeds from this camera is send to both students and teachers interface. The Beacon is connected via Bluetooth to both these interfaces. The attendance is then send to the cloud via Raspberry Pi for further reference and consolidation purpose. Both the teachers and students here could connect with their respective portal through, Raspberry Pi which could be hosted in a local network of the school or college. The usage of the mobile data is not required, only bluetooth connection with the Intranet device is enough for the proposed system.

Beacon
Beacons [8] can be mounted at the top of a tall building or similar site or anywhere, here it would be mounted in every classroom. They need not be paired like a traditional bluetooth device, as they work by the principle of Bluetooth Low energy which gets automatically paired once a user walks into its range. Beacons can be programmed with an app where once users walks onto a range of a beacon, information/data can be displayed or manipulated according to it. Figure 2 shows the model of beacon and its components. It ranges in various sizes and shapes. This system uses beacons which looks similar to carrom coins and any number of devices can be connected to the range of a beacon. The coverage area of each beacon is 70-120m. Since we have used it inside the classroom for attendance purpose, the beacon is programmed for the coverage to 30-50m.

Class web session interface for teachers and students
A Beacon would be installed on every class. The details of teachers, students and class timetable of that particular class would be enrolled in the web interface. Every student would be registered with their registration numbers and teachers registered will be assigned class to teach their subject. Every class session/period the teacher would open the web interface in the classroom and start a session for their subject (E.g. For Programming in C period, the teacher for C Programming would host a session for that class) to a class. Beacons, which act as proximity sensors would be programmed according to a location. As only the students inside that class should be able to join the session, the attendance of each student would be stored.

Crawling the class
The camera used for obtaining the image of the class is a surveillance camera that can be accessed through the internet provided the camera has been connected to a working WiFi. To automate the process of clicking the image, the entire accessing of the camera has been implemented through web crawling using the puppeteer tool. Web crawler [9] automatically loads the surveillance camera website and enters the user credential for accessing the camera. Once credentials are entered, the snapshot of the surveillance video is taken and saved into the server after which the picture gets saved and the python script containing the pre-trained Mask R-CNN [7] model is run.

Mask R-CNN
Mask R-CNN [7] is simple region based CNN for object instance segmentation. This model is easy to train and generalize to other tasks, adding only a small overhead to Faster R-CNN. Mask R-CNN also generates a high-quality segmentation mask for each object it detects. The model uses ConvNet to extract the features from image. The features are passed through two stages. The first stage is Region Proposal Network (RPN) that is used to propose the possible object bounding boxes. The second stage is Fast R-CNN that extracts the features from bounding boxes which performs the classification using RoI pooling layers. In addition to this it also produces the binary mask for each region of interest. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone. This model detects 91 classes as listed in Table 1 consisting of people, animals, cars, chairs, phones, etc which is trained on the MS COCO dataset (Microsoft Common objects in Context) [23] grouped into 11 super categories. The dataset consists of 2.5 million labelled instances of 328k images. The pre-trained weight vectors are used further to train and test our data. For our implementation, the classification of people alone are obtained along with their positioning. Also a loop is run to project these predictions on the image. Figure  3 shows the output of Mask R-CNN. The head of the persons are identified and marked in the given image.

Head Identification
The Mask R-CNN image processing model is used to identify the number of people and the exact position of these people. Once the model runs on the classroom image and it detects the people, the coordinates for their exact position is obtained and a python script is run to draw rectangles at these positions using OpenCV [11] libraries. Processed image obtained from the model is saved in a folder. Also as an addendum, another file is saved containing the URL of image, width and height of the image, count of people and the coordinates of the rectangles plotted on the image. All these data along with the image gets saved in the firebase [10].

Cloud data storage
Firebase is used for storing the students and teachers details and also the specific attendance details for each session. The image and coordinates for the image retrieval by the students are also stored in one collection, flags related to the presence of phones in the beacon vicinity is also triggered in the database. Using this database, we can consolidate and generate the students attendance report.

Teacher Companion App
The main purpose of the app is to host a class session and end the session according to the teacher's decision. The moment the teacher starts the session the class crawling activity starts and the whole model runs automatically. Only after the start of the session, the students are allowed to attend these sessions and mark attendance. The attendance for the class will be updated in teacher companion app, this will be verified by the teacher inorder to find any cases of proxy attendance.

Student Companion App
The app opens with a page for firebase authentication [12] and inputs for the student to enter their credentials for successfully logging into the app. Once the user logins in the mobile app, it is made to scan beacons in its vicinity. A thread is running to keep scanning in the background, such that the beacon could keep a check on the connected devices. When a student leaves the vicinity after the start of the session, the scanning thread detects the missing beacon and updates the firebase to inform the admin/teacher about the defaulting student. At the beginning of the session, an image is provided to each connected student with a clickable box on each students head (provided from the model). Once the students click on these free boxes the firebase is notified of the presence of the respective students. The image will be updated with the heads selected along with an unique student ID . This app can run in the background for continuous scanning of beacons. Any Student without a mobile phone or a dead phone will be able to provide attendance wherein a provision in the student app allows another student (bench partner) to give his/her attendance, where both of their names will be notified to the teacher for verification at end of that hour. If a head is selected by more than one student, then the unique ID of these selected mobile phones and their will be notified to the teacher companion app for verification.

Test Case results
The attendance system is tested in our classroom for the following scenarios. The system is tested for correct attendance entry, head count mismatch, proxy attendance and coverage outside classroom. Figure 4 shows the working of various test cases in detail.  The output of various scenarios are listed below: (i) Student is physically present in class -Attendance is awarded (ii) Student is physically not present in class -Attendance is not awarded (iii) Student has not brought mobile or no charge in battery -Attendance entry is made in classmate's student companion app verified by teacher for awarding attendance (iv) Student gave phone to his friend for proxy attendance -Can be identified in head selection and verified by teacher for not awarding the attendance for those students (v) Mobile phones of students standing outside the classroom are covered by beacon -Can be identified in head selection and only selected students inside the class will be awarded the attendance

Conclusion
Efficacy of the attendance system has always been a critical problem for any educational institution. Time is used efficiently in making roll-call, proxy attendance, mistakes, sound clarity due to noise and manual entry for report generation are some of the issues. In automatic biometric or face recognition too, students has to make manual entry for each session, also after entering their biometric they may leave the class without attending it. Efficient usage of the allocated hour is achieved when this proposed system is deployed in any classroom. Teachers, when they activate the session an image is captured by the back end, processed using a deep learning model for head detection, and send to the students interface for selection. At the end of an hour, the teacher would receive any information regarding the discrepancy in the attendance reported, and the required action can be taken, after which the attendance is stored in a database.
The system is tested for various test case scenarios and it is working perfectly for all the cases. The system was tested in classrooms of different landscapes and during majority of instance, the system generated a high quality segmentation over the heads of each students, proving the systems high efficacy. Further improvements to the teacher and student interface like making it more intuitive by adding a dynamic time table and chat box for effective student-teacher communication and also create a system wherein unwanted usage of mobile phones in the class during teaching would be informed by the system to the teacher can be made.