The combined mode of operation of a digital range-gated surveillance device for operating in low illumination conditions

The paper presents the results of researching the method to improve digital range-gated surveillance device base of CCD image sensor for reduced illumination conditions. Due to the rejection of the image intensifier the CCD-base device cannot get images of sufficient quality in such conditions if it is applied in range-gated mode. The reason is insufficient exposure time for each frame. The presented solution of the problem is a combination of both passive and range-gated observation modes. It allows reducing the lower threshold of operating illumination. The paper describes exactly how to combine the two modes at once without framerate decreasing.


Introduction
Range-gated surveillance devices are intended to provide observation in low visibility conditions and via turbid environments [1]. Their principle of operation is based on the limitation of the speed of the light that leads to the fact that when the area is illuminated by short laser pulses, the light reflected from objects located at different ranges to the observer returns at different times proportional to distance. The range-gated device realizes so called gated viewing. It perceives the scene image with a delay to laser pulse so it gets the reflected light only from objects located at specific distance and filters the light reflected from the objects located nearer it [2]. Also such devices can be used for searching of optical devices in the area by glares as a result of the reflection of the laser light [3,4] and for distance measurements [5,6].
Most of range gated surveillance devices are based on CMOS or CCD camera and an image intensifier. The last one is used as an external fast shutter [7] which allows realizing gated viewing and processing delays between laser pulse and image detection by the sensor with less than 10 ns accuracy. The article [8] describes the way to realize the device on interline CCD image sensor without the use of an image intensifier and by this to significantly reduce the weight and size characteristics of the device. The articles [9] and [10] give examples of successful implementation of proposed method.
The lack of abandoning the image intensifier is that the intensifier performs not only the function of a shutter but also intensifies the light subsequently perceived by image sensor, so the system without an image intensifier turns out to be less suitable for work in low illumination conditions. It manifests itself as follows. In case of observating the targets by their glares in low illumination we have enough laser backlighting light to detect them but not enough light and not enough exposure to get the scene image. The viewed image in this case looks like shown in Figure 1. The targets are well detected. However the determination of their locations relative to the terrain or any landmarks is difficult due to too dark background. The increase in exposure by increasing the number of laser pulses doesn't give the desired effect and only makes glares brighter. Moreover the number of laser pulses can not be increased to infinity as it leads to loss of the framerate in case pulses duration cumulative over time of CCD vertical charge transfer exceeds 40 ms, i.e. period of one frame.

Combination of passive and range-gated modes
The difficulties in increasing the number of laser pulses associated with minimum pulse repetition period. For modern small power pulse lasers (such as Pernim Elmer, Laser Components and similar ones) maximum repetition frequency does not exceed 15 kHz. For 150 ns pulse duration the total glow time in one frame is about 22 µs. The value is obtained from consideration that frame period is 40 ms and 30 ms is used for vertical charges transfer (for Sony ICX445 sensor operating in center cutout mode), so we have only 10 ms for laser backlighting. The exposure time, of course, exceeds this value. The exposure time at single pulse is about 1 µs, so for 150 pulses the total exposure value will be 150 µs which is also not enough to perceive the background image in details.
At the same time we have about 30 ms of idle time of photosensitive section of the sensor during vertical charges transfer. This time can be used to perceive the background image in passive mode to further fold the resulting frame with the frame separately obtained in range-gated mode. Thus it will be possible to combine the background image without targes glares with the image containing glares whithout the background.

The technique to combine both subframes
The summation of passive frame with range-gated frame can be processed the same approach as combining several subframes of separate laser pulses, described in the work [8]. The differece is only that the first subframe in combined viewing mode is placed from photosensitive section of the sensor to its buffer section without executing an erase signal applied to each of subframes in range-gated viewing mode. The other parts of image receiving procces are the same including the idea that vertical transfer of charges is initiated only after the last sequence of erasing so called "noise charges" in photosensitive section and shifting "usefull charges" into the storage section of an interline CCD image sensor and not after the first readout procedure as it is recommended by the sensor documentation. Thus, subframes are summed into general image frame in analog mode by charges summation before digitizing of accumulated data, and this is why the time spending on vertical transfer and on data readout is requied once per general frame and not per each of subframes. At that fact we can combine the background image accumulated in passive mode with target glares range-gated image witout the framerate decrease.
The sample waveforms to contorl interline CCD in such mode are given in Figure 2. At this figure they are presented for Sony ICX618 sensor but waveforms for other sensors look similar and differ only in order to apply high voltage to corresponding inputs while processing charges shifting from sensetive section into vertical transfer section. Laser is the signal launching laser emitter. Noise is a nominal signal which means laser light return from distances less that requied, i.e. reflected from dust, smoke, etc. Reflected pulse shows time of detecting of useful light reflected from target objects.
The process of sensor controlling and image persiving can be conventionally divided into three stages. At the first stage we apply vertical transfer of previous image frame. At the same time new image is viewed in passive mode during requied exposure time. At the second stage the accumulated charges in sensetive section are shifted into vertical transfer section where they are being stored during viewing the reflected light of laser pulses. After that the third stage starts. Several sequences of Laser and SUB pulses are applied with requied delays to each other depending on the viewing distance. At SUB pulse all accumulated noise data is erased in sensetive area. Hight voltage (usually +15V) pulse at corresponding V-phases shiftes usefull data from sensetive area into vertical transfer area where this data is combined with already storaged one. After the last sequence the combined general image is ready to be transferred by vertical transfer procedure, and the process repeats.

Mode application
Required modifications of digital range-gated surveillance devices concern only their software updating and only in part responsible for the formation of contorl signal for CCD sensor and CCD processor. All other modules and software parts including image processing module remain the same. The hardware of the the device is not changed too. This allows not only applying new mode of viewing in future perspective digital range-gated surveillance devices but also upgrading already  Figure 3 presents sample image viewed in same illumination conditions as the image given in Figure 1. As it is seen the operator of the device can recognise terrain details and to determine their location in the field of view relative to well detected landmarks. For better recognition on printed picture target glares are circled. The conducted tests and experiments showed that combined mode allows reducing the lower threshold of operating illumination of range-gated survaillance device in comparison to the device operating only in gated viewing mode. For example, the picture given at Figure 3 is taken in 0.01 lux illuminance at 25 fps framerate.

Conclusion
The designed operating mode of digital range-gated surveillance device without an image intensifier in its construction allows operating it in lower illumination conditions than in just gated-viewing mode. It is necessary to note that this mode can be applied not instead of gated-viewing mode but alternating to it depending on specific view conditions automatically or manually by the operator. The experiments have confirmed mode applicability in practice.