CN110634155A - A method and device for target detection based on deep learning - Google Patents

Abstract

The invention discloses a deep learning-based target detection method and device, and relates to the technical field of computers. A specific implementation of the method includes: using a deep learning network to determine the detection target in the initial frame image in each cycle, wherein, taking each consecutive number of frames in the continuously collected multi-frame images as a cycle, the first frame image in each cycle One frame of image is the start frame image; for each frame image except the start frame image in the cycle, use the characteristics of the detected target in the previous frame image, and carry out target tracking according to the dynamic tracking algorithm to determine the target frame except for the cycle. The detection target in each frame image other than the initial frame image; after the detection target in each frame image is determined, the determined detection target is output. This embodiment can greatly reduce the amount of calculation without reducing the recognition accuracy, thereby well meeting the needs of the field of unmanned driving or robotics, and is easy to deploy on hardware.

Description

A method and device for target detection based on deep learning

technical field

The present invention relates to the field of computer technology, in particular to a deep learning-based target detection method and device.

Background technique

Object detection methods based on deep learning are currently widely used in time-critical video analysis (Time-critical video analysis) scenarios, such as robot navigation and autonomous driving. These methods will detect vehicles and pedestrians in each frame of the image, which is a basic technology for computer vision applications in the field of robot navigation and automatic driving.

The bounding boxes output by existing target detection algorithms rely on the Non-max Suppression (non-maximum value suppression) method to output a bounding box with the highest probability. The Non-max Suppression method needs to first calculate the IoU (Intersection-over-Union) value of the detected frame and the reference frame, and select the largest frame output frame from it.

In the course of realizing the present invention, the inventor finds that there are at least the following problems in the prior art:

Existing target detection methods need to detect each frame of image, and the amount of calculation is huge, which cannot well meet the needs of the field of unmanned driving or robotics.

Contents of the invention

In view of this, the embodiments of the present invention provide a method and device for target detection based on deep learning, which can greatly reduce the amount of calculation without reducing the recognition accuracy, so as to well meet the needs of unmanned driving or robotics. requirements, and is easy to deploy on hardware.

In order to achieve the above object, according to an aspect of the embodiments of the present invention, a method for object detection based on deep learning is provided.

A target detection method based on deep learning, including: using a deep learning network to determine the detection target in the initial frame image in each cycle, wherein, taking each consecutive number of frames in the continuously collected multi-frame images as a cycle, each The first frame image in the cycle is the start frame image; for each frame image in the cycle except the start frame image, use the characteristics of the detected target in the previous frame image, and perform target tracking according to the dynamic tracking algorithm to determine the The detection target in each frame image except the initial frame image in the cycle; after each detection target in a frame image is determined, the determined detection target is output.

Optionally, the dynamic tracking algorithm is a particle filter algorithm.

Optionally, for each frame image in the period except the initial frame image, use the features of the detected target in the previous frame image, and perform target tracking according to the dynamic tracking algorithm, so as to determine the frame image in the period except the initial frame image The step of detecting the target in each frame image outside includes: calculating the feature histogram of the detection target in the initial frame image in the cycle; on the second frame image plane in the cycle, deploying according to preset rules Particles, and calculate the feature histogram of the position of each particle; according to the similarity between the feature histogram of the position of each particle and the feature histogram of the detection target, determine the detection target in the second frame image; from The third frame image in the period starts to calculate the feature histogram of the detected target in the previous frame image of each frame, and according to the position of the detected target in the previous frame image of each frame, on the image plane of each frame Deploy the particles, and then calculate the feature histogram of the position of each particle on the image plane of each frame, according to the similarity between the feature histogram of the position of each particle in each frame and the feature histogram of the detection target in the previous frame, determine the Describe the detection target in each frame of image.

Optionally, the deep learning network is based on a faster image region-based convolutional neural network or a YOLO object detection deep network.

According to another aspect of the embodiments of the present invention, an object detection device based on deep learning is provided.

A target detection device based on deep learning, comprising: a detection module, configured to use a deep learning network to determine the detection target in the initial frame image in each cycle, wherein each consecutive number of frames in the continuously collected multi-frame images As a cycle, the first frame image in each cycle is the start frame image; the tracking module is used to use the features of the detected target in the previous frame image for each frame image in the cycle except the start frame image, according to the dynamic The tracking algorithm performs target tracking to determine the detection target in each frame image except the initial frame image in the period; the output module is used to output the determined detection target after each determination of the detection target in a frame image Target.

Optionally, the dynamic tracking algorithm is a particle filter algorithm.

Optionally, the tracking module is further configured to: calculate the feature histogram of the detected target in the initial frame image within the period; deploy particles according to preset rules on the image plane of the second frame within the period, And calculate the feature histogram of the position of each particle; according to the similarity between the feature histogram of the position of each particle and the feature histogram of the detection target, determine the detection target in the second frame image; from the Starting from the third frame image in the cycle, calculate the feature histogram of the detected target in the previous frame image of each frame, and deploy particles on the image plane of each frame according to the position of the detected target in the previous frame image of each frame , and then calculate the feature histogram of the position of each particle on the image plane of each frame, and determine the feature histogram of each particle position according to the similarity between the feature histogram of the position of each particle in each frame and the feature histogram of the detection target in the previous frame. Detected objects in frame images.

Optionally, the deep learning network is based on a faster image region-based convolutional neural network or a YOLO object detection deep network.

According to yet another aspect of the embodiments of the present invention, an electronic device is provided.

An electronic device, comprising: one or more processors; a memory for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more Multiple processors realize the object detection method based on deep learning provided by the present invention.

According to yet another aspect of the embodiments of the present invention, a computer-readable medium is provided.

A computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, the object detection method based on deep learning provided by the present invention is implemented.

An embodiment of the above-mentioned invention has the following advantages or beneficial effects: use the deep learning network to determine the detection target in the initial frame image in each cycle, wherein each consecutive number of frames in the continuously collected multi-frame images is taken as a cycle , the first frame image in each period is the initial frame image; for each frame image in the period except the initial frame image, use the characteristics of the detected target in the previous frame image, and perform target tracking according to the dynamic tracking algorithm to determine The detection target in each frame image except the initial frame image in the cycle. It can greatly reduce the amount of calculation without reducing the recognition accuracy, so as to well meet the needs of unmanned driving or robot fields, and is easy to deploy on hardware.

The further effects of the above-mentioned non-conventional alternatives will be described below in conjunction with specific embodiments.

Description of drawings

The accompanying drawings are used to better understand the present invention, and do not constitute improper limitations to the present invention. in:

Fig. 1 is a schematic diagram of main steps of a target detection method based on deep learning according to an embodiment of the present invention;

2 is a schematic diagram of main modules of a deep learning-based target detection device according to an embodiment of the present invention;

FIG. 3 is an exemplary system architecture diagram to which an embodiment of the present invention can be applied;

Fig. 4 is a schematic structural diagram of a computer system suitable for implementing a terminal device or a server according to an embodiment of the present invention.

Detailed ways

Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

FIG. 1 is a schematic diagram of main steps of a deep learning-based object detection method according to an embodiment of the present invention.

As shown in FIG. 1 , the object detection method based on deep learning in the embodiment of the present invention mainly includes steps S101 to S103.

Step S101: Use the deep learning network to determine the detection target in the initial frame image in each cycle.

Wherein, each consecutive frame of multiple frames of images collected continuously is taken as a cycle, and the first frame image in each cycle is the initial frame image.

The deep learning network can be based on Faster R-CNN (a faster convolutional neural network based on image regions) or YOLO object detection deep network. YOLO is You Only Look Once, which is a deep object detection network.

Step S102: For each frame image in the period except the initial frame image, use the features of the detected target in the previous frame image to track the target according to the dynamic tracking algorithm to determine the target frame image in the period except the initial frame image Detection objects in each frame of image.

Specifically, the dynamic tracking algorithm may be a particle filter algorithm.

The feature of the detection target may be any pixel feature among various pixel features in the frame corresponding to the detection target, such as a color feature.

Step S102 may specifically include: calculating the feature histogram of the detected target in the initial frame image within one cycle; deploying particles according to preset rules on the second frame image plane within this cycle, and calculating the feature of the position of each particle Histogram; according to the similarity between the feature histogram of the position of each particle and the feature histogram of the detection target in the initial frame image, determine the detection target in the second frame image; start from the third frame image in this period, calculate The feature histogram of the detected target in the previous frame image of each frame, and according to the position of the detected target in the previous frame image of each frame, deploy particles on the image plane of each frame, and then calculate the position of each particle on the image plane of each frame According to the similarity between the feature histogram of the position of each particle in each frame and the feature histogram of the detection target in the previous frame, the detection target in each frame of image is determined.

On the image plane of the second frame in this period, when the particles are deployed according to the preset rules, the preset rules can be that the particles are uniformly deployed at each position of the image plane of the second frame, or that the particles are deployed on the plane of the second image frame More particles are deployed on and near the position corresponding to the detection target in the initial frame image, and fewer particles are deployed on the plane of the second frame image and away from the position corresponding to the detection target in the initial frame image Particles, wherein the measure of the distance between the position of the deployed particles and the corresponding position of the detection target in the initial frame image, and the measure of the number of deployed particles can be set as required.

Wherein, if the position of the detection target in the initial frame image is position I, then the position corresponding to the detection target in the initial frame image on the plane of the second frame image is: on the second frame image plane and with Position I corresponds to the same position.

Starting from the third frame image in this period, according to the position of the detection target in the previous frame image of each frame, the particles are deployed on the image plane of each frame, specifically, the corresponding The similarity has been normalized, and the corresponding preset number of particles with the highest similarity is selected from the particles in the image plane of the previous frame as the selected particles. When the particles are deployed on the image plane of this frame, the More particles are deployed near the positions corresponding to the selected particles, and fewer particles are deployed far away from the positions corresponding to the selected particles on the image plane of the frame. Wherein, assuming that the position of the selected particle in the image plane of the previous frame is position II, the position corresponding to the selected particle on the image plane of this frame is: the same position corresponding to position II on the image plane of this frame.

Wherein, the similarity corresponding to a certain particle on the image plane of a certain frame is the similarity between the feature histogram of the position of the particle on the image plane of the frame and the feature histogram of the detection target in the previous frame.

According to the similarity between the feature histogram of the position of each particle in each frame and the feature histogram of the detection target in the previous frame, determine the detection target in each frame of image, specifically, the feature histogram of the position of each particle in each frame After the similarity between the graph and the feature histogram of the detected target in the previous frame is normalized, determine each Each determined detection target in the frame image corresponds to a frame, and the frame is parallel to the boundary of the collected image. The frame can also be called a detection frame or a rectangular frame. The corresponding pixel points at the positions of the preset number of particles with the highest similarity degree constitute the size range of the bounding box corresponding to the detection target. The corresponding pixel at the location of the example with the largest similarity is also the pixel with the largest posterior probability. Pixel is the probability of detecting the target pixel.

Step S103: After each determined detection target in a frame of image, output the determined detection target.

It should be noted that the determined detection target of each frame of image is output immediately. Taking a specific application scenario, such as an unmanned driving scene as an example, after determining the detection target in each frame of image, the output of the frame of image Detection target, for example, in step S101, after determining the detection target in the initial frame image in a certain period through the deep learning network, promptly output the detection target in the initial frame image, in step S102, every time a frame image is determined After the detection target in , output the determined detection target.

Since the existing processing method is to detect each frame, although such a processing method has a high accuracy rate, it requires a very large amount of calculation. This amount of calculation is currently unbearable in the field of unmanned driving or robotics. The embodiment of the present invention uses the image recognition (or image detection) algorithm based on the deep learning network combined with the method of dynamic tracking, especially the particle filter technology, to improve the existing method of target detection using the deep learning network target detection algorithm , after using the deep learning network to detect the result of a frame in the period, then use the particle filter to track and detect, and use the result of the particle filter to replace the result of the subsequent frame in the period. In the new period, the first frame uses the deep learning network again The detection is carried out, and the subsequent frames are tracked and detected by particle filter, and the above process is repeated, so that the amount of calculation is significantly reduced when performing target detection. Taking the detection targets as vehicles and pedestrians as an example, the target detection process of the embodiment of the invention will be described in detail below.

Firstly, train the neural network. Specifically, commonly used neural network training methods can be used, such as backpropagation and gradient descent methods, to obtain parameters for identifying vehicles or pedestrians, mainly the weight W and bias b of each layer of the neural network. are the parameters of the neural network. The neural network can use Faster R-CNN or YOLO deep learning network.

Receive multiple frames of images collected continuously, and use the trained neural network to detect the first frame of images, so as to detect vehicles and pedestrians in the frame of images with high accuracy. Specifically, what is detected is the bounding box corresponding to each vehicle and pedestrian.

Starting from the second frame image, a dynamic tracking algorithm is used, specifically, a particle filter method is used to track the detection target in the second frame image and subsequent frame images. Specifically, according to the color features in the frame corresponding to each vehicle and pedestrian in the above-mentioned first frame image, the tone histogram is calculated to obtain the feature histogram of each vehicle and pedestrian. Distribute particles evenly at each position of the second frame image plane, or deploy more particles near the positions corresponding to the vehicles and pedestrians in the first frame image on the plane of the second frame image, and deploy particles farther away from the corresponding first frame image plane. The position of each vehicle and pedestrian in the image deploys fewer particles, and the position distance of the above-mentioned deployed particles corresponds to the measurement standard of the position distance of each vehicle and pedestrian in the first frame image, and the measurement standard of the number of deployed particles can be set as required Certainly.

According to the color feature of the position of each particle deployed on the image plane of the second frame, respectively calculate the feature histogram of the position of each particle on the image plane of the second frame, according to the position of each particle on the image plane of the second frame and the feature histograms of the vehicles and pedestrians in the first frame image, respectively calculate the similarity between the feature histograms of the positions of the particles in the second frame image and the feature histograms of the vehicles and pedestrians in the first frame, In order to obtain the similarity corresponding to each particle on the image plane of the second frame, the detection target in the second image frame is determined according to the similarity corresponding to each particle on the image plane of the second frame.

Each of the feature histograms above may specifically be a tone histogram. After normalizing the similarity corresponding to each particle in the second frame image, select the corresponding preset number of particles with the highest similarity as the selected particles, and the pixels at the positions of these selected particles constitute the corresponding detection target. border. Calculate the feature histogram of each vehicle and pedestrian in the second frame image. Deploy more particles near the positions corresponding to the above-mentioned selected particles on the image plane of the third frame of the image, and deploy fewer particles at positions far away from the positions corresponding to the above-mentioned selected particles. Similarly, on the image plane of the third frame The distance between the position of the deployed particle and the position corresponding to the selected particle in the second frame image, and the measurement standard of the number of deployed particles can be set as required. This process is a resampling process.

According to the method similar to determining the detection target in the second frame image, according to the feature histogram of the position of each particle on the third frame image plane, and the feature histogram of each vehicle and pedestrian in the second frame image, respectively calculate the third The similarity between the feature histogram of the position of each particle in the frame image and the feature histogram of each vehicle and pedestrian in the second frame is used to obtain the corresponding similarity of each particle on the image plane of the third frame, and then determine the corresponding similarity according to each similarity. The detection target in the third frame image takes consecutive frames as a cycle, for example, takes 20 frames as a cycle, and in the 4th to 20th frame, it is the same as the process of determining the detection target in the third frame image above, no longer repeat.

Each cycle repeats the above-mentioned target detection and tracking process of the previous cycle, that is, the starting frame of each cycle uses the deep learning network to determine the detection target, and the other frames in a cycle use the particle filter algorithm for target tracking. For example, the process of using the deep learning network to determine the detection target and the particle filter algorithm for target tracking is repeated every 20 frames. The image detection method based on deep learning can accurately identify vehicles and pedestrians in the image, while the dynamic tracking method can track the target with less calculation, and can use a period of 20 to 30 frames. There are about 50 frames of images. Through the deep learning-based target detection method of the embodiment of the present invention, it is enough to detect a new vehicle or pedestrian entering the camera within half a second, and a high recognition accuracy can be achieved.

The object detection method based on deep learning in the embodiment of the present invention closely combines the image detection technology based on deep learning and the dynamic tracking method, which not only achieves high recognition accuracy but also reduces the amount of calculation, and is easy to deploy on hardware.

Fig. 2 is a schematic diagram of main modules of an object detection device based on deep learning according to an embodiment of the present invention.

The object detection device 200 based on deep learning in the embodiment of the present invention mainly includes: a detection module 201 , a tracking module 202 , and an output module 203 .

The detection module 201 is used to use the deep learning network to determine the detection target in the initial frame image in each cycle, wherein, taking every consecutive number of frames in the continuously collected multi-frame images as a cycle, the first frame image in each cycle is The starting frame image.

The deep learning network can be a faster image region-based convolutional neural network or a YOLO object detection deep network.

The tracking module 202 is used to use the features of the detected target in the previous frame image to track the target according to the dynamic tracking algorithm for each frame image in the period except the initial frame image, so as to determine the frame image in the period except the initial frame image. The detection target in each frame image outside.

The dynamic tracking algorithm may be a particle filter algorithm.

The tracking module 202 can be specifically used to: calculate the feature histogram of the detected target in the initial frame image within one cycle; deploy particles according to preset rules on the second frame image plane within this cycle, and calculate the position of each particle The feature histogram of the feature histogram; according to the similarity between the feature histogram of the position of each particle and the feature histogram of the detection target in the initial frame image, determine the detection target in the second frame image; from the third frame image in this period At the beginning, calculate the feature histogram of the detected target in the previous frame image of each frame, and according to the position of the detected target in the previous frame image of each frame, deploy particles on the image plane of each frame, and then calculate each frame on the image plane The feature histogram of the particle location, according to the similarity between the feature histogram of each particle location in each frame and the feature histogram of the detection target in the previous frame, determine the detection target in each frame of image.

The output module 203 is configured to output the determined detection target after each determination of the detection target in a frame of image.

In addition, the specific implementation content of the object detection device based on deep learning in the embodiment of the present invention has been described in detail in the above-mentioned object detection method based on deep learning, so the repeated content will not be described here.

FIG. 3 shows an exemplary system architecture 300 to which the deep learning-based object detection method or the deep learning-based object detection apparatus according to the embodiment of the present invention can be applied.

As shown in FIG. 3 , the system architecture 300 may include terminal devices 301 , 302 , and 303 , a network 304 and a server 305 . The network 304 is used as a medium for providing communication links between the terminal devices 301 , 302 , 303 and the server 305 . Network 304 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 301 , 302 , 303 to interact with server 305 via network 304 to receive or send messages and the like. Various communication client applications can be installed on the terminal devices 301, 302, 303, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, and the like.

The terminal devices 301, 302, 303 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers and the like.

The server 305 may be a server that provides various services, such as a background management server that provides support for shopping websites browsed by users using the terminal devices 301 , 302 , and 303 . The background management server can analyze and process the received data such as product information query requests, and feed back the processing results (such as target push information, product information) to the terminal device.

It should be noted that the object detection method based on deep learning provided by the embodiment of the present invention can be executed by the server 305 or the terminal equipment 301, 302, 303, and correspondingly, the object detection device based on deep learning can be set on the server 305 or the terminal equipment 301, 302, 303 in.

It should be understood that the numbers of terminal devices, networks and servers in Fig. 3 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Referring now to FIG. 4 , it shows a schematic structural diagram of a computer system 400 suitable for implementing a terminal device or a server according to an embodiment of the present application. The terminal device or server shown in FIG. 4 is only an example, and should not limit the functions and application scope of this embodiment of the present application.

As shown in FIG. 4 , a computer system 400 includes a central processing unit (CPU) 401 that can be programmed according to a program stored in a read-only memory (ROM) 402 or loaded from a storage section 408 into a random-access memory (RAM) 403 Instead, various appropriate actions and processes are performed. In the RAM 403, various programs and data necessary for the operation of the system 400 are also stored. The CPU 401 , ROM 402 , and RAM 403 are connected to each other through a bus 404 . An input/output (I/O) interface 405 is also connected to bus 404 .

The following components are connected to the I/O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 408 including a hard disk, etc. and a communication section 409 including a network interface card such as a LAN card, a modem, or the like. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I/O interface 405 as needed. A removable medium 411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 410 as necessary so that a computer program read therefrom is installed into the storage section 408 as necessary.

In particular, according to the disclosed embodiments of the present invention, the processes described above with reference to the schematic diagrams of main steps can be implemented as computer software programs. For example, the disclosed embodiments of the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the schematic diagram of main steps. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 409 and/or installed from removable media 411 . When this computer program is executed by a central processing unit (CPU) 401, the above-mentioned functions defined in the system of the present application are performed.

It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The schematic diagrams and block diagrams of main steps in the drawings illustrate the architecture, functions and operations of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the principal step diagram or block diagram may represent a module, program segment, or portion of code that contains one or more logic components for implementing the specified Executable instructions for a function. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also to be noted that each block of the block diagrams or major step illustrations, and combinations of blocks in the block diagrams or major step illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or Implementation can be by a combination of special purpose hardware and computer instructions.

The modules involved in the embodiments described in the present invention may be implemented by software or by hardware. The described modules can also be set in a processor, for example, it can be described as: a processor includes a detection module 201 , a tracking module 202 , and an output module 203 . Wherein, the names of these modules do not constitute a limitation on the module itself under certain circumstances. For example, the detection module 201 can also be described as "used to use a deep learning network to determine the module to detect objects".

As another aspect, the present invention also provides a computer-readable medium. The computer-readable medium may be contained in the device described in the above embodiments, or it may exist independently without being assembled into the device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by one of the devices, the device includes: using a deep learning network to determine the detection target in the initial frame image in each cycle , wherein, taking each consecutive number of frames in the continuously collected multi-frame images as a period, the first frame image in each period is the initial frame image; for each frame image in the period except the initial frame image, use the previous The characteristics of the detection target in the frame image are carried out according to the dynamic tracking algorithm to determine the detection target in each frame image except the initial frame image in the cycle; after each determination of the detection target in a frame image, The determined detection target is output.

According to the technical solution of the embodiment of the present invention, a deep learning network is used to determine the detection target in the initial frame image in each cycle, wherein, taking each consecutive number of frames in the continuously collected multi-frame images as a cycle, the first frame image in each cycle One frame of image is the start frame image; for each frame image in the period except the start frame image, use the characteristics of the detected target in the previous frame image, and perform target tracking according to the dynamic tracking algorithm to determine the period except the start frame image. The detection target in each frame image other than the frame image. It can greatly reduce the amount of calculation without reducing the recognition accuracy, so as to well meet the needs of unmanned driving or robot fields, and is easy to deploy on hardware.

The above specific implementation methods do not constitute a limitation to the protection scope of the present invention. It should be apparent to those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A target detection method based on deep learning is characterized by comprising the following steps:

determining a detection target in an initial frame image in each period by utilizing a deep learning network, wherein each continuous frame in a plurality of continuously acquired frames of images is taken as a period, and a first frame image in each period is taken as an initial frame image;

for each frame image except the initial frame image in the period, utilizing the characteristics of the detected target in the previous frame image to track the target according to a dynamic tracking algorithm so as to determine the detected target in each frame image except the initial frame image in the period;

after each determination of a detection target in one frame image, the determined detection target is output.

2. The method of claim 1, wherein the dynamic tracking algorithm is a particle filtering algorithm.

3. The method according to claim 2, wherein the step of determining the detection target in each frame image except the initial frame image in the period by performing target tracking according to a dynamic tracking algorithm using the feature of the detection target in the previous frame image for each frame image except the initial frame image in the period comprises:

calculating a characteristic histogram of a detection target in the initial frame image in the period;

deploying particles on a second frame of image plane in the period according to a preset rule, and calculating a characteristic histogram of the position of each particle;

determining the detection target in the second frame of image according to the similarity between the characteristic histogram of the position of each particle and the characteristic histogram of the detection target;

starting from the third frame image in the period, calculating a feature histogram of a detection target in the previous frame image of each frame, deploying particles on each frame image plane according to the position of the detection target in the previous frame image of each frame, then calculating the feature histogram of the position of each particle on each frame image plane, and determining the detection target in each frame image according to the similarity between the feature histogram of the position of each particle of each frame and the feature histogram of the detection target in the previous frame.

4. The method of claim 1, wherein the deep learning network is a faster image region-based convolutional neural network or a YOLO object detection deep network.

5. An object detection device based on deep learning, characterized by comprising:

the detection module is used for determining a detection target in an initial frame image in each period by utilizing a deep learning network, wherein each continuous frame in a plurality of continuously acquired frames of images is taken as one period, and a first frame image in each period is taken as an initial frame image;

the tracking module is used for tracking the target of each frame image except the initial frame image in the period by utilizing the characteristics of the detected target in the previous frame image according to a dynamic tracking algorithm so as to determine the detected target in each frame image except the initial frame image in the period;

and the output module is used for outputting the determined detection target after each detection target in one frame of image is determined.

6. The apparatus of claim 5, wherein the dynamic tracking algorithm is a particle filtering algorithm.

7. The apparatus of claim 6, wherein the tracking module is further configured to:

calculating a characteristic histogram of a detection target in the initial frame image in the period;

deploying particles on a second frame of image plane in the period according to a preset rule, and calculating a characteristic histogram of the position of each particle;

determining the detection target in the second frame of image according to the similarity between the characteristic histogram of the position of each particle and the characteristic histogram of the detection target;

starting from the third frame image in the period, calculating a feature histogram of a detection target in the previous frame image of each frame, deploying particles on each frame image plane according to the position of the detection target in the previous frame image of each frame, then calculating the feature histogram of the position of each particle on each frame image plane, and determining the detection target in each frame image according to the similarity between the feature histogram of the position of each particle of each frame and the feature histogram of the detection target in the previous frame.

8. The apparatus of claim 5, wherein the deep learning network is a faster image region-based convolutional neural network or a YOLO object detection deep network.

9. An electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method recited in any of claims 1-4.

10. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.

Images