CN116092134A - A Fingerprint Liveness Detection Method Based on Deep Learning and Feature Fusion - Google Patents

Abstract

A fingerprint living body detection method based on deep learning and feature fusion comprises the following steps: 1) Establishing a basic data set: a large number of fingerprint image datasets are created, including live fingerprint images and false fingerprint images. 2) Constructing a deep neural network model: the structure of the mobile network is finely adjusted by taking the MobileNet V2 model as a basic network, and a lightweight deep neural network model suitable for fingerprint living body detection is designed. 3) Feature extraction: respectively preparing a gray value map, a direction field map and a Local Binary Pattern (LBP) map of the fingerprint, inputting the three maps into a deep neural network model, and respectively extracting features by using the deep neural network model. 4) Feature fusion: and 3) carrying out fusion calculation on the features extracted in the step 3) and different feature layers. 5) And (3) completing deep neural network model training: training the deep neural network model by using the training set of the basic data set in the step 1), so that the performance of the deep neural network model is optimal. 6) Classifying the true and false fingerprints: and (3) classifying by using the test set of the basic data set in the step (1) and using the optimal deep neural network model trained in the step (5), and finally obtaining the accurate result of fingerprint living body detection. The invention can provide a fingerprint living body detection method based on deep learning and feature fusion, which adopts a deep learning algorithm to design a convolutional neural network with feature fusion on the basis of not increasing the hardware cost of products, namely, different feature graphs are fused by using a cascade fusion function in a convolutional layer, and multi-layer information features of fingerprint images are extracted by fully utilizing complementary information so as to ensure the accuracy and safety of identities and rapidly and accurately detect the fingerprints of human living bodies.

Description

A Fingerprint Liveness Detection Method Based on Deep Learning and Feature Fusion

technical field

The invention relates to the fields of image processing, fingerprint recognition technology, etc., and in particular to a fingerprint living body detection method based on deep learning and feature fusion.

Background technique

At present, fingerprint identification technology is being accelerated in the fields of finance, telecommunications, information security, and e-government, involving many application scenarios such as information security and identity verification. However, with the advancement of technology, the authentication scheme relying only on fingerprints is not safe enough, and it lacks advanced technologies such as "living detection". On the one hand, fingerprint liveness detection technology can effectively avoid the security risks of ordinary fingerprint authentication technology, effectively prevent credential database attacks, and reduce the chance of fingerprint information leakage; on the other hand, using fingerprint liveness detection technology can also realize identity detection and identify illegal intrusion behavior, anti-collision, and meet the quality requirements of safety management. Fingerprint liveness detection is an important technology, which provides users with more accurate and reliable security verification, and helps to ensure the integrity of information security and personal privacy, so its importance cannot be ignored.

Fingerprint liveness detection methods can be divided into hardware detection method and software detection method at present. The hardware detection method is to detect the characteristics of the finger by adding additional hardware, which will lead to an increase in product cost, and is inconvenient to operate, which is not conducive to popularization. Compared with the hardware detection method, the software detection method is a cheaper method based on image processing, which is easier to implement, and the ability can be improved by updating the software. At present, most of the algorithms at the software level are based on the shallow manual feature design SVM (Support Vector Machine) solution. The number of extracted features is relatively small and the feature type is single, resulting in low accuracy and slow detection speed of live fingerprints. There is also an algorithm combining convolutional neural network and SVM (Support Vector Machine), but this algorithm divides feature extraction and classification recognition into two parts, so that the performance of the algorithm for fingerprint liveness detection cannot be optimal.

Contents of the invention

In order to overcome the problems of the current live fingerprint detection technology, such as the small number of extracted features, single feature type, low accuracy, slow detection speed, and low performance. The present invention aims to provide a fingerprint biometric detection method based on deep learning and feature fusion. On the basis of no additional increase in product hardware costs, a deep learning algorithm is used to design a convolutional neural network for feature fusion, that is, a cascaded neural network is used in the convolution layer. The fusion function fuses different feature maps, and makes full use of complementary information to extract the multi-layer information features of the fingerprint image to ensure the accuracy and security of the identity, and can quickly and accurately detect human living fingerprints.

The present invention proposes the following technical solutions: a fingerprint living body detection method based on deep learning and feature fusion, comprising the following steps:

1) Establish basic data sets: establish a large number of fingerprint image data sets, including living fingerprint images and false fingerprint images.

2) Build a deep neural network model: Based on the MobileNetV2 model, fine-tune its structure and design a lightweight deep neural network model suitable for fingerprint liveness detection.

3) Feature extraction: Prepare the gray value map, direction field map and local binary pattern (LBP) map of the fingerprint respectively, input the three kinds of maps into the deep neural network model, and use the deep neural network model to perform feature extraction respectively.

4) Feature fusion: For the features extracted in step 3), fusion calculations are performed on different feature layers.

5) Complete the training of the deep neural network model: use the training set of the basic data set in step 1) to train the deep neural network model to achieve optimal performance.

6) Classify true and false fingerprints: use the test set of the basic data set in step 1), use the optimal deep neural network model trained in step 5) to classify, and finally obtain the accurate result of fingerprint liveness detection.

Further, in the step 1), the establishment of the basic data set: the fingerprint data set used in this paper comes from LivDet2013 and LivDet2015, and these two data sets are official data sets of the global fingerprint liveness detection competition. Among them, LivDet2013 contains 4 sub-datasets: Biometrika, CrossMatch, Italdata and Swipe. LivDet2015 contains 4 sub-datasets: CrossMatch, Digital_Persona, GreenBit, Hi_Scan. The LivDet dataset has two parts: training set and test set.

In the step 2), the method for constructing a deep neural network model is as follows: for the accuracy, real-time performance and generalization ability of fingerprint living detection, the determination of the network structure is the most important. First of all, it is necessary to determine the input size and the number of convolutional layers of the single-channel convolutional network. Based on the structure of MobileNetV2, the present invention fine-tunes its structure and adds the idea of feature fusion to design a lightweight network model suitable for fingerprint liveness detection. The model includes convolution layer 1 (Convolution1), inverted residual block 1 (bottleneck1), inverted residual block 2 (bottleneck2), inverted residual block 3 (bottleneck3), inverted residual block 4 (bottleneck4), global mean pooling layer (Pooling AVE), convolutional layer 2 (Convolution2) and output layer (Softmax).

In the step 3), the feature extraction method is as follows: using the inverted residual block 1 (bottleneck1) and the inverted residual block 2 (bottleneck2) from the gray value map, direction field map and local binary pattern of the fingerprint respectively (LBP) to extract features from the graph. in,

Gray value map: The gray value information of fingerprints is very important for fingerprint identification. The gray value map of the present invention uses the original fingerprint image to directly tile the pixels, and extracts column vectors as the extracted fingerprint features.

Direction field map: The fingerprint image has a relatively clear direction field, which describes the direction pattern information of fingerprint ridges and bone lines. As a global and reliable feature of fingerprints, the direction field is a very important part of the existing mainstream fingerprint recognition technology. Many methods are used to estimate the fingerprint direction field. The present invention uses a gradient-based method to estimate the direction field. The specific algorithm process is:

a) Divide the fingerprint image I into a series of non-overlapping blocks of size W×W.

b) Calculate the gradient vector [G _x (x, y), G _y (x, y)] ^T of each point in the block along the X and Y directions respectively. The calculation method is shown in formula (1).

c) Calculate the block gradient vector [ _GBx ,G _By ] ^T of each block according to formula (2), and convert it into block direction θ (0≤θ<π) according to formula (3).

Local Binary Pattern (LBP) map: It is used to describe the local texture features of the image. The LBP algorithm has low computational complexity and has grayscale invariance and rotation invariance, so it has been widely used. The specific process can be expressed as:

Among them: (x _c , y _c ) represents the center; p _c represents the pixel of the center point; p _i represents the pixel of the surrounding point; P represents the number of surrounding pixel points;

In the step 4), the feature fusion method is as follows: performing feature fusion on different feature layers in the step 3), and then using the inverted residual block 3 (bottleneck3) and the inverted residual block 4 (bottleneck4) from the fusion The distinguishing information is further extracted from the feature map, and the location with specific features is located as a reference for fingerprint liveness detection to ensure the accuracy of identity. Through the fusion of feature layers, a variety of information can be fully utilized to overcome the lack of single information, and it helps to improve the accuracy and generalization ability of the model.

In the step 5), the described method of completing the training of the deep neural network model is: in the step 4), the fused feature vector adopts global mean pooling, quantizes the feature map of the last convolutional layer, and then the convolution The layer is connected to the fully connected layer, followed by a Softmax logistic regression classification layer to achieve classification. This network structure connects the convolutional layer and the traditional neural network layer together. The convolutional layer can be regarded as a feature extractor, and the obtained features are then classified by the traditional neural network layer, and finally the accuracy of fingerprint liveness detection is obtained. result. Utilize the training set of the basic data set in step 1) to train the deep neural network model to make its performance optimal.

The beneficial effects of the present invention are that, firstly, the present invention utilizes the convolution technology to effectively reduce the amount of computation and improve the detection speed; secondly, the present invention uses the MobileNetV2 model as the basic network, and since MobileNetV2 belongs to a lightweight network, it can guarantee its performance At the same time, by fine-tuning its structure and adding the idea of feature fusion, it can run on the embedded platform in real time with less calculation, which improves the accuracy, real-time and generalization ability of fingerprint liveness detection; finally, the present invention can Simultaneously realize feature extraction and classification recognition, and directly extract features through the deep learning model, and the features learned in a data-driven manner are more general, so that they can be applied to various deception attacks, and can greatly improve the robustness and versatility of the present invention. The ability to optimize the performance of fingerprint biometric detection.

Description of drawings

Figure 1 is the parameter table of LivDet2013;

Figure 2 is the parameter table of LivDet2015;

Fig. 3 is the deep neural network structural diagram of the present invention;

Figure 4 is a flowchart of the implementation of the entire algorithm.

Detailed ways

Example 1

In order to facilitate a better understanding, the present invention will be further described below in conjunction with the accompanying drawings.

Referring to Figures 1 to 4, a fingerprint liveness detection method based on deep learning and feature fusion includes the following steps:

1) Establish basic data sets: The fingerprint data sets used in this paper come from LivDet2013 and LivDet2015, which are the official data sets of the global fingerprint liveness detection competition. Among them, LivDet2013 contains 4 sub-datasets: Biometrika, CrossMatch, Italdata and Swipe. LivDet2015 contains 4 sub-datasets: CrossMatch, Digital_Persona, GreenBit, Hi_Scan. The LivDet dataset has two parts: training set and test set. Figure 1 and Figure 2 respectively show the parameters of LivDet2013 and LivDet2015, including the image size of fingerprints, the number of real samples, the number of fake samples, and the number of materials used to prepare fake fingerprints.

2) Building a deep neural network model: For the accuracy, real-time and generalization capabilities of fingerprint liveness detection, the determination of the network structure is the most important. First of all, it is necessary to determine the input size and the number of convolutional layers of the single-channel convolutional network. Based on the structure of MobileNetV2, the present invention fine-tunes its structure and adds the idea of feature fusion to design a lightweight network model suitable for fingerprint liveness detection. The model includes convolution layer 1 (Convolution1), inverted residual block 1 (bottleneck1), inverted residual block 2 (bottleneck2), inverted residual block 3 (bottleneck3), inverted residual block 4 (bottleneck4), global mean pooling layer (Pooling AVE), convolutional layer 2 (Convolution2) and output layer (Softmax). Fig. 3 is a structural diagram of the deep neural network of the present invention.

3) Feature extraction: Prepare the gray value map, direction field map and local binary pattern (LBP) map of the fingerprint respectively, input the three kinds of maps into the deep neural network model, and use the inverted residual block 1 (bottleneck1) and inverted residual block 1 (bottleneck1) and inverted residual The difference block 2 (bottleneck2) extracts features from the gray value map, direction field map and local binary pattern (LBP) map of the fingerprint respectively. in,

Gray value map: The gray value information of fingerprints is very important for fingerprint identification. The gray value map of the present invention uses the original fingerprint image to directly tile the pixels, and extracts column vectors as the extracted fingerprint features.

Direction field map: The fingerprint image has a relatively clear direction field, which describes the direction pattern information of fingerprint ridges and bone lines. As a global and reliable feature of fingerprints, the direction field is a very important part of the existing mainstream fingerprint recognition technology. Many methods are used to estimate the fingerprint direction field. The present invention uses a gradient-based method to estimate the direction field. The specific algorithm process is:

a) Divide the fingerprint image I into a series of non-overlapping blocks of size W×W.

b) Calculate the gradient vector [G _x (x, y), G _y (x, y)] ^T of each point in the block along the X and Y directions respectively. The calculation method is shown in formula (1).

c) Calculate the block gradient vector [ _GBx ,G _By ] ^T of each block according to formula (2), and convert it into block direction θ (0≤θ<π) according to formula (3).

Local Binary Pattern (LBP) map: It is used to describe the local texture features of the image. The LBP algorithm has low computational complexity and has grayscale invariance and rotation invariance, so it has been widely used. The specific process can be expressed as:

Among them: (x _c , y _c ) represents the center; p _c represents the pixel of the center point; p _i represents the pixel of the surrounding point; P represents the number of surrounding pixel points;

4) Feature fusion: perform feature fusion on different feature layers in step 3), and then use the inverted residual block 3 (bottleneck3) and inverted residual block 4 (bottleneck4) to further extract distinguishing features from the fused feature map Information, located at a location with specific characteristics, as a reference for fingerprint liveness detection to ensure identity accuracy. Through the fusion of feature layers, a variety of information can be fully utilized to overcome the lack of single information, and it helps to improve the accuracy and generalization ability of the model.

5) Complete the training of the deep neural network model: use global mean pooling for the fused feature vector in step 4), quantize the feature map of the last convolutional layer, and then connect the convolutional layer with the fully connected layer, followed by a Softmax logistic regression classification layer implements classification. This network structure connects the convolutional layer and the traditional neural network layer together. The convolutional layer can be regarded as a feature extractor, and the obtained features are then classified by the traditional neural network layer, and finally the accuracy of fingerprint liveness detection is obtained. result. Utilize the training set of the basic data set in step 1) to train the deep neural network model to make its performance optimal.

6) Classify true and false fingerprints: Utilize the test set of the basic data set in step 1), and use the optimal deep neural network model trained in step 5) to classify, and the accuracy rate of the final test reaches a better effect.

Claims (5)

1. The fingerprint living body detection method based on deep learning and feature fusion is characterized by comprising the following steps of:

1) Establishing a basic data set: a large number of fingerprint image datasets are created, including live fingerprint images and false fingerprint images.

2) Constructing a deep neural network model: the structure of the mobile network is finely adjusted by taking the MobileNet V2 model as a basic network, and a lightweight deep neural network model suitable for fingerprint living body detection is designed.

3) Feature extraction: respectively preparing a gray value map, a direction field map and a Local Binary Pattern (LBP) map of the fingerprint, inputting the three maps into a deep neural network model, and respectively extracting features by using the deep neural network model.

4) Feature fusion: and 3) carrying out fusion calculation on the features extracted in the step 3) and different feature layers.

5) And (3) completing deep neural network model training: training the deep neural network model by using the training set of the basic data set in the step 1), so that the performance of the deep neural network model is optimal.

6) Classifying the true and false fingerprints: and (3) classifying by using the test set of the basic data set in the step (1) and using the optimal deep neural network model trained in the step (5), and finally obtaining the accurate result of fingerprint living body detection.

2. The fingerprint living body detection method based on deep learning and feature fusion according to claim 1, wherein in the step 2), the method for constructing the deep neural network model is as follows: the determination of the network structure is important for the accuracy, real-time and generalization capability of fingerprint live detection. Firstly, the input size and the number of convolution layers of a single-path convolution network are required to be determined, the structure of the single-path convolution network is finely adjusted based on the structure of the MobileNet V2, a feature fusion idea is added, and a lightweight network model suitable for fingerprint living body detection is designed. The model includes Convolution layer 1 (Convolition 1), inverse residual block 1 (bottlebeck 1), inverse residual block 2 (bottlebeck 2), inverse residual block 3 (bottlebeck 3), inverse residual block 4 (bottlebeck 4), global average Pooling layer (Pooling AVE), convolution layer 2 (Convoltion 2), and output layer (Softmax).

3. The fingerprint living body detection method based on deep learning and feature fusion according to claim 1, wherein in step 3), the feature extraction method is as follows: features are extracted from the gray value map, the direction field map, and the Local Binary Pattern (LBP) map of the fingerprint using the inverse residual block 1 (bottleneck 1) and the inverse residual block 2 (bottleneck 2), respectively.

4. The fingerprint living body detection method based on deep learning and feature fusion according to claim 1, wherein in step 4), the feature fusion method is as follows: and 3) carrying out feature fusion on different feature layers in the step 3), and then further extracting distinguishing information from the fused feature map by using a reverse residual block 3 (bottleneck 3) and a reverse residual block 4 (bottleneck 4), and positioning the information to a position with specific features as a reference for fingerprint living body detection so as to ensure identity accuracy. Through fusion of the feature layers, multiple information can be fully utilized, the defect of single information is overcome, and the accuracy and generalization capability of the model are improved.

5. The fingerprint living body detection method based on deep learning and feature fusion according to claim 1, wherein in step 5), the method for training the deep neural network model is as follows: and (3) global mean pooling is adopted for the feature vectors after fusion in the step (4), the feature map of the last convolution layer is quantized, then the convolution layer is connected with a full connection layer, and then a Softmax logistic regression classification layer is connected to realize classification. The network structure enables the convolution layer and the traditional neural network layer to be connected together, the convolution layer can be regarded as a feature extractor, the obtained features are classified by the traditional neural network layer, and finally the accurate result of fingerprint living body detection is obtained. Training the deep neural network model by using the training set of the basic data set in the step 1), so that the performance of the deep neural network model is optimal.

Images