CN111091492B - A light transfer method for face images based on convolutional neural network - Google Patents

Abstract

A face image light transfer method based on convolutional neural network, which realizes the light transfer of face image by using convolutional neural network CNN. The method is mainly implemented by two parts: lighting model training and lighting classification, lighting matching and lighting transfer based on image style transfer. First, by combining the convolutional neural networks VGG19 and VGG16 to complete the illumination classification on the Yale Face face data set and the PIE face data set, a model that can classify the illumination of face images is obtained; The illumination matching of face images can obtain images similar to the given face image illumination from the face image illumination data set; finally, through the illumination classification model, the given reference face illumination images are extracted and processed. The migration to the input face image realizes the overall migration of the illumination of a single face image.

Description

A light transfer method for face images based on convolutional neural network

technical field

The invention relates to a method for light migration of a face image based on a convolutional neural network, and belongs to the field of computer vision.

Background technique

The lighting effect of images is a research hotspot in many research directions in the field of computer vision, and it is also very important for portrait images themselves. The effect of light and shadow has a wide range of application requirements in modern digital film and television production, portrait photography beautification, advertising and other artistic design. One of the challenges in film and television production is how to capture the best shot performance of the actors, while the lighting in the scene is optimal. To solve this problem from the aspect of improving the shooting process is faced with expensive costs, and to solve this problem from the aspect of post-production, it is faced with complex technical processing methods. In the aspect of portrait photography and beautification, it is faced with similar problems as film and television production. The ideal camera lighting scene for professional camera settings is too expensive, and it is also affected by factors such as the cameraman’s personal ability, and the use of professional image processing software in the later stage also requires time and manpower. cost, very complicated.

Illumination transfer for face images will provide a simple solution to the above problems. In the problem of image lighting, it only needs to give an image with ideal lighting effect, and then the lighting effect can be transferred to the target image, so that the target image can obtain the ideal lighting effect. In the video image problem, it needs to give an image or video with ideal lighting effect, and transfer the lighting to the target image or video, one or both of which are video files. Light migration aims to generate target images or videos with ideal lighting effects in one step only by providing ideal lighting effects, without complex operation methods and methods, saving time, manpower, and resource costs. At present, the illumination transfer methods of face images only change the illumination of the face of the face image, and do not deal with the illumination of the neck and background of the face image. In practical application scenarios, the face part and the non-face part are inseparable, and only changing the illumination of the face part does not fully meet the actual application requirements. Aiming at this problem, the present invention studies a method capable of performing illumination migration on the entire face image. It can make the target image get the ideal lighting effect, and at the same time, it is closer to the actual lighting transfer application scenario.

SUMMARY OF THE INVENTION

The technology of the present invention solves the problem: on the basis of traditional face makeup and style transfer, a face image illumination transfer based on a convolutional neural network is proposed. After combining the deep neural network, the effect of lighting migration is better and closer to the real image lighting effect, the process is simple and the function is more powerful.

The technical solution of the present invention: a method for migrating face image illumination based on convolutional neural network, comprising the following steps:

Step 1. Data set preparation and establishment;

Step 2. Preparation of the pre-training model;

Step 3. Obtain the illumination classification model;

Step 4. Illumination matching based on illumination classification model;

Step 5. Illumination migration based on the illumination classification model, as follows:

Among them, the step 1, the preparation and establishment of the data set: At present, the data sets with more detailed lighting classification include the YaleFace data set and the PIE data set. Yale Face dataset B contains 10 photographic subjects, 9 males and 1 female, all black and white images. Each camera object contains 9 poses, each pose has 64 lighting effects, a total of 5760 face lighting images including the background. The two datasets have a total of 21,888 face images. The PIE face dataset is a color face dataset with a total of 68 photographic subjects, including male, female and all races. Among them, there are 13 poses of face illumination images without background lights, 3 poses of face illumination images with background lights, each pose contains 21 kinds of lights, and the camera's x-direction, y-direction, z-direction location to classify. The face illumination image part of this dataset has a total of 22,848 pictures, including pictures with background lights and poses facing the camera, a total of 1,428 pictures.

The above two datasets are relatively small. For the Yale Face face dataset, the face data of 28 photographic subjects are used for training, and the face data of 10 photographic subjects are used for testing. For the PIE face dataset, the face data of 50 photographic subjects are used for training, and the face data of 18 photographic subjects are used for testing. The background lighting of all images containing background lighting in the PIE dataset is similar, which interferes with the learning and classification of face lighting, so the image data is matted.

Among them, the step 2, the preparation of the pre-training model: there are currently two methods for training classification tasks, one is to start training from an initial state, and the other is to transfer learning. On the basis of small datasets, the model effect obtained using pre-trained model transfer learning is better than the effect of random initialization. Therefore, the present invention adopts the method of migration learning to train the illumination classification model. For the Yale Face face data set, the VGG19 convolutional neural network is used, and the pre-training model of the object classification data set ImageNet provided by Matconvnet. For the PIE face dataset, use the convolutional neural network VGG16, the pre-trained model of the face recognition model Vgg Face provided by Matconvnet. Matconvnet is a toolbox that can use convolutional neural networks in Matlab, and there are rich pre-training model resources.

Wherein, in the step 3, the illumination classification model is obtained: the training illumination classification model first removes the last fully connected layer of the original network according to the classification task, and adds a new softmax layer, top1-error layer and top5-error layer for training , add a new fully connected layer according to the number of categories in the dataset, 64 categories for the Yale Face dataset and 21 categories for the PIE dataset. Then fine-tune the entire network, fine-tune the weights of the first few layers of the network, set the learning rate to 1* ^10-4 , and iterate 300 rounds to complete the face image illumination classification task. The illumination classification model trained by the VGG16 network and the VGG Face pre-trained model has an average accuracy of 62.8571% on the PIE face dataset test set. The lighting classification model trained by the VGG19 network and the ImageNet pre-training model has an average accuracy of 94.375% on the Yale Face face dataset test set. Among 64 types of lighting, 48 types of lighting effects have a classification accuracy of 100%. %.

In the step 4, lighting matching based on the lighting classification model: lighting matching is to give a face lighting input image, find an image with the same lighting effect as the input image in the face lighting image data set, and output the image with the same lighting. The key to lighting matching is how to calculate the lighting information representing the image. The VGG neural network adopted in the present invention is composed of a convolution layer, a fully connected layer, a pooling layer and a softmax layer. The output of the convolutional layer operation is image feature information with large data, and it is difficult to extract illumination information from it. The last fully connected layer of the VGG network, the output data dimension is consistent with the number of categories of the image, and it is easier to find the correlation with the image lighting information. Therefore, on the basis of the illumination classification model, the model trained by the YaleFace dataset with high illumination classification accuracy and more detailed categories is used to find the correlation between the illumination information and the output of the last fully connected layer of the model, and complete the matching Illumination matching for face illumination images. By observing the output of a large number of images through the fully connected layer of the network and the label of the image category, it is found that there is a certain rule between the two. The dimension sequence number corresponding to the maximum value in the output 64-dimensional data is the same as the sequence number of the image category label. Therefore, the present invention creatively proposes an illumination matching algorithm based on an illumination classification model, as shown in FIG. 3 .

Among them, the step 5, illumination migration based on illumination classification model: on the basis of illumination classification model, inspired by Neural Style, combined with the quotient image method of traditional illumination migration research, the end-to-end single face image illumination migration is completed. The traditional Neural Style uses the input content map and style map, respectively calculates the loss of the content map and style map, and finally linearly combines the losses of the content map and style map to obtain the final style transfer map. Based on this idea, the present invention proposes illumination migration based on illumination classification model, which is also the input image and reference image required for preparing illumination migration. Input the input image and the reference image into the light transfer network (VGG19), get the feature matrix value of the two images, and then solve the light and shadow quotient according to the feature matrix value. Return the light and shadow quotient and the expected result image to the transfer network to minimize the transfer loss function, and output the result image after 1000 iterations. In specific experiments, it was found that the style transfer part of the traditional Neural Style could not transfer and learn the lighting information, so finally only the content transfer part was used to transfer the lighting information of the image.

Further, the light and shadow quotient solution process is as follows:

The light migration method based on the light classification model calculates the ratio of the reference image and the target image after learning by the VGG network before calculating the light and shadow quotient. The formula is as follows:

Among them, F _l [I] is the feature matrix of the input image I at the l-layer convolutional layer, F _l [E] is the feature matrix of the reference image E at the l-layered convolutional layer; ε is a constant, set is 0.0001, the feature matrix value of the reference image is divided by the feature value of the input image to obtain the ratio S _l of the reference image and the target image after learning by the VGG network, and the input image feature matrix is multiplied by this ratio to obtain the light and shadow quotient. The formula is as follows:

F _l [M]=F _l [I]×S _l

所示的约束改进。其中，约束值r_ij为0.4、5经实验得出，在该约束范围内即能较好的迁移光照信息，也不会过多的把参考图像的结构和内容迁移到输入图像上。Among them, the present invention makes the formula for F _l [M] (F _l [M] is the light and shadow quotient of the image M at the l-layer convolutional layer)

Constraint improvements shown. Among them, the constraint value r _ij is 0.4 and 5. It is obtained through experiments that within this constraint range, the illumination information can be transferred well, and the structure and content of the reference image will not be transferred to the input image too much.

Further, the calculation process of the migration loss function is as follows:

Neural Style creatively proposes the content loss (content loss) and style loss (style loss) of the image, which are used to represent the transfer loss of the convolutional neural network to learn the content and style of the image. The formula is:

where _l denotes that at the _lth convolutional layer, there are Nl filters of a vectorized feature map of size Dl ( _Dl is the number of elements in the filter response). F _l [.]∈R ^Nl×Dl is the obtained feature matrix, and (i,j) is the index of the feature matrix. (Fl[ _I ], _Fl [E], _Fl [O]) are the feature matrix of the input image I, the feature matrix of the reference painting style image E and the result image O of the expected output of the lth convolutional layer, respectively feature matrix. L is the total number of layers in the network being examined, where (α _l , β _l ) are the configured weight parameters. Γ is a weight reconciling between the completeness of the input image content and the amount of painting style transfer.

The migration loss function of the migration method of the present invention is:

F _l [O] is the expected result image, F _l [M] is the light and shadow quotient, the distance between the two light and shadow quotients is calculated and the gap between the two is continuously narrowed to obtain natural light migration. After the experiment, the content transfer part in Neural Style is retained to transfer the illumination information of the image. The convolutional layers used for illumination migration are determined by experiments as the convolutional layers 1_2 and 2_1. The lower convolutional layers are sensitive to the content and structure information of the image and can retain the content information of the input image. The illumination migration method based on the illumination classification model can migrate illumination in multiple directions on the same camera object, including positive light source, left light source, and right light source, and can migrate illumination, and the illumination is natural.

The advantages of the present invention compared with the prior art are:

(1) The rise of deep learning has brought breakthroughs to many studies in the field of computer vision. Style transfer is a typical example. Inspired by style transfer, the present invention provides a method for light transfer using convolutional neural network, which is more effective than traditional methods to complete face makeup transfer and style transfer, and makes The light transfer effect of the face image is closer to the result of the light effect of the real image.

(2) In the present invention, the pre-training model is used to train the illumination classification model of migration learning on the Yale Face data set and the PIE data set, which are more detailed in illumination classification. Different pre-trained models are used for different datasets and the neural structure of the pre-trained models is improved, fine-tuned and then trained to achieve a better illumination classification model.

(3) The present invention performs illumination matching based on the illumination classification model. For a given face illumination input image, an image with the same illumination effect as the input image is found in the face illumination image data set, and the image with the same illumination is output. . In order to obtain the above goals, a large number of experiments were conducted to find the correlation between the illumination information in the neural network and the output of the last fully connected layer of the model, to complete the illumination matching of the face illumination image and propose an illumination matching algorithm.

(4) The present invention performs illumination matching of the illumination classification model on the basis of obtaining a well-trained illumination classification model, and innovatively proposes illumination migration in combination with image style migration, and proposes an improved loss value function for traditional style migration. The content migration part is used to migrate the illumination information of the image, and finally a face image illumination migration method based on convolutional neural network is obtained.

Description of drawings

Fig. 1 is the framework diagram of the present invention;

Fig. 2 is the network structure diagram of the convolutional neural network VGG16 and VGG19 used in the present invention;

Fig. 3 is the flow chart of the illumination matching algorithm based on illumination classification model of the present invention;

FIG. 4 is a flowchart of the illumination migration algorithm based on the illumination classification model of the present invention.

Detailed ways

For a better understanding of the present invention, some basic concepts are first explained.

Convolutional Neural Networks: Convolutional Nerual Networks, CNN is a class of neural networks that contain convolutional computations and are often used in deep learning.

Convolution layer: extract image features through convolution calculation;

Pooling layer: compress the input feature image to simplify the computational complexity of the network;

Fully connected layer: connect all features and send the output value to the softmax layer;

Transfer learning: Instead of starting training from the initial state, transfer the weights in a network model trained to perform a certain classification task to a new network model trained for another target classification.

Illumination matching: Illumination matching means that given a face illumination input image, find an image with the same illumination effect as the input image in the face illumination image dataset, and output the image with the same illumination.

Matconvnet: A toolbox that can use convolutional neural networks in Matlab, with rich pre-trained model resources.

Referring to Fig. 1, the whole realization process of the present invention is as follows:

(1) The first is the preparation and establishment of the data set, select the Yale Face data set and the PIE data set. For the YaleFace face dataset, the face data of 28 photographic subjects are used for training, and the face data of 10 photographic subjects are used for testing. For the PIE face dataset, the face data of 50 photographic subjects are used for training, and the face data of 18 photographic subjects are used for testing.

(2) Use different pretrained models for the two selected datasets. The Yale Face dataset uses the convolutional neural network VGG19, the ImageNet pre-training model provided by Matconvnet, the convolutional neural network VGG16 for the PIE dataset, and the Vgg Face pre-training model provided by Matconvnet, as shown in Figure 2 for VGG16 and Neural network structure diagram of VGG19.

(3) Obtain the illumination classification model. Modify the network structure of VGG16 and VGG19, remove the last fully connected layer of the original network according to the classification task, add a new softmax layer, top1-error layer and top5-error layer for training, and add new ones according to the number of categories in the dataset The fully connected layer of the Yale Face dataset has 64 classes and the PIE dataset has 21 classes. Then fine tune the entire network, fine-tune the weights of the first few layers of the network, set the learning rate to 300 iterations, and complete the face image illumination classification task. The illumination classification model trained by the VGG16 network and the VGG Face pre-trained model has an average accuracy of 62.8571% on the PIE face dataset test set. The lighting classification model trained by the VGG19 network and the ImageNet pre-training model has an average accuracy of 94.375% on the Yale Face face dataset test set. Among 64 types of lighting, 48 types of lighting effects have a classification accuracy of 100%. %.

(4) Illumination matching is performed on the basis of illumination classification model. The key to lighting matching is how to calculate the lighting information representing the image. After a large number of experiments, it is found that there is a certain pattern between the output of the image through the fully connected layer of the network and the label of the image category. The dimension sequence number corresponding to the maximum value in the output 64-dimensional data is the same as the sequence number of the image category label. Therefore, a lighting matching algorithm based on the lighting classification model is proposed, and the specific algorithm flow is shown in Figure 3.

(5) Inspired by Neural Style, combined with the quotient image method of traditional illumination migration research, modify its loss function, retain only its content migration part to migrate the illumination information of the image, and complete the end-to-end single face image illumination migration. The specific algorithm The process is shown in Figure 4.

The specific implementation process of the above steps is as follows:

The network structure diagram of the convolutional neural network VGG16 and the convolutional neural network VGG19 used in the step (1) The core neural network structure diagram used in the present invention is shown in Figure 2:

(1.1) VGG16 contains 16 weight layers, VGG19 contains 19 weight layers, of which the weight layers include convolutional layers and fully connected layers;

(1.2) VGG16 and VGG19 input 224*224RGB images;

(1.3) VGG16 contains 13 convolutional layers, which are represented by conv3-xxx respectively, where 3 means that the convolution kernel size is 3*3, and -xxx means the number of convolutions; 3 fully connected layers are represented by FC-xxx respectively; 5 A pooling layer, represented by maxpool; 1 sofmax layer;

(1.4) VGG19 contains 16 convolutional layers, which are represented by conv3-xxx respectively, where 3 means that the convolution kernel size is 3*3, and -xxx means the number of convolutions; 3 fully connected layers are represented by FC-xxx respectively; 5 A pooling layer, represented by maxpool; 1 sofmax layer.

The specific algorithm flow of illumination matching based on illumination classification model in the step (4)

On the basis of training the illumination classification model, the care matching is realized. The flow of the illumination matching algorithm is shown in Figure 3:

(4.1) Begin;

(4.2) Input the face illumination image to the illumination classification model: To train the illumination classification model, first remove the last fully connected layer of the original network according to the classification task, and add a new softmax layer, top1-error layer and top5- for training. The error layer adds a new fully connected layer according to the number of categories in the dataset. The Yale Face dataset has 64 categories and the PIE dataset has 21 categories. Then fine tune the entire network, fine-tune the weights of the first few layers of the network, set the learning rate to 300 iterations, and complete the face image illumination classification task;

(4.3) Take out the output value of the last fully connected layer of the model: the VGG neural network used in the present invention is composed of a convolution layer, a fully connected layer, a pooling layer, and a softmax layer. The output of the convolutional layer operation is image feature information with large data, and it is difficult to extract illumination information from it. The last fully connected layer of the VGG network, the output data dimension is consistent with the number of categories of the image, and it is easier to find the correlation with the image lighting information. Therefore, on the basis of the illumination classification model, the model trained by the YaleFace dataset with high illumination classification accuracy and more detailed categories is used to find the correlation between the illumination information and the output of the last fully connected layer of the model, and complete the matching Illumination matching of face illumination images;

(4.4) Obtain the maximum value and the serial number of its dimension: By observing the output of a large number of images passing through the fully connected layer of the network and the label of the image category, it is found that there is a certain rule between the two. The maximum value in the output 64-dimensional data corresponds to The dimension serial number is the same as the serial number of the image category label, so the maximum value and the serial number of the dimension are obtained later to find the image with the same dimension serial number as the input image;

(4.5) Find the image with the same dimension serial number as the input image in the face dataset: after the above-mentioned steps, find the image with the same dimension number as the input image in the face dataset so as to achieve the matching of illumination;

(4.6) Compare the maximum value of all images with the same dimension serial number: The maximum image obtained by comparing the maximum value of all images with the same dimension serial number in the image with the same dimension serial number as the input image found in the face dataset is the image in the face dataset. the image that best matches the lighting of the input image;

(4.7) The image whose output maximum value is closest to the input image is the matching image;

(4.8) End.

The specific algorithm flow of illumination migration based on illumination classification model in the step 5

Based on the illumination classification model, inspired by Neural Style, combined with the quotient image method of traditional illumination migration research, the end-to-end illumination migration of a single face image is completed. The traditional Neural Style uses the input content map and style map, respectively calculates the loss of the content map and style map, and finally linearly combines the losses of the content map and style map to obtain the final style transfer map. Based on this idea, the present invention proposes illumination migration based on illumination classification model. The implementation process of the specific algorithm is shown in Figure 4:

(5.1) Begin;

(5.2) Input the reference image and the input image into the migration network: the input image is generally an image with uniform frontal illumination, and the reference image is generally an image with obvious light and shadow difference, and the input image and the reference image are input into the light migration network (VGG19);

(5.3) Obtain the eigenvalues of the input image and the reference image: the input image and the reference image obtain the eigenvalues of the image through the calculation of the illumination migration network, which is convenient for calculating the light and shadow quotient of the image;

(5.4) Solving the light and shadow quotient: Before calculating the light and shadow quotient, calculate the ratio of the reference image and the target image after learning by the VGG network, and the formula is as follows:

Among them, F _l [I] is the feature matrix of the input image I at the l-layer convolutional layer, F _l [E] is the feature matrix of the reference image E at the l-layered convolutional layer; ε is a constant, set is 0.0001, the feature matrix value of the reference image is divided by the feature value of the input image to obtain the ratio S _l of the reference image and the target image after learning by the VGG network, and the input image feature matrix is multiplied by this ratio to obtain the light and shadow quotient. The formula is as follows:

F _l [M]=F _l [I]×S _l

所示的约束改进。其中，约束值r_ij为0.4、5经实验得出，在该约束范围内即能较好的迁移光照信息，也不会过多的把参考图像的结构和内容迁移到输入图像上；Wherein the present invention makes the formula for F _l [M] (F _l [M] is the light and shadow quotient of the image M at the l-layer convolutional layer)

Constraint improvements shown. Among them, the constraint value r _ij is 0.4 and 5. It is obtained through experiments that within this constraint range, the illumination information can be transferred well, and the structure and content of the reference image will not be transferred to the input image too much;

(5.5) Obtain the result image through the transfer loss function: Neural Style creatively proposes the contentloss (content loss) and style loss (style loss) of the image, which are used to represent the transfer loss of the convolutional neural network to learn the content and style of the image, Its formula is:

where _l denotes that at the _lth convolutional layer, there are Nl filters of a vectorized feature map of size Dl ( _Dl is the number of elements in the filter response). F _l [.]∈R ^Nl×Dl is the obtained feature matrix, and (i,j) is the index of the feature matrix. (Fl[ _I ], _Fl [E], _Fl [O]) are the feature matrix of the input image I, the feature matrix of the reference painting style image E and the result image O of the expected output of the lth convolutional layer, respectively feature matrix. L is the total number of layers in the inspected network, where (αl, β _l ) are the configured weight parameters. Γ is a weight reconciling between the completeness of the input image content and the amount of painting style transfer.

The transfer loss function of the light transfer method is:

F _l [O] is the expected result image, F _l [M] is the light and shadow quotient, the distance between the two light and shadow quotients is calculated and the gap between the two is continuously narrowed to obtain natural light migration. After the experiment, the content transfer part in Neural Style is retained to transfer the illumination information of the image. The convolutional layers used for illumination migration are determined by experiments as the convolutional layers 1_2 and 2_1. The lower convolutional layers are sensitive to the content and structure information of the image and can retain the content information of the input image. The illumination migration method based on the illumination classification model can migrate illumination in multiple directions on the same camera object, including positive light source, left light source, and right light source, and can migrate illumination, and the illumination is natural;

(5.6) Return the result image;

(5.7) END.

Although illustrative specific embodiments of the present invention have been described above to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited in scope to the specific embodiments, to those skilled in the art, As long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the inventive concept are included in the protection list.

Claims (7)

1. a face image illumination migration method based on convolutional neural network, is characterized in that, comprises the steps: Step 1. Data set preparation and establishment; Step 2. Preparation of the pre-training model; Step 3. Obtain the illumination classification model; Step 4. Illumination matching based on illumination classification model; Step 5. Illumination migration based on illumination classification model; Step (4) The lighting matching based on the lighting classification model specifically includes the following steps: (4.1) Begin; (4.2) Input the face illumination image to the illumination classification model: To train the illumination classification model, first remove the last fully connected layer of the original network according to the classification task, and add a new softmax layer, top1-error layer and top5- for training. error layer, add a new fully connected layer according to the number of categories in the dataset, the Yale Face dataset is 64 categories, and the PIE dataset is 21 categories; then fine-tune the entire network, fine-tune the weights of the first few layers of the network, and set the learning rate to 1* ^10-4 , iterate 300 rounds, complete the task of face image illumination classification; (4.3) Take out the output value of the last fully connected layer of the model: the VGG neural network used is composed of a convolutional layer, a fully connected layer, a pooling layer, and a softmax layer. The output image feature information of the convolutional layer operation is used in the illumination classification. On the basis of the model, the model trained by the Yale Face dataset is used to find the correlation between the illumination information and the output of the last fully connected layer of the model to complete the illumination matching of the face illumination image; (4.4) Obtain the maximum value and the serial number of the dimension: the image passes through the output of the fully connected layer of the network and the label of the image category. The serial number of the dimension corresponding to the maximum value in the output 64-dimensional data is the same as the serial number of the image category label, and the maximum value is obtained. and the serial number of the dimension; (4.5) Use step (4.4) to obtain the maximum value and the serial number of the dimension where it is located. Find the image with the same dimension serial number as the input image in the face data set: find the image with the same dimension serial number as the input image in the face data set in order to achieve the illumination effect. match; (4.6) Compare the maximum value of all images with the same dimension serial number: In the face data set found in the image with the same dimension serial number as the input image, the maximum image obtained by comparing the maximum value of all the images with the same dimension serial number is the face data set. The image that best matches the lighting of the input image; (4.7) output the maximum image as the matching image of the image closest to the input image; (4.8) End. 2. a kind of face image illumination migration method based on convolutional neural network according to claim 1, is characterized in that, the preparation and establishment of step (1) data set, comprise the steps: For the Yale Face face data set, the face data of 28 photographic subjects are used for training, and the face data of 10 photographic subjects are used for testing; For the PIE face data set, the face data of 50 photographic subjects is used for training, and the face data of 18 photographic subjects is used for testing; the image data is matted. 3. a kind of face image illumination migration method based on convolutional neural network according to claim 1, is characterized in that, the preparation of step (2) pre-training model, comprises: For the Yale Face face data set, the VGG19 convolutional neural network is used, and the pre-training model of the object classification data set ImageNet provided by Matconvnet; For the PIE face dataset, use the convolutional neural network VGG16, the pre-trained model of the face recognition model VggFace provided by Matconvnet. 4. a kind of face image illumination migration method based on convolutional neural network according to claim 1, is characterized in that, step (3) obtains illumination classification model as follows: To train the illumination classification model, first remove the last fully connected layer of the original network according to the classification task, add a new softmax layer, top1-error layer and top5-error layer for training, and add a new fully connected layer according to the number of categories in the dataset layer, the Yale Face dataset is 64 categories, and the PIE dataset is 21 categories; then fine tune the entire network, fine-tune the weights of the first few layers of the network, set the learning rate to 1*10 ^-4 , and iterate 300 rounds to complete the face image illumination For the classification task, the illumination classification model is trained by the VGG16 network and the VGGFace pre-trained model. 5. a kind of face image illumination migration method based on convolutional neural network according to claim 1, is characterized in that, the illumination matching of step (4) based on illumination classification model is as follows: Find the image with the same lighting effect as the input image in the face illumination image dataset, and output the image with the same illumination. On the basis of the illumination classification model, use the model trained from the Yale Face dataset to complete the illumination of the face illumination image. match. 6. a kind of face image illumination migration method based on convolutional neural network according to claim 1, is characterized in that, step (5) is based on the illumination migration of illumination classification model: First, prepare the input image and reference image required for light migration. The input image is a uniform frontal illumination image, and the reference image is an image with obvious light and shadow difference. Input the input image and reference image into the light migration network, and get the two images. The feature matrix value is obtained, and then the light and shadow quotient is obtained according to the feature matrix value; the light and shadow quotient and the expected result image are returned to the transfer network to minimize the transfer loss function, and the result image is output after multiple iterations. 7. a kind of face image illumination migration method based on convolutional neural network according to claim 6, is characterized in that, the illumination migration of step (5) based on illumination classification model specifically comprises: (5.1) Begin; (5.2) Input the reference image and the input image into the migration network: the input image is a uniform frontal illumination image, the reference image is an image with obvious light and shadow difference, and the input image and the reference image are input into the illumination migration network VGG19; (5.3) Obtain the eigenvalues of the input image and the reference image: the input image and the reference image obtain the eigenvalues of the image through the calculation of the illumination migration network, which is convenient for calculating the light and shadow quotient of the image; (5.4) Solving the light and shadow quotient: Before calculating the light and shadow quotient, calculate the ratio of the reference image and the target image after learning by the VGG network, and the formula is as follows:

Among them, F _l [I] is the feature matrix of the input image I at the l-layer convolutional layer, F _l [E] is the feature matrix of the reference image E at the l-layered convolutional layer; ε is a constant, set is 0.0001, the feature matrix value of the reference image is divided by the feature value of the input image to obtain the ratio S _l of the reference image and the target image after learning by the VGG network, and the input image feature matrix is multiplied by this ratio to obtain the light and shadow quotient. The formula is as follows: F _l [M]=F _l [I]×S _l Fl[M] is the light and shadow quotient of the image M at the _lth convolutional layer, where the constraint improvement on Fl[M] is as follows:

Wherein, the constraint value r _ij is in the range of 0.4 to 5; (5.5) Obtain the result image through the transfer loss function: the content loss and style loss of the image in the Neural Style method are used to represent the transfer loss of the convolutional neural network to learn the content and style of the image. The formula is:

where l denotes that at the lth convolutional layer, there are N _l filters of vectorized feature maps of size D _l , D _l is the number of elements in the filter response, F _l [.]∈R ^{Nl× Dl} is the obtained feature matrix, (i, j) is the index of the feature matrix, F _l [I], F _l [E], F _l [O] are the feature matrix of the input image I and the reference painting style image E respectively. The feature matrix and the feature matrix of the resulting image O of the expected output of the lth convolutional layer; L is the total number of layers in the inspected network, where α _l , β _l are the configured weight parameters; Γ is the input image content The weight of the reconciliation between the integrity of the painting style and the transfer amount of painting style; The transfer loss function of the light transfer method is:

F _l [O] is the expected result image, F _l [M] is the light and shadow quotient, calculate the distance between the two light and shadow quotients and continuously narrow the gap between the two to obtain natural light migration; (5.6) Return the result image; (5.7) END.

Images