CN114219753B - Deep learning-based power equipment surface defect detection method and terminal - Google Patents

Deep learning-based power equipment surface defect detection method and terminal Download PDF

Info

Publication number
CN114219753B
CN114219753B CN202111256280.9A CN202111256280A CN114219753B CN 114219753 B CN114219753 B CN 114219753B CN 202111256280 A CN202111256280 A CN 202111256280A CN 114219753 B CN114219753 B CN 114219753B
Authority
CN
China
Prior art keywords
real
deep learning
template library
equipment
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111256280.9A
Other languages
Chinese (zh)
Other versions
CN114219753A (en
Inventor
何志甘
范彦琨
张锦吉
付胜宪
陈德兴
李冠颖
林剑平
吕小伟
徐显烨
杨宏毅
李升晖
姚国华
林石
彭质斌
熊旭
张舒雅
黄东方
张颜真
许卉
严欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Original Assignee
Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Maintenance Branch of State Grid Fujian Electric Power Co Ltd filed Critical Maintenance Branch of State Grid Fujian Electric Power Co Ltd
Priority to CN202111256280.9A priority Critical patent/CN114219753B/en
Publication of CN114219753A publication Critical patent/CN114219753A/en
Application granted granted Critical
Publication of CN114219753B publication Critical patent/CN114219753B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Energy or water supply
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C1/00—Registering, indicating or recording the time of events or elapsed time, e.g. time-recorders for work people
    • G07C1/20—Checking timed patrols, e.g. of watchman
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Health & Medical Sciences (AREA)
  • Marketing (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Public Health (AREA)
  • Water Supply & Treatment (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Primary Health Care (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Operations Research (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method and a terminal for detecting surface defects of electric equipment based on deep learning, wherein the method comprises the steps of obtaining real-time pictures automatically and regularly collected by an electric intelligent inspection system, judging the real-time pictures by a deep learning template library, judging the real-time pictures by an image recognition mode if judging failure, judging the real-time pictures by a manual marking process if judging failure, automatically marking the type, the position and the outline of equipment by the deep learning template library or the real-time pictures successfully judged by the image recognition mode, importing the real-time pictures after marking into the deep learning template library for deep learning, and continuously recognizing the next real-time pictures after updating the deep learning template library. The invention realizes the self-feedback image recognition processing method, fully utilizes a large number of pictures acquired by the intelligent inspection system, completes the automatic maintenance and automatic updating of the power equipment template library, and realizes the more accurate intelligent judgment of the working state of the power equipment.

Description

Deep learning-based power equipment surface defect detection method and terminal
Technical Field
The invention relates to the technical field of power grid transmission and transformation equipment state monitoring systems, in particular to a deep learning-based power equipment surface defect detection method and a terminal.
Background
With the rapid development of automation technology, many links requiring manual operation are gradually changed to be completed by machines in industrial production, and industrial production automation also releases more and more workers from boring work, so that the workers play a greater value.
The surface defect detection of the power equipment is an important link in the safety production of the power grid, is a key step of automatic inspection of the intelligent inspection system, and can effectively improve the intelligent inspection quality and efficiency of the intelligent inspection system by means of the surface defect detection technology. The traditional surface defect detection algorithm structure obtains an image which is convenient to detect through image preprocessing, and then extracts image characteristics by means of a statistical machine learning method, so that the defect detection target is realized. The image preprocessing step comprises histogram equalization, filtering denoising, gray level binarization and re-filtering to obtain simplified image information with separated foreground and background, and then marking and detecting defects by using algorithms such as mathematical morphology, fourier transformation, gabor transformation and the like and a machine learning model.
However, the traditional surface defect detection algorithm needs to manually design a complex algorithm flow, so that the surface defect detection of the power equipment based on the deep learning algorithm occurs.
The deep learning algorithm adopts a Deep Convolutional Neural Network (DCNNs) and target detection algorithms such as SSD (Single Shot MultiBox Detector) and Yolo (You Only Look Once) to construct a cascade detection network from coarse to fine, and comprises the following specific steps of equipment positioning, defect detection and classification:
(1) Extracting the surface of equipment, namely positioning cantilever nodes in an image by means of an SSD frame with good speed and precision, and positioning a sleeve based on a quick localization framework of Yolo frames;
(2) And (3) detecting and classifying the defects on the surface of the equipment, namely judging the defects according to the detection on the surface of the equipment in the second stage, and classifying the defects through 4 convolution layers by means of DCNN.
The deep learning algorithm avoids the complex algorithm flow which is required to be designed manually by the traditional algorithm, has extremely high robustness and precision, but the essence of the deep learning is machine learning, and the most fundamental principle is statistics, namely that enough data is required to be used, and the processes are defined, analyzed, data collected, injected and trained to improve the final output structure of the model, and are circularly executed to continuously improve the precision. In short, the deep learning algorithm needs to provide a sufficient number of various device image templates for training modeling the algorithm.
The device image template is required to train the device mark on the picture by manpower, and the mark is required to be manually operated, namely, the device is marked on each picture, and the training of the deep learning template is completed through the marked picture. In the process of adopting the deep learning template, the process from template picture training, template image recognition is a unidirectional process, in the recognition process, only a pre-trained template which can be used easily causes inaccurate recognition, and for the picture which can not be recognized by the template, a manual recognition mode is needed, so that the recognition efficiency is reduced.
Disclosure of Invention
The invention aims to solve the technical problem of providing a deep learning-based power equipment surface defect detection method and a terminal, which can realize more accurate intelligent judgment of the working state of power equipment by automatically maintaining and updating a power equipment template library.
In order to solve the technical problems, the technical scheme adopted by the invention is that the method and the terminal for detecting the surface defects of the power equipment based on deep learning comprise the following steps:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
And S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture.
In order to solve the technical problem, the invention adopts another technical scheme that the deep learning-based power equipment surface defect detection terminal comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the following steps when executing the computer program:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
And S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture.
The invention has the beneficial effects that the method and the terminal for detecting the surface defects of the electric equipment based on the deep learning have the advantages that firstly, the real-time pictures acquired by the electric intelligent inspection system at automatic timing are identified based on the pre-obtained deep learning template library, the pictures which cannot be identified are identified by adopting the image identification mode, the type, the position and the outline of the marking equipment can be automatically identified, the pictures which cannot be identified by the image identification mode are manually marked, the marked real-time pictures are imported into the deep learning template library for training after each marking, the deep learning template library is updated, the self-feedback image identification processing method is realized, a large number of pictures acquired by the intelligent inspection system are fully utilized, the automatic maintenance and the automatic updating of the electric equipment template library are completed, and the intelligent identification of the working state of the electric equipment is realized.
Drawings
Fig. 1 is a main flowchart of a method for detecting surface defects of electrical equipment based on deep learning according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for detecting surface defects of electrical equipment based on deep learning according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of automatic marking on a real-time picture according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of marking a real-time picture with manual marking according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a deep learning-based power equipment surface defect detection terminal according to an embodiment of the present invention.
Description of the reference numerals:
1. a deep learning-based power equipment surface defect detection terminal, 2, a memory, 3, and a processor.
Detailed Description
In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.
Referring to fig. 1 to 4, a method for detecting surface defects of electrical equipment based on deep learning includes the steps of:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
And S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture.
The method has the advantages that firstly, the real-time pictures acquired by the electric power intelligent inspection system at automatic timing are identified based on the pre-obtained deep learning template library, the pictures which cannot be identified are identified by adopting an image identification mode, the type, the position and the outline of the marking equipment can be automatically identified, the pictures which cannot be identified by the image identification mode are manually marked, the marked real-time pictures are imported into the deep learning template library for training after each marking, the deep learning template library is updated, a self-feedback image identification processing method is realized, a large number of pictures acquired by the intelligent inspection system are fully utilized, the automatic maintenance and the automatic updating of the electric power equipment template library are completed, and the intelligent identification of the working state of the electric power equipment is realized.
Further, in the step S3, an image recognition mode is adopted to distinguish the real-time picture, specifically:
And carrying out image recognition on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of equipment in the real-time pictures, wherein the image recognition template library is formed by distinguishing all historical pictures automatically and regularly acquired by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
From the above description, for the real-time pictures which cannot be identified by the deep learning template library, the real-time pictures are further identified by the image identification template library, so that the type, the position and the outline of the equipment in the real-time pictures can be marked, and the accurate intelligent identification of the working state of the power equipment is further realized.
Further, in the step S3, the real-time picture after the completion of the manual marking is imported into the deep learning template library to perform deep learning, specifically:
and receiving the type and the position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the type, the position and the outline of the real-time picture and the corresponding equipment into the image recognition template library and the deep learning template library to carry out deep learning.
It can be seen from the above description that, for the real-time pictures that cannot be identified by the image recognition template library, the staff performs manual marking for the type, position and outline of the equipment in the real-time pictures, so as to further realize intelligent identification of the working state of the power equipment, and meanwhile, the marked real-time pictures are imported into the image recognition template library to further enrich the template materials in the image recognition template library, namely, the image recognition template library is continuously perfected, and the next time similar real-time pictures are encountered again, the real-time pictures are not required to be marked by complex machine identification, but can be directly identified by the image recognition template library, so that the intelligent identification efficiency is further improved.
Further, in S4, the marking the type, the position and the outline of the device on the real-time picture automatically further includes:
And importing the real-time pictures and the types and the outlines of the corresponding devices into the image recognition template library.
From the above description, it can be known that, after each marking of the device type, position and outline is performed on the real-time image, the marked real-time image is imported into the image recognition template library, so that the template materials in the image recognition template library can be further enriched, that is, the image recognition template library is continuously perfected, so as to improve the intelligent discrimination efficiency of the power device.
Further, in S4, the type, the position and the outline of the device are automatically marked on the real-time picture, which specifically includes the following steps:
s41, obtaining the type of equipment in the real-time picture by using the history picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, taking the history picture as a reference image, and taking the real-time picture as an image to be processed;
S42, selecting four vertexes of a polygonal outline of the equipment on the reference image as characteristic points P1, P2, P3 and P4;
S43, taking a certain characteristic point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating a data block with highest similarity with the data block T in the image to be processed by adopting a global searching method, wherein the calculation formula is as follows:
Wherein P i,j represents image brightness data of M pixel points in the image to be processed, the length and width of the image to be processed taking the position of a coordinate (i, j) as the center are respectively (M/2) - (W-M/2), the value range of j is (M/2) - (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed, which is matched with the characteristic point in the reference image, and repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4, which are matched with the characteristic points P1, P2, P3 and P4;
S46, according to the positions of the four target position points T1, T2, T3 and T4, obtaining the position of the equipment in the image to be processed, and combining the polygonal outline of the equipment in the reference image to obtain the polygonal outline of the equipment in the image to be processed.
According to the description, the minimum rectangle capable of accommodating all polygon vertexes is calculated by carrying out coordinate operation on the polygons of the equipment body outline in the real-time picture based on four feature points, so that the data for marking the equipment position in the deep learning template can be obtained, the equipment position and outline can be automatically and rapidly marked, meanwhile, the equipment position data and outline data can be applied to continuous training of the deep learning template library, and the deep learning template library is further updated, so that the intelligent discrimination accuracy of the electric equipment is improved.
Referring to fig. 5, a deep learning-based power equipment surface defect detection terminal includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
And S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture.
According to the technical scheme, the method for detecting the surface defects of the power equipment based on the deep learning is matched with the method for detecting the surface defects of the power equipment based on the deep learning, firstly, real-time pictures acquired by an intelligent power inspection system automatically and regularly are identified based on a pre-obtained deep learning template library, the pictures which cannot be identified are identified by adopting an image identification mode, the type, the position and the outline of the marking equipment can be automatically identified, the pictures which cannot be identified by the image identification mode are manually marked, the marked real-time pictures are imported into the deep learning template library again after each marking, the deep learning template library is trained to be updated, the self-feedback image identification processing method is realized, a large number of pictures acquired by the intelligent inspection system are fully utilized, the automatic maintenance and the automatic updating of the power equipment template library are completed, and the intelligent identification of the working state of the power equipment is realized.
Further, in the step S3, an image recognition mode is adopted to distinguish the real-time picture, specifically:
And carrying out image recognition on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library so as to determine the type, the position and the outline of equipment in the real-time pictures, wherein the image recognition template library is formed by distinguishing all historical pictures automatically and regularly acquired by an electric intelligent inspection system, and each historical picture comprises the type, the position and the outline of the corresponding equipment.
From the above description, for the real-time pictures which cannot be identified by the deep learning template library, the real-time pictures are further identified by the image identification template library, so that the type, the position and the outline of the equipment in the real-time pictures can be marked, and the accurate intelligent identification of the working state of the power equipment is further realized.
Further, in the step S3, the real-time picture after the completion of the manual marking is imported into the deep learning template library to perform deep learning, specifically:
and receiving the type and the position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the type, the position and the outline of the real-time picture and the corresponding equipment into the image recognition template library and the deep learning template library to carry out deep learning.
It can be seen from the above description that, for the real-time pictures that cannot be identified by the image recognition template library, the staff performs manual marking for the type, position and outline of the equipment in the real-time pictures, so as to further realize intelligent identification of the working state of the power equipment, and meanwhile, the marked real-time pictures are imported into the image recognition template library to further enrich the template materials in the image recognition template library, namely, the image recognition template library is continuously perfected, and the next time similar real-time pictures are encountered again, the real-time pictures are not required to be marked by complex machine identification, but can be directly identified by the image recognition template library, so that the intelligent identification efficiency is further improved.
Further, in S4, the marking the type, the position and the outline of the device on the real-time picture automatically further includes:
And importing the real-time pictures and the types and the outlines of the corresponding devices into the image recognition template library.
From the above description, it can be known that, after each marking of the device type, position and outline is performed on the real-time image, the marked real-time image is imported into the image recognition template library, so that the template materials in the image recognition template library can be further enriched, that is, the image recognition template library is continuously perfected, so as to improve the intelligent discrimination efficiency of the power device.
Further, in S4, the type, the position and the outline of the device are automatically marked on the real-time picture, which specifically includes the following steps:
s41, obtaining the type of equipment in the real-time picture by using the history picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, taking the history picture as a reference image, and taking the real-time picture as an image to be processed;
S42, selecting four vertexes of a polygonal outline of the equipment on the reference image as characteristic points P1, P2, P3 and P4;
S43, taking a certain characteristic point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating a data block with highest similarity with the data block T in the image to be processed by adopting a global searching method, wherein the calculation formula is as follows:
Wherein P i,j represents image brightness data of M pixel points in the image to be processed, the length and width of the image to be processed taking the position of a coordinate (i, j) as the center are respectively (M/2) - (W-M/2), the value range of j is (M/2) - (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed, which is matched with the characteristic point in the reference image, and repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4, which are matched with the characteristic points P1, P2, P3 and P4;
S46, according to the positions of the four target position points T1, T2, T3 and T4, obtaining the position of the equipment in the image to be processed, and combining the polygonal outline of the equipment in the reference image to obtain the polygonal outline of the equipment in the image to be processed.
According to the description, the minimum rectangle capable of accommodating all polygon vertexes is calculated by carrying out coordinate operation on the polygons of the equipment body outline in the real-time picture based on four feature points, so that the data for marking the equipment position in the deep learning template can be obtained, the equipment position and outline can be automatically and rapidly marked, meanwhile, the equipment position data and outline data can be applied to continuous training of the deep learning template library, and the deep learning template library is further updated, so that the intelligent discrimination accuracy of the electric equipment is improved.
Referring to fig. 1, a first embodiment of the present invention is as follows:
a method for detecting surface defects of power equipment based on deep learning comprises the following steps:
S1, acquiring real-time pictures acquired by the electric power intelligent inspection system at automatic timing.
In this embodiment, a large number of intelligent patrol apparatuses such as high-definition cameras, robots or unmanned aerial vehicles are installed on the electric intelligent patrol system, and the intelligent patrol apparatuses collect working images of various apparatuses at regular time and fixed points every day, where in this embodiment, there may be a plurality of apparatuses or only one apparatus in each real-time image, but one real-time image includes at least one apparatus.
S2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4.
In this embodiment, the first deep learning template library is a pre-trained template, and the real-time picture is analyzed and judged through the pre-trained deep learning template library to determine the position, outline and type of the equipment in the picture, thereby judging the equipment defect.
And S3, judging the real-time picture by adopting an image recognition mode, if judging the real-time picture fails, leading the real-time picture after the manual marking into a deep learning template library for deep learning, updating the deep learning template library, returning to the step S1 for recognizing the next real-time picture, and otherwise, entering the step S4.
In this embodiment, the image recognition mode is used to recognize the real-time pictures which cannot be distinguished by the deep learning template library obtained by training in advance, and the manual marking process is used to further intelligently judge the device as a whole.
S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, guiding the marked real-time picture into a deep learning template library for deep learning, updating the deep learning template library, and returning to S1 for identifying the next real-time picture.
In this embodiment, each time a real-time picture is determined, the device type, position and contour of the real-time picture are marked, and the real-time picture is reintroduced into the deep learning template library for further deep learning training, so that the existing mode that the determined picture only enters the historical image library for historical data query without other use is replaced, the deep learning template library can be continuously updated and perfected, and the self-feedback image recognition processing method is realized. The intelligent inspection system is used for acquiring a large number of pictures, automatic maintenance and automatic updating of the power equipment template library are completed, and more accurate intelligent judgment of the working state of the power equipment is realized.
Referring to fig. 2 to 4, a second embodiment of the present invention is as follows:
In the first embodiment, a specific flowchart of a method for detecting surface defects of electrical equipment based on deep learning is shown in fig. 2.
In step S3, an image recognition mode is adopted to determine a real-time picture, which specifically includes:
And carrying out image recognition on the real-time pictures which cannot be recognized in the deep learning template library by adopting the image recognition template library so as to determine the type, the position and the outline of the equipment in the real-time pictures. In this embodiment, the image recognition template library is formed by distinguishing all history pictures automatically and regularly acquired by the electric power intelligent inspection system, and each history picture includes the type, position and outline of the corresponding device.
In this embodiment, for the real-time picture that cannot be identified by the deep learning template library, the image recognition template library is used for further identifying, so that the type, the position and the contour of the equipment in the real-time picture can be marked, and the accurate intelligent identification of the working state of the power equipment is further realized.
In step S3, the real-time picture after the completion of the manual marking is imported into a deep learning template library for deep learning, specifically:
And receiving the type and the position of equipment marked on the real-time picture by manpower and the outline of the equipment obtained by copying the real-time picture by the aid of the polygons by manpower, and importing the real-time picture and the type, the position and the outline of the corresponding equipment into an image recognition template library and a deep learning template library to perform deep learning.
In this embodiment, for the real-time pictures that the image recognition template library cannot recognize, the staff marks the type, position and outline of the device in the real-time pictures manually, as shown in fig. 5, the staff copies the outline of the device in the real-time pictures by using polygons, that is, further realizes the intelligent discrimination of the working state of the power device, and meanwhile, the marked real-time pictures are imported into the image recognition template library to further enrich the template materials in the image recognition template library, that is, the image recognition template library is continuously perfected, and the staff does not need to mark by complex machine recognition when touching similar real-time pictures again next time, but can directly discriminate by the image recognition template library, and further improves the intelligent discrimination efficiency.
In step S4, the type, the position and the outline of the device are automatically marked on the real-time picture, and the method further includes:
and importing the real-time pictures and the types and the outlines of the corresponding devices into an image recognition template library.
In this embodiment, as in the marking process in the step S3, the real-time picture obtained by marking the device type, the position and the contour each time is imported into the image recognition template library, so as to further enrich the template materials in the image recognition template library, i.e. to continuously perfect the image recognition template library, and improve the intelligent discrimination efficiency of the power device.
In step S4, the type, position and outline of the device are automatically marked on the real-time picture, which specifically includes the following steps:
s41, obtaining the type of equipment in the real-time picture by using the history picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, taking the history picture as a reference image, and taking the real-time picture as an image to be processed.
S42, selecting four vertexes of a polygonal outline of the device on the reference image as characteristic points P1, P2, P3 and P4.
S43, taking a certain characteristic point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement;
In this embodiment, the value of M affects the computation strength, the larger the value of M, the larger the consumed computation resource, the slower the computation speed, but the higher the accuracy of recognition, and the smaller the value of M, the faster the computation speed, but the lower the accuracy of recognition. Therefore, in practical use, the value of M may be adjusted as needed, and generally the value of M may be 34 to 40 pixels, and in this embodiment, the value of M is set to 37.
S44, calculating a data block with highest similarity with the data block T in the image to be processed by adopting a global searching method, wherein the calculation formula is as follows:
Wherein P i,j represents the image brightness data of M pixel points with the length and width centered on the position of the coordinate (i, j) in the image to be processed, wherein the value range of i is (M/2) - (W-M/2), the value range of j is (M/2) - (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed.
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed, which is matched with the characteristic point in the reference image, and repeating the steps S43 to S44 to obtain target position points T1, T2, T3 and T4, which are matched with the characteristic points P1, P2, P3 and P4 respectively.
S46, according to the positions of the four target position points T1, T2, T3 and T4, obtaining the position of the equipment in the image to be processed, and combining the polygonal contour of the equipment in the reference image to obtain the polygonal contour of the equipment in the image to be processed.
In this embodiment, as shown in fig. 3, the minimum rectangle capable of accommodating all polygon vertices is calculated by performing coordinate operation on the polygon of the device body contour in the real-time picture based on four feature points, so that the data for marking the device position in the deep learning template can be obtained, thereby realizing automatic and rapid marking of the device position and contour, and meanwhile, the position data and contour data of the device can be applied to continuous training of the deep learning template library, and further updating of the deep learning template library is performed, so as to improve the accuracy of intelligent discrimination of the power device. In this embodiment, as shown in fig. 2, for a real-time image processed by manual marking, the marked outline, position, etc. of the image are also converted into a minimum rectangle by the coordinate operation based on the four feature points, so as to provide training data for deep learning, and further update of a deep learning template library is realized.
The data imported into the deep learning template library for further deep learning training further comprises type data of the equipment, and in the embodiment, the picture data are acquired automatically and regularly for training, so that the timing updating and maintenance of the deep learning template library are realized, and the accuracy of intelligent discrimination of the working state of the power equipment is further improved.
Referring to fig. 5, a third embodiment of the present invention is as follows:
The deep learning-based power equipment surface defect detection terminal 1 comprises a memory 2, a processor 3 and a computer program stored on the memory 2 and executable on the processor 3, wherein the steps in the first embodiment or the second embodiment are realized when the processor 3 executes the computer program.
In summary, according to the deep learning-based power equipment surface defect detection method and terminal provided by the invention, firstly, real-time pictures acquired automatically and regularly by an intelligent power inspection system are identified based on a pre-obtained deep learning template library, the pictures which cannot be identified are further identified by an image identification template library, so that the accurate intelligent judgment of the working state of the power equipment is further realized, and the pictures which cannot be judged by the image identification template library are subjected to manual marking processing, so that the intelligent judgment of the working state of the power equipment is further realized. The method comprises the steps of marking a real-time picture, importing an image recognition template library to enrich template materials in the image recognition template library, improving intelligent distinguishing efficiency of the power equipment, importing the real-time picture subjected to distinguishing each time into a deep learning template library again to perform further deep learning training, replacing the mode that the existing distinguished picture only enters a historical image library to be used for historical data inquiry and does not serve other purposes, and continuously updating and perfecting the deep learning template library to realize a self-feedback image recognition processing method. The intelligent inspection system is used for acquiring a large number of pictures, automatic maintenance and automatic updating of the power equipment template library are completed, and more accurate intelligent judgment of the working state of the power equipment is realized.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.

Claims (6)

1. The method for detecting the surface defects of the power equipment based on deep learning is characterized by comprising the following steps:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture;
And in the step S3, an image recognition mode is adopted to judge the real-time picture, specifically:
Performing image recognition on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library to determine the type, the position and the outline of equipment in the real-time pictures, wherein the image recognition template library is formed by distinguishing all history pictures automatically and regularly acquired by an electric intelligent inspection system, and each history picture comprises the type, the position and the outline of the corresponding equipment;
and in the step S4, the type, the position and the outline of the equipment are automatically marked on the real-time picture, and the method specifically comprises the following steps:
s41, obtaining the type of equipment in the real-time picture by using the history picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, taking the history picture as a reference image, and taking the real-time picture as an image to be processed;
S42, selecting four vertexes of a polygonal outline of the equipment on the reference image as characteristic points P1, P2, P3 and P4;
S43, taking a certain characteristic point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating a data block with highest similarity with the data block T in the image to be processed by adopting a global searching method, wherein the calculation formula is as follows:
Wherein P i,j represents image brightness data of M pixel points in the image to be processed, the length and width of the image to be processed taking the position of a coordinate (i, j) as the center are respectively (M/2) - (W-M/2), the value range of j is (M/2) - (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed, which is matched with the characteristic point in the reference image, and repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4, which are matched with the characteristic points P1, P2, P3 and P4;
S46, according to the positions of the four target position points T1, T2, T3 and T4, obtaining the position of the equipment in the image to be processed, and combining the polygonal outline of the equipment in the reference image to obtain the polygonal outline of the equipment in the image to be processed.
2. The method for detecting the surface defects of the power equipment based on the deep learning according to claim 1, wherein the step S3 is to guide the real-time picture after the completion of the manual marking into the deep learning template library for the deep learning, specifically:
and receiving the type and the position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the type, the position and the outline of the real-time picture and the corresponding equipment into the image recognition template library and the deep learning template library to carry out deep learning.
3. The method for detecting surface defects of electrical equipment based on deep learning according to claim 1, wherein the step S4 of automatically marking the type, position and contour of the equipment on the real-time picture further comprises:
And importing the real-time pictures and the types and the outlines of the corresponding devices into the image recognition template library.
4. A deep learning-based power equipment surface defect detection terminal, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
S1, acquiring a real-time picture acquired by an electric power intelligent inspection system at an automatic timing;
s2, judging the real-time picture by adopting a deep learning template library, if the judgment fails, entering a step S3, otherwise, entering a step S4;
S3, judging the real-time picture by adopting an image recognition mode, if judging fails, importing the real-time picture after the completion of manual marking into the deep learning template library for deep learning, updating the deep learning template library, returning to the S1 for recognizing the next real-time picture, and otherwise, entering the step S4;
S4, automatically marking the type, the position and the outline of the equipment on the real-time picture, importing the marked real-time picture into the deep learning template library for deep learning, updating the deep learning template library, and returning to the S1 for identifying the next real-time picture;
And in the step S3, an image recognition mode is adopted to judge the real-time picture, specifically:
Performing image recognition on the real-time pictures which cannot be recognized in the deep learning template library by adopting an image recognition template library to determine the type, the position and the outline of equipment in the real-time pictures, wherein the image recognition template library is formed by distinguishing all history pictures automatically and regularly acquired by an electric intelligent inspection system, and each history picture comprises the type, the position and the outline of the corresponding equipment;
and in the step S4, the type, the position and the outline of the equipment are automatically marked on the real-time picture, and the method specifically comprises the following steps:
s41, obtaining the type of equipment in the real-time picture by using the history picture corresponding to the real-time picture in the image recognition template library recognized in the image recognition mode, taking the history picture as a reference image, and taking the real-time picture as an image to be processed;
S42, selecting four vertexes of a polygonal outline of the equipment on the reference image as characteristic points P1, P2, P3 and P4;
S43, taking a certain characteristic point as a center, selecting a brightness data block T with the length and width of M pixel points as a data calculation basis for calculating similarity measurement, wherein the value of M is 34-40 pixels;
s44, calculating a data block with highest similarity with the data block T in the image to be processed by adopting a global searching method, wherein the calculation formula is as follows:
Wherein P i,j represents image brightness data of M pixel points in the image to be processed, the length and width of the image to be processed taking the position of a coordinate (i, j) as the center are respectively (M/2) - (W-M/2), the value range of j is (M/2) - (H-M/2), W is the width of the image to be processed, and H is the height of the image to be processed;
S45, counting the maximum value of all R (i, j), wherein (i, j) is the position point on the image to be processed, which is matched with the characteristic point in the reference image, and repeating the steps S43 to S44 to respectively obtain target position points T1, T2, T3 and T4, which are matched with the characteristic points P1, P2, P3 and P4;
S46, according to the positions of the four target position points T1, T2, T3 and T4, obtaining the position of the equipment in the image to be processed, and combining the polygonal outline of the equipment in the reference image to obtain the polygonal outline of the equipment in the image to be processed.
5. The deep learning-based power equipment surface defect detection terminal according to claim 4, wherein the step S3 is to import the real-time picture after the completion of the manual marking into the deep learning template library for deep learning, specifically:
and receiving the type and the position of equipment marked on the real-time picture manually, copying the outline of the equipment obtained by the real-time picture manually by adopting a polygon, and importing the type, the position and the outline of the real-time picture and the corresponding equipment into the image recognition template library and the deep learning template library to carry out deep learning.
6. The deep learning-based power equipment surface defect detection terminal of claim 4, wherein the step S4 of automatically marking the type, position and contour of the equipment on the real-time picture further comprises:
And importing the real-time pictures and the types and the outlines of the corresponding devices into the image recognition template library.
CN202111256280.9A 2021-10-27 2021-10-27 Deep learning-based power equipment surface defect detection method and terminal Active CN114219753B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111256280.9A CN114219753B (en) 2021-10-27 2021-10-27 Deep learning-based power equipment surface defect detection method and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111256280.9A CN114219753B (en) 2021-10-27 2021-10-27 Deep learning-based power equipment surface defect detection method and terminal

Publications (2)

Publication Number Publication Date
CN114219753A CN114219753A (en) 2022-03-22
CN114219753B true CN114219753B (en) 2025-01-17

Family

ID=80696182

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111256280.9A Active CN114219753B (en) 2021-10-27 2021-10-27 Deep learning-based power equipment surface defect detection method and terminal

Country Status (1)

Country Link
CN (1) CN114219753B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115393258A (en) * 2022-04-25 2022-11-25 佳源科技股份有限公司 Visual defect detection method based on transformer substation instrument
CN116704267B (en) * 2023-08-01 2023-10-27 成都斐正能达科技有限责任公司 Deep learning 3D printing defect detection method based on improved YOLOX algorithm
CN117725942A (en) * 2024-02-06 2024-03-19 浙江码尚科技股份有限公司 Identification and early warning method and system for label texture anti-counterfeiting
CN119762802A (en) * 2025-01-03 2025-04-04 广州市派客朴食信息科技有限责任公司 Tableware contour learning and identifying method and system based on deep neural network

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111666957A (en) * 2020-07-17 2020-09-15 湖南华威金安企业管理有限公司 Image authenticity recognition method and device
CN112990335A (en) * 2021-03-31 2021-06-18 江苏方天电力技术有限公司 Intelligent recognition self-learning training method and system for power grid unmanned aerial vehicle inspection image defects

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105119910A (en) * 2015-07-23 2015-12-02 浙江大学 Template-based online social network rubbish information real-time detecting method
CN108447051B (en) * 2018-03-09 2019-12-24 东北大学 A Computer Vision-Based Appraisal Method for Surface Defect Grades of Metal Products
CN110378869B (en) * 2019-06-05 2021-05-11 北京交通大学 Steel rail fastener abnormity detection method with automatic sample marking function

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111666957A (en) * 2020-07-17 2020-09-15 湖南华威金安企业管理有限公司 Image authenticity recognition method and device
CN112990335A (en) * 2021-03-31 2021-06-18 江苏方天电力技术有限公司 Intelligent recognition self-learning training method and system for power grid unmanned aerial vehicle inspection image defects

Also Published As

Publication number Publication date
CN114219753A (en) 2022-03-22

Similar Documents

Publication Publication Date Title
CN114219753B (en) Deep learning-based power equipment surface defect detection method and terminal
Li et al. Automatic pavement crack detection by multi-scale image fusion
CN112270310A (en) Cross-camera pedestrian multi-target tracking method and device based on deep learning
CN110969166A (en) Small target identification method and system in inspection scene
CN101957325A (en) Substation equipment appearance abnormality recognition method based on substation inspection robot
CN111754422A (en) A visual edge patrol method based on EDLines and LSM
CN117853492B (en) Intelligent industrial defect detection method and system based on fusion model
CN113706455B (en) Rapid detection method for damage of 330kV cable porcelain insulator sleeve
CN116363573A (en) Transformer substation equipment state anomaly identification method and system
CN111563896A (en) Image processing method for catenary anomaly detection
CN114972736B (en) A method and system for identifying abnormality of square hole locks of rail vehicles
CN111091534A (en) A method for detecting and locating PCB board defects based on target detection
CN111144237A (en) AI-based water pollution identification method and system
CN111950556A (en) A detection method of number plate printing quality based on deep learning
CN110287970B (en) A Weakly Supervised Object Localization Method Based on CAM and Masking
CN103353941A (en) Natural marker registration method based on viewpoint classification
CN118967555B (en) Metal surface defect detection method and system for segmentation model with noise label
KR102887123B1 (en) Position determination apparatus and method of display manufacturing facility
CN118587496A (en) Automatic identification system and method of parts processing accuracy based on computer vision
CN119169389A (en) A carton classification method and device based on data processing
CN114821598B (en) Steel plate blank character recognition method, equipment, storage medium and device
CN113554018B (en) Meter automatic modeling method based on machine vision
Tang et al. A YOLOv3-based framework for weed detection in agricultural fields
CN118470293B (en) Anti-offset two-dimensional code detection positioning method
CN120689699B (en) An AI visual inspection method based on supervised and unsupervised deep learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant