Detailed Description
A human eye visual angle limited space operation remote monitoring system based on AR intelligent glasses comprises AR augmented reality glasses, a ground operation monitoring mobile terminal and a remote control terminal; the AR augmented reality glasses comprise cameras, intelligent lenses and intelligent controllers, wherein the cameras, the intelligent lenses and the intelligent controllers are installed on the glasses frames of the AR augmented reality glasses; the camera is used for capturing a front image and generating a front video image; the intelligent lens is used for providing a user interaction interface, and the user interaction interface provides menu options and displays a front video image in real time;
the intelligent controller comprises a control chip and a wireless communication module; the control chip receives the front video image through the camera interface and dynamically superposes the front video image to generate an AR end enhanced image; the control chip transmits the AR end enhanced image to the ground operation monitoring mobile end or the remote control terminal through the wireless communication module;
the ground operation monitoring mobile terminal is used for marking the AR terminal enhanced image with guide information through dynamic superposition processing to generate a mobile terminal marking image and respectively transmitting the mobile terminal marking image to AR augmented reality glasses or a remote control terminal;
the remote control terminal is used for generating a remote end annotation image for the AR end enhanced image annotation data information through dynamic superposition processing; the remote control terminal can also be used for generating a mixed annotation image by dynamically overlaying the annotation data information of the mobile end annotation image; and the remote control terminal transmits the remote end annotation image and the mixed annotation image to AR augmented reality glasses or a ground operation monitoring mobile end.
The intelligent controller also comprises a voice recognition module; the intelligent controller selectively displays a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image through voice control.
As a preferred technical scheme, a partial discharge sensor electrically connected with an intelligent controller is further installed on a spectacle frame of the AR augmented reality spectacles; the partial discharge inductor monitors live line information in the electrical equipment and transmits the live line information to the intelligent controller; the control chip marks the charged circuit information to the front video image through dynamic superposition processing to generate an AR end enhanced image.
As a preferred technical scheme, the control chip is electrically connected with the voice recognition module and is used for recognizing the voice command into a control command; the control chip controls the image page turning, the amplification, the reduction and the visual angle movement of the image set according to the control instruction; the image set comprises a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image.
As a preferred technical scheme, the voice recognition module is in communication connection with the bluetooth headset through the bluetooth module, and is used for receiving a voice instruction or voice interaction.
As a preferred technical scheme, a storage unit is further arranged in the intelligent controller; the control chip controls the camera to record video and stores the video record to the storage unit; the control chip also controls a video recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through a streaming media technology; and the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live broadcast.
As a preferred technical scheme, the AR augmented reality glasses, the ground operation monitoring mobile terminal and the remote control terminal are all in communication connection with a server; the server is internally provided with a digital book library for storing a test standardized process and detailed information of tested equipment; the ground operation monitoring mobile terminal and the remote control terminal acquire equipment information for being marked on a marked image of the mobile terminal by inquiring the digital book library.
As a preferred technical solution, the control chip intercepts the operation video from the video of the storage unit and transmits the operation video to the digital book library of the server for VR training.
The invention has the following advantages:
1) the partial discharge inductor detects live line information in the electrical equipment, marks the live line information on a front video image through dynamic superposition processing to generate an AR end enhanced image, realizes the combination of an equipment maintenance operation technology and an augmented reality virtual and intelligent sensing technology, realizes the remote monitoring of the visual angle limited space operation of human eyes, and can effectively reduce the danger of ascending maintainers;
2) according to the invention, after observing the AR end enhanced image and retrieving related data, a ground guardian labels related guidance information on the AR end enhanced image, generates a mobile labeled image through dynamic superposition processing, and finally transmits the mobile labeled image to AR augmented reality glasses and displays the mobile labeled image through a user interaction interface so as to help a climber to overhaul equipment and improve the efficiency;
3) the remote end labeled image is a service center person, and is formed by inquiring more detailed information, combining maintenance and inspection suggestions given by the AR end enhanced image and dynamically overlaying the information on the AR end enhanced image; similarly, the mixed annotation image is formed based on the mobile end annotation image and dynamic superposition maintenance and inspection suggestions, so that the ascending personnel can avoid danger and can overhaul more efficiently;
4) the control chip of the invention also controls the video recording recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through the streaming media technology; the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live; the multi-person multi-place cooperative operation of the primary equipment maintenance operation of the transformer substation is really realized, and the maintenance efficiency is improved.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Example 1
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention.
Referring to fig. 1-2, the invention relates to an AR intelligent glasses-based remote monitoring system for limited space operation of human eye visual angle, comprising: AR augmented reality glasses, a ground operation monitoring mobile terminal and a remote control terminal.
The AR augmented reality glasses comprise cameras, intelligent lenses and intelligent controllers, wherein the cameras, the intelligent lenses and the intelligent controllers are installed on the glasses frame; a camera for capturing a front image and generating a front video image; specifically, the camera here can adopt a binocular depth camera, the binocular depth camera is embedded in the middle of the spectacle frame, the binocular depth camera estimates the depth by using a stereoscopic vision principle (stereo, MVS), the depth is obtained by using a parallax principle, a pattern is projected, and then comparison is performed, so that the binocular depth camera can play a role in enhancing reality, and is connected with the control chip through a camera interface; the remote monitoring system has the advantages that the equipment maintenance operation technology is combined with the augmented reality virtual and intelligent sensing technology, and the remote monitoring of the eye visual angle limited space operation is realized.
The intelligent lens is used for providing a user interaction interface, and the user interaction interface provides menu options and displays the front video image in real time; specifically, the control chip is electrically connected with the voice recognition module and is used for recognizing the voice command into a control command; specifically, the user interaction interface comprises my application, a camera and my file; opening my application program, selecting items after rotating the head, specifically comprising: equipment tour, equipment standing book, area search and visual display.
The intelligent controller comprises a control chip and a wireless communication module; the control chip receives the front video image through the camera interface and dynamically superposes the front video image to generate an AR end enhanced image; specifically, the mirror bracket is also provided with a partial discharge sensor electrically connected with the intelligent controller; the partial discharge sensor monitors live line information in the electrical equipment and transmits the live line information to the intelligent controller; the control chip labels the charged circuit information to a front video image through dynamic superposition processing to generate an AR end enhanced image; in actual use, the wireless transmission module is a 4G module, which is convenient for realizing remote text, image and video transmission, and the detection range of the partial discharge inductor is as follows:
a live line is arranged in 10kV electrical equipment, and the detection distance is 0-0.7 m; the detection distance of a live line in 35kV electrical equipment is 0.7-1.0 meter; a live line is arranged in 110kV electrical equipment, and the detection distance is 1.0-1.5 m; a live line in 220kV electrical equipment is detected at a distance of 1.5-3.0 m; here, the partial discharge inductor detects live-line and dynamic superposition to the video image in the place ahead in electrical equipment, can effectually make the maintainer of ascending a height reduce danger.
In addition, the control chip transmits the AR end enhanced image to the ground operation monitoring mobile end or the remote control terminal through a wireless communication module; the ground operation monitoring mobile terminal is used for marking the guide information on the AR end enhanced image to generate a mobile terminal marking image through dynamic superposition processing, and respectively transmitting the mobile terminal marking image to AR augmented reality glasses or a remote control terminal; the system comprises a digital library, a database and a database server, wherein the digital library is used for remotely inquiring a digital book library and storing a test standardized process and detailed information of tested equipment; the real-time video is used for displaying the video image captured by the camera in real time; the picture management is used for storing or displaying a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image; here, after ground guardians observe the AR end augmented image and retrieve relevant data, the AR end augmented image is labeled with relevant known information and is processed through dynamic superposition to generate a mobile labeling image, and the mobile labeling image is finally transmitted to the AR augmented reality glasses and is displayed through a user interaction interface so as to help climbers to overhaul equipment and improve efficiency.
The remote control terminal is used for generating a remote end annotation image for the AR end enhanced image annotation data information through dynamic superposition processing; the remote control terminal is used for marking data information of the mobile terminal marked image to generate a mixed marked image through dynamic superposition processing; the remote end annotation image is formed by inquiring more detailed information, combining maintenance and inspection suggestions given by the AR end enhancement image and dynamically overlaying the information on the AR end enhancement image by the service center personnel; similarly, the hybrid annotation images are formed based on mobile-side annotation images, dynamic overlay maintenance and inspection recommendations, so that the climber avoids danger and more efficient maintenance.
The remote control terminal transmits the remote end annotation image and the mixed annotation image to AR augmented reality glasses or a ground operation monitoring mobile end; the intelligent controller selectively displays a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image through voice control; in fact, the voice recognition module is in communication connection with the bluetooth headset through the bluetooth module and is used for receiving voice instructions or voice interaction; the control chip controls the image page turning, the amplification, the reduction and the visual angle movement of the image set according to the control instruction; the image set comprises a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image; in addition, the voice recognition module recognizing the common control instruction comprises: real-time transmission, video recording, electronic file viewing and file transmission, a function menu does not need to be opened step by step, and the operation is convenient.
In addition, a storage unit is arranged in the intelligent controller; the control chip controls the camera to record video and stores the video record to the storage unit; the control chip also controls the video recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through the streaming media technology; the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live; and multi-person multi-place cooperative operation of primary equipment maintenance operation of the transformer substation is really realized.
The AR augmented reality glasses, the ground operation monitoring mobile terminal and the remote control terminal are in communication connection with the server; the server is provided with a digital book library for storing a test standardized process and detailed information of tested equipment; the ground operation monitoring mobile terminal and the remote control terminal acquire equipment information for being marked on a marked image of the mobile terminal by inquiring the digital book library; the remote AR picture has a real-time marking function, and can quickly and effectively monitor and guide the aerial work; the monitoring requirements of different roles in different scene operation sites are met. The control chip intercepts the misoperation video from the video of the storage unit and transmits the misoperation video to a digital book library of the server for VR training, and the VR training system is used as a teaching material to improve VR training effect.
When the system is actually used, the partial discharge inductor detects live line information in the electrical equipment, and marks the live line information on a front video image through dynamic superposition processing to generate an AR end enhanced image, so that the combination of an equipment maintenance operation technology and an augmented reality virtual and intelligent sensing technology is realized, the remote monitoring of the visual angle limited space operation of human eyes is realized, and the danger of ascending maintenance personnel can be effectively reduced; meanwhile, after the ground guardian observes the AR end enhanced image and retrieves related data, the AR end enhanced image is labeled with related known information, a mobile labeled image is generated through dynamic superposition processing, and finally the information is transmitted to the AR augmented reality glasses and displayed through a user interaction interface so as to help the climber to overhaul the equipment and improve the efficiency.
In addition, in order to ensure better maintenance effect, the remote end annotation image is taken as a service center personnel, and is formed by inquiring more detailed data, combining maintenance and inspection suggestions given by the AR end enhancement image and dynamically overlaying the data on the AR end enhancement image; similarly, the hybrid annotation images are formed based on mobile-side annotation images, dynamic overlay maintenance and inspection recommendations, so that the climber avoids danger and more efficient maintenance.
It should be noted that, in the above system embodiment, each included unit is only divided according to functional logic, but is not limited to the above division as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
In addition, it is understood by those skilled in the art that all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing associated hardware, and the corresponding program may be stored in a computer-readable storage medium.
The preferred embodiments of the present invention disclosed above are intended only to aid in the description of the invention; the preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed.