CN112153267B - Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses - Google Patents

Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses Download PDF

Info

Publication number
CN112153267B
CN112153267B CN202011138066.9A CN202011138066A CN112153267B CN 112153267 B CN112153267 B CN 112153267B CN 202011138066 A CN202011138066 A CN 202011138066A CN 112153267 B CN112153267 B CN 112153267B
Authority
CN
China
Prior art keywords
image
remote
mobile terminal
terminal
remote control
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011138066.9A
Other languages
Chinese (zh)
Other versions
CN112153267A (en
Inventor
汪祝年
张钰金
陈通
张丹青
李星宇
胡航
李静
陈永明
包磊
孙咸凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhenjiang Power Supply Co of State Grid Jiangsu Electric Power Co Ltd
Original Assignee
Zhenjiang Power Supply Co of State Grid Jiangsu Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhenjiang Power Supply Co of State Grid Jiangsu Electric Power Co Ltd filed Critical Zhenjiang Power Supply Co of State Grid Jiangsu Electric Power Co Ltd
Priority to CN202011138066.9A priority Critical patent/CN112153267B/en
Publication of CN112153267A publication Critical patent/CN112153267A/en
Application granted granted Critical
Publication of CN112153267B publication Critical patent/CN112153267B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223Execution procedure of a spoken command

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明公开了基于AR智能眼镜的人眼视角局限空间作业远程监护系统,涉及AR眼镜远程监控技术领域;本发明包括AR增强现实眼镜,地面作业监护移动端,远程控制终端;所述AR增强现实眼镜包括安装在镜架上的摄像头、智能镜片、智能控制器;所述摄像头用于捕捉前方图像并生成前方视频图像;所述智能镜片用于提供用户交互界面,所述用户交互界面提供菜单选项并实时显示前方视频图像;所述智能控制器包括控制芯片、无线通信模块;所述地面作业监护移动端用于生成移动端标注图像;所述远程控制终端用于生成远程端标注图像;所述远程控制终端还用于生成混合标注图像。本发明用于实现人眼视角局限空间作业远程监护,能够使登高检修人员减少危险。

Figure 202011138066

The invention discloses a remote monitoring system for space operations with limited human eye perspective based on AR smart glasses, and relates to the technical field of AR glasses remote monitoring; the invention includes AR augmented reality glasses, a ground operation monitoring mobile terminal, and a remote control terminal; the AR augmented reality The glasses include a camera mounted on the frame, a smart lens, and a smart controller; the camera is used to capture a front image and generate a front video image; the smart lens is used to provide a user interaction interface that provides menu options and display the front video image in real time; the intelligent controller includes a control chip and a wireless communication module; the ground operation monitoring mobile terminal is used to generate an annotation image of the mobile terminal; the remote control terminal is used to generate an annotation image of the remote terminal; the The remote control terminal is also used to generate hybrid annotated images. The present invention is used for realizing remote monitoring of space operations with limited viewing angle of human eyes, and can reduce dangers for maintenance personnel who climb up.

Figure 202011138066

Description

Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses
Technical Field
The invention belongs to the technical field of AR (augmented reality) glasses remote monitoring, and particularly relates to an AR intelligent glasses-based remote monitoring system for limited-space operation of human eye visual angle.
Background
In the process of overhauling the transformer substation equipment, due to the limitation of an operation field and space, some operations can be finished by a single person, and the purpose of simultaneously monitoring the operations by a plurality of persons cannot be realized; for example: the ground personnel can not effectively know the operation state of the ascending wiring personnel in the ascending operation, and the outside-cabinet guardian can not know the operation state of the inside-cabinet operator in the switch cabinet operation.
Taking a 220kV current transformer test in the electrical test specialty as an example: when a capacitance dielectric loss test is carried out on a 220kV current transformer in a secondary junction box, test wiring work of a secondary lead is required.
The secondary line concentration box is 5-6 meters away from the ground and the equipment interval is close, so that lifting equipment such as a bucket arm vehicle cannot be used, and only a single person can ascend by using a single ladder or a herringbone ladder to perform wiring operation.
During wiring, ground personnel cannot directly observe the wiring condition in the wiring box; if equipment is unusual, the personnel of ascending a height can only feed back the box interior situation to ground personnel through the mode of singlehanded camera shooting, still need the second person to ascend a height in turn after the completion of connecing, taking out stitches and rechecking, and ground personnel are difficult directly to carry out effective management and control to operation condition and operation quality, and the equipment abnormal conditions also is difficult to convey to the distant place in real time.
In addition, the operators and guardians who ascend the height cannot know and solve all the problems, and under more conditions, the operators and guardians need to inquire data and digital book libraries to acquire failure problems or solutions; how to realize the convenient acquisition of relevant data by the ascending operators and the guardians in the maintenance is also an urgent problem to be solved.
In order to solve the problems, the invention provides a human eye visual angle limited space operation remote monitoring system based on AR intelligent glasses.
Disclosure of Invention
The invention provides an AR intelligent glasses-based remote monitoring system for human eye visual angle confined space operation, and aims to realize remote monitoring of human eye visual angle confined space operation.
The technical solution of the invention is as follows: a human eye visual angle limited space operation remote monitoring system based on AR intelligent glasses comprises AR augmented reality glasses, a ground operation monitoring mobile terminal and a remote control terminal; the AR augmented reality glasses comprise cameras, intelligent lenses and intelligent controllers, wherein the cameras, the intelligent lenses and the intelligent controllers are installed on the glasses frame; the camera is used for capturing a front image and generating a front video image; the intelligent lens is used for providing a user interaction interface, and the user interaction interface provides menu options and displays a front video image in real time;
the intelligent controller comprises a control chip and a wireless communication module; the control chip receives the front video image through the camera interface and dynamically superposes the front video image to generate an AR end enhanced image; the control chip transmits the AR end enhanced image to the ground operation monitoring mobile end or the remote control terminal through the wireless communication module;
the ground operation monitoring mobile terminal is used for marking the AR terminal enhanced image with guide information through dynamic superposition processing to generate a mobile terminal marking image and respectively transmitting the mobile terminal marking image to AR augmented reality glasses or a remote control terminal;
the remote control terminal is used for generating a remote end annotation image for the AR end enhanced image annotation data information through dynamic superposition processing; the remote control terminal can also be used for generating a mixed annotation image by dynamically overlaying the annotation data information of the mobile end annotation image; and the remote control terminal transmits the remote end annotation image and the mixed annotation image to AR augmented reality glasses or a ground operation monitoring mobile end.
The invention has the beneficial effects that:
1) the remote monitoring of the spatial operation with the limited visual angle of the human eyes can be realized, and the danger of the ascending maintainers can be effectively reduced;
2) the user interaction interface display can help the climber to overhaul the equipment and improve the efficiency;
3) the multi-person multi-place cooperative operation of the primary equipment maintenance operation of the transformer substation can be really realized, and the maintenance efficiency is improved.
Drawings
Fig. 1 is a schematic structural diagram of AR augmented reality glasses according to the present invention.
Fig. 2 is a schematic structural diagram of the human eye visual angle limited space operation remote monitoring system based on the AR intelligent glasses.
Detailed Description
A human eye visual angle limited space operation remote monitoring system based on AR intelligent glasses comprises AR augmented reality glasses, a ground operation monitoring mobile terminal and a remote control terminal; the AR augmented reality glasses comprise cameras, intelligent lenses and intelligent controllers, wherein the cameras, the intelligent lenses and the intelligent controllers are installed on the glasses frames of the AR augmented reality glasses; the camera is used for capturing a front image and generating a front video image; the intelligent lens is used for providing a user interaction interface, and the user interaction interface provides menu options and displays a front video image in real time;
the intelligent controller comprises a control chip and a wireless communication module; the control chip receives the front video image through the camera interface and dynamically superposes the front video image to generate an AR end enhanced image; the control chip transmits the AR end enhanced image to the ground operation monitoring mobile end or the remote control terminal through the wireless communication module;
the ground operation monitoring mobile terminal is used for marking the AR terminal enhanced image with guide information through dynamic superposition processing to generate a mobile terminal marking image and respectively transmitting the mobile terminal marking image to AR augmented reality glasses or a remote control terminal;
the remote control terminal is used for generating a remote end annotation image for the AR end enhanced image annotation data information through dynamic superposition processing; the remote control terminal can also be used for generating a mixed annotation image by dynamically overlaying the annotation data information of the mobile end annotation image; and the remote control terminal transmits the remote end annotation image and the mixed annotation image to AR augmented reality glasses or a ground operation monitoring mobile end.
The intelligent controller also comprises a voice recognition module; the intelligent controller selectively displays a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image through voice control.
As a preferred technical scheme, a partial discharge sensor electrically connected with an intelligent controller is further installed on a spectacle frame of the AR augmented reality spectacles; the partial discharge inductor monitors live line information in the electrical equipment and transmits the live line information to the intelligent controller; the control chip marks the charged circuit information to the front video image through dynamic superposition processing to generate an AR end enhanced image.
As a preferred technical scheme, the control chip is electrically connected with the voice recognition module and is used for recognizing the voice command into a control command; the control chip controls the image page turning, the amplification, the reduction and the visual angle movement of the image set according to the control instruction; the image set comprises a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image.
As a preferred technical scheme, the voice recognition module is in communication connection with the bluetooth headset through the bluetooth module, and is used for receiving a voice instruction or voice interaction.
As a preferred technical scheme, a storage unit is further arranged in the intelligent controller; the control chip controls the camera to record video and stores the video record to the storage unit; the control chip also controls a video recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through a streaming media technology; and the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live broadcast.
As a preferred technical scheme, the AR augmented reality glasses, the ground operation monitoring mobile terminal and the remote control terminal are all in communication connection with a server; the server is internally provided with a digital book library for storing a test standardized process and detailed information of tested equipment; the ground operation monitoring mobile terminal and the remote control terminal acquire equipment information for being marked on a marked image of the mobile terminal by inquiring the digital book library.
As a preferred technical solution, the control chip intercepts the operation video from the video of the storage unit and transmits the operation video to the digital book library of the server for VR training.
The invention has the following advantages:
1) the partial discharge inductor detects live line information in the electrical equipment, marks the live line information on a front video image through dynamic superposition processing to generate an AR end enhanced image, realizes the combination of an equipment maintenance operation technology and an augmented reality virtual and intelligent sensing technology, realizes the remote monitoring of the visual angle limited space operation of human eyes, and can effectively reduce the danger of ascending maintainers;
2) according to the invention, after observing the AR end enhanced image and retrieving related data, a ground guardian labels related guidance information on the AR end enhanced image, generates a mobile labeled image through dynamic superposition processing, and finally transmits the mobile labeled image to AR augmented reality glasses and displays the mobile labeled image through a user interaction interface so as to help a climber to overhaul equipment and improve the efficiency;
3) the remote end labeled image is a service center person, and is formed by inquiring more detailed information, combining maintenance and inspection suggestions given by the AR end enhanced image and dynamically overlaying the information on the AR end enhanced image; similarly, the mixed annotation image is formed based on the mobile end annotation image and dynamic superposition maintenance and inspection suggestions, so that the ascending personnel can avoid danger and can overhaul more efficiently;
4) the control chip of the invention also controls the video recording recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through the streaming media technology; the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live; the multi-person multi-place cooperative operation of the primary equipment maintenance operation of the transformer substation is really realized, and the maintenance efficiency is improved.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Example 1
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the present invention.
Referring to fig. 1-2, the invention relates to an AR intelligent glasses-based remote monitoring system for limited space operation of human eye visual angle, comprising: AR augmented reality glasses, a ground operation monitoring mobile terminal and a remote control terminal.
The AR augmented reality glasses comprise cameras, intelligent lenses and intelligent controllers, wherein the cameras, the intelligent lenses and the intelligent controllers are installed on the glasses frame; a camera for capturing a front image and generating a front video image; specifically, the camera here can adopt a binocular depth camera, the binocular depth camera is embedded in the middle of the spectacle frame, the binocular depth camera estimates the depth by using a stereoscopic vision principle (stereo, MVS), the depth is obtained by using a parallax principle, a pattern is projected, and then comparison is performed, so that the binocular depth camera can play a role in enhancing reality, and is connected with the control chip through a camera interface; the remote monitoring system has the advantages that the equipment maintenance operation technology is combined with the augmented reality virtual and intelligent sensing technology, and the remote monitoring of the eye visual angle limited space operation is realized.
The intelligent lens is used for providing a user interaction interface, and the user interaction interface provides menu options and displays the front video image in real time; specifically, the control chip is electrically connected with the voice recognition module and is used for recognizing the voice command into a control command; specifically, the user interaction interface comprises my application, a camera and my file; opening my application program, selecting items after rotating the head, specifically comprising: equipment tour, equipment standing book, area search and visual display.
The intelligent controller comprises a control chip and a wireless communication module; the control chip receives the front video image through the camera interface and dynamically superposes the front video image to generate an AR end enhanced image; specifically, the mirror bracket is also provided with a partial discharge sensor electrically connected with the intelligent controller; the partial discharge sensor monitors live line information in the electrical equipment and transmits the live line information to the intelligent controller; the control chip labels the charged circuit information to a front video image through dynamic superposition processing to generate an AR end enhanced image; in actual use, the wireless transmission module is a 4G module, which is convenient for realizing remote text, image and video transmission, and the detection range of the partial discharge inductor is as follows:
a live line is arranged in 10kV electrical equipment, and the detection distance is 0-0.7 m; the detection distance of a live line in 35kV electrical equipment is 0.7-1.0 meter; a live line is arranged in 110kV electrical equipment, and the detection distance is 1.0-1.5 m; a live line in 220kV electrical equipment is detected at a distance of 1.5-3.0 m; here, the partial discharge inductor detects live-line and dynamic superposition to the video image in the place ahead in electrical equipment, can effectually make the maintainer of ascending a height reduce danger.
In addition, the control chip transmits the AR end enhanced image to the ground operation monitoring mobile end or the remote control terminal through a wireless communication module; the ground operation monitoring mobile terminal is used for marking the guide information on the AR end enhanced image to generate a mobile terminal marking image through dynamic superposition processing, and respectively transmitting the mobile terminal marking image to AR augmented reality glasses or a remote control terminal; the system comprises a digital library, a database and a database server, wherein the digital library is used for remotely inquiring a digital book library and storing a test standardized process and detailed information of tested equipment; the real-time video is used for displaying the video image captured by the camera in real time; the picture management is used for storing or displaying a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image; here, after ground guardians observe the AR end augmented image and retrieve relevant data, the AR end augmented image is labeled with relevant known information and is processed through dynamic superposition to generate a mobile labeling image, and the mobile labeling image is finally transmitted to the AR augmented reality glasses and is displayed through a user interaction interface so as to help climbers to overhaul equipment and improve efficiency.
The remote control terminal is used for generating a remote end annotation image for the AR end enhanced image annotation data information through dynamic superposition processing; the remote control terminal is used for marking data information of the mobile terminal marked image to generate a mixed marked image through dynamic superposition processing; the remote end annotation image is formed by inquiring more detailed information, combining maintenance and inspection suggestions given by the AR end enhancement image and dynamically overlaying the information on the AR end enhancement image by the service center personnel; similarly, the hybrid annotation images are formed based on mobile-side annotation images, dynamic overlay maintenance and inspection recommendations, so that the climber avoids danger and more efficient maintenance.
The remote control terminal transmits the remote end annotation image and the mixed annotation image to AR augmented reality glasses or a ground operation monitoring mobile end; the intelligent controller selectively displays a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image through voice control; in fact, the voice recognition module is in communication connection with the bluetooth headset through the bluetooth module and is used for receiving voice instructions or voice interaction; the control chip controls the image page turning, the amplification, the reduction and the visual angle movement of the image set according to the control instruction; the image set comprises a front video image, an AR end enhanced image, a mobile end annotation image, a remote end annotation image or a mixed annotation image; in addition, the voice recognition module recognizing the common control instruction comprises: real-time transmission, video recording, electronic file viewing and file transmission, a function menu does not need to be opened step by step, and the operation is convenient.
In addition, a storage unit is arranged in the intelligent controller; the control chip controls the camera to record video and stores the video record to the storage unit; the control chip also controls the video recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or the remote control terminal through the streaming media technology; the ground operation monitoring mobile terminal or the remote control terminal broadcasts the video live; and multi-person multi-place cooperative operation of primary equipment maintenance operation of the transformer substation is really realized.
The AR augmented reality glasses, the ground operation monitoring mobile terminal and the remote control terminal are in communication connection with the server; the server is provided with a digital book library for storing a test standardized process and detailed information of tested equipment; the ground operation monitoring mobile terminal and the remote control terminal acquire equipment information for being marked on a marked image of the mobile terminal by inquiring the digital book library; the remote AR picture has a real-time marking function, and can quickly and effectively monitor and guide the aerial work; the monitoring requirements of different roles in different scene operation sites are met. The control chip intercepts the misoperation video from the video of the storage unit and transmits the misoperation video to a digital book library of the server for VR training, and the VR training system is used as a teaching material to improve VR training effect.
When the system is actually used, the partial discharge inductor detects live line information in the electrical equipment, and marks the live line information on a front video image through dynamic superposition processing to generate an AR end enhanced image, so that the combination of an equipment maintenance operation technology and an augmented reality virtual and intelligent sensing technology is realized, the remote monitoring of the visual angle limited space operation of human eyes is realized, and the danger of ascending maintenance personnel can be effectively reduced; meanwhile, after the ground guardian observes the AR end enhanced image and retrieves related data, the AR end enhanced image is labeled with related known information, a mobile labeled image is generated through dynamic superposition processing, and finally the information is transmitted to the AR augmented reality glasses and displayed through a user interaction interface so as to help the climber to overhaul the equipment and improve the efficiency.
In addition, in order to ensure better maintenance effect, the remote end annotation image is taken as a service center personnel, and is formed by inquiring more detailed data, combining maintenance and inspection suggestions given by the AR end enhancement image and dynamically overlaying the data on the AR end enhancement image; similarly, the hybrid annotation images are formed based on mobile-side annotation images, dynamic overlay maintenance and inspection recommendations, so that the climber avoids danger and more efficient maintenance.
It should be noted that, in the above system embodiment, each included unit is only divided according to functional logic, but is not limited to the above division as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
In addition, it is understood by those skilled in the art that all or part of the steps in the method for implementing the embodiments described above may be implemented by a program instructing associated hardware, and the corresponding program may be stored in a computer-readable storage medium.
The preferred embodiments of the present invention disclosed above are intended only to aid in the description of the invention; the preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed.

Claims (6)

1.基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是该系统包括AR增强现实眼镜,地面作业监护移动端,远程控制终端;1. A remote monitoring system for space operations with limited human eye perspective based on AR smart glasses, characterized in that the system includes AR augmented reality glasses, a mobile terminal for ground operation monitoring, and a remote control terminal; 所述AR增强现实眼镜包括安装在镜架上的摄像头、智能镜片、智能控制器;所述摄像头用于捕捉前方图像并生成前方视频图像;所述智能镜片用于提供用户交互界面,所述用户交互界面提供菜单选项并实时显示前方视频图像;The AR augmented reality glasses include a camera mounted on the frame, a smart lens, and an intelligent controller; the camera is used to capture a front image and generate a front video image; the smart lens is used to provide a user interaction interface, and the user The interactive interface provides menu options and displays the front video image in real time; 所述智能控制器包括控制芯片、无线通信模块;所述控制芯片通过摄像头接口接收前方视频图像,并对前方视频图像动态叠加处理生成AR端增强图像;所述控制芯片通过无线通信模块向地面作业监护移动端或远程控制终端传递AR端增强图像;The intelligent controller includes a control chip and a wireless communication module; the control chip receives the front video image through the camera interface, and dynamically superimposes the front video image to generate an AR-side enhanced image; the control chip operates on the ground through the wireless communication module The monitoring mobile terminal or the remote control terminal transmits the augmented image of the AR terminal; 所述地面作业监护移动端,用于通过动态叠加处理对所述AR端增强图像标注指导信息生成移动端标注图像,并分别传递移动端标注图像至AR增强现实眼镜或远程控制终端;The ground operation monitoring mobile terminal is used to generate a mobile terminal marked image for the AR terminal enhanced image marking guidance information through dynamic superposition processing, and respectively transmit the mobile terminal marked image to the AR augmented reality glasses or the remote control terminal; 所述远程控制终端用于通过动态叠加处理对AR端增强图像标注数据信息生成远程端标注图像;所述远程控制终端还用于通过动态叠加处理对移动端标注图像标注数据信息生成混合标注图像;所述远程控制终端传递远程端标注图像、混合标注图像至AR增强现实眼镜或地面作业监护移动端;The remote control terminal is configured to generate a remote-end labeled image from the AR-side enhanced image annotation data information through dynamic superposition processing; the remote-control terminal is further configured to generate a mixed-labeled image from the mobile-end labeled image annotation data information through dynamic superimposition processing; The remote control terminal transmits the remote end labeled image and the mixed labeled image to the AR augmented reality glasses or the ground operation monitoring mobile end; 所述智能控制器还包括语音识别模块;所述智能控制器通过语音控制选择显示前方视频图像、AR端增强图像、移动端标注图像、远程端标注图像或混合标注图像;The intelligent controller further includes a voice recognition module; the intelligent controller selects and displays the front video image, the AR-side enhanced image, the mobile-side marked image, the remote-side marked image or the mixed marked image through voice control; 所述AR增强现实眼镜的镜架上还安装有与智能控制器电连接的局放感应器;所述局放感应器监测电气设备中带电线路信息并传递至智能控制器;所述控制芯片通过动态叠加处理将带电线路信息标注到前方视频图像以生成AR端增强图像。The frame of the AR augmented reality glasses is also provided with a partial discharge sensor electrically connected with the intelligent controller; the partial discharge sensor monitors the live line information in the electrical equipment and transmits it to the intelligent controller; the control chip passes the The dynamic superposition process annotates the live line information to the front video image to generate an AR-side enhanced image. 2.根据权利要求1所述的基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是所述控制芯片通过与语音识别模块电连接,用于将语音指令识别成控制指令;所述控制芯片根据控制指令控制图像集的图像翻页、放大、缩小以及视角移动;所述图像集包括前方视频图像、AR端增强图像、移动端标注图像、远程端标注图像或混合标注图像。2. The remote monitoring system for space operations with limited human eye perspective based on AR smart glasses according to claim 1, is characterized in that the control chip is electrically connected with the voice recognition module for recognizing the voice command as a control command; The control chip controls the image page-turning, zoom-in, zoom-out and viewing angle movement of the image set according to the control instruction; the image set includes the front video image, the AR-side enhanced image, the mobile-end annotation image, the remote-end annotation image or the mixed annotation image. 3.根据权利要求1所述的基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是所述语音识别模块通过蓝牙模块与蓝牙耳机通信连接,用于接收语音指令或语音���互。3. The remote monitoring system for space operations with limited human eye perspective based on AR smart glasses according to claim 1, wherein the voice recognition module is communicated with a bluetooth headset through a bluetooth module for receiving voice commands or voice interaction. 4.根据权利要求1所述的基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是所述智能控制器内还设置有存储单元;所述控制芯片控制摄像头进行视频录制并存储视频录像至存储单元;所述控制芯片还通过流媒体技术控制摄像头录制的视频录像传递至地面作业监护移动端或远程控制终端;所述地面作业监护移动端或远程控制终端对视频录像直播播放。4. The remote monitoring system for space operations with limited human eye perspective based on AR smart glasses according to claim 1, is characterized in that a storage unit is also provided in the intelligent controller; the control chip controls the camera to record video and store The video recording is recorded to the storage unit; the control chip also controls the video recording recorded by the camera to be transmitted to the ground operation monitoring mobile terminal or remote control terminal through the streaming media technology; the ground operation monitoring mobile terminal or remote control terminal broadcasts the video recording live. 5.根据权利要求4所述的基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是所述AR增强现实眼镜、地面作业监护移动端、远程控制终端均与服务器通信连接;所述服务器中设有数字图书库,用于存储试验标准化流程以及被试验设备详细资料;所述地面作业监护移动端以及远程控制终端通过查询数字图书库,获取设备信息用于标注到移动端标注图像上。5. The AR smart glasses-based remote monitoring system for limited space operations of human eyes according to claim 4, wherein the AR augmented reality glasses, the ground operation monitoring mobile terminal, and the remote control terminal are all connected to the server in communication; There is a digital book library in the server for storing the test standardization process and the detailed information of the tested equipment; the ground operation monitoring mobile terminal and the remote control terminal obtain the equipment information by querying the digital book library and use it to mark images on the mobile terminal. superior. 6.根据权利要求5所述的基于AR智能眼镜的人眼视角局限空间作业远程监护系统,其特征是所述控制芯片从存储单元的视频录像中截取操作视频录像并传递至服务器的数字图书库中,用于VR培训。6. The human eye view-limited space operation remote monitoring system based on AR smart glasses according to claim 5 is characterized in that the control chip intercepts the operation video recording from the video recording of the storage unit and transmits it to the digital library of the server , for VR training.
CN202011138066.9A 2020-10-22 2020-10-22 Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses Active CN112153267B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011138066.9A CN112153267B (en) 2020-10-22 2020-10-22 Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011138066.9A CN112153267B (en) 2020-10-22 2020-10-22 Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses

Publications (2)

Publication Number Publication Date
CN112153267A CN112153267A (en) 2020-12-29
CN112153267B true CN112153267B (en) 2022-02-22

Family

ID=73954568

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011138066.9A Active CN112153267B (en) 2020-10-22 2020-10-22 Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses

Country Status (1)

Country Link
CN (1) CN112153267B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113610987B (en) * 2021-07-13 2024-02-20 北京远舢智能科技有限公司 Mixed reality space labeling method and system based on three-dimensional reconstruction
CN113556530A (en) * 2021-07-22 2021-10-26 深圳市绿色智城科技有限公司 Intelligent rail transit maintenance system and method based on wearable AR equipment
CN114510171B (en) * 2022-02-14 2023-10-24 广州塔普鱼网络科技有限公司 Portable three-dimensional interaction system based on image processing technology
CN114863068A (en) * 2022-04-22 2022-08-05 镇江电力设计院有限公司 AR intelligent glasses for maintenance work and remote monitoring maintenance system
CN115278144A (en) * 2022-06-06 2022-11-01 国网浙江省电力有限公司武义县供电公司 Multi-party video call and safety management and control system

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016164342A1 (en) * 2015-04-06 2016-10-13 Scope Technologies Us Inc. Methods and apparatus for augmented reality applications
WO2017072616A1 (en) * 2015-10-29 2017-05-04 Koninklijke Philips N.V. Remote assistance workstation, method and system with a user interface for remote assistance with spatial placement tasks via augmented reality glasses
CN105573500B (en) * 2015-12-22 2018-08-10 王占奎 The intelligent AR glasses devices of eye movement control
CN107277451A (en) * 2017-07-14 2017-10-20 福建铁工机智能机器人有限公司 A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure
CN207280605U (en) * 2017-11-02 2018-04-27 国家电网公司 A kind of power equipment wireless and passive temperature monitoring system of smart grid-oriented
CN108170273A (en) * 2017-12-28 2018-06-15 南京华讯方舟通信设备有限公司 A kind of expert's remote guide system and implementation method based on hololens glasses
WO2019226688A1 (en) * 2018-05-22 2019-11-28 Agilent Technologies, Inc. Method and system for implementing augmented reality (ar)-based assistance within work environment
WO2020054905A1 (en) * 2018-09-13 2020-03-19 주식회사 이엠아이티 Augmented reality facility assembly guide system using smart glasses
CN109638959B (en) * 2018-12-06 2022-06-17 杭州意能电力技术有限公司 Power equipment remote signaling function debugging method and system based on AR and deep learning
CN110212451B (en) * 2019-05-28 2021-09-10 国网江西省电力有限公司电力科学研究院 Electric power AR intelligence inspection device

Also Published As

Publication number Publication date
CN112153267A (en) 2020-12-29

Similar Documents

Publication Publication Date Title
CN112153267B (en) Human eye visual angle limitation space operation remote monitoring system based on AR intelligent glasses
CN112018892B (en) Electric power operation and maintenance remote guidance system
CN116911823A (en) AR (augmented reality) glasses-based substation inspection interaction method and system
CN102967373B (en) Three-dimensional (3D) infrared temperature monitoring method and system for substation
CN112887651A (en) AR-based remote command method and system
CN110212451A (en) A kind of electric power AR intelligent patrol detection device
CN110490339A (en) A kind of auto repair auxiliary system and method based on augmented reality
CN113452962B (en) Data center enhanced inspection system and method with space collaborative perception
CN113115008B (en) A pipe gallery master-slave operation inspection system and method
CN110231061A (en) A kind of data center's infrastructure management system based on robot
CN108156033A (en) The wearable maintenance support of unmanned plane supports system
CN207718444U (en) The wearable equipment of inspection operation is moved for substation equipment maintenance and repair
CN214704735U (en) Comprehensive pipe rack wisdom system of patrolling and examining
CN114663996A (en) Intelligent equipment inspection system based on AR augmented reality technology
CN116384963A (en) AR equipment-based equipment inspection guiding method
CN118868399A (en) A secondary equipment inspection and remote guidance method, system and on-site AR device
CN114825616A (en) AR first visual angle remote diagnosis method and device, storage medium and electronic device
CN116912722A (en) A remote communication method and system for equipment inspection process based on AR equipment
CN117765209A (en) A method, device, equipment and medium for fusion of virtual and real objects under mixed reality
CN116824102A (en) AR equipment-based intelligent equipment inspection method
WO2019085945A1 (en) Detection device, detection system, and detection method
CN112581836A (en) AR-based power transmission field operation training system
CN112104086B (en) Transformer substation remote live-action holographic exploration system and method
CN207491089U (en) A kind of unmanned machine head device for intelligently switching
CN113907475A (en) Image acquisition device and method based on video semantics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant