KR101556158B1 - The social service system based on real image using smart fitting apparatus - Google Patents

Abstract

The present invention relates to a social service system using a real-time image-based smart fitting device that provides a three-dimensional real-time image through a display device and can share an image of a wearer with surrounding persons, A background modeling unit for photographing a background space without a shape of a buyer through each camera and converting the photographed background image into a feature image to set a background image; A background removal unit which removes backgrounds excluding a purchaser's body from images input from a plurality of depth cameras; A three-dimensional image modeling unit for generating a three-dimensional image by aligning three-dimensional spatial coordinates using calibration information obtained from a camera adjusting unit; A central control unit for controlling display and storage of the three-dimensional image and controlling transmission of the stored image; And a display unit for displaying a three-dimensional image on the display device under the control of the central control unit. Thus, it is possible to provide a three-dimensional real image through a display device, I will help you.

Description

[0001] The present invention relates to a social service system using a real-image-based smart fitting device,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a social service system using a smart image-based smart fitting device. More particularly, the present invention relates to a social service system using a display device (for example, digital The present invention relates to a social service system using a real-time image-based smart fitting device that provides a three-dimensional real-time image through a sign-up image (signage)

In general, a buyer who purchases clothes at a clothing store observes his / her appearance wearing clothes in various directions in front of a mirror after wearing clothes, and listens to the opinions of surrounding people about his / her appearance. In addition, they are compared with each other by wearing different clothes and comparing them to each other.

However, this method has a restriction that it is possible to exchange opinions only when the surrounding people are present together in the store.

Meanwhile, in recent years, in the course of purchasing clothes, a technique has been developed in which buyers can wear a variety of clothes suitable for their own, and compare and select them.

The existing technology related to this is virtual clothing (costume) wearing technology, and a 3D fitting system is a typical example. This technology is a technique of modeling 3D human body from a video shot by a human body camera, creating clothes virtually using graphic technology, and synthesizing and displaying virtual clothing on live-action video.

Conventional techniques for the digital virtual fitting system as described above are disclosed in the following Patent Document 1: Korean Patent Laid-Open Publication No. 10-2012-0087265 (2012.08.07.) And Patent Document 2: Patent Publication No. 10-2012-0040565 (published on Apr. 27, 2012) and Patent Document 3 (Korean Patent Laid-Open Publication No. 10-2013-0007011 (published on January 18, 2013)).

The disclosed patent document 1 searches for a fitting content matched with a user's body form in a user image and copies the fitted content unit area divided into a plurality of areas in advance to a content area to be modified according to a user's request for modification, . Then, the modified fitting contents are subjected to virtual fitting according to the user's body shape. As a result, it is possible to prevent an unnatural combination due to the resizing or spreading of the image, and the fitting contents can be naturally transformed into the user's image without loss of image quality.

In addition, Patent Document 2 discloses a user terminal equipped with an application capable of creating and modifying a body desired by the user and capable of three-dimensional virtual dressing of clothes provided on the web through the Internet, A web server for providing a three-dimensional attachment service of the generated apparel file, an integrated clipping program for creating an apparel file to be appended to a virtual body implemented in the application, and the apparel file is uploaded to the web server And a merchant terminal for selling merchandise. By using the three-dimensional virtual fitting system, the clothes can be freely three-dimensionally fitted to the body set in the application of the user terminal.

In addition, Patent Document 3 stores image information of an apparel product matched with an identification code assigned for each apparel, and can fetch and coordinate a plurality of image information according to a user's selection. In addition, the whole body of the user is photographed and overlapped with the whole body of the user directly or is displayed as an augmented reality, so that the effect of wearing the clothes is facilitated, so that the user can easily select clothes suitable for him.

Korean Patent Publication No. 10-2012-0087265 (2012.08.07. Disclosed) Korean Patent Publication No. 10-2012-0040565 (published on April 27, 2012) Korean Patent Publication No. 10-2013-0007011 (published on January 18, 2013)

However, since the above-mentioned prior arts need to modify the size of the virtual clothing according to the human body, the data on the size of the human body and the size of the virtual clothing must be managed and the size variation of the virtual clothing must be calculated. Therefore, There is a disadvantage that real time processing is difficult.

In addition, since the conventional technology is a method in which a camera image outputted to a display device is outputted as a two-dimensional image and a real image is not worn, a sense of reality is not much known because the costume does not fit itself, There are disadvantages that it is difficult to utilize.

The object of the present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide a three-dimensional real image through a digital signage, To provide a social service system using a real-image-based smart fitting device.

It is another object of the present invention to provide a social service system using a real-time image-based smart fitting device capable of simultaneously comparing images of wearing different clothes through the function of recording a three-dimensional image.

It is still another object of the present invention to provide a real-image-based smart phone capable of transmitting and sharing actual three-dimensional images of clothes with clothing to exchange purchasing decisions between buyers and surrounding people in real time, And to provide a social service system using a fitting device.

delete

delete

delete

In order to achieve the above object, a social service system using a real-time image-based smart fitting device according to the present invention is a smart service system that acquires a real image and acquires a three-dimensional image by modeling the obtained real image with a three- A fitting device; A social service server connected to the smart fitting device through a network for receiving a three-dimensional image based on a real image, registering the three-dimensional image in a database, and providing a three-dimensional image registered in the database through the network; A user terminal for downloading and displaying a three-dimensional image provided by the social service server or the smart fitting device through the network; And an acquaintance terminal for downloading and displaying a three-dimensional image of a user provided by the social service server through the network on a screen.
The smart fitting device includes a camera interface unit for connecting a plurality of depth cameras for photographing a wear state of a user (purchaser); A camera calibration unit for adjusting the positions of the plurality of depth cameras to set a reference space coordinate system; An image acquisition unit for photographing a wearer's clothing wearing state from the plurality of depth cameras; A background modeling unit for photographing a background space without a shape of a user (buyer) through each camera and converting the photographed background image; A background removal unit configured to remove backgrounds excluding the purchaser's body from images input from the plurality of depth cameras; A three-dimensional image modeling unit for generating a three-dimensional image by aligning three-dimensional space coordinates using calibration information obtained by the camera adjusting unit; A central controller for controlling display and storage of the generated three-dimensional image and controlling transmission of the stored image; An image display unit for displaying a three-dimensional image on a display device under the control of the central control unit; A three-dimensional image control unit for controlling a three-dimensional image screen by gesture recognition of a user or interaction with a user through a menu method; A three-dimensional image recording unit for recording (recording) a clothing wearing state of the purchaser displayed on the display device under the control of the central control unit; A three-dimensional image storage unit for registering a video image of a wearer's clothing wearing state in a database; And a three-dimensional image transmission unit for transmitting the recorded image to the outside.

The user terminal accesses the called terminal through the network, converts the user's opinion into data, and transmits the data to the called terminal through the network.

The terminal is connected to the user terminal through the network, and transmits the opinion input by the acquaintance to the user terminal through the network.

delete

delete

According to the present invention, by allowing a wearer to observe an actual appearance of a wearer wearing a garment as a three-dimensional image, it is possible to provide a mirror having a mirror in front, rear, left, Therefore, it is possible to increase the satisfaction of shopping for clothes.

In addition, according to the present invention, a real three-dimensional image of a wearer wearing a garment can be transmitted to nearby people to be shared, thereby enabling a buyer to exchange opinions with nearby people in real time, It has the effect of promoting real-time social communication service.

1 is a block diagram of a smart-image-based smart fitting device applied to the present invention;
FIG. 2 is a block diagram of an embodiment of the background modeling unit of FIG. 1;
FIGS. 3A to 3D illustrate a mirror function example when a real-world three-dimensional garment is worn,
FIGS. 4A to 4D are views showing a comparative example of a clothing wearing state of the present invention,
FIG. 5 is a diagram illustrating an example of image acquisition by a plurality of depth cameras in the present invention,
6 is a block diagram of a social service system using a real-time image-based smart fitting device according to the present invention.

Hereinafter, a social service system using a real-image-based smart fitting device according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a smart-image-based smart fitting device applied to the present invention.

The smart-image-based smart fitting apparatus 100 according to the present invention includes a camera interface unit 110, a camera adjustment unit 120, an image acquisition unit 125, a background modeling unit 130, a background removal unit 140, The three-dimensional image modeling unit 150, the central control unit 160, the user interface unit 165, the three-dimensional image control unit 170, the image display unit 175, the three-dimensional image recording unit 180, A three-dimensional image transmission unit 190, and a database 195. The three-

The camera interface unit 110 serves to connect a plurality of depth cameras 101 to 101 + N for photographing the clothes of the user (buyer), and the camera adjusting unit 120 is connected to the reference space coordinate system And adjusts the positions of the plurality of depth cameras for setting.

The image acquiring unit 125 acquires an image of an image of a wearer wearing a clothes from the plurality of depth cameras.

In addition, the background modeling unit 130 photographs a background space without a user (buyer) through each camera, and converts the background image into a feature image.

As shown in FIG. 2, the background modeling unit 130 includes a background space acquiring unit 131 for acquiring a background space image without a user (buyer) image. And a feature modeling unit 132 for modeling a feature of the background space image acquired by the background space acquisition unit 131 and setting a background image.

In addition, the background removing unit 140 removes the background excluding the purchaser's body from the images input from the plurality of depth cameras, and the 3D image modeling unit 150 removes backgrounds from the images obtained from the camera adjusting unit 120 Dimensional space coordinates by using the calibration information to generate a three-dimensional image.

The central control unit 160 controls display and storage of the generated three-dimensional image, and controls transmission of the stored image. The central control unit 160 is preferably implemented as a control device such as a microprocessor, a microcomputer, a controller, and a central processing unit.

The image display unit 175 displays a three-dimensional image on the display device under the control of the central control unit 160. The three-dimensional image control unit 170 controls the three-dimensional image control unit 170 based on the user's gesture recognition, And controls the three-dimensional image screen.

In addition, the three-dimensional image recording unit 180 records (records) a clothing wearing state of the buyer displayed on the display device under the control of the central control unit 160. [

In addition, the three-dimensional image storage unit 185 registers a video image of a wearer's wear-wearing state in the database 195, and the three-dimensional image transmission unit 190 transmits the three- To the outside.

In addition, the user interface unit 165 serves as an interface with the user.

The operation of the real-image-based smart fitting apparatus according to the present invention will now be described.

First, a plurality of depth cameras (101 to 101 + N) are installed in a specific space in order to photograph a wearer wearing clothes by a three-dimensional image. FIG. 5 is a view showing an installation position of a plurality of depth cameras and image acquisition. The specific space may be a fitting room.

When a plurality of depth cameras and a smart fitting device 100 are connected by using a connector or the like, the camera interface unit 110 interfaces data between the central control unit 160 and a plurality of depth cameras.

A kind of camera sensor, such as a Kinect sensor from Microsoft, may be used as a kind of depth camera. The Kinect sensor basically consists of three lenses, one for RGB recognition and the other for projecting infrared rays in pixel units. A lens that projects infrared rays is also called an infrared project. And the other recognizes that a myriad of dots sent to the infrared project are reflected from the subject being shot. For example, an infrared ray is transmitted on a pixel basis, and an infrared ray camera collects the pixel as distance information to produce an image and recognize the object.

In addition, when a plurality of depth cameras are installed in a specific space as described above, camera adjustment is required for obtaining an optimum spatial coordinate. That is, an image acquired through a plurality of depth cameras is displayed on a display device (for example, a digital signage) through the image display unit 175. The user inputs the adjustment information through an input device (not shown in the drawing) while viewing the image displayed on the display device, or changes the position of the camera. According to this operation, the camera adjusting unit 120 adjusts the three-dimensional space coordinate (x, y, z) inputted from each camera coordinate system in order to construct a three-dimensional space of the photographing point when photographing the same space (buyer) simultaneously through a plurality of depth cameras Values are converted into a reference spatial coordinate system to obtain all coincident spatial coordinates in the three-dimensional space. The camera adjustment is performed only when the system is installed for the first time or when the installation space is changed.

In the state where the adjustment of the camera is completed, the manager photographs the background space without the shape of the user (buyer) through each depth camera in advance. The photographed image is transmitted to the background modeling unit 130 under the control of the central control unit 160. The background space acquisition unit 131 of the background modeling unit 130 acquires the input image as a background space image without a user (buyer) and transmits the background space image to the feature modeling unit 132. The feature modeling unit 132 models a feature of the background space image acquired by the background space obtaining unit 131 to set a background image. The background image thus set is stored in the database 195. The background can be stored through the background modeling unit even if the shooting space is changed in the store or the location of the store is changed. Such a background image is used to remove a background image from a subsequent image. By eliminating unnecessary backgrounds, the processing speed of the three-dimensional image modeling can be improved and the user's appearance can be emphasized.

Then, when the buyer wears the clothes to be purchased and then inputs a shooting start command at a predetermined position, the purchaser is photographed in front, rear, left, and right directions from a plurality of depth cameras. The image acquisition unit 125 acquires images photographed by a plurality of depth cameras under the control of the central control unit 160. [

The image of the user acquired by the image acquisition unit 125 is transmitted to the background removal unit 140. The background removal unit 140 receives the user's clothes photographed by the respective depth cameras through the image acquisition unit 125 The wear image is compared with the background image registered in the database 195 to remove the same feature and extract the remaining image of the user (buyer).

The thus-extracted clothes-wearing image of the user is transmitted to the three-dimensional image modeling unit 150. The three-dimensional image modeling unit 150 arranges the user images photographed by the cameras removed to the background into three-dimensional space coordinates using the calibration information of each camera acquired by the camera adjustment unit 120 Thereby generating a three-dimensional image.

The thus generated three-dimensional image is displayed on the display device through the image display unit 175 under the control of the central control unit 160. In this case, the display device may be a display device installed in a store such as a digital signage, a telescreen, or a smart TV.

3A to 3D are examples of a three-dimensional image displayed on the display device. The image is displayed as if it were viewed through a mirror. FIG. 3A shows an actual front image, FIG. 3B shows an actual rear image, FIG. 3C shows an actual side image, and FIG. 3D shows the front, rear, and side images simultaneously on one screen .

The user can manipulate the three-dimensional image in a state in which the three-dimensional image is displayed on the display device, and the operation signal of the user is transmitted to the central control unit 160 through the user interface unit 165. [ For example, the user interface unit 165 is for interacting with a display device and provides a function for a user to control the display device. Here, the function control can be performed by a remote control or a menu-based function control by a touch method or by a function control method by a user's gesture recognition.

When a user operation command is received through the user interface unit 165, the central control unit 160 transmits an operation command to the three-dimensional image control unit 170 to control the three-dimensional image screen. For example, the three-dimensional image control unit 170 controls a three-dimensional image screen displayed on the display device. The three-dimensional image control unit 170 includes a function of observing a clothes line while rotating a three- , The front view, the back view, and the side view on one screen simultaneously to observe the entire clothes at a glance, and the ability to simultaneously observe and compare the appearance of wearing different kinds of clothing on one screen. In other words, the display of the three-dimensional image is controlled in response to the user inputting the operation command.

FIGS. 4A to 4D show the state of actual 3-dimensional clothing wearing on the display device according to the user interface. FIG. 4A is a view showing a state of wearing of the clothing A, FIG. 4B is a display of a state of wearing the clothes B, FIG. 4C is a view of wearing the clothes C, So that three clothing-wearing images are simultaneously displayed on the screen.

On the other hand, as a feature of the present invention, it is possible to record an image output on the display device. For example, the three-dimensional image recording unit 180 records (records, records) a user's (wearer's) clothes-wearing image output to the display device. Recording is progressed during a user's desired time in response to the user selecting the <Start recording> and the <End recording> buttons through the user interface unit 165.

The image recorded by the three-dimensional image recording unit 180 is temporarily stored in the internal memory of the central control unit 160 under the control of the three-dimensional image storage unit 185, and then stored in the other storage device. Here, the storage device may include an external storage device such as a hard disk drive (HDD), a USB, a CD, a DVD, a cloud, and the like.

According to another aspect of the present invention, a recorded image stored in the internal memory can be stored in a server or the like via another external device or the Internet. For example, the three-dimensional image transmission unit 190 transmits a recording image temporarily stored in the memory of the central control unit 160 or an image stored in the storage device by the three-dimensional image recording unit 180 to a central server Or downloaded directly to the buyer's smart device via wireless communication.

FIG. 6 is a block diagram of a social service system using a real-image-based smart fitting device according to the present invention.

As shown in FIG. 6, the social service system using the real-image-based smart fitting device according to the present invention includes a smart fitting that acquires a real image and acquires a three-dimensional image by modeling the obtained real image as a three- Apparatus (100); Dimensional image registered in the database through the network 300. The smart-fitting device 100 is connected to the smart-fitting device 100 through the network 300, receives the three-dimensional image based on the real- (200); A user terminal 400 for downloading the three-dimensional image provided from the social service server 200 or the smart fitting device 100 through the network 300 and displaying the downloaded three-dimensional image on the screen; And an acquaintance terminal 500 for downloading a 3D image of a user provided by the social service server 200 through the network 300 and displaying the downloaded 3D image on a screen.

Here, the smart fitting device 100 has the same configuration as that shown in FIG. 1, and functions in the same way.

The operation of the social service system using the real-image-based smart fitting apparatus according to the present invention will now be described.

First, the smart fitting device 100 transmits a three-dimensional image stored in a memory or a storage device to the social service server 200 through the network 300 via the three-dimensional image transmission unit 190. Here, the network 300 is collectively referred to as a network such as the Internet and a mobile communication network.

The social service server 200 registers the three-dimensional image transmitted from the smart fitting device 100 in an internal database. Here, since the 3D image data includes identification information for identifying the user (buyer), the 3D image data can be distinguished through the identification information. The identification information may be a user's cell phone number or a name. The social service server 200 establishes a site for a smart fitting through a network such as the Internet, and connects with each smart fitting device and the smart device through a network. The three-dimensional image transmitted from the smart fitting device is registered in the database and is downloaded to the user terminal 400 or the local terminal 500. The three-dimensional image uploaded from the user terminal 400 is registered in the database. To the smart fitting device 100 or to the smartphone device 500. The social service server 200 is preferably implemented as a server-class computer.

In addition, the social service system of the present invention can download a three-dimensional image directly from the smart fitting device 100 to a user terminal 400 such as a smart phone. For example, the 3D image transmitter 190 and the user terminal 400 of the smart fitting device 100 transmit 3D image data through wireless communication such as Bluetooth, WiFi, WCDMA, Long Term Evolution (LTE), or the like. For this purpose, it is also necessary to implement an access point (AP) as a relay apparatus.

Also, the acquaintances carrying the acquaintance terminal 500 can view a three-dimensional image of a user (purchaser) downloaded to the acquaintance terminal 500, and can transmit the opinion on the user in the form of a text message. The opinions of the acquaintances transmitted in this way are transmitted to the user terminal 400 through the social service server 200. Therefore, the user can collect the opinions of the acquaintances about the clothes wearing at present by simply wearing the clothes to be purchased, shooting, and transmitting them at the store where the clothes are to be purchased. In addition, the user's own opinion is formed in the form of a text message, and transmitted to the acquaintance terminal 500 via the social service server 200 as described above, so that the opinion of the purchaser can be transmitted to the acquaintances. Through this process, the purchaser can purchase clothes while sharing opinions with the remote acquaintances about the clothing to be purchased.

delete

delete

delete

delete

delete

delete

delete

delete

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

The present invention is applied to a real image based clothing fitting technique in a clothing store. In particular, it is possible to transmit the image wearing the clothing to be purchased to a remote location, effectively communicate the purchase with the wearer who can remotely exchange opinions with the wearer,

100 ... Smart Fitting Device
130 ... Background modeling unit
140 ... Background removal
150 ... The 3D image modeling unit
160 ... The central control unit
170 ... The three-
180 ... The three-
200 ... Social service server
400 ... User terminal
500 ... Acquaintance terminal

Claims (14)

delete delete delete delete delete delete delete A smart fitting device for acquiring a real image and modeling the obtained real image as a three-dimensional image to acquire a three-dimensional image;
A social service server connected to the smart fitting device through a network for receiving a three-dimensional image based on a real image, registering the three-dimensional image in a database, and providing a three-dimensional image registered in the database through the network;
A user terminal for downloading and displaying a three-dimensional image provided by the social service server or the smart fitting device through the network;
And an acquaintance terminal for downloading and displaying a three-dimensional image of a user provided by the social service server through the network on a screen;
The smart fitting device includes a camera interface unit for connecting a plurality of depth cameras for photographing a wear state of a user (purchaser); A camera calibration unit for adjusting the positions of the plurality of depth cameras to set a reference space coordinate system; A background modeling unit for photographing a background space without a shape of a user (buyer) through each camera and converting a photographed background image to set a background image; A background removal unit configured to remove backgrounds excluding the purchaser's body from images input from the plurality of depth cameras; A three-dimensional image modeling unit for generating a three-dimensional image by aligning three-dimensional space coordinates using calibration information obtained by the camera adjusting unit; A central controller for controlling display and storage of the generated three-dimensional image and controlling transmission of the stored image; An image display unit for displaying a three-dimensional image on a display device under the control of the central control unit; And a three-dimensional image control unit for controlling a three-dimensional image screen by a user's gesture recognition or a menu method by interaction with a user.
delete The smart fitting apparatus according to claim 8, wherein the user terminal accesses the home terminal through the network, converts the user's opinion into character data, and transmits the character data to the home terminal through the network. Used social service system.
9. The smart-phone-based smart fitting according to claim 8, wherein the acquaintance terminal is connected to the user terminal through the network, converts the opinion inputted by the acquaintance into character data, and transmits the character data to the user terminal through the network. Social service system using device.


delete delete delete

Images