KR102399673B1 - Method and apparatus for recognizing object based on vocabulary tree - Google Patents

Abstract

A method and apparatus for recognizing an object according to an embodiment obtains feature points and descriptors corresponding to each of the feature points from an input image, determines indices of the feature points using the descriptors, and calculates the density distribution of the feature points for each index. and recognizes at least one object included in the input image based on the prediction result of the density distribution.

Description

Method and device for recognizing objects based on vocabulary tree

The following embodiments relate to a method and an apparatus for recognizing an object based on a vocabulary tree.

In order to identify an object included in one input image, various methods for recognizing whether images previously registered in a database are included in the input image may be used.

These recognition methods may erroneously recognize an object when a part of an image included in the input image is obscured by occlusion or the like. Also, it is possible to recognize one object included in the input image, but it is difficult to recognize a plurality of objects. In addition, many training images are required for accurate object recognition, but many training images are not easy to apply to a mobile application because the size of the data is very large.

According to one aspect, a method for recognizing an object includes: obtaining feature points and descriptors corresponding to each of the feature points from an input image; determining indices of the feature points using the descriptors; predicting a density distribution of feature points for each of the indices; and recognizing at least one object included in the input image based on a prediction result of the density distribution.

Determining the indexes of the key points may include determining the indexes of the key points by applying the descriptors to a pre-learned vocabulary tree.

The determining of the indexes of the feature points may include: calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree; and determining indices of the feature points based on the similarity scores.

Calculating the similarity scores may include calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree using an Lp-norm.

The determining of the indexes of the feature points based on the similarity scores may include: sorting the similarity scores; and determining an index corresponding to a feature vector having a highest similarity score among the sorted similarity scores as indices of the feature points.

Recognizing the at least one object may include: determining a representative index corresponding to at least one object area (blob) included in the input image based on a prediction result of the density distribution; and recognizing at least one object included in the input image based on the representative index.

The determining of the representative index may include determining the representative index based on a weighted sum of similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree. .

Predicting the density distribution of the feature points may include predicting the density distribution of the feature points using a kernel density estimation (KDE) technique.

Recognizing the at least one object may include segmenting the at least one object area based on a prediction result of the density distribution.

The dividing of the at least one object region may include dividing the at least one object region included in the input image by using a bounding box based on a prediction result of the density distribution.

The acquiring of the feature points and the descriptors may include: limiting an area for extracting the feature points from the input image; and obtaining the feature points and the descriptors from the limited area.

According to one aspect, a method for recognizing an object includes determining coordinates of each of the feature points extracted from an input image; classifying at least one object region included in the input image based on the coordinates of each of the feature points; determining indices of the feature points using descriptors corresponding to the feature points belonging to the at least one object area; and recognizing at least one object included in the input image by using the indexes of the feature points.

The dividing of the at least one object region may include classifying the at least one object region included in the input image by clustering the feature points based on the coordinates of each of the feature points.

The determining of the indexes of the feature points may include determining the indexes of the feature points by applying descriptors corresponding to the feature points belonging to the at least one object area to a pre-learned vocabulary tree.

The determining of the indexes of the feature points may include: calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree; and determining indices of the feature points based on the similarity scores.

Recognizing the at least one object may include: determining a representative index corresponding to the at least one object area; and recognizing at least one object included in the input image based on the representative index.

The determining of the representative index may include counting the number of indices of the feature points; and determining a representative index corresponding to the at least one object area based on the number of each of the indices.

According to one side, an apparatus for recognizing an object includes: a communication interface for receiving an input image; a memory for storing the pre-learned vocabulary tree; and obtaining feature points and descriptors corresponding to each of the feature points from the input image, determining indices of the feature points by applying the descriptors to the vocabulary tree, and a density distribution of feature points belonging to each of the indices of the feature points and a processor that predicts and recognizes at least one object included in the input image.

The processor may calculate similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree, and determine indices of the feature points based on the similarity scores.

1 is a flowchart illustrating a method of recognizing an object according to an embodiment;
2 is a diagram illustrating an input image according to an exemplary embodiment;
3 to 6 are diagrams for explaining a method of recognizing an object according to an embodiment.
3 is a diagram for describing a method of determining indexes of feature points according to an embodiment;
4 to 5 are diagrams for explaining a method of determining a representative index according to embodiments;
6 is a diagram for describing a method of recognizing at least one object included in an input image, according to an exemplary embodiment;
7 is a flowchart illustrating a method for recognizing an object according to another embodiment.
8 is a flowchart illustrating a method of recognizing an object according to another embodiment.
9 to 11 are diagrams for explaining a method of recognizing an object according to another embodiment.
12 is a flowchart illustrating a method of constructing a vocabulary tree according to an embodiment.
13 is a diagram for describing descriptors corresponding to each of feature points according to an exemplary embodiment;
14 is a diagram for explaining a method of constructing a vocabulary tree according to an embodiment;
15 is a block diagram of an apparatus for recognizing an object according to an exemplary embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference numerals in each figure indicate like elements.

Various modifications may be made to the embodiments described below. It should be understood that the embodiments described below are not intended to limit the embodiments, and include all modifications, equivalents, and substitutes thereto.

Terms used in the examples are used only to describe specific examples, and are not intended to limit the examples. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the overlapping description thereof will be omitted. In describing the embodiment, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the embodiment, the detailed description thereof will be omitted.

1 is a flowchart illustrating a method of recognizing an object according to an exemplary embodiment. Referring to FIG. 1 , a process of recognizing an object through a process of segmenting an object region corresponding to the identified object after the recognition device identifies at least one object included in an input image will be described.

An apparatus for recognizing an object (hereinafter, 'recognition apparatus') according to an embodiment obtains feature points and descriptors corresponding to each of the feature points from an input image ( 110 ). The recognition apparatus may extract, for example, feature points of an object included in the input image shown in FIG. 2 . The feature points may be used to distinguish a corresponding object when tracking or recognizing an object in an image, such as a corner point.

In order for the recognition device to extract the feature points from the input image, for example, known feature point detection methods such as scale invariant feature transform (SIFT), speeded up robust features (SURF), binary robust independent elementary features (BRIEF), etc. may be used. .

The descriptors correspond to each of the feature points and may be called a 'feature vector' or a 'visual word'. The recognition apparatus according to an embodiment may obtain Oriented Fast and Rotated BRIEF (ORB) descriptors corresponding to each of the key points by the above-described key point detection method.

According to an embodiment, the recognition apparatus may limit a region from which feature points are extracted from an input image, and may acquire feature points and descriptors from the limited region.

The recognition apparatus marks, as a bounding box, a candidate group of a region in which an object to be recognized is likely to exist among the entire region of the input image by an object proposal algorithm, and the region corresponding to the bounding box, for example. It is also possible to extract feature points from Through this, the recognition device can reduce the amount of computation and computation time to facilitate real-time implementation of a method for recognizing an object.

The recognition apparatus may determine indices of the feature points using the descriptors ( 120 ). The recognition apparatus may determine indices of feature points by applying descriptors to a pre-learned vocabulary tree. The recognition apparatus may calculate similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree, and determine indices of the feature points based on the similarity scores. The recognition apparatus may calculate similarity scores between feature points and a feature vector corresponding to each node of the vocabulary tree using, for example, an Lp-norm.

According to an embodiment, the recognition apparatus may calculate an index and a similarity score for each feature point, and store information about a {index, similarity score} pair corresponding to the feature point. A method for the recognition apparatus to determine the index of the feature points will be described in detail with reference to FIGS. 3 to 4 .

The recognition apparatus predicts the density distribution of feature points for each index ( 130 ). The recognition apparatus may predict the density distribution of feature points through non-parametric density estimation. The non-parametric density estimation may be, for example, Kernel Density Estimation (KDE).

The recognition apparatus recognizes at least one object included in the input image based on the prediction result of the density distribution ( 140 ). The recognition apparatus may determine a representative index corresponding to at least one object region included in the input image based on the prediction result of the density distribution.

The recognition apparatus may determine the representative index by predicting the density distribution of feature points belonging to each of the indices. Alternatively, the recognition apparatus may determine the representative index based on a weighted sum of similarity scores between the previously calculated feature points and a feature vector corresponding to each node of the vocabulary tree.

Alternatively, the recognition apparatus may cluster the feature points and determine a representative index of at least one object area corresponding to the clustered feature points by using the indexes of the clustered feature points. In this case, the at least one object region is a region including clustered feature points and may correspond to a partial region of the input image.

The recognition apparatus may count the number of clustered feature points for each index, and determine the representative index based on the number of each index. The recognition apparatus may determine, for example, an index having the greatest number of counted indexes as the representative index. Specific methods for the recognition device to determine the representative index will be described with reference to FIG. 5 .

The recognition apparatus may recognize at least one object by dividing the at least one object area based on the prediction result of the density distribution. The recognition apparatus may segment at least one object region included in the input image by using, for example, a bounding box based on a prediction result of the density distribution.

The recognition apparatus may recognize at least one object included in the input image based on the representative index. A method for the recognition device to recognize at least one object will be described in detail with reference to FIG. 6 .

In general, object recognition refers to a process of determining whether a given object (object) has been touched in advance and distinguishing what the object is (eg, whether it is a shoe or a doll). In addition, object retrieval refers to a process of finding information on a related object in a pre-prepared database based on information on a recognized object. As used herein, the term 'object recognition' may be understood to include both object recognition and object search.

2 is a diagram illustrating an input image according to an exemplary embodiment. Referring to FIG. 2 , an input image 200 including two objects 210 and 230 is shown.

The input image 200 may be, for example, a grayscale 2D image.

The input image 200 may include a plurality of objects, such as the toy car 210 and the Minnie mouse doll 230 shown in FIG. 2 , or may include a single object.

3 to 4 are diagrams for explaining a method of determining indexes of feature points according to an embodiment. Referring to FIG. 3 , a toy car 310 and a Minnie Mouse doll 330 with index values labeled are shown.

Descriptors corresponding to the feature points obtained in step 110 may be transferred to the last node based on a dot-product between feature vectors corresponding to each node of the pre-learned vocabulary tree. The recognition apparatus may determine the index of each feature point based on the similarity between the pre-stored feature points of the target objects (or objects) and the movement paths of the feature points acquired from the image.

For example, the recognition apparatus may calculate similarity scores with pre-stored target objects according to a movement path of individual feature points, and sort the similarity scores. The recognition apparatus may determine an index corresponding to the object having the highest similarity score among the sorted similarity scores as indices of the feature points. Through this, each of the feature points may have an associated index value.

The recognition apparatus may calculate similarity scores using, for example, an Lp-norm as in Equation 1 below.

Here, q may correspond to a query, that is, a feature vector composed of features corresponding to a movement path (moving node i) of a feature point extracted from an input image. d is a class in the database stored in the movement path. ) may correspond to a feature vector of Also, p may represent a constant for defining a function space. Here, the 'class' may correspond to a key frame or an object to be found.

For example, when L1-norm with p=1 is used and multiple classes are stored in the database, [Equation 1] can be expressed as [Equation 2] below.

In this case, the feature value (or feature vector) q _i of the query corresponding to the node i is defined as a weight w _i of the corresponding node in the vocabulary tree, and the weight may have a scalar value. In addition, the feature value d _ji of the j-th class in the node may be defined as (ratio of j-th class feature points corresponding to the node, for example, 0 to 1) x (weight of the node, w _i ).

For example, a thresholding operation used in an embodiment may show a distribution such as “soft thresholding” when L1-norm is used. Here, when d _ji and q _i are the same, the absolute output value becomes the maximum, and when d _ji becomes smaller than q _i , the absolute output value may gradually decrease to 0. In other words, as the ratio of the j-th class feature points in node i increases (closer to 1), d _ji has a value similar to q _i , which is a feature value (or feature vector) corresponding to the corresponding node in the vocabulary tree. This means that q _i is most likely to be the jth class.

The recognition apparatus according to an embodiment may attribute the result of [Equation 1] below as a characteristic of each individual characteristic point without summation of all n characteristic points, for example. Through this, each feature point has a value corresponding to a related class, and this value may be an index of the feature points.

For example, if the value corresponding to the class of the toy car 310 is '8' and the value corresponding to the class of the Minnie Mouse doll 330 is '9', only feature points corresponding to classes 8 and 9 are selected. When illustrated, it may be displayed as shown in FIG. 4 .

5 is a diagram for explaining a method of determining a representative index according to an embodiment. Referring to FIG. 5 , a density distribution 510 of feature points having an index (Idx) #8 and a density distribution 530 of feature points having an index #9 are illustrated.

Indices of individual feature points as shown in FIGS. 3 and 4 may be determined as incorrect values due to noise generated by various factors. In this case, various factors may correspond to, for example, erroneous learning, insufficient number of learning times, and lack of robustness of technicians due to rotation and/or lighting change.

According to an embodiment, noise caused by the various factors described above may be removed by determining a representative index corresponding to at least one object region of the input image.

The recognition apparatus may determine a representative index corresponding to at least one object region of the input image based on the prediction result of the density distribution. An index having the highest value in the density distribution of feature points may be determined as a representative index corresponding to at least one object area.

For example, through the above-mentioned density estimation, the index (Idx) #8 is the most 81 points (pts) in the region where the density distribution 510 of the feature points corresponds to the 250-300 points on the X-axis and the 250-300 points on the Y-axis. Let's say we have a large estimate.

In this case, the representative index of the corresponding object may be determined as #8 through the density distribution 510 . An object whose representative index corresponds to #8 is a car toy. Accordingly, the recognition device may determine that there is a high probability that the car toy exists in the region corresponding to the points 250 to 300 of the X axis and the points 250 to 300 of the Y axis.

In the density distribution 510 , values appearing in other portions except for regions corresponding to points 250 to 300 of the X axis and points of point 250 to 300 of the Y axis may correspond to a distribution caused by misrecognition. The recognition device may remove the density distribution below a certain value through thresholding.

For example, through the above-described density estimation, the index (Idx) #9 is the most 84 points (pts) in the region where the density distribution 530 of the feature points corresponds to 450 to 600 points on the X axis and 220 to 300 points on the Y axis. Let's say we have a large estimate.

In this case, the representative index of the corresponding object may be determined as #9 through the density distribution 530 . The object whose representative index corresponds to #9 is the Minnie Mouse doll. Accordingly, the recognition device may determine that there is a high probability that the mini-mouse doll exists in a region corresponding to a point 250 to 300 of the X axis and a point 250 to 300 of the Y axis.

6 is a diagram for describing a method of recognizing at least one object included in an input image, according to an exemplary embodiment. Referring to FIG. 6 , a car toy 210 and a minnie mouse doll 230 separated by a bounding box 650 are shown.

The recognition apparatus may segment at least one object area by analyzing the prediction result of the density distribution described above with reference to FIG. 5 . The recognition device divides at least one object region included in the input image using the bounding box 650 based on the prediction result of the density distribution to recognize at least one object (the car toy 210 and the Minnie mouse doll 230). can recognize

Since the bounding box 650 can be tracked to the next frame, it is not necessary to calculate it for every frame. For tracking the bounding box 650 , for example, a tracking-learning-detection (TLD) tracking framework, a Kernelized Correlation Filters (KCF) tracker, or the like may be used.

7 is a flowchart illustrating a method of recognizing an object according to another exemplary embodiment. Referring to FIG. 7 , the recognition apparatus according to an embodiment may receive an input image ( 710 ). The input image may be, for example, a 2D image.

The recognition apparatus may detect feature points from the input image (S720). The recognition apparatus may detect, for example, 500 feature points in one frame of the input image.

The recognition apparatus may calculate (calculate) descriptors corresponding to each of the feature points ( 730 ).

The recognition apparatus may pass the feature points detected in step 720 through the pre-learned vocabulary tree 780 (according to the movement path of) to calculate a similarity score for each feature point ( 740 ). In this case, the vocabulary tree 780 may be stored in advance through an off-line process. The vocabulary tree is, for example, a database by loading a tree (data structure) (781) and applying features (features) of a key frame corresponding to an object to be searched to the tree loaded in step (781). (database) may be configured through the process of creating (783). In this case, the vocabulary tree may be constructed by hierarchically quantizing descriptors corresponding to the features (feature points) of the key frame from the root to the leaves of the tree.

The recognition apparatus may determine indices of the feature points by sequentially arranging similarity scores and removing an index having a low similarity score ( S750 ).

The recognition device may calculate a probability density for each index ( 760 ). The recognition apparatus may calculate a probability density for each index using, for example, a probability density function.

The recognition apparatus may recognize at least one object included in the input image by dividing the object based on the probability density for each index calculated in operation 760 (operation 770).

In an embodiment, elements such as occlusion and scale may be overcome by recognizing an object based on individual features (or feature points) rather than on a scene basis.

8 is a flowchart illustrating a method of recognizing an object according to another exemplary embodiment. Referring to FIG. 8 , a process of recognizing the divided object area after the recognition apparatus classifies at least one object area included in an input image will be described.

The recognition apparatus according to an embodiment determines the coordinates of each of the feature points extracted from the input image ( S810 ). In this case, the input image may be, for example, a 2D image, and the coordinates may be (x,y) coordinates. The coordinates of each of the feature points extracted from the input image may be expressed as in the graph 900 of FIG. 9 .

The recognition apparatus classifies at least one object region included in the input image based on the coordinates of each of the feature points determined in operation 810 (operation 820). The recognition apparatus may classify at least one object region included in the input image by clustering the key points based on the coordinates of each of the key points.

The recognition device may cluster the feature points through unsupervised clustering, such as K-mean clustering, for example. Autonomous clustering classifies objects without any knowledge of each class to be classified, and corresponds to a method of classifying clusters based on similarity. At this time, in order to evaluate the similarity between clusters, for example, Euclidean distance, Mahalanobis distance, Lance-Williams distance, and Hamming distance. Various distance (similarity) measurement functions may be used, such as

According to an embodiment, when the value of k is unknown in K-means clustering, the recognition apparatus may perform clustering based on the peak value of the density distribution as shown in FIG. 10 .

The recognition apparatus determines indices of the feature points by using descriptors corresponding to the feature points belonging to at least one object area ( 830 ). The recognition apparatus may determine indices of the feature points by applying descriptors corresponding to feature points belonging to at least one object area to a pre-learned vocabulary tree.

For example, the recognition apparatus may calculate similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree, and determine indices of the feature points based on the similarity scores.

For example, the recognition apparatus may sort similarity scores, and determine an index corresponding to a feature vector having a highest similarity score among the sorted similarity scores as indices of the feature points. A pair of {index, similarity score} may be obtained by using descriptors of feature points belonging to at least one recognition object region.

The recognition apparatus recognizes at least one object included in the input image using the indexes of the feature points determined in operation 830 (operation 840). The recognition apparatus may determine a representative index corresponding to at least one object region, and recognize at least one object included in the input image based on the representative index.

The recognition apparatus may, for example, count the number of indexes of the feature points, and recognize at least one object corresponding to at least one object area based on the number of indexes.

9 is a diagram illustrating feature points extracted from an input image according to an exemplary embodiment. Referring to FIG. 9 , a graph 900 showing coordinates of feature points is shown.

10 is a diagram for describing a method of classifying at least one object region included in an input image, according to an exemplary embodiment. Referring to FIG. 10 , a graph 1000 indicated by '+' at the peak of the density distribution of feature points is shown.

When the k value is unknown in the aforementioned K-means clustering, the recognition apparatus may classify at least one object region using a peak value of a density distribution of feature points.

As shown in FIG. 10 , the recognition apparatus may classify an object region by dividing a predetermined distance from a point representing the peak value of the density distribution as the center of the cluster.

11 is a diagram for describing a method of recognizing at least one object included in an input image, according to an exemplary embodiment. Referring to FIG. 11 , a toy car recognized as index #8 and a Minnie mouse doll recognized as index #9 are illustrated.

When at least one object area is identified through the aforementioned clustering, the recognition apparatus may determine a representative index corresponding to the corresponding object area. The recognition device may determine the representative index by, for example, counting the number of indices included in the corresponding object region or accumulating similarity scores of feature points included in the corresponding object region by weighted sum or the like.

12 is a flowchart illustrating a method of constructing a vocabulary tree according to an embodiment. Referring to FIG. 12 , the recognition apparatus according to an embodiment extracts feature points from patch images of at least one key frame to be recognized ( S1210 ).

The recognition apparatus determines descriptors corresponding to the feature points ( 1220 ), and constructs a vocabulary tree by hierarchically propagating the descriptors to the tree structure ( 1230 ).

A method for the recognition device to construct a vocabulary tree will be described in detail with reference to FIG. 14 .

13 is a diagram for describing descriptors corresponding to each of feature points according to an exemplary embodiment. Referring to FIG. 13 , at least one object 1310 included in an input image, feature points 1315 extracted from the at least one object, and descriptors corresponding to the feature points 1315 are illustrated.

The descriptors may correspond to a feature vector that describes the feature of the feature points 1315 . Descriptors may have, for example, a 256-bit binary form such as "001001000...01011" or "010111110...01010", but is not limited thereto.

14 is a diagram for explaining a method of constructing a vocabulary tree according to an embodiment. Referring to FIG. 14 , a process of adding an image to a vocabulary tree by applying a descriptor to the tree database structure is illustrated.

For example, an object 1420 included in a key frame 1410 corresponding to an image to be searched includes a feature point 1430 , and a descriptor corresponding to the feature point 1430 is “001001000...01011”. lets do it.

The recognition device may hierarchically propagate the descriptor (“001001000...01011”) from a root node to a leaf node of the tree (database structure) 1450 prepared in advance.

The recognition apparatus may calculate a similarity between each node of the vocabulary tree and a descriptor through a Hamming distance, and map a corresponding descriptor to a node having a high similarity.

In this case, the vocabulary tree is constructed by storing at least one key frame in a leaf node of the structure ( 1240 ).

15 is a block diagram of an apparatus for recognizing an object according to an exemplary embodiment. Referring to FIG. 15 , the recognition apparatus 1500 according to an embodiment includes a memory 1510 , a processor 1520 , and a communication interface 1530 . The memory 1510 , the processor 1520 , and the communication interface 1530 may communicate with each other through the communication bus 1540 .

The memory 1510 stores a pre-learned vocabulary tree.

The processor 1520 obtains feature points and descriptors corresponding to each of the feature points from the input image, and applies the descriptors to the vocabulary tree to determine indices of the feature points. The processor 1520 recognizes at least one object included in the input image by predicting the density distribution of key points belonging to each index of the key points.

The processor 1520 may calculate similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree. The processor 1520 may determine indices of the feature points based on the similarity scores.

The communication interface 1530 receives an input image.

In addition, the processor 1520 may perform at least one method described above with reference to FIGS. 1 to 14 . The processor 1520 may execute a program and control the recognition device 1500 . The program code executed by the processor 1520 may be stored in the memory 1510 .

Also, the memory 1510 may store information received through the communication interface 1530 . The memory 1510 may be a volatile memory or a non-volatile memory.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

obtaining feature points and descriptors corresponding to each of the feature points from the input image;
determining indices of the feature points using the descriptors, each of the indices indicating a respective object class among a plurality of object classes;
predicting a density distribution of feature points for each of the indices; and
Recognizing at least one object included in the input image as one object class among the plurality of object classes based on a prediction result of the density distribution
A method for recognizing an object, comprising: The method of claim 1,
The step of determining the indexes of the feature points
determining indices of the feature points by applying the descriptors to a pre-learned vocabulary tree;
A method for recognizing an object, comprising: 3. The method of claim 2,
The step of determining the indexes of the feature points
calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree; and
determining indices of the feature points based on the similarity scores;
A method for recognizing an object, comprising: 4. The method of claim 3,
Calculating the similarity scores includes:
calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree using an Lp-norm
A method for recognizing an object, comprising: 4. The method of claim 3,
The step of determining the indexes of the feature points based on the similarity scores includes:
sorting the similarity scores; and
determining an index corresponding to a feature vector having the highest similarity score among the sorted similarity scores as indices of the feature points
A method for recognizing an object, comprising: According to claim 1,
The recognizing step
determining a representative index corresponding to at least one object region (blob) included in the input image based on a prediction result of the density distribution; and
Recognizing at least one object included in the input image as one object class among the plurality of object classes based on the representative index
A method for recognizing an object, comprising: 7. The method of claim 6,
The step of determining the representative index is
determining the representative index based on a weighted sum of similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree;
A method for recognizing an object, comprising: According to claim 1,
Predicting the density distribution of the feature points includes:
Predicting the density distribution of the feature points using a Kernel Density Estimation (KDE) technique
A method for recognizing an object, comprising: 9. The method of claim 8,
The recognizing step
segmenting the at least one object region based on a prediction result of the density distribution;
A method for recognizing an object, comprising: 9. The method of claim 8,
The step of dividing the at least one object area includes:
segmenting at least one object region included in the input image using a bounding box based on a prediction result of the density distribution;
A method for recognizing an object, comprising: According to claim 1,
The step of obtaining the feature points and the descriptors is
limiting an area for extracting the feature points from the input image; and
obtaining the feature points and the descriptors from the limited area;
A method for recognizing an object, comprising: determining the coordinates of each of the feature points extracted from the input image;
classifying at least one object region included in the input image based on the coordinates of each of the feature points;
determining indices of the feature points using descriptors corresponding to the feature points belonging to the at least one object area, each of the indices indicating a respective object class among a plurality of object classes; and
Recognizing at least one object included in the input image as one object class among the plurality of object classes using a representative index selected from among the indexes of the feature points
A method for recognizing an object, comprising: 13. The method of claim 12,
The step of classifying the at least one object area includes:
classifying at least one object region included in the input image by clustering the feature points based on the coordinates of each of the feature points
A method for recognizing an object, comprising: 13. The method of claim 12,
The step of determining the indexes of the feature points
determining indices of the feature points by applying descriptors corresponding to feature points belonging to the at least one object area to a pre-learned vocabulary tree;
A method for recognizing an object, comprising: 15. The method of claim 14,
The step of determining the indexes of the feature points
calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree; and
determining indices of the feature points based on the similarity scores;
A method for recognizing an object, comprising: 13. The method of claim 12,
The recognizing step
determining a representative index corresponding to the at least one object area; and
Recognizing at least one object included in the input image as one object class among the plurality of object classes based on the representative index
A method for recognizing an object, comprising: 17. The method of claim 16,
The step of determining the representative index is
counting the number of each indices of the feature points; and
determining a representative index corresponding to the at least one object area based on the number of each of the indexes;
A method for recognizing an object, comprising: A computer program stored in a medium for executing the method of any one of claims 1 to 17 in combination with hardware. a communication interface for receiving an input image;
a memory for storing the pre-learned vocabulary tree; and
Obtaining feature points and descriptors corresponding to each of the feature points from the input image, and applying the descriptors to the vocabulary tree, indexes of the feature points - Each of the indexes represents a respective object class among a plurality of object classes A processor that determines - and recognizes at least one object included in the input image as one object class among the plurality of object classes by predicting the density distribution of feature points belonging to each of the indexes of the feature points
A device for recognizing an object, including. 20. The method of claim 19,
the processor is
An apparatus for recognizing an object, calculating similarity scores between the feature points and a feature vector corresponding to each node of the vocabulary tree, and determining indices of the feature points based on the similarity scores.

Images