KR20260059626A - Tobacco classification method using hyperspectral imaging - Google Patents

Abstract

A method for classifying cigarettes includes the step of visualizing cigarettes using an imaging system to acquire images, and classifying cigarettes as very low nicotine (VLN) or regular cigarettes based on the acquired images. The imaging system may include a hyperspectral camera and an image processing system.

Description

Tobacco classification method using hyperspectral imaging

The present disclosure relates to the classification of tobacco using hyperspectral imaging and analysis.

Cross-reference regarding related applications

This application claims priority to U.S. Patent Application No. 18/462,793 filed on September 7, 2023, the entire contents of which are incorporated herein by reference.

A novel and useful system, apparatus, and method for classifying tobacco is described in this specification.

In at least one embodiment, a method for classifying cigarettes is described. The method may include the step of imaging cigarettes using an imaging system to acquire images, and classifying cigarettes as very low nicotine (VLN) cigarettes or regular cigarettes based on the acquired images. The imaging system may include a hyperspectral camera and an image processing system.

In at least one embodiment, the method may further include the step of placing a cigarette on a conveyor belt configured to pass under a hyperspectral camera. In at least one embodiment, imaging of the cigarette may be performed while the cigarette moves in a straight line under the hyperspectral camera along the conveyor belt. The movement of the cigarette may be tracked by the image processing system to generate a consistent image of the cigarette. In at least one embodiment, classification of the cigarette may include analyzing the image in real time while the cigarette moves in a straight line under the hyperspectral camera.

In at least one embodiment, the hyperspectral camera may be configured to image a cigarette using short-wave infrared (SWIR) imaging. In at least one embodiment, SWIR imaging may operate in the range of about 900 nanometers (nm) to about 2500 nanometers (nm).

In at least one embodiment, a relevant feature of the tobacco can be extracted from the acquired image.

In at least one embodiment, the classification may be performed by a machine learning algorithm. In at least one embodiment, the machine learning algorithm may be at least one of logistic regression or linear discriminant analysis. In at least one embodiment, the method may further include the step of training the machine learning algorithm using a plurality of cigarette images known to be classified as VLN cigarettes or regular cigarettes.

In at least one embodiment, the hyperspectral camera can image the cigarette to form a two-dimensional image of the cigarette surface for each spectral wavelength.

In at least one embodiment, the image may include a plurality of pixels. Each pixel may include a plurality of spectral measurements. In at least one embodiment, each pixel may include at least 160 spectral measurements.

In at least one embodiment, the step of classifying the tobacco as VLN or regular tobacco can be performed non-invasively.

In at least one embodiment, a rectangular area of the cigarette may be imaged by the hyperspectral camera. In at least one embodiment, the rectangular area may be a cigarette area approximately 12 inches × 30 inches in size.

In at least one embodiment, the method may further include the step of placing a reflectance material on the cigarette for preprocessing. In at least one embodiment, the method may further include the step of defining a region of interest (ROI) for the reflectance material, the step of defining a second region of interest for the cigarette, and the step of removing pixels with a response too low within the second region of interest after acquiring an image with the hyperspectral camera. In at least one embodiment, the method may further include the step of generating a mean spectral vector for each pixel along the x-axis of the first region of interest by averaging along the y-axis corresponding to each pixel on the x-axis. In at least one embodiment, the method may further include the step of correcting each pixel within the second region of interest with the mean spectral vector to generate the mean spectra of the cigarette. In at least one embodiment, the cigarette may be classified as very low nicotine (VLN) or regular cigarette based on the mean spectra. In at least one embodiment, the step of correcting each pixel within the second region of interest may include discarding the corresponding pixel in the second region of interest if there is no element corresponding to the average spectrum vector. In at least one embodiment, the reflectance material may have an appropriate reflectance. In at least one embodiment, the reflectance material may be a SPECTRALON® 40% reflectance standard material.

Additionally, the present specification describes a method for analyzing tobacco. The method may include the step of imaging the tobacco with a hyperspectral camera to acquire an image, and analyzing the acquired image to quantify the amount of nicotine in the tobacco.

Additionally, the present specification describes a method for quantifying at least one chemical component in tobacco. The method may include the step of imaging tobacco with a hyperspectral camera to acquire an image, and analyzing the image to quantify the amount of said at least one chemical component.

In at least one embodiment, the at least one chemical component may be at least one of propylene glycol or glycerin.

The various features and advantages of the non-limiting embodiments described herein may become clearer by reviewing the detailed description together with the accompanying drawings. The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of the claims. Unless otherwise noted, the accompanying drawings should not be construed as drawn to scale. For clarity, some dimensions in the drawings may be exaggerated.
FIG. 1 is a schematic diagram illustrating a method for imaging and analyzing tobacco using hyperspectral imaging according to one embodiment.
FIG. 2 is a flowchart illustrating a method for classifying cigarettes into very low nicotine (VLN) cigarettes or regular cigarettes using hyperspectral imaging according to one embodiment.
FIG. 3 is a side view of a system for tobacco imaging and analysis using hyperspectral imaging according to one embodiment.
FIG. 4 is a side view of the system device of FIG. 3 used for tobacco imaging using hyperspectral imaging according to one embodiment.
FIG. 5 is a flowchart illustrating a method for classifying cigarettes into very low nicotine (VLN) cigarettes or regular cigarettes using hyperspectral imaging according to one embodiment.
FIGS. 6A and 6B are images of the surface of a cigarette bale containing a reflective material for imaging a cigarette through hyperspectral imaging according to one embodiment.
FIG. 7 is a graph showing the spectral profile of a tobacco veil according to one embodiment.
FIG. 8 is a graphical user interface (GUI) that can implement a machine learning algorithm to classify tobacco into VLN tobacco or regular tobacco through hyperspectral imaging according to one embodiment.
FIG. 9 is a graph showing the results of linear discriminant analysis (LDA) for different tobacco variety datasets imaged through hyperspectral imaging according to one embodiment.
FIG. 10 is a graph showing the results of linear discriminant analysis for additional different tobacco variety data sets imaged through hyperspectral imaging according to one embodiment.
FIG. 11 is a three-dimensional graph of linear discriminant analysis for different tobacco variety datasets imaged through hyperspectral imaging according to one embodiment.
FIG. 12 is a graph showing the visualization of t-distribution stochastic neighborhood embeddings (t-SNE) of VLN and general tobacco variety datasets imaged through hyperspectral imaging according to one embodiment.
FIG. 13 is a graph showing the t-SNE visualization of additional VLN and general tobacco variety datasets imaged through hyperspectral imaging according to one embodiment.
14 is a flowchart illustrating a tobacco analysis method for determining the nicotine content in tobacco according to one embodiment.
FIG. 15 is a flowchart illustrating another tobacco analysis method for determining the nicotine content in tobacco according to one embodiment.
FIG. 16 is a graph showing the root mean square error (RMSE) against the partial least squares (PLS) regression component for a tobacco dataset imaged through hyperspectral imaging according to one embodiment.
FIG. 17 is a graph showing the predicted nicotine percentage and the actual measured nicotine percentage for a tobacco data set imaged through hyperspectral imaging according to one embodiment.
FIG. 18 is a graph showing regression coefficients relative to predictors for a tobacco dataset imaged through hyperspectral imaging according to one embodiment.
FIG. 19 is a graph showing regression coefficients relative to predictors for another tobacco dataset imaged through hyperspectral imaging according to one embodiment.
FIG. 20 is a graph showing the spectral profile of a tobacco veil according to one embodiment.
Figure 21 is a chart showing the data included in the graph of Figure 20.
FIG. 22 is a flowchart illustrating an analysis method for determining the content of at least one chemical component in tobacco according to one embodiment.
FIG. 23 is a flowchart illustrating another analysis method for determining the content of at least one chemical component in tobacco according to one embodiment.
FIG. 24 is a block diagram of a computer system for implementing one of the methods described in this specification according to one embodiment.

Some detailed embodiments are disclosed herein. However, the specific structural and functional details disclosed herein are merely representative for illustrating the embodiments. Embodiments may be implemented in various other forms and should not be construed as being limited only to the embodiments presented herein.

Accordingly, various modifications and alternative forms are possible for the embodiments, and examples thereof are presented and described in detail in the drawings herein. However, there is no intention to limit the embodiments to the specifically disclosed forms; rather, the embodiments should be understood to include all variations, equivalents, and alternatives that fall within the scope thereof. Throughout the description of the drawings, the same reference numerals refer to the same components.

Where one element or layer is referred to as being "on," "engaged," "connected," "combined," or covering another element or layer, it may be directly on, directly engaged with, directly connected with, directly combined with, or directly covering the other element or layer, or an intermediate element or layer may exist. Conversely, where one element is referred to as being "directly on," "directly engaged with," "directly connected with," or "directly combined" with another element or layer, there may be no intermediate element or layer. As used in this application specification, the term "and/or" includes any combination of one or more of the items enumerated in relation thereto and all combinations thereof.

In this application specification, terms such as first, second, third, etc., may be used to describe various elements, regions, layers, and/or sections, but such elements, regions, layers, and/or sections should not be limited by such terms. Such terms may be used only to distinguish one element, region, layer, or section from another element, region, layer, or section. Therefore, the first element, first region, first layer, or first section discussed below may refer to the second element, second region, second layer, or second section without departing from the teachings of the embodiments.

Spatial relative terms such as "below," "below," "lower," "above," and "upper" may be used in this application specification to facilitate the description of the relationship that one element or feature has with respect to other element(s) or feature(s) depicted in the drawings. Spatial relative terms may be intended to include other orientations of the device during use or operation in addition to the orientation shown in the drawings. For example, if the device is inverted in the drawings, an element described as "below" or "below" of another element or feature becomes "above" of that other element or feature. Therefore, the exemplary term "below" may include both the lower and upper directions. The device may be oriented in other ways (rotated 90 degrees or in other directions), and spatial descriptions used in this application specification may be interpreted accordingly.

The terms used in this specification are for the purpose of describing various embodiments and are not intended to limit the embodiments. As used in this application specification, the singular form is intended to likewise include the plural form unless the context clearly indicates otherwise. The terms “include,” “comprising,” “having,” and “having” are in a comprehensive sense and indicate that the mentioned features, integers, steps, operations, and/or elements are present, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and/or groups thereof.

Where the terms “about” or “substantially” are used in relation to numerical values in this specification, it implies that manufacturing or operating tolerances (e.g., ±10%) are included around the specified numerical value. Furthermore, where the terms “generally” or “substantially” are used in relation to geometric shapes, it implies that the accuracy of such shapes is not required and that a certain range of variation is included within the scope of this disclosure. Moreover, where numerical values or shapes are modified by terms such as “about,” “generally,” or “substantially,” these numerical values or shapes should be interpreted as including manufacturing or operating tolerances (e.g., ±10%) with respect to the mentioned numerical values or shapes.

Unless otherwise defined, all terms used in this document (including technical and scientific terms) shall have the same meaning as generally understood by a person of ordinary skill in the relevant technical field. Additionally, terms defined in commonly used dictionaries shall also be interpreted consistently with their meanings in the relevant technical field and shall not be interpreted in an ideal or overly formal sense unless explicitly defined in this document.

As used herein, the term "bonded" includes both cases where they are detachably bonded and cases where they are permanently bonded. For example, if an elastic layer and a support layer are detachably bonded, the elastic layer and the support layer can be separated from each other when sufficient force is applied.

Hardware may be implemented using processing or control circuits including one or more processors, one or more central processing units (CPUs), one or more microcontrollers, one or more arithmetic logic units (ALUs), one or more digital signal processors (DSPs), one or more microcomputers, one or more field programmable gate arrays (FPGAs), one or more system-on-chip (SoCs), one or more programmable logic units (PLUs), one or more microprocessors, one or more application-specific integrated circuits (ASICs), or other devices capable of responding to and/or executing instructions in a defined manner.

For illustrative purposes, this system is described based on its application in the tobacco processing and product development process, but substantially the same system can be applied to the processing and product development of other agricultural products. Tobacco is packaged in the form of tobacco bales and purchased from growers after being graded. A tobacco bale is a large, generally rectangular package in which tobacco leaves and stems are tightly bound with a tight string or wire, or loosely packed in a box. A typical Burley or Virginia tobacco bale has sides of at least 1.2 meters in length, and the corresponding volume can be about 1.7 cubic meters.

In hyperspectral imaging, an image is acquired by exposing a sample to electromagnetic radiation and scanning its field of view. During the hyperspectral scanning and imaging process, multiple spectral images of objects emitting electromagnetic radiation at various wavelengths and frequencies are generated and collected one at a time in a very rapid, continuous manner, accumulating a vast amount of data; in this context, the wavelengths and frequencies relate to different selected portions or bands of the entire hyperspectral spectrum emitted by the object. Hyperspectral imaging and analysis systems operate at very high speeds, enabling them to provide spectral and spatial data and information with very high resolution and precision—which is fundamentally difficult to obtain with conventional spectral imaging and analysis methods.

Generally, when electromagnetic radiation, such as the light used in hyperspectral imaging, is incident on an object, that radiation is influenced by the physical, chemical, and/or biological components constituting the object. These influences can manifest through a combination of mechanisms such as absorption, diffusion, reflection, diffraction, scattering, and/or transmission of the electromagnetic radiation. Additionally, objects containing organic chemical components typically exhibit some degree of fluorescence and/or phosphorescence when irradiated with electromagnetic radiation or light, such as ultraviolet (UV), visible light (VIS), and infrared (IR). The electromagnetic radiation thus affected (i.e., electromagnetic radiation emitted via diffusion, reflection, diffraction, scattering, and/or transmission from the object) is generally directly and uniquely associated with the object's physical, chemical, and/or biological properties, particularly its chemical composition. Consequently, this forms a unique spectral fingerprint or signature pattern for identifying and characterizing the object.

A typical spectral imaging system consists of an automated measurement system and analysis software. The automated measurement system includes optical devices, mechanical devices, electronic devices, peripheral hardware, and software; generally, it illuminates a scene or sample using an illuminating source, measures and collects light emitted from objects within that scene or sample—for example, in the form of fluorescence—and applies appropriate calibration techniques to extract desired results from the measurement results. The analysis software includes software and mathematical algorithms for analyzing, displaying, and providing useful results regarding objects within the scene or sample in a meaningful form.

Hyperspectral images of a scene or sample can be acquired using a commercially available hyperspectral camera or a hyperspectral camera custom-made according to user requirements.

Each spectral image is a three-dimensional dataset composed of voxels (volume pixels), where two dimensions represent spatial coordinates or positions (x, y) within an object, and the third dimension represents the wavelength (A) of light emitted or reflected from the object. Thus, the coordinates of each voxel in the spectral image can be expressed as (x, y, A). A specific wavelength (A) of the object's imaged light is associated with a set of spectral images representing the object's spectral features in two dimensions (e.g., x-axis and y-axis), and the voxels with that wavelength value constitute the pixels of a monochromatic image of the object at that wavelength. Each spectral image, characterized by a range of wavelengths of the object's imaged light, is analyzed to generate a two-dimensional map of one or more physicochemical properties, such as the geometric shape, form or configuration, dimensions, and/or chemical composition of the object and/or object components in a scene or sample.

In hyperspectral imaging, multiple images of each object are generated from the electromagnetic radiation emitted or reflected by the object, having wavelengths and frequencies associated with different selected parts or bands of the entire spectrum emitted or reflected by the object. For example, hyperspectral imaging of an object is generated from the electromagnetic radiation emitted or reflected by the object, having wavelengths and frequencies associated with one or more of the following bands of the entire spectrum emitted or reflected by the object: a visible light band with a wavelength range of about 400 to 700 nanometers, an infrared band with a wavelength range of about 700 to 3000 nanometers, and a deep infrared band with a wavelength range of about 3 to 12 microns. By using appropriate wavelengths and wavelength ranges in hyperspectral imaging, the data and information of hyperspectral imaging can be optimally utilized to identify, distinguish, classify, and quantify the imaged object and/or material, for example, by analyzing the various signature spectra present in the pixels of the hyperspectral image.

High-speed hyperspectral imaging systems are often required to analyze various repeatable or non-repetitive chemical and physical processes occurring on time scales of less than 100 milliseconds. These processes are often difficult to study using conventional hyperspectral imaging techniques. Examples include combustion reactions, impulse spectroscopic electrochemical experiments, and inelastic polymer deformation. Furthermore, high-speed hyperspectral imaging is also necessary for remote sensing distant objects from fast-moving platforms, such as satellites or aircraft, as the observation targets are rapidly changing and difficult to repeat.

Referring to FIG. 1, a process (100) for imaging a tobacco bale (105) is illustrated. The tobacco bale (105) may be loaded onto a conveyor belt (110) after being delivered from a grower. The conveyor belt (110) may include rollers configured to move the tobacco bale (105). As the conveyor belt (110) moves, the tobacco bale (105) passes under a near-infrared (NIR) light source (115) and a hyperspectral camera (120). Further description of the NIR hyperspectral imaging of the tobacco bale (105) is provided below. After the tobacco bale (105) passes through the hyperspectral camera (120), it may further pass under and/or above a moisture meter (125) and a weight scale (130), and subsequently be inspected by a grader (135) and/or grower (140). A commercially available moisture meter (125) can be used. The tobacco bale (105) can then be moved on the conveyor belt (110) to a crop protection holding area (CPA hold) (145) or a transport container (150).

Referring to FIG. 2, a flowchart of a method (200) for classifying tobacco is illustrated. In some embodiments, the tobacco to be classified may be in the form of a veil, such as a tobacco veil (105), and in other embodiments, it may be in a form other than a veil. The method (200) may be initiated when the tobacco is loaded onto a conveyor belt (110) for image acquisition. In step 205, the tobacco is imaged by a hyperspectral camera (120). An image of the tobacco may be acquired in this step 205. After image acquisition, the tobacco is classified in step 210. Based on the acquired image, the tobacco may be classified as very-low nicotine (VLN) tobacco or traditional tobacco. This classification may be performed by an image processing system comprising at least a processor and memory.

FIG. 3 illustrates a system (300) capable of performing hyperspectral imaging and analysis of a tobacco bale (105). The system (300) may be an image processing system and may include one or more light sources (305) for providing light to an imaging processor. In some embodiments, one or more light sources (305) may be mounted on an arm (310) to adjust the position of the bale when the tobacco bale (105) is located on a platform (315) which is part of a conveyor belt (110). In some embodiments, the arm (310) may be mounted on a frame (320) of a cabinet (325). The arm (310) may be fixed or movably positioned on the frame (320) of the cabinet (325).

In at least one embodiment, one or more light sources (305) may provide a beam of electromagnetic radiation at one or more wavelengths. In some embodiments, one or more light sources (305) may be tungsten, halogen, mercury, ultraviolet (UV), or xenon light sources.

The system (300) may also include a hyperspectral camera (330). In some embodiments, the hyperspectral camera (330) may be the same or similar as the hyperspectral camera (120) described above. The hyperspectral camera (330) may be configured to image the tobacco veil (105) when the tobacco veil (105) is positioned on the platform (315) below the hyperspectral camera (330).

The system (300) may also include a computer (335) equipped with a processor and memory capable of rapidly processing system data. In some embodiments, the computer (335) may be configured to classify the tobacco veil (105) as very low nicotine (VLN) or regular cigarette. The computer (335) may be responsible for the operation of the system (300) and the position control of its components (e.g., one or more light sources (305) and hyperspectral cameras (330)). A power source (340) may provide continuous power to the computer (335), and such a device may be readily available commercially. Generally, the computer (335) may include a keyboard (345) and a monitor (350) to allow a user (355) to perform input and system monitoring. The power source (340) may be provided to supply stable and controlled power to the system (300).

Referring to FIG. 4, a system (300) in which a cigarette bale (105) is located within a cabinet (325) is illustrated. One or more light sources (305) may be two light sources, and a hyperspectral camera (330) may be positioned above the cigarette bale (105) and configured to image at least a portion of the cigarette bale. Further details regarding cigarette imaging and classification are described with reference to FIG. 5, 6A and 6B.

Referring to FIG. 5, a flowchart of a tobacco imaging and classification method (500) is illustrated. The method (500) may describe the method (200) in more detail. In at least one embodiment, the method (500) may be performed by the image processing system described above. In step 505, a reflectance material may be placed over the tobacco. In at least one embodiment, the tobacco may be a tobacco veil (105), and the reflectance material may be a SPECTRALON® material. The reflectance material may be placed by a person such as a grader (135) or through a machine or device of the system (300). As illustrated in FIG. 6A, the reflectance material (605) may be placed in a rectangular area (610) of the tobacco veil (105). In some embodiments, the reflectance material (605) may be a SPECTRALON® 40% reflectance standard material.

After the reflective material (605) is placed on the cigarette veil (105), a first region of interest may be defined by a processor of an image processing system in step 510. In at least one embodiment, the image processing system may be a computer system such as the computer (335) of FIG. 3. As shown in FIG. 6B, the first region of interest (615) may be a part of the reflective material (605) placed on the cigarette.

Returning to FIG. 5, in step 515, a second region of interest may be defined by the processor. As illustrated in FIG. 6B, the second region of interest (620) may be a part or region of the cigarette that does not contain the reflective material (605).

In step 520, the cigarette may be imaged by a hyperspectral camera (330). In some embodiments, one or more light sources (305) may be illuminated while the cigarette is being imaged by the hyperspectral camera (330). In some embodiments, the hyperspectral camera (330) may be configured to image a rectangular area (610) of the cigarette. In at least one embodiment, the rectangular area (610) may be an area of about 12 inches × 30 inches of the cigarette. Both the first region of interest (615) and the second region of interest (620) may be located within the rectangular area (610) of the cigarette.

As the cigarette moves in a straight line under the hyperspectral camera (330), a line of images is acquired. The hyperspectral camera (330) acquires a line of images while the cigarette moves under the camera and continues this process until a rectangular area (610) is fully imaged (captured). The line of images thus obtained are combined by a processor to generate a hyperspectral image of the cigarette. The hyperspectral image may represent a two-dimensional image of the cigarette surface for each spectral wavelength captured by the hyperspectral camera. The image processing system may also ensure that a consistent image is acquired by tracking the movement of the cigarette moving under the hyperspectral camera on a conveyor belt. The hyperspectral image may refer herein to an image or an image acquired by the hyperspectral camera. In at least one embodiment, the image may undergo preprocessing to remove the first four and last four spectral bands to reduce the total number of spectral bands in the image. For example, the total number of spectral bands can be reduced from 168 to 160. Processors that obtain images from hyperspectral cameras are a technology known to relevant technicians.

Imaging of the tobacco veil (105) by the hyperspectral camera (330) can generate an image including a reflection spectral grading signature and a spectral chemical signature. As will be understood by those skilled in the art, each pixel captured by the hyperspectral camera (330) may contain multiple spectral measurements, for example, about 160 or more spectral measurements at different wavelengths. In some embodiments, during imaging, the hyperspectral camera (330) may provide a three-dimensional hyperspectral image cube with a range of 640 × 1024 pixels, but is not limited thereto. The image acquired by the hyperspectral camera may contain relevant features that can be extracted to perform analysis. In some embodiments, the different wavelengths may be in the shortwave infrared (SWIR) range of the electromagnetic spectrum and may be about 900 nm to 2500 nm. To obtain an image of a cigarette in the short-wave infrared range of the electromagnetic spectrum, a short-wave infrared imaging camera or a camera having a spectral response including the SWIR region can be used.

In step 525, a second region of interest of the image is analyzed by a processor to remove pixels whose response value is below a threshold. In some embodiments, this process is known as dark subtraction. If a pixel in the second region of interest (620) has a response value lower than the threshold due to a shadow on the cigarette surface, that pixel is excluded from further analysis.

In step 530, a mean spectral vector for each pixel along the x-axis of the first region of interest (615) of the image is generated by the processor. The mean spectral vector can be generated by averaging the y-axis pixels corresponding to the x-axis pixels for each pixel along the x-axis.

After an average spectrum vector for each pixel along the x-axis of the first region of interest (615) is generated, the processor can correct each pixel of the second region of interest (620) with the corresponding average spectrum vector. When correcting a pixel of the second region of interest, the pixel is divided by the average spectrum vector for the corresponding point on the x-axis of the first region of interest (615). If a pixel of the second region of interest (620) is removed in step 525, that pixel remains uncorrected and removed. After all remaining pixels of the second region of interest (620) are corrected, the pixels of the second region of interest are averaged and normalized to generate the mean spectrum of the cigarette.

In step 540, the processor can classify the tobacco based on the average spectrum of the tobacco. In at least one embodiment, the tobacco may be classified as very low nicotine (VLN) tobacco or regular tobacco based on the average spectrum.

The method (500) may be a non-invasive method for classifying tobacco. The method (500) can classify tobacco in real time and facilitates verification of the tobacco classification. For example, when a shipment of tobacco arrives, the sender can classify the tobacco. The method (500) allows for easy analysis of whether the classification performed by the sender is accurate for the received tobacco. Additionally, the method (500) classifies previously unclassified tobacco, allowing the tobacco to be sold, marketed, and/or used based on the classification results. The classification performed by the method (500) ensures that the tobacco is accurately priced and identified, and allows for confidence that the tobacco has been accurately classified as VLN tobacco or regular tobacco for future use, transport, storage, and/or use within tobacco products.

Referring to FIG. 7, a chart (700) showing the spectral profile of a cigarette veil is illustrated. The chart (700) displays the wavelength index on the x-axis and the normalized average reflectance on the y-axis. The spectral profile of the cigarette veil represents the normalized average reflectance at each wavelength for an image taken by a hyperspectral camera.

FIG. 8 shows a graphical user interface (GUI) (800) of a software program that can be used to implement at least part of the method (500) of FIG. 5. The GUI (800) can be output to a display such as a monitor (350) of a computer (335). The GUI (800) can receive input through an input device connected to the computer (335), such as a keyboard (345), and can operate through the processor and memory of the computer (335).

The GUI (800) may display an image (805) captured by a hyperspectral camera. In at least one embodiment, the image (805) may capture a rectangular area (610) that may include a first region of interest (615) and a second region of interest (620). The GUI (800) may also include information fields such as a file name, an output directory, and a threshold. This threshold is a reference value for determining whether to remove pixels from the second region of interest (620). For example, as described in step 525 of FIG. 5, pixels may be removed if they are shadows or dark pixels. The GUI (800) may also include an option to use a prior reference. The prior reference may include the first region of interest (615) and may be applied to a subsequent cigarette image that does not include the first region having a reflective material (605).

The GUI (800) may include options for the user to acquire and analyze images. If the user selects the image acquisition option, the user may instruct a system, such as the system (300), to acquire images of cigarettes using a hyperspectral camera. If the user selects the image analysis option, the user may analyze the captured images to check image-related information. For example, image analysis may include determining whether the cigarette is a VLN cigarette or a regular cigarette. The GUI may be configured to output the classification results of the cigarettes, which may be whether the cigarette is a VLN cigarette or a regular cigarette. The GUI (800) may be designed so that a grader or analyst can classify cigarettes efficiently and easily.

In at least one embodiment, the task of classifying a cigarette as a VLN cigarette or a regular cigarette may be performed by a machine learning algorithm. A trained machine learning algorithm receives an image acquired by a hyperspectral camera as input and can predict whether the imaged (captured) cigarette is a VLN cigarette or a regular cigarette. In at least one embodiment, the machine learning algorithm may be configured to classify cigarettes in real-time. For example, when a cigarette moves in a straight line under the hyperspectral camera, the machine learning algorithm may classify the cigarette. In at least one embodiment, the machine learning algorithm may be implemented through a GUI (800).

Machine learning algorithms may include ensemble techniques such as random forests, for example, linear regression and/or logistic regression (e.g., partial least squares regression), statistical clustering, Bayesian classification, decision trees, dimensionality reduction (e.g., principal component analysis (PCA)), other machine learning models (e.g., expert systems), and/or combinations thereof. Machine learning algorithms may be used to provide various services such as image classification and tobacco classification, and may be installed and run on other electronic devices.

Figures 9���13 show linear discriminant analysis graphs of various data used to train and test machine learning algorithms. The first set of data includes 359 VLN tobacco veils and 83 regular tobacco veils. The 359 VLN tobacco veils include 167 flue-cured veils and 192 burley veils. All 83 regular veils are burley veils. The second set of data includes 1,211 VLN tobacco veils and 684 regular tobacco veils. The 1,211 VLN tobacco veils include 513 flue-cured veils and 698 burley veils, and the 684 regular tobacco veils include 311 flue-cured veils and 373 burley veils. Of the 698 VLN Burley bales in the second dataset, 560 are Gen 2 and 138 are Gen 3. Of the 513 VLN Flu-cured bales in the second dataset, 370 are Gen 2 and 143 are Gen 3.

In at least one embodiment, the machine learning algorithm described above can be trained using a dataset containing regular cigarettes and VLN cigarettes. The dataset can be divided into approximately 75% as a training dataset and approximately 25% as a testing dataset. For example, if the dataset consists of approximately 1,895 observations, approximately 1,420 observations can be used as the training dataset and approximately 475 observations can be used as the testing dataset. The training dataset inputs known classification information for each observation into the machine learning algorithm. By inputting observations that are pre-classified as either VLN cigarettes or regular cigarettes, the machine learning algorithm can be trained to classify whether a new cigarette image is a VLN cigarette or a regular cigarette.

After the machine learning algorithm is trained with training observations, a test dataset can be input into the machine learning algorithm. The machine learning algorithm receives test observations and determines whether each test observation should be classified as a VLN cigarette or a regular cigarette. Since the actual classification of each test observation is already known, the classification results performed by the algorithm can be verified. In at least one test of the machine learning algorithm, the algorithm was able to accurately classify all observations in the training dataset as either VLN cigarettes or regular cigarettes.

FIG. 9 shows a linear discriminant analysis graph (900) performed on a portion of a second dataset. FIG. 9 includes VLN burly and flu-cured Gen 3 veil data and general burly and flu-cured data. In the graph (900), the VLN burly data is displayed overlapping at least a portion of the general burly data.

FIG. 10 shows a linear discriminant analysis graph (1000) performed on a portion of the second data set. FIG. 10 includes VLN Burley and Flu-cured Gen 2 and Gen 3 veil data, but does not include regular cigarette data. There is no overlap between the VLN cigarette data in the graph (1000).

FIG. 11 shows a linear discriminant analysis graph (1100) with a third dimension introduced. The graph (1100) may contain the same data as the graph (900). By introducing the third dimension, a clear separation between each tobacco type appears.

FIG. 12 shows a two-dimensional t-SNE (t-distributed stochastic neighbor embedding) visualization graph (1200) for a first dataset. FIG. 12 includes VLN burly and flu-cured data and general data. The general data is displayed overlapping at least partially with the VLN burly and flu-cured data.

FIG. 13 shows a linear discriminant analysis graph (1300) performed on a first data set. FIG. 13 includes VLN Burley and flu-cured data and general data. In the graph, general data is displayed overlapping with some VLN Burley data, while VLN flu-cured data is distinguished from VLN Burley and general data. It can be understood that if three dimensions are introduced into the data visualization of FIG. 12 or 13, VLN Burley, VLN flu-cured, and general cigarette data can be clearly distinguished without overlapping with each other, just as in FIG. 11.

The image processing system described in this specification may be configured to analyze and quantify the nicotine content of tobacco.

FIG. 14 shows a flowchart of a method (1400) for analyzing tobacco. In some embodiments, the tobacco to be analyzed may be in the form of a tobacco bale (105), and in other embodiments, it may be in a form other than a bale. The method (1400) may begin at the point when the tobacco is loaded onto a conveyor belt (110) for imaging of the tobacco. In step 1405, the tobacco is imaged using a hyperspectral camera (120). After the tobacco image is acquired, in step 1410, the image is analyzed to quantify the nicotine content of the tobacco. The method (1400) may be performed through the system (300) described above.

FIG. 15 shows a flowchart of a method (1500) for visualizing and analyzing tobacco. The method (1500) may describe the method (1400) in more detail. Steps 1505–1535 of the method (1500) may be performed identically to steps 505–535 of FIG. 5. In step 1540, an analysis is performed to quantify the nicotine content of the tobacco. In at least one embodiment, the nicotine content of the tobacco may be determined based on the average spectrum determined in step 1535.

The method (1500) may be a method for non-invasively analyzing tobacco. The method (1500) enables real-time analysis of tobacco. The method (1500) can ensure accurate pricing and identification based on the nicotine content of the tobacco. For example, using the method (1500) allows tobacco to be sold, marketed, or used based on its nicotine content.

In at least one embodiment, the GUI (800) may be configured to implement part of the method (1500) of FIG. 15. As previously described, the GUI (800) may display an image (805) captured by a hyperspectral camera. The GUI (800) may output the nicotine level of the cigarette captured by the hyperspectral camera using a machine learning algorithm.

A machine learning algorithm may be configured to receive an image of a cigarette as input and determine the nicotine level of the cigarette. In at least one embodiment, the machine learning algorithm may be configured to analyze the cigarette in real time. For example, when the cigarette moves in a straight line under a hyperspectral camera, the machine learning algorithm may analyze it.

Machine learning algorithms may include ensemble techniques such as random forests, for example, linear regression and/or logistic regression, for example, partial least squares regression, statistical clustering, Bayesian classification, decision trees, dimensionality reduction such as principal component analysis (PCA), expert systems, and other types of machine learning models, and/or combinations thereof. These machine learning algorithms provide services such as image classification and tobacco classification, and can be installed and run on other electronic devices.

FIGS. 16–21 show charts and graphs analyzing various data used to train and test a machine learning algorithm to determine the nicotine content of tobacco. In at least one embodiment, the nicotine content of tobacco can be determined by testing and training a machine learning algorithm using the first data set and the second data set described above.

In at least one embodiment, a machine learning algorithm can be trained as previously described using a dataset containing regular cigarettes and VLN cigarettes. The dataset can be divided into approximately 75% as a training dataset and approximately 25% as a test dataset. For example, if the dataset consists of approximately 513 observations, approximately 385 observations can be used for training and approximately 128 observations for testing. The training dataset is input into the machine learning algorithm along with the known nicotine content of each observation. By inputting observations with known nicotine content, the machine learning algorithm can be trained to determine the nicotine content of cigarette samples using the training method described above.

After a machine learning algorithm is trained with training observations, a test data set can be input into the machine learning algorithm. The machine learning algorithm receives the test observations and can determine the nicotine content of each test observation. Since the actual nicotine content of each test observation is already known, the nicotine content calculated by the machine learning algorithm can be verified. In at least one embodiment, the machine learning algorithm was able to accurately classify each observation of the training data set as either VLN cigarettes or regular cigarettes.

FIG. 16 shows a partial least squares (PLS) regression graph (1600) for tobacco data such as the first dataset and/or the second dataset. The data to be analyzed is. In FIG. 16, the x-axis represents the number of PLS components, and the y-axis represents the root mean square error (RMSECV) of the cross-validation. The minimum RMSECV corresponds to approximately 26 PLS components.

FIG. 17 shows a partial least squares (PLS) regression graph (1700) for tobacco data such as a first data set and/or a second data set. The x-axis represents the measured nicotine percentage, and the y-axis represents the predicted nicotine percentage. In at least one embodiment, the predicted nicotine percentage is calculated by a machine learning algorithm.

In at least one embodiment, using 26 PLS components, the RMSE of the calibration was 0.0314, the RMSE of the prediction was 0.0332, the ^R² of the calibration was 0.915, and the ^R² of the prediction was 0.894.

FIG. 18 is a graph (1800) showing the regression coefficients of each predictor variable for a second dataset. FIG. 19 is a graph (1900) showing the regression coefficients of each predictor variable for a first dataset. In both the graphs of FIG. 18 and FIG. 19, the x-axis represents the predictor variable that may be related to the intercept and wavelength index, and the y-axis represents the regression coefficient. In at least one embodiment, the coefficient with the largest absolute value may have the greatest influence on the final model output. The top 10 coefficients are indicated by vertical lines in the graph.

FIG. 20 is a graph (2000) showing example spectra of VLN burrito tobacco data of a first data set and a second data set. The graph (2000) includes two highlighted portions corresponding to different wavelength regions. The first highlighted portion represents wavelengths in the range of approximately 1700 nm to 1750 nm, and the second highlighted portion represents wavelengths in the range of approximately 2200 nm to 2350 nm.

FIG. 21 is a chart (2100) showing a portion of the data used to generate the graph (2000) of FIG. 20. The first data set may contain approximately 160 spectral bands, and the second data set may contain approximately 270 spectral bands. The first data set and the second data set were acquired from different hyperspectral cameras or images. The first data set was acquired with a hyperspectral camera having approximately 160 spectral bands, and the second data set was acquired with a hyperspectral camera having approximately 270 spectral bands. As can be seen in the embodiment of FIG. 21, the prediction and classification system and method described herein are robust and can be implemented on various hardware. Thus, as technology advances, newer and more powerful systems with higher resolution can be used for classification and prediction.

The image processing system described herein may be further configured to analyze the content of at least one chemical constituent of tobacco. In at least one embodiment, at least one chemical constituent may be either propylene glycol or glycerin. In other embodiments, additional chemical constituents may be analyzed in the tobacco image by a hyperspectral camera.

FIG. 22 is a flowchart of a method (2200) for analyzing tobacco. In some embodiments, the tobacco to be analyzed may be in the form of a veil, such as a tobacco veil (105). In other embodiments, the tobacco to be analyzed may be in a form other than a veil. The tobacco may be a reconstituted tobacco product (RLD) and may be analyzed. The method (2200) begins when the tobacco is loaded onto a conveyor belt (110) for imaging of the tobacco (to obtain an image of the tobacco). In step 2205, the tobacco is imaged with a hyperspectral camera (120). In step 2205, an image of the tobacco may be acquired with the hyperspectral camera (120). After acquiring the image of the tobacco, in step 2210, the image is analyzed to quantify the content of at least one chemical component. The method (2200) may be performed by the system (300) described above.

FIG. 23 is a flowchart of a method (2300) for obtaining and analyzing images of a cigarette. The method (2300) further describes the method (2200). Steps 2305–2335 of this method (2300) can be performed substantially identically to steps 505–535 of FIG. 5. In step 2340, the cigarette may be analyzed to quantify the content of at least one chemical component of the cigarette. In at least one embodiment, the content of at least one chemical component of the cigarette may be determined based on the average spectrum determined in step 2335.

The method (2300) may be a method for non-invasively analyzing tobacco. The method (2300) enables real-time analysis of tobacco. The method (2300) can ensure that tobacco contains an appropriate chemical component content, thereby ensuring tobacco quality and enabling the sale, marketing, and use of tobacco based on at least one chemical component content. The method (2300) can also be used for product optimization in product development by utilizing the analysis of at least one chemical component content.

In at least one embodiment, the GUI (800) may be configured to implement part of the method (2300) of FIG. 23. As previously described, the GUI (800) may display an image (805) captured by a hyperspectral camera. The GUI (800) may use a machine learning algorithm to output at least one level of chemical components of the tobacco captured by the hyperspectral camera.

A machine learning algorithm may be configured to receive an image of a cigarette as input and determine at least one level of chemical components. In at least one embodiment, the machine learning algorithm may be configured to analyze the cigarette in real time. For example, the machine learning algorithm may analyze the cigarette as it moves in a straight line under a hyperspectral camera.

Machine learning algorithms may include ensemble techniques such as random forests, for example, linear regression and/or logistic regression, for example, partial least squares regression, statistical clustering, Bayesian classification, decision trees, dimensionality reduction, for example, principal component analysis (PCA), expert systems, and other types of machine learning models, and/or combinations thereof. Machine learning algorithms provide various services such as image classification and tobacco classification and can be installed and run on other electronic devices.

In at least one embodiment, the machine learning algorithm may be trained with a dataset containing 130 unique spectral sets. These 130 unique sets may include 10 base sheet images, 40 LW 0488 images, 40 MH 0808 images, and 40 MH 0809 images. Two images may be acquired per sample of each of the 40 LW 0488 images, 40 MH 0808 images, and 40 MH 0809 images.

If base sheet images are included, approximately 100 of the 130 unique sets can be used as training data and approximately 30 as test data. If base sheet images are excluded, approximately 93 of the 130 unique sets can be used as training data and approximately 27 as test data. In at least one embodiment, the machine learning algorithm can be trained using leave-one-out cross-validation. Additionally, the machine learning algorithm can be trained using a wide range of latent variables. The variable with the lowest cross-validation error can be selected as the final parameter for training and testing the machine learning algorithm.

In at least one embodiment, there may be at least 14 latent variables. Using the variable with the lowest cross-validation error, the following training data can be obtained for the propylene glycol chemical component: calibration RMSE 0.0533, prediction RMSE 0.0952, calibration ^R² 0.981, prediction ^R² 0.921, calibration RPD 4.82, prediction RPD 3.61, cross-validation RMSE 0.0799. Additionally, using the lowest cross-validation error variable, the following training data can be obtained for the chemical component glycerin: corrected RMSE 0.0586, predicted RMSE 0.0969, corrected ^R² 0.972, predicted ^R² 0.900, corrected RPD 4.09, predicted RPD 3.22, RMSE cross-validation 0.0864.

FIG. 24 illustrates an example of a computer system (2400) capable of analyzing the hyperspectral images described above. In some embodiments, the computer system (2400) may be the system (300) described earlier or an image processing system. The structure shown in FIG. 24 may also represent other computer components described above.

As illustrated in FIG. 24, a computer system (2400) may include a memory (2405), a processor (2410) connected to the memory (2405), and at least one communication interface (2415) connected to the processor (2410). The at least one communication interface (2415) may constitute a transceiver for transmitting and receiving data with other computer components. As is understood, depending on the implementation method of the computer system (2400), the computer system (2400) may include additional general configurations. For illustrative purposes, the embodiment shown in FIG. 24 will be discussed in relation to the processor (2410). However, it should be understood that the computer system (2400) shown in FIG. 24 may include one or more processors or other processing circuits such as one or more application-specific integrated circuits (ASICs).

Memory (2405) may generally be a computer-readable storage medium including random access memory (RAM), read-only memory (ROM), and/or a permanent mass storage device such as a disk drive. Memory (2405) may store an operating system executed by the processor (2410) and other routines/modules/applications that provide the functions of network nodes (including UPF, CPF, MPF, etc.). These software components may be loaded into memory (2405) from a separate computer-readable storage medium using a drive mechanism (not shown). This separate computer-readable storage medium may include a disk, tape, DVD/CD-ROM drive, memory card, or other computer-readable storage medium (not shown). In some embodiments, software components may be loaded into memory (2405) through one or more communication interfaces (2415), and loading through a computer-readable storage medium is not required.

A processor (2410) or other processing circuit may be configured to perform arithmetic operations, logical operations, and input/output operations of the system to execute instructions of a computer program. Instructions may be provided to the processor (2410) by memory (2405).

At least one communication interface (2415) may be wired and may include a configuration connecting the processor (2410) and other input/output configurations. As is understood, at least one communication interface (2415) and a program stored in memory (2405) set up special purpose functionalities of the computer, which may vary depending on the implementation method of the computer.

At least one communication interface (2415) may include one or more user input devices (e.g., keyboard, keypad, mouse, etc.) and user output devices (e.g., display, speaker, etc.).

The appended claims define new and creative aspects of the subject matter described above, but the claims may also include additional subject matter that is not specifically detailed. For example, some features, elements, or aspects may be omitted from the claims if they are not necessary to distinguish the new and creative features of the invention from those already known to those skilled in the art. Features, elements, or aspects described in the context of some embodiments may also be omitted, combined, or replaced with alternative features that perform the same, equivalent, or similar purposes, provided that they do not depart from the scope of the invention as defined in the appended claims.

Claims (23)

As a method for classifying tobacco, the above method is:
A step of imaging a cigarette using a hyperspectral imaging system including a hyperspectral camera and an image processing system to acquire an image; and
A step of classifying a cigarette as a very low nicotine (VLN) cigarette or a regular cigarette based on an acquired image; comprising
method. In paragraph 1,
The above method further includes the step of placing cigarettes on a conveyor belt configured to pass under the hyperspectral camera,
method. In paragraph 2,
The imaging of the above cigarette is performed while the cigarette moves in a straight line under the hyperspectral camera via the conveyor belt, and the movement of the cigarette is tracked by the image processing system to provide a consistent image.
method. In paragraph 2,
The classification of the above cigarette includes analyzing the image in real time while the cigarette moves in a straight line under the hyperspectral camera,
method. In paragraph 1,
The above hyperspectral camera is configured to image cigarettes using short-wave infrared (SWIR) imaging,
method. In paragraph 5,
The above SWIR imaging operates in the range of about 900 nanometers (nm) to about 2500 nanometers (nm),
method. In paragraph 1,
The above classification includes extracting relevant features from the above-mentioned acquired image,
method. In paragraph 1,
The above classification is performed by a machine learning algorithm,
method. In paragraph 8,
The above machine learning algorithm is at least one of logistic regression or linear discriminant analysis,
method. In paragraph 8,
The above method further includes the step of training the machine learning algorithm using a plurality of cigarette images known to be classified as VLN cigarettes or regular cigarettes.
method. In paragraph 1,
The hyperspectral camera images the cigarette to construct a two-dimensional image of the cigarette surface for each spectral wavelength,
method. In paragraph 1,
The above image includes a plurality of pixels, and each pixel of the plurality of pixels includes a plurality of spectral measurement values,
method. In Paragraph 12,
Each pixel of the plurality of pixels comprises at least 160 spectral measurements defined by the hyperspectral camera,
method. In paragraph 1,
The step of classifying the above tobacco as VLN tobacco or regular tobacco is performed non-invasively,
method. In paragraph 1,
A rectangular area of the above cigarette is imaged by the hyperspectral camera,
method. In paragraph 15,
The above rectangular area is approximately 12 inches × 30 inches in size,
method. In paragraph 1,
The above method further includes the step of placing a reflective material on the cigarette for pretreatment.
method. In Paragraph 17,
The above method includes the step of defining a first region of interest for the reflectance material;
A step of defining a second region of interest for the above tobacco; and,
The method further comprises the step of removing pixels with a response that is too low within the second region of interest after acquiring the image with the hyperspectral camera.
method. In Paragraph 18,
The above method further comprises the step of generating an average spectrum vector for each pixel along the x-axis of the first region of interest by averaging the first region of interest along the y-axis corresponding to each pixel on the x-axis.
method. In Paragraph 19,
The above method further comprises the step of correcting each pixel within the second region of interest with the average spectrum vector to generate the average spectrum of the tobacco.
method. In paragraph 20,
The above cigarette is classified as a very low nicotine (VLN) cigarette or a regular cigarette based on the above average spectrum,
method. In paragraph 20,
The step of correcting each pixel within the second region of interest includes discarding the corresponding pixel in the second region of interest when there is no corresponding element of the average spectrum vector.
method. In Paragraph 17,
The above reflective material is a material having an appropriate reflectiveness,