CN111062475B - Method and apparatus for quantizing parameters of a neural network - Google Patents

Method and apparatus for quantizing parameters of a neural network Download PDF

Info

Publication number
CN111062475B
CN111062475B CN201910822654.5A CN201910822654A CN111062475B CN 111062475 B CN111062475 B CN 111062475B CN 201910822654 A CN201910822654 A CN 201910822654A CN 111062475 B CN111062475 B CN 111062475B
Authority
CN
China
Prior art keywords
parameter
point format
shift value
bit shift
neural network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910822654.5A
Other languages
Chinese (zh)
Other versions
CN111062475A (en
Inventor
朴炫宣
李俊行
姜信行
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Publication of CN111062475A publication Critical patent/CN111062475A/en
Application granted granted Critical
Publication of CN111062475B publication Critical patent/CN111062475B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Neurology (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Image Analysis (AREA)
  • Complex Calculations (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种量化神经网络的参数的方法包括：针对每个参数来计算比特移位值，该比特移位值指示在用于对参数进行量化的定点格式的比特范围之外的程度；基于所计算的参数的比特移位值来更新定点格式；以及根据经更新的定点格式来量化在学习或推断过程中更新的参数。

A method of quantizing parameters of a neural network includes: calculating a bit shift value for each parameter, the bit shift value indicating a degree outside a bit range of a fixed-point format used to quantize the parameter; updating the fixed-point format based on the calculated bit shift value of the parameter; and quantizing the parameters updated in a learning or inference process according to the updated fixed-point format.

Description

Method and device for quantifying parameters of a neural network
Cross Reference to Related Applications
The present application claims priority from korean patent application No. 10-2018-0123977, filed in the korean intellectual property office on 10 month 17 of 2018, the entire disclosure of which is incorporated herein by reference.
Technical Field
The following description relates to methods and apparatus for quantifying parameters of a neural network.
Background
Neural networks refer to computational architecture that models the brain of a living organism. With recent progress in neural network technology, a great deal of research has been conducted on analyzing input data and extracting effective information by using a neural network device in various electronic systems.
The neural network device performs a large number of operations on the input data. There is a need for a technique to efficiently handle such neural network operations.
Disclosure of Invention
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
In one general aspect, a method of quantizing parameters of a neural network includes, for each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter, updating the fixed point format using the calculated bit shift value of the parameter, and quantizing the parameter updated in a learning or inference process according to the updated fixed point format.
Calculating the bit shift value may include detecting, for each parameter, a most significant bit having a value of "1", and determining, for each parameter, a difference in number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed point format as the bit shift value.
Detecting the most significant bits may include searching for bits within a particular range based on the most significant bits of the integer part of the fixed point format for each parameter and detecting the most significant bits having a value of "1".
Updating the fixed-point format may include determining a number of occurrences of overflow and a maximum bit shift value according to the calculated bit shift value, and updating the fixed-point format based on the number of occurrences of overflow and the maximum bit shift value.
Updating the fixed-point format may include updating the fixed-point format by reducing a fractional length of the fixed-point format by a maximum bit shift value in the case that the number of occurrences of overflow is greater than a specific value.
The particular value may be based on the number of parameters.
The updated parameter may be a parameter updated during the t+1th learning or inference process, the parameter may be a parameter updated during the t-th learning or inference process, the fixed-point format may be a fixed-point format updated based on the parameter updated during the t-1 th learning or inference process, and t may be a natural number greater than or equal to 2.
Calculating the bit-shift value may include calculating a bit-shift value for each parameter during quantization of the parameter according to the fixed-point format.
The parameters may be weights or activations on the same layer in the neural network.
A computer-readable recording medium may store a program for causing a computer to execute the method.
In another general aspect, an apparatus for quantizing parameters of a neural network includes a memory storing at least one program, and a processor configured to calculate, for each parameter, a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter, update the fixed point format using the calculated bit shift value of the parameter, and quantize the parameter updated in a learning or inference process according to the updated fixed point format by executing the at least one program.
The processor may detect a most significant bit having a value of "1" for each parameter, and determine, for each parameter, a difference in number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed point format as the bit shift value.
The processor may search for bits within a particular range based on the most significant bits of the integer portion of the fixed point format and detect the most significant bits having a value of "1".
The processor may determine the number of occurrences of overflow and a maximum bit shift value from the calculated bit shift value, and update the fixed point format using the number of occurrences of overflow and the maximum bit shift value.
The processor may update the fixed point format by decreasing the fractional length of the fixed point format by a maximum bit shift value in case that the number of occurrences of overflow is greater than a predetermined value.
The processor may calculate a bit shift value for each parameter during quantization of the parameter according to the fixed point format.
In another general aspect, a method includes calculating, for each parameter updated in a t-th learning or inference process of a neural network, a bit shift value based on a fixed point format used to quantize the parameter, determining a number of occurrences of overflow and a maximum bit shift value from the calculated bit shift values, updating the fixed point format based on the number of occurrences of overflow and the maximum bit shift value, and quantizing the parameter in a t+1th learning or inference process of the neural network based on the updated fixed point format, and t is a natural number greater than or equal to 2.
Determining the number of occurrences of overflow may include determining whether a bit shift value of each parameter is greater than 0 and increasing the number of occurrences of overflow by 1 for each bit shift value greater than 0.
Determining the maximum bit-shift value may include comparing the calculated bit-shift values of the parameters updated in the t-th learning or inference process with each other and determining the maximum value of the bit-shift values updated in the t-th learning or inference process as the maximum bit-shift value.
In another general aspect, a computer-readable recording medium has one or more programs recorded thereon, the one or more programs including instructions for performing a method of quantifying parameters of a neural network.
Other features and aspects will be apparent from the following detailed description, the drawings, and the claims.
Drawings
Fig. 1 is a diagram for explaining a neural network learning device and a neural network inference device according to an example.
Fig. 2 is a diagram illustrating an example of a neural network according to an example.
Fig. 3A is a diagram illustrating floating point format parameters according to an example, fig. 3B is a diagram illustrating fixed point format parameters according to an example, and fig. 3C is a diagram illustrating fixed point format parameters according to another example.
Fig. 4 is a block diagram of a hardware configuration of a neural network device according to an example.
FIG. 5 illustrates an example of a processor operating in a iterative learning or inference process.
Fig. 6 shows an example in which a processor uses bit-shifted values of parameters to update a fixed point format.
Fig. 7 shows an example of a bit shift value of a processor calculated parameter.
Fig. 8 shows an example of the processor detecting the most significant bit with "1" in the parameter.
Fig. 9 shows an example in which the processor determines the maximum bit shift value and the number of occurrences of overflow.
Fig. 10 is a diagram illustrating an algorithm for a processor to update a fractional length of a fixed point format according to an example.
Fig. 11 is a diagram illustrating an algorithm for a processor to update a fractional length of a fixed point format according to another example.
Fig. 12 is a block diagram of an electronic system according to an example.
Fig. 13 is a diagram for explaining a method of operating a neural network device according to an example.
Throughout the drawings and detailed description, unless otherwise described or provided, like reference numerals will be understood to refer to like elements, features and structures. The figures may not be drawn to scale and the relative sizes, proportions, and depictions of elements in the figures may be exaggerated for clarity, illustration, and convenience.
Detailed Description
Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, the embodiments presented may take different forms and should not be construed as limited to the descriptions set forth herein. Accordingly, the embodiments are described below merely by referring to the drawings to explain various aspects. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of the list of elements" modify the entire list of elements before it, rather than modifying individual elements in the list.
The terms "comprising" or "including," and the like, as used herein, should not be construed as necessarily including all of the various elements or operations described in the specification, and should exclude some of them or may be construed as including additional components or operations.
Furthermore, as used herein, various elements may be described using terms including ordinal numbers such as "first" or "second," but the elements should not be limited to the terms. These terms are only used to distinguish one element from another element.
Examples relate to a method and apparatus for processing parameters of a neural network, and detailed descriptions thereof will be omitted with respect to matters widely known to those of ordinary skill in the art.
Fig. 1 is a diagram for explaining a neural network learning device 10 and a neural network inference device 20 according to an example.
Referring to fig. 1, a neural network learning apparatus 10 may correspond to a computing device having various processing functions, such as generating a neural network, training or learning a neural network, quantizing a floating point format neural network to a fixed point format neural network, quantizing a fixed point format neural network to another fixed point format neural network, or retraining a neural network. For example, the neural network learning apparatus 10 may be implemented as various types of devices, such as a Personal Computer (PC), a server device, a mobile device, and the like. Meanwhile, parameter quantization may represent conversion of a floating-point format parameter to a fixed-point format parameter, or conversion of a fixed-point format parameter having a specific bit width to another fixed-point format parameter having another bit width.
The neural network learning device 10 can generate the trained neural network 11 by repeatedly training (learning) a given initial neural network. In this state, the initial neural network may have floating point format parameters, for example, parameters of 32-bit floating point precision, in terms of ensuring processing accuracy of the neural network. The parameters may include various types of data input to or output from the neural network, such as input/output activation of the neural network, weights, biases, and the like. As the neural network is trained repeatedly, floating point parameters of the neural network may be adjusted to output a more accurate output relative to a given input.
The neural network learning device 10 may process parameters according to a fixed-point format in the course of repeatedly learning (training) the initial neural network. Specifically, the neural network learning device 10 can process parameters according to an 8-bit or 16-bit fixed-point format to learn the neural network within an allowable accuracy loss while sufficiently reducing the number of operations. Thus, the neural network learning apparatus 10 may be implemented in a smart phone, a tablet computer, or a wearable device having relatively low processing power for device-side learning.
The neural network learning device 10 may send the trained neural network 11 to a hardware accelerator (e.g., the neural network inference device 20). The neural network inference means 20 may be included in a mobile device, an embedded device, or the like. The neural network inference means 20 is dedicated hardware for driving the quantized neural network 21. Since the neural network inference means 20 is implemented with relatively low power or low performance, the neural network inference means 20 may be implemented to be more suitable for fixed point operations than floating point operations. The neural network inference means 20 may correspond to, but is not limited to, a Tensor Processing Unit (TPU), a neural engine, etc., which is a dedicated module for driving the neural network.
The neural network inference means 20 for driving the quantized neural network 21 may be implemented in a separate device independent of the neural network learning means 10. However, the present disclosure is not limited thereto, and the neural network inference means 20 may be implemented in the same device as the neural network learning means 10.
The neural network inference apparatus 20 deploying the quantized neural network 21 may be included in, for example, an autonomous vehicle, a robot, a smart phone, a tablet device, an Augmented Reality (AR) device, an internet of things (IoT) device, or the like, which performs voice recognition, image recognition, or the like by using the neural network, but the present disclosure is not limited thereto.
Fig. 2 is a diagram showing an example of the neural network 2.
Referring to fig. 2, the neural network 2 may have a structure including an input layer, a hidden layer, and an output layer, may perform an operation based on received input data (e.g., I 1 and I 2), and may generate output data (e.g., O 1 and O 2) based on a result of the operation.
The neural network 2 may be a Deep Neural Network (DNN) or an n-layer neural network including one or more hidden layers. For example, as shown in fig. 2, the neural network 2 may be DNN including an input layer 1, two hidden layers (layer 2 and layer 3), and an output layer 4. The DNNs may include, but are not limited to, convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), deep belief networks, limited boltzmann machines, and the like.
Although the neural network 2 is shown as including four layers, this is merely exemplary, and the neural network 2 may include more or fewer layers or more or fewer nodes. Further, the neural network 2 may include layers having various structures different from the structure shown in fig. 2. For example, the neural network 2 may include a convolutional layer, a pooling layer, and a fully-connected layer as DNNs.
Each layer included in the neural network 2 may include a plurality of artificial nodes called neurons, processing units (PEs), units, or similar terms. For example, as shown in fig. 2, layer 1 may include two nodes and layer 2 may include three nodes. However, this is merely exemplary, and each layer included in the neural network 2 may include a variety of numbers of nodes.
Nodes included in the various layers of the neural network 2 may be connected to each other in order to process data. For example, one node may receive data from other nodes and process the data, and may output the operation result to the other nodes.
The output value of each node may be referred to as active. The activation may be the output of one node and is the input value of the node included in the next layer. Each node may determine its own activation based on the activation received from the node included in the previous layer and the weight. The weight is a parameter for calculating activation in each node, and may be a value assigned to a connection relationship between nodes.
Each node may be processed by a computing unit that receives input and output activations, and may map the input and output. At sigma, the activation function is represented,Represents a weight from a kth node included in the (i-1) th layer to a jth node included in the i-th layer,Representing the offset sum of the jth node included in the ith layerWhen representing activation of the j-th node of the i-th layer, the activation may be calculated by using the following equation 1
Equation 1
As shown in fig. 2, the activation of the first node in the second layer 2 may be expressed asIn addition, according to equation 1,May have a value ofHowever, the above equation 1 is merely an example for describing activation, weight, and bias for processing data in the neural network 2, and the present disclosure is not limited thereto. The activation may be a value obtained by allowing a value obtained by applying an activation function to a weighted sum of activations received from a previous layer to pass through a modified linear unit (ReLU).
As described above, in the neural network 2, a large number of data sets are exchanged between a plurality of interconnected nodes and these data sets undergo many calculation processes in each layer.
Fig. 3A is a diagram illustrating floating point format parameters 30 according to an example. Fig. 3B is a diagram illustrating fixed point format parameters 35 according to an example. Fig. 3C is a diagram illustrating fixed point format parameters according to another example.
Referring to FIG. 3A, floating point format parameter 30 may include sign bit 310, exponent portion 320, mantissa portion 330, and offset 340. Floating point indications divide a number into a portion that indicates a significant digit portion (i.e., mantissa) and a portion that indicates a decimal point location.
Mantissa portion 330 may correspond to a portion that indicates a significant digit portion. The index portion 320 may correspond to a portion indicating the position of the decimal point. Sign bit 310 may determine the sign of floating point format parameter 30. The bias 340 may be a value added to the exponent portion 320 or a value subtracted from the exponent portion 320 and determined to be a value representing a negative exponent. Floating point format parameter 30 may include sign bits 310, bits corresponding to exponent portion 320, and bits corresponding to mantissa portion 330. The offset 340 may be predetermined with respect to the floating point format parameters 30 and stored separately.
When sign bit 310 represents a sign, exponent portion 320 represents an exponent, mantissa portion 330 represents a mantissa, and offset 340 represents an offset, floating point format parameter 30 may be a value according to equation 2 below.
Equation 2
Floating point value=(-1)sign·2exponent-bias·mantissa
Referring to fig. 3B, the fixed point format parameter 35 may include a sign bit 315, an integer portion 325, a fractional portion 335, and a fractional point 345. The fixed point indicates a sign that represents a decimal of a fixed number of bits using a decimal point.
Sign bit 315 may determine the sign of fixed point format parameter 35. Integer portion 325 may correspond to a portion of an integer representing fixed point format parameter 35. The fractional portion 335 may correspond to a portion representing the fractional portion of the fixed point format parameter 35. The decimal point 345 may indicate a point that is a reference for distinguishing the integer portion 325 and the decimal portion 335 of the fixed point format parameter 35.
The values represented by the fixed point format parameters 35 may be described with reference to fig. 3C. Referring to fig. 3C, the fixed-point format parameter 35 may be an 8-bit fixed-point value. The fixed point format parameter 35 may also include an integer portion 325, a fractional portion 335, and a fractional point 345.
Each bit representing the integer portion 325 and the fractional portion 335 may have a value of 1 or 0. As shown in fig. 3C, bits representing the integer portion 325 and the fractional portion 335 may have values of-8, +4, +2, +1, +0.5, +0.25, +0.125, and +0.0625 sequentially. When the most significant bit of the integer part 325 is 1, since the value represented by the most significant bit of the integer part 325 is-8, the value represented by the fixed point format parameter 35 may be negative even if the other bits included in the integer part 325 and the fractional part 335 have specific values. The most significant bits of integer portion 325 may correspond to symbol bits 315 of the symbol that determines fixed-point format parameter 35.
However, fig. 3C is only an example, and the fixed-point format parameter 35 may be a 16-bit fixed-point value, and may be a fixed-point value having any suitable number of bits. In addition, the fixed-point format parameter 35 may represent a negative number by any one of the encoding representation method, the 1-complement method, and the 2-complement method.
Fig. 4 is a block diagram of a hardware configuration of the neural network device 100 according to an example.
The neural network device 100 may operate by being included in at least one of the neural network learning device 10 and the neural network inference device 20, or may operate as a separate third hardware accelerator.
Referring to fig. 4, the neural network device 100 may include a processor 110 and a memory 120. In fig. 4, only the constituent elements of the neural network device 100 of the present example are shown. It will thus be apparent to those of ordinary skill in the art that the neural network device 100 may also include general constituent elements in addition to those illustrated in fig. 4.
The processor 110 performs all functions to control the neural network device 100. For example, the processor 110 controls all functions of the neural network device 100 by executing programs stored in the memory 120 in the neural network device 100. The processor 110 may be implemented by a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Application Processor (AP), or the like provided in the neural network device 100. However, the present disclosure is not limited thereto.
The memory 120 is hardware for storing pieces of data handled in the neural network device 100. For example, the memory 120 may store data processed in the neural network device 100 or to be processed in the neural network device 100. Further, the memory 120 may store applications, drivers, and the like to be driven by the neural network device 100. The memory 120 may be a (dynamic random access memory) DRAM, but the present disclosure is not limited thereto. The memory 120 may include at least one of volatile memory or nonvolatile memory.
The processor 110 may generate a trained neural network by iteratively learning the initial neural network. The processor 110 may repeatedly update the parameters of the neural network by repeatedly learning the neural network. For example, the weights in the neural network may be updated repeatedly during the learning process, and the activation may also be updated repeatedly due to the operation using the weights. Each time the parameters of the neural network are repeatedly updated, the processor 110 may quantize the updated parameters in a fixed point format. Further, the processor 110 may repeatedly update the fixed point format each time the parameters of the neural network are repeatedly updated. When quantizing variable parameters with a fixed point format, the processor 110 may update the fixed point format to quantize the parameters, reducing accuracy loss while reducing the number of operations, as accuracy loss may occur during the learning process. In particular, the processor 110 may update the fixed point format to correspond to the distribution of fixed point values of the updated parameters. For example, the processor 110 may update the decimal point location in the fixed point format to correspond to the parameter having the greatest value among the updated parameters.
The processor 110 may repeatedly update the parameters of the neural network, even during the inference process that drives the learned neural network to obtain the result value. For example, during the inference process, data may be repeatedly input to the learned neural network, and thus the activation in the learned neural network may be repeatedly updated. Thus, similar to the learning process, the processor 110 may quantify the updated parameters according to a fixed point format each time the parameters of the neural network are repeatedly updated, even during the inference process. Further, similar to the learning process, the processor 110 may repeatedly update the fixed point format each time the parameters of the neural network are repeatedly updated, even during the inference process.
The processor 110 may update the parameters during the t-th learning or inference process and may then quantify the updated parameters. At this point, the processor 110 may quantify the updated parameters of the t-th learning or inference process according to a particular first setpoint format. For example, the first setpoint format may be an updated setpoint format based on parameters updated during the t-1 th learning or inference process. The processor 110 may update the existing first fixed point format to the second fixed point format based on the parameters updated during the t-th learning or inference. The processor 110 may then quantify the parameters updated during the t+1st learning or inference process according to the second fixed point format.
Thus, when quantizing parameters updated during the t-th learning or inference process, the processor 110 may quantize the parameters according to a particular fixed point format, thereby reducing the time to scan all parameters updated during the t-th learning or inference process and the hardware overhead for determining the fixed point format.
Fig. 5 illustrates an example of the operation of the processor 110 in a iterative learning or inference process.
The processor 110 may quantize the parameter t updated during the t-th learning or inference process according to a fixed point format having a fractional length frac length t-1 to generate a quantized parameter q_parameter t. In other words, the processor 110 may pre-determine the fixed-point format having the fractional length frac_length t-1 based on the parameter t-1 updated during the t-1 th learning or inference process, and quantize the parameter t according to the fixed-point format having the fractional length frac_length t-1 during the t-1 th learning or inference process.
The processor 110 may update the existing fractional length frac_length t-1 to a fractional length frac_length t based on the parameter t. In other words, the processor 110 may predetermine the fractional length frac_length t for quantifying the parameter t+1 updated during the t+1st learning or inference process.
The processor 110 may update the parameter t to the parameter t+1 during the t+1st learning or inference process. In addition, the processor 110 may quantize the parameter t+1 according to a fixed point format having a fractional length frac_length t to generate a quantized parameter q_parameter t+1.
In fig. 5, according to an example, parameters parameter t-1、parametert and parameter t+1 may be data tensors, which are, for example, weights or input/output activations on the same layer in a neural network, and according to another example, weights or input/output activations on the same channel in the neural network. Further, t may be a natural number of 1 or more, and when t is 1, a fixed-point format for quantizing the parameters updated in the first learning or inference process may be preset by the user.
Referring again to fig. 4, the processor 110 may calculate a bit shift value for each parameter, the bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter. The bit shift value may be a value indicating the extent to which the bit range representing the parameter value is outside the bit range that may be covered by the fixed point format. According to an example, in case the fixed point format may cover up to 3 bit integer parts, the bit shift value may be 3 when up to 6 bit number integer parts are needed to represent parameter values having the fixed point format. As another example, in the case where the fixed point format may cover up to 4-bit fractional parts, the bit shift value may be-2 when up to 6-bit fractional parts are required to represent parameter values having the fixed point format. The bit shift value also indicates the degree to which overflow or underflow occurs when quantizing parameters having a fixed point format. For example, when the bit shift value is 3, it can be seen that 3-bit overflow occurs, and when the bit shift value is-2, it can be seen that 2-bit underflow occurs. Thus, the processor 110 may calculate the bit-shift value of the parameter to determine the total number of occurrences of overflow or underflow associated with the parameter. Further, the processor 110 may store the bit shift value, the parameter, the number of occurrences of overflow, and the number of occurrences of underflow in the memory 120.
Processor 110 may update the fixed point format using the bit-shifted values of the parameters. According to an example, when the number of occurrences of overflow is greater than a predetermined value, the processor 110 may update the fixed point format by reducing the length of the fractional part in the fixed point format by a maximum bit shift value of the bit shift values. According to another example, when the number of occurrences of overflow is greater than a predetermined value, the processor 110 may update the fixed point format using a maximum bit shift value and a minimum bit shift value among the bit shift values. Further, the processor 110 may store information regarding the updated fixed-point format in the memory 120.
The processor 110 may then quantify the parameters updated during the learning or inference process according to the updated fixed-point format. In particular, the processor 110 may quantify the parameter updated during the t+1st learning or inference process according to a fixed point format updated based on the bit shift value of the parameter updated during the t-th learning or inference process.
Accordingly, the processor 110 can update the fixed-point format using the maximum bit shift value and the number of occurrences of overflow of the parameter, thereby updating the fixed-point format through relatively simple operations with less operations, and thus hardware overhead can be reduced.
Fig. 6 shows an example in which the processor 110 uses the bit-shifted values of the parameters to update the fixed point format.
The processor 110 may calculate a bit shift value for each of the parameters parameter t (1) through parameter t (i) (where i is a natural number greater than or equal to 2) updated during the t-th learning or inference process. Specifically, the processor 110 may calculate a bit shift value of each of the parameters parameter t (1) to parameter t (i) (where i is a natural number greater than or equal to 2) with respect to a bit range of a fixed-point format having a predetermined fractional length frac_length t-1.
Fig. 7 shows an example in which the processor 110 calculates a bit shift value of the i-th parameter t (i).
The processor 110 may calculate a bit shift value of the i-th parameter t (i) among the parameters updated in the t-th learning or inference process.
The processor 110 may detect the most significant bit in the i-th parameter t (i) having a value of "1". Referring to fig. 7, the processor 110 may detect the sixth bit of the integer part in the i-th parameter t (i) as the most significant bit.
Subsequently, the processor 110 may determine a difference in bit number between the most significant bit of the integer part having the fractional length frac_length t-1 of the ith parameter t (i) and the most significant bit of the previously detected ith parameter t (i) as a bit-shift value of the ith parameter t (i). Referring to fig. 7, since the third bit of the integer part of the fixed point format having the fractional length frac_length t-1 is the most significant bit, the difference in the number of bits between the sixth bit and the third bit is 3, and thus the processor 110 may calculate the bit shift value of the i-th parameter t (i) as 3. A bit shift detector for detecting a difference in the number of bits may be included in the processor 110. In addition, the processor 110 may determine that a 3-bit overflow has occurred with respect to the i-th parameter t (i).
The processor 110 may quantize the i-th parameter t (i) according to a fixed point format having a fractional length frac_length t-1 to generate a quantized parameter q_parameter t (i) to calculate a bit-shift value of the i-th parameter t (i) during quantization of the i-th parameter t (i). In other words, the processor 110 may perform a process of calculating the bit shift value of the i-th parameter t (i) together with a process of quantizing the i-th parameter t (i).
In fig. 7, the i-th parameter t (i) updated in the t-th learning or inference process is shown as a 48-bit fixed-point value, and the quantized parameter q_parameter t (i) is shown as a fixed-point value having a bit width of 8 and a fractional length of 4, but is not limited thereto.
Fig. 8 shows an example in which the processor 110 detects the most significant bit having "1" in the parameter t (i).
The processor 110 may sequentially read the bit values in the bit-down direction starting from the most significant bit in the parameter t (i) to detect the most significant bit with a "1" in the parameter t (i).
According to another example, the processor 110 may read bits within a predetermined range relative to predetermined bits in the parameter t (i) to detect the most significant bit in the parameter t (i) that has a "1". For example, as shown in fig. 8, the processor 110 may read bits in the 8-bit or 16-bit range related to the most significant bit of the integer part of the fixed-point format having the fractional length frac_length t-1 to detect the most significant bit having "1" in the parameter t (i). Thus, the processor 110 may read bits within a predetermined range, instead of all bits in the parameter t (i), thereby reducing the hardware overhead for scanning.
Referring again to fig. 6, the processor 110 may determine a maximum bit shift value and the occurrence number of overflows according to the bit shift values of the parameters parameter t (1) to parameter t (i). Specifically, the processor 110 may determine the maximum value of the bit shift values as the maximum bit shift value and the number of bit shift values corresponding to the positive number of the bit shift values as the occurrence frequency of overflow.
Processor 110 may then update the fixed point format based on the maximum bit shift value and the number of occurrences of overflow. In other words, the processor 110 may update the fixed point format having the fractional length frac_length t-1 to the fixed point format having the fractional length frac_length t based on the maximum bit shift value and the number of occurrences of overflow. Specifically, when the number of occurrences of overflow is greater than a specific value, the processor 110 may decrease the fractional length frac_length t-1 by the maximum bit-shift value and update the fractional length from the existing fractional length frac_length t-1 to a new fractional length frac_length t.
Thus, the processor 110 may quantify the parameters updated during the t+1st learning or inference process according to a fixed-point format having a fractional length frac_length t.
Fig. 9 shows an example in which the processor 110 determines the maximum bit shift value and the number of occurrences of overflow.
Similar to the logical operation shown in fig. 9, the processor 110 may determine the maximum bit shift value and the number of occurrences of overflow by performing a logical operation on a total of N parameters, which are parameters updated in the t-th learning or inference process. The processor 110 may include a logic operator for performing the logic operations shown in fig. 9.
In operation s910, the processor 110 may determine whether the bit shift value bit_shift (i) of the i-th parameter among the parameters is greater than 0. In other words, the processor 110 may determine whether the ith parameter is a parameter corresponding to overflow. When the bit shift value bit_shift (i) of the i-th parameter is greater than 0, the processor 110 may increase the number of occurrences of overflow by 1 (s 920). Thereafter, the processor 110 may determine whether the bit shift value bit_shift (i+1) of the i+1th parameter is greater than 0 to determine the number of occurrences of overflow. As a result, the processor 110 may sequentially determine whether each bit shift value of the N parameters updated during the t-th learning or inference process is greater than 0, and determine the total number of occurrences of overflow during the t-th learning or inference process.
In operation s930, the processor 110 may compare the bit shift value bit_shift (i) of the i-th parameter with the existing maximum bit shift value max_bit_shift t. When the bit shift value bit_shift (i) is greater than the existing maximum bit shift value max_bit_shift t, the processor 110 may update the bit shift value bit_shift (i) to the maximum bit shift value max_bit_shift t (s 940). The processor 110 may then determine whether to update the maximum bit shift value max_bit_shift t by comparing the bit shift value bit_shift (i+1) of the i+1th parameter with the updated maximum bit shift value max_bit_shift t. As a result, the processor 110 may compare the bit shift values of the N parameters updated in the t-th learning or inference process with each other, thereby determining the maximum value of the bit shift values as the maximum bit shift value max_bit_shift t.
The processor 110 may determine a minimum bit shift value corresponding to an underflow among the bit shift values of the N parameters. In particular, the processor 110 may compare the bit shift values of the N parameters with each other to determine a minimum bit shift value, which is the minimum value of the bit shift values having a value less than or equal to 0.
Fig. 10 is a diagram illustrating an algorithm for the processor 110 to update the fractional length frac_length t-1 of the fixed-point format according to an example.
In operation s1010, the processor 110 may obtain a maximum bit shift value max_bit_shift t, a total number of occurrences of overflow #of overflow t, a fractional length frac_length t-1 of a fixed-point format for quantizing parameters updated in a t-TH learning or inference process, a total number N of parameters updated in a t-TH learning or inference process, and an outlier data rate TH. The processor 110 may determine the maximum bit shift value max_bit_shift t, the total number of occurrences of overflow #of overflow t by the logical operation shown in fig. 9. Further, the user may set the outlier data rate TH to any number between 0 and 1.
In operation s1020, when the total number of occurrences of overflow #of flow t is greater than Nx TH, the processor 110 may update the fractional length frac_length t-1 through equation 1030. In equation 1030, a represents a specific weight. For example, when a is 1, the processor 110 may perform an operation of subtracting the maximum bit shift value max_bit_shift t from the fractional length frac_length t-1 to update the fractional length frac_length t-1. In other words, the processor 110 may perform an operation of subtracting the maximum bit shift value max_bit_shift t from the fractional length frac_length t-1 to update the fractional length from the existing fractional length frac_length t-1 to a new fractional length frac_length t. Thus, the processor 110 may determine the fractional length frac_length t of the fixed-point format used to quantify the parameters updated during the t+1st learning or inference process.
When the overflow total #of overflow t is not greater than Nx TH, the processor 110 may not update and maintain the fractional length frac_length t-1. In other words, the processor 110 may determine the fractional length frac_length t-1 for quantizing the parameters updated in the t-th learning or inference process as the fractional length frac_length t for quantizing the parameters updated in the t+1th learning or inference process. As a result, the processor 110 may set the outlier data rate TH to a value between 0 and 1, and thus may not update the fractional length frac_length t-1 again when determining that the total number of overflows #of overflow t is relatively small compared to the total number of parameters N.
Fig. 11 is a diagram showing an algorithm for the processor 110 to update the fractional length frac_length t-1 of the fixed-point format according to another example.
Operations s1110 and s1120 of fig. 11 may correspond to operations s1010 and s1020 of fig. 10, and thus redundant description thereof will be omitted.
In operation s1110, the processor 110 may additionally obtain a minimum bit shift value min_bit_shift t in addition to the signal obtained in operation s1010 of fig. 10. In other words, the processor 110 may obtain a minimum bit shift value min_bit_shift t corresponding to an underflow in the bit shift values of the parameters updated in the t-th learning or inference process.
In operation s1120, when the overflow total #of overflow t is greater than NXTH, the processor 110 may update the fractional length frac_length t-1 through equation 1130. In equation 1130, a and b are values representing specific weights, and the symbol [ x ] is a gaussian function symbol representing the maximum integer not exceeding x. For example, the user may set a greater weight to a than to b to determine the fractional length frac_length t.
Fig. 12 is a block diagram of an electronic system 1200 according to an example.
Referring to fig. 12, the electronic system 1200 may extract effective information by analyzing input data in real time based on a neural network, determine a condition based on the extracted information, or control elements of an electronic device on which the electronic system 1200 is mounted. For example, the electronic system 1200 may be applied to robotic devices (e.g., unmanned aerial vehicles, advanced Driving Assistance Systems (ADAS), etc.), smart TVs, smart phones, medical devices, mobile devices, image display devices, measurement devices, ioT devices, etc., and may also be mounted on at least one of various types of electronic devices.
Electronic system 1200 can include processor 1210, RAM 1220, neural network device 1230, memory 1240, sensor module 1250, and communication (TX/RX) module 1260. The electronic system 1200 may also include input/output modules, security modules, power control devices, and the like. Some hardware components of electronic system 1200 may be mounted on at least one semiconductor chip. The neural network device 1230 may include the neural network apparatus 100 described above or a neural network-specific hardware accelerator or apparatus including the same.
Processor 1210 controls all operations of electronic system 1200. Processor 1210 may include one processor core (single core) or multiple processor cores (multi-core). Processor 1210 may process or execute programs and/or data stored in memory 1240. The processor 1210 may control the functions of the neural network device 1230 by executing programs stored in the memory 1240. Processor 1210 may be implemented by CPU, GPU, AP or the like.
RAM 1220 may temporarily store programs, data, or instructions. For example, programs and/or data stored in the memory 1240 may be temporarily stored in the RAM 1220 according to a control of the processor 1210 or the boot code. The RAM 1220 may be implemented by a memory such as a Dynamic RAM (DRAM) or a Static RAM (SRAM), etc.
The neural network device 1230 may perform an operation of the neural network based on the received input data, and generate an information signal based on a result of the operation. The neural network may include, but is not limited to, CNN, RNN, deep belief network, boltzman-machine (Boltzman) limited, and the like. The neural network device 1230 is hardware that drives the above neural network for classification, and may correspond to a neural network-specific hardware accelerator.
The information signal may comprise one of various types of recognition signals, such as a speech recognition signal, an object recognition signal, an image recognition signal, a biometric information recognition signal, etc. For example, the neural network device 1230 may receive frame data included in a video stream as input data, and generate an identification signal for an object included in an image indicated by the frame data according to the frame data. However, the present disclosure is not limited thereto, and the neural network device 1230 may receive various types of input data according to the type or function of the electronic device on which the electronic system 1200 is mounted, and generate an identification signal according to the input data.
The memory 1240 is a storage device for storing data, such as an Operating System (OS), various programs, and various data. In an embodiment, the memory 1240 may store intermediate results generated during the execution of the operations of the neural network device 1230, such as an output feature map, as an output feature list or an external feature matrix. In an embodiment, the memory 1240 may store the compressed output profile. Further, the memory 1240 may store quantized neural network data, such as parameters, weight maps, or weight lists, for use by the neural network device 1230.
The memory 1240 may be a DRAM, but the present disclosure is not limited thereto. The memory 1240 may include at least one of a volatile memory and a nonvolatile memory. The non-volatile memory may include ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, and the like. Volatile memory can include DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, feRAM or the like. In an embodiment, the memory 1240 may include at least one of HDD, SSD, CF, SD, micro SD, mini SD, xD, and memory stick.
The sensor module 1250 may collect information regarding the periphery of the electronic device on which the electronic system 1200 is installed. The sensor module 1250 may sense or receive signals from the outside of the electronic device, such as image signals, voice signals, magnetic signals, biometric signals, touch signals, etc., and convert the sensed or received signals into data. To this end, the sensor module 1250 may include at least one of various types of sensing devices, such as a microphone, an imaging device, an image sensor, a light detection and ranging (LIDAR) sensor, an ultrasonic sensor, an infrared sensor, a biological sensor, a touch sensor, and the like.
The sensor module 1250 may provide the converted data to the neural network device 1230 as input data. For example, the sensor module 1250 may include an image sensor, and may generate a video stream by photographing an external environment of the electronic device and provide consecutive data frames of the video stream to the neural network device 1230 in the order of input data. However, the present disclosure is not limited thereto, and the sensor module 1250 may provide various types of data to the neural network device 1230.
The communication module 1260 may include various wired or wireless interfaces capable of communicating with external devices. For example, the communication module 1260 may include a Local Area Network (LAN), a Wireless Local Area Network (WLAN) such as wireless fidelity (Wi-Fi), a Wireless Personal Area Network (WPAN) such as bluetooth, a wireless Universal Serial Bus (USB), zigBee, near Field Communication (NFC), radio Frequency Identification (RFID), power Line Communication (PLC), or a communication interface capable of connecting to a mobile cellular network (e.g., 3 rd generation (3G), 4 th generation (4G), long Term Evolution (LTE), etc.).
Fig. 13 is a diagram for explaining a method of operating the neural network device 100 according to the embodiment.
The method shown in fig. 13 may be performed by the elements of the neural network device 100 of fig. 4 or the electronic system 1200 of fig. 12, and redundant explanation is omitted.
In operation s1310, the neural network device 100 may calculate a bit shift value indicating a degree outside a bit range of a fixed point format for quantizing the parameters, for each parameter. Specifically, the neural network device 100 may calculate a bit shift value of each parameter updated in the t-th learning or inference process with respect to a bit range of a predetermined fixed point format based on the parameters updated in the t-1 th learning or inference process. The neural network device 100 may calculate a bit shift value of the parameter to determine a total number of occurrences of overflow or underflow of the parameter.
The neural network device 100 may detect the most significant bit having a value of "1" for each parameter, and calculate a difference in the number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed point format as a bit shift value for each parameter. Further, for each parameter, the neural network device 100 may search for bits in a predetermined range based on the most significant bit of the integer part of the fixed-point format, and detect the most significant bit having a value of "1".
In operation s1320, the neural network device 100 may update the fixed point format using the bit shift value of the parameter. The neural network device 100 may update the existing fixed-point format used to quantify the parameters in the t-th learning or inference process to a new fixed-point format. For example, when the number of occurrences of overflow is greater than a predetermined value, the neural network device 100 may update the fixed-point format by reducing the fractional length of the fixed-point format by the maximum bit shift value of the bit shift values.
In operation s1330, the neural network device 100 may quantify parameters updated in the learning or inference process according to the updated fixed-point format. Specifically, the neural network device 100 may quantize the parameter updated in the t+1st learning or inference process according to a fixed point format updated by the bit shift value of the parameter updated in the t-th learning or inference process.
According to an embodiment, the neural network device may update a fixed point format for quantifying the parameters updated in the learning or inference process according to the change of the updated parameters, thereby reducing the amount of computation and reducing the accuracy loss. In addition, when quantizing the parameters updated in the t-th learning or inference process, the neural network device may quantize the parameters according to a predetermined fixed-point format, thereby reducing the time to scan all the parameters updated in the t-th learning or inference process and the hardware overhead for determining the fixed-point format. In addition, since the neural network device updates the fixed-point format using the number of occurrences of overflow and the maximum bit shift value of the parameter, the neural network device can update the fixed-point format through relatively simple and small-scale calculation, thereby reducing hardware overhead.
The apparatus described herein may include a processor, memory (persistent storage such as a disk drive) for storing program data to be executed by the processor, a communication port for handling communications with external devices, and a user interface device (including a display, keys, etc.). When software modules are involved, they may be stored as program instructions or computer readable code executable by a processor on non-transitory computer readable media such as read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium may be readable by a computer, may be stored in a memory, and may be executed by a processor.
Embodiments may be described in the context of functional block components and various processing steps. Such functional blocks may be implemented by any number of hardware and/or software components configured to perform the specified functions. In embodiments, the present disclosure may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, etc., which may perform various functions under the control of one or more microprocessors or other control devices. Similarly, where elements of the disclosure are implemented using software programming or software elements, the disclosure may be implemented using any programming or scripting language (e.g., C, C ++, java, assembly language, etc.) as well as various algorithms that are implemented using any combination of data structures, objects, processes, routines, or other programming elements. The functional aspects may be implemented as algorithms executing on one or more processors. Further, the present disclosure may employ any number of techniques depending on the relevant fields for electronic configuration, signal processing and/or control, data processing, and the like. The terms "mechanism" and "element" are used broadly and are not limited to mechanical or physical embodiments, but may include software routines in combination with a processor or the like.
The particular implementations shown and described herein are illustrative examples of the present disclosure and are not intended to otherwise limit the scope of the present disclosure in any way. For brevity, electronic, control systems, software development and other functional aspects of the systems (and components of the individual operating components of the systems) according to the related art may not be described in detail. Furthermore, the connecting lines or connectors shown in the various figures presented are intended to represent exemplary functional relationships and/or physical or logical couplings between the various elements. It should be noted that there may be many alternative or additional functional relationships, physical connections or logical connections in a practical device.
The use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosure (especially in the context of the following claims) are to be construed to cover both the singular and the plural. Furthermore, recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. Finally, the steps of all methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., "such as (e.g)") provided herein, is intended merely to better illuminate the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. Various modifications and adaptations may become apparent to those skilled in the relevant art without departing from the spirit and scope of the disclosure.
It is to be understood that the embodiments described herein should be considered in descriptive sense only and not for purposes of limitation. The description of features or aspects in each embodiment should generally be taken to be applicable to other similar features or aspects in other embodiments.
Although one or more embodiments have been described with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope as defined by the following claims.
While this disclosure includes particular examples, it will be obvious to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered as illustrative only and not for the purpose of limitation. The descriptions of features or aspects in each example should be considered as applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques were performed in a different order and/or if components in the described systems, architectures, devices or circuits were combined in a different manner and/or replaced or supplemented by other components or their equivalents. Therefore, the scope of the present disclosure is defined not by the detailed description but by the claims and their equivalents, and all changes within the scope of the claims and their equivalents should be construed as being included in the present disclosure.

Claims (17)

1. A processor-implemented method of quantifying parameters of a neural network for performing speech recognition and/or image recognition, the method comprising:
for each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter;
updating the fixed point format based on the calculated bit shift value of the parameter, and
The parameters updated during the learning or inference process are quantized according to the updated fixed-point format,
Wherein calculating the bit shift value comprises:
For each parameter, detecting the most significant bit having a value of "1", and
For each parameter, a difference in number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed point format is determined as the bit shift value.
2. The method of claim 1, wherein detecting the most significant bit comprises:
for each parameter, bits within a predetermined range are searched based on the most significant bits of the integer part of the fixed point format, and the most significant bits having a value of "1" are detected.
3. The method of claim 1, wherein updating the fixed point format comprises:
Determining the number of occurrences of overflow and a maximum bit shift value based on the calculated bit shift value, and
The fixed point format is updated based on the number of occurrences of the overflow and the maximum bit shift value.
4. The method of claim 3, wherein updating the fixed point format comprises:
In case that the number of occurrences of overflow is greater than a predetermined value, the fixed point format is updated by decreasing the fractional length of the fixed point format by the maximum bit shift value.
5. The method of claim 4, wherein the predetermined value is based on a number of parameters.
6. The method according to claim 1,
Wherein the updated parameter is the parameter updated during the t+1st learning or inference,
Wherein the parameter is a parameter updated during a t-th learning or inference process,
Wherein the fixed-point format is a fixed-point format updated based on parameters updated in the t-1 th learning or inference process, and
T is a natural number greater than or equal to 2.
7. The method of claim 1, wherein calculating the bit shift value comprises:
the bit shift value of each parameter is calculated during quantization of the parameter according to the fixed point format.
8. The method of claim 1, wherein the parameter is a weight or activation on the same layer in the neural network.
9. A computer-readable recording medium storing a program for causing a computer to execute the method according to claim 1.
10. An apparatus for quantifying parameters of a neural network for performing speech recognition and/or image recognition, the apparatus comprising:
a memory storing at least one program, and
A processor configured to execute the at least one program by:
for each parameter, calculating a bit shift value indicating a degree outside a bit range of a fixed point format used to quantize the parameter;
Updating the fixed point format using the calculated bit shift value of the parameter, and
The parameters updated during the learning or inference process are quantized according to the updated fixed-point format,
Wherein the processor is configured to:
For each parameter, detecting the most significant bit having a value of "1", and
For each parameter, a difference in number of bits between the detected most significant bit and the most significant bit of the integer part of the fixed point format is determined as the bit shift value.
11. The apparatus of claim 10, wherein the processor is configured to search for bits within a predetermined range based on the most significant bits of the integer portion of the fixed point format and to detect the most significant bits having a value of "1".
12. The apparatus of claim 10, wherein the processor is configured to determine a number of occurrences of overflow and a maximum bit shift value from the calculated bit shift value, and to update the fixed point format using the number of occurrences of overflow and the maximum bit shift value.
13. The apparatus of claim 12, wherein the processor is configured to update the fixed point format by reducing a fractional length of the fixed point format by the maximum bit shift value if the number of occurrences of overflow is greater than a predetermined value.
14. The apparatus of claim 13, wherein the predetermined value is based on a number of parameters.
15. The device according to claim 10,
Wherein the updated parameter is the parameter updated during the t+1st learning or inference,
Wherein the parameter is a parameter updated during a t-th learning or inference process,
Wherein the fixed-point format is a fixed-point format updated based on parameters updated in the t-1 th learning or inference process, and
T is a natural number greater than or equal to 2.
16. The apparatus of claim 10, wherein the processor is configured to calculate the bit shift value for each parameter during quantization of the parameter according to the fixed point format.
17. The apparatus of claim 10, wherein the parameter is a weight or activation on a same layer in the neural network.
CN201910822654.5A 2018-10-17 2019-08-30 Method and apparatus for quantizing parameters of a neural network Active CN111062475B (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020180123927A KR102917770B1 (en) 2018-10-17 2018-10-17 Method and apparatus for quantizing neural network parameters
KR10-2018-0123927 2018-10-17

Publications (2)

Publication Number Publication Date
CN111062475A CN111062475A (en) 2020-04-24
CN111062475B true CN111062475B (en) 2025-05-02

Family

ID=67777176

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910822654.5A Active CN111062475B (en) 2018-10-17 2019-08-30 Method and apparatus for quantizing parameters of a neural network

Country Status (5)

Country Link
US (1) US12026611B2 (en)
EP (1) EP3640858B1 (en)
JP (1) JP7117280B2 (en)
KR (1) KR102917770B1 (en)
CN (1) CN111062475B (en)

Families Citing this family (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6540725B2 (en) * 2017-01-30 2019-07-10 富士通株式会社 Arithmetic processing device, method, and program
US10592799B1 (en) * 2019-01-23 2020-03-17 StradVision, Inc. Determining FL value by using weighted quantization loss values to thereby quantize CNN parameters and feature values to be used for optimizing hardware applicable to mobile devices or compact networks with high precision
US11783917B2 (en) 2019-03-21 2023-10-10 Illumina, Inc. Artificial intelligence-based base calling
US11210554B2 (en) 2019-03-21 2021-12-28 Illumina, Inc. Artificial intelligence-based generation of sequencing metadata
US11423306B2 (en) 2019-05-16 2022-08-23 Illumina, Inc. Systems and devices for characterization and performance analysis of pixel-based sequencing
US11593649B2 (en) 2019-05-16 2023-02-28 Illumina, Inc. Base calling using convolutions
JP7272121B2 (en) * 2019-06-07 2023-05-12 富士通株式会社 Arithmetic processing device, control method, and control program
JP7371499B2 (en) * 2020-01-07 2023-10-31 富士通株式会社 Arithmetic processing unit, control method for the arithmetic processing unit, and arithmetic processing program
JP2021124849A (en) * 2020-02-04 2021-08-30 富士通株式会社 Arithmetic processing device, arithmetic processing device control method, and arithmetic processing program
US12592298B2 (en) 2020-02-20 2026-03-31 Illumina, Inc. Hardware execution and acceleration of artificial intelligence-based base caller
US12354008B2 (en) 2020-02-20 2025-07-08 Illumina, Inc. Knowledge distillation and gradient pruning-based compression of artificial intelligence-based base caller
US11861467B2 (en) * 2020-03-05 2024-01-02 Qualcomm Incorporated Adaptive quantization for execution of machine learning models
FR3109651B1 (en) * 2020-04-23 2022-12-16 St Microelectronics Rousset METHOD FOR IMPLEMENTING AN ARTIFICIAL NEURON NETWORK IN AN INTEGRATED CIRCUIT
US12572797B2 (en) 2020-05-19 2026-03-10 Samsung Electronics Co., Ltd. Electronic apparatus implementing artificial intelligence model having low resource utilization and control method thereof
EP4177795B8 (en) * 2020-07-01 2025-10-29 NTT, Inc. Data processing device and data processing method
JP7524667B2 (en) * 2020-08-19 2024-07-30 富士通株式会社 Information processing device, machine learning method, and machine learning program
US12400103B2 (en) * 2021-03-05 2025-08-26 Qualcomm Incorporated Variable quantization for neural networks
US12525320B2 (en) 2021-03-16 2026-01-13 Illumina, Inc. Neural network parameter quantization for base calling
WO2022197752A1 (en) * 2021-03-16 2022-09-22 Illumina, Inc. Tile location and/or cycle based weight set selection for base calling
WO2022201352A1 (en) * 2021-03-24 2022-09-29 三菱電機株式会社 Inference device, inference method, and inference program
US12217829B2 (en) 2021-04-15 2025-02-04 Illumina, Inc. Artificial intelligence-based analysis of protein three-dimensional (3D) structures
US12444482B2 (en) 2021-04-15 2025-10-14 Illumina, Inc. Multi-channel protein voxelization to predict variant pathogenicity using deep convolutional neural networks
US12530882B2 (en) 2021-07-01 2026-01-20 Illumina, Inc. Efficient artificial intelligence-based base calling of index sequences
KR20250023867A (en) * 2023-08-10 2025-02-18 삼성전자주식회사 Display device and operating method thereof
KR102724199B1 (en) * 2023-11-28 2024-11-01 한국과학기술원 Method and system for multi-region precision quantization using scalable bias with hardware optimization

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018140294A1 (en) * 2017-01-25 2018-08-02 Microsoft Technology Licensing, Llc Neural network based on fixed-point operations

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6263022B1 (en) 1999-07-06 2001-07-17 Philips Electronics North America Corp. System and method for fine granular scalable video with selective quality enhancement
US7068717B2 (en) 2000-07-12 2006-06-27 Koninklijke Philips Electronics N.V. Method and apparatus for dynamic allocation of scalable selective enhanced fine granular encoded images
US7305539B2 (en) * 2004-10-14 2007-12-04 Sandisk Il Ltd. Adaptive booting from mass storage device
KR101885258B1 (en) 2010-05-14 2018-08-06 삼성전자주식회사 Method and apparatus for video encoding, and method and apparatus for video decoding
CN102118803A (en) * 2011-04-14 2011-07-06 北京邮电大学 Video cross-layer scheduling method of mobile communication system on basis of QoE prediction
US10803381B2 (en) 2014-09-09 2020-10-13 Intel Corporation Fixed point integer implementations for neural networks
US10373050B2 (en) 2015-05-08 2019-08-06 Qualcomm Incorporated Fixed point neural network based on floating point neural network quantization
US10262259B2 (en) 2015-05-08 2019-04-16 Qualcomm Incorporated Bit width selection for fixed point neural networks
US20160328645A1 (en) * 2015-05-08 2016-11-10 Qualcomm Incorporated Reduced computational complexity for fixed point neural network
US10083395B2 (en) 2015-05-21 2018-09-25 Google Llc Batch processing in a neural network processor
US9747546B2 (en) 2015-05-21 2017-08-29 Google Inc. Neural network processor
US10417555B2 (en) 2015-05-29 2019-09-17 Samsung Electronics Co., Ltd. Data-optimized neural network traversal
US20160358069A1 (en) 2015-06-03 2016-12-08 Samsung Electronics Co., Ltd. Neural network suppression
US10540588B2 (en) 2015-06-29 2020-01-21 Microsoft Technology Licensing, Llc Deep neural network processing on hardware accelerators with stacked memory
US11106973B2 (en) 2016-03-16 2021-08-31 Hong Kong Applied Science and Technology Research Institute Company Limited Method and system for bit-depth reduction in artificial neural networks
US20180082181A1 (en) 2016-05-13 2018-03-22 Samsung Electronics, Co. Ltd. Neural Network Reordering, Weight Compression, and Processing
CN107526709A (en) 2016-06-15 2017-12-29 辉达公司 Handled using the tensor of low precision format
US11222263B2 (en) 2016-07-28 2022-01-11 Samsung Electronics Co., Ltd. Neural network method and apparatus
US20180107451A1 (en) 2016-10-14 2018-04-19 International Business Machines Corporation Automatic scaling for fixed point implementation of deep neural networks
US10997492B2 (en) * 2017-01-20 2021-05-04 Nvidia Corporation Automated methods for conversions to a lower precision data format
JP6540725B2 (en) 2017-01-30 2019-07-10 富士通株式会社 Arithmetic processing device, method, and program
CN108364061B (en) 2018-02-13 2020-05-05 北京旷视科技有限公司 Arithmetic device, arithmetic execution apparatus, and arithmetic execution method
JP7056225B2 (en) * 2018-02-26 2022-04-19 富士通株式会社 Arithmetic processing unit, information processing unit, information processing method, and program

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018140294A1 (en) * 2017-01-25 2018-08-02 Microsoft Technology Licensing, Llc Neural network based on fixed-point operations

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Overflow Detection and Correction in a Fixed-Point Multiplier;Arthurs A等;《IEEE》;20070421;正文第82-83页、第三节 *

Also Published As

Publication number Publication date
KR102917770B1 (en) 2026-01-23
CN111062475A (en) 2020-04-24
EP3640858B1 (en) 2025-07-16
US12026611B2 (en) 2024-07-02
US20200125947A1 (en) 2020-04-23
JP2020064635A (en) 2020-04-23
KR20200043169A (en) 2020-04-27
EP3640858A1 (en) 2020-04-22
JP7117280B2 (en) 2022-08-12

Similar Documents

Publication Publication Date Title
CN111062475B (en) Method and apparatus for quantizing parameters of a neural network
CN110689109B (en) Neural network method and device
US12511535B2 (en) Neural network method and apparatus
CN109389219B (en) Method and apparatus for quantizing parameters of a neural network
CN109754066B (en) Method and apparatus for generating a fixed-point neural network
EP3474194B1 (en) Method and apparatus with neural network parameter quantization
KR20200086581A (en) Method and apparatus for neural network quantization
KR20210121946A (en) Method and apparatus for neural network quantization
KR20210083935A (en) Method and apparatus for quantizing parameters of neural network
JP7329352B2 (en) Method and apparatus for processing parameters in neural networks for classification
KR102852287B1 (en) Processing apparatus and method for processing floating point operation thereof
US12373681B2 (en) Neuromorphic method and apparatus with multi-bit neuromorphic operation
KR102581471B1 (en) Method and apparatus for processing parameter in neural network for classification
KR102896842B1 (en) Method and apparatus for quantizing parameter in neural network
KR20240035013A (en) Sparse data based convolution calculate method and apparatus using artificial neural network
KR20250065119A (en) Neural network parameter quantization method and device based on layer characteristics

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant