CN111950733A - Information flow sorting method, device and computer storage medium - Google Patents

Abstract

The embodiments of the present invention disclose a method, a device and a computer storage medium for sorting information flow. The method includes: obtaining an information flow recommendation list according to user characteristics and current state; obtaining a score for each information flow material in the information flow recommendation list according to the information flow recommendation list and information flow characteristics; The information flow in the information flow recommendation list described above is sorted. According to the embodiments of the present invention, long-term benefits can be considered in the process of information flow sorting, and historical accumulation can be maximized, thereby improving user experience.

Description

Information flow sorting method, device and computer storage medium

technical field

The present invention relates to the technical field of machine learning, and more particularly, to an information flow sorting method, device and computer storage medium.

Background technique

In the information flow short video scene, every time the user refreshes the information flow short video list, he can see many short videos arranged in order. Generally, short videos that are more likely to be of interest to users need to be displayed at the higher position in the short video list, so as to attract users to click. If the short videos that the user is most interested in cannot be ranked first, the user needs to drop down the short video list to find the most interesting short videos, which will increase the user cost. It can be seen that the ordering of information flow is very important.

In the prior art, in the ordering stage of the information flow, the click-through rate estimation model is usually used to realize the ordering of the information flow. However, the CTR estimation model mainly considers the maximum click possibility in the current scenario. That is to say, using the CTR estimation model to sort the information flow can only maintain short-term maximization, and the given information flow ordering It is not necessarily what users are most interested in, and the user experience is poor.

Therefore, the inventor believes that it is necessary to improve at least one of the problems existing in the above-mentioned prior art.

SUMMARY OF THE INVENTION

An object of the embodiments of the present invention is to provide a new technical solution for sorting information flow.

According to a first aspect of the embodiments of the present invention, there is provided a method for sorting information flow, the method comprising:

According to the user characteristics and the current state of the moment, get the information flow recommendation list;

According to the information flow recommendation list and the information flow characteristics, get the score of each information flow material in the information flow recommendation list;

Sort the information streams in the information stream recommendation list according to the scores.

Optionally, according to the user characteristics and the current state of the moment, an information flow recommendation list is obtained, including:

Calculate the evaluation value according to the current moment state, the reward value at the current moment, the next moment state, the user action and the first mapping function;

According to the user feature, the current state, the evaluation value and the second mapping function, the information flow recommendation list is obtained by calculation.

Optionally, the reward value reward at the current moment is calculated by the following formula:

为位置调权因子；N为信息流数量。Among them, click+β*read_time is the user action, β is the adjustment factor,

is the location adjustment factor; N is the number of information flows.

Optionally, the expression of the loss function critic_loss of the first mapping function is:

critic_loss=reward+gamma*v _t+1 -v _t ;

Wherein, reward is the reward value at the current moment; gamma is a smoothing factor; v _t is the current moment state; v _t+1 is the next moment state.

Optionally, the expression of the loss function actor_loss of the second mapping function is:

actor_loss=reward_gain*td_error;

Wherein, td_error is the evaluation value; reward_gain=reward-origin_reward is the reward value gain at the current moment, origin_reward is the original reward value, and reward is the reward value at the current moment.

Optionally, the method further includes:

Get log information;

updating the first mapping function according to the current state of the log information, the reward value of the current time, the state of the next time, and the user action;

The second mapping function is updated according to the user characteristics, the current state and the evaluation value in the log information.

According to a second aspect of the embodiments of the present invention, an apparatus for sorting information flow is provided, and the apparatus includes:

The acquisition module is used to obtain the information flow recommendation list according to the user characteristics and the current state of the moment;

The scoring module is used to obtain the score of each information flow material in the information flow recommendation list according to the information flow recommendation list and the information flow characteristics;

A sorting module, configured to sort the information flow in the information flow recommendation list according to the score.

Optionally, the obtaining module is specifically used for:

Calculate the evaluation value according to the current moment state, the reward value at the current moment, the next moment state, the user action and the first mapping function;

According to the user feature, the current state, the evaluation value and the second mapping function, the information flow recommendation list is obtained by calculation.

According to a third aspect of the embodiments of the present invention, an apparatus for sorting information flow is provided, the apparatus includes: a memory and a processor, where the memory is used to store instructions; the instructions are used to control the processor to operate , so as to execute the method for sorting information flow according to any one of the first aspect of the embodiments of the present invention.

According to a fourth aspect of the embodiments of the present invention, there is provided a computer storage medium on which a computer program is stored, and the computer program, when executed by a processor, implements any one of the first aspects of the embodiments of the present invention. The sorting method of the information flow described above.

One beneficial effect of the present invention is that long-term benefits can be considered in the process of information flow sorting, and historical accumulation can be maximized, thereby improving user experience.

Other features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

FIG. 1 is a block diagram showing a hardware configuration of a client 1000 that can be used to implement an embodiment of the present invention.

FIG. 2 is a schematic flowchart of a method for sorting information flows according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of constructing an AC model according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of position adjustment of an information flow according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of AC model update according to an embodiment of the present invention.

FIG. 6 is a schematic structural diagram of an information flow sorting apparatus 600 according to the present invention.

FIG. 7 is a schematic diagram of a hardware structure of an information flow sorting apparatus 700 according to another embodiment.

FIG. 8 is a schematic diagram of information flow ordering obtained without using the method of the embodiment of the present invention.

FIG. 9 is a schematic diagram of information flow ordering obtained by a method according to an embodiment of the present invention.

Detailed ways

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the invention unless specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.

In all examples shown and discussed herein, any specific values should be construed as illustrative only and not limiting. Accordingly, other instances of the exemplary embodiment may have different values.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further discussion in subsequent figures.

Hereinafter, various embodiments and examples according to embodiments of the present invention will be described with reference to the accompanying drawings.

<Hardware configuration>

FIG. 1 is a block diagram showing a hardware configuration of a client 1000 that can be used to implement an embodiment of the present invention.

As shown in FIG. 1 , the client 1000 in this embodiment may be a portable computer, a desktop computer, a mobile phone, a tablet computer, or the like.

As shown in FIG. 1, the client 1000 may include a processor 1010, a memory 1020, an interface device 1030, a communication device 1040, a display device 1050, an input device 1060, a speaker 1070, a microphone 1080, and the like. The processor 1010 may be a central processing unit CPU, a microprocessor MCU, or the like. The memory 1020 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, and the like. The interface device 1030 includes, for example, a USB interface, an earphone interface, and the like. The communication device 1040 is capable of wired or wireless communication, for example. The display device 1050 is, for example, a liquid crystal display, a touch display, or the like. The input device 1060 may include, for example, a touch screen, a keyboard, and the like. The user can input/output voice information through the speaker 1070 and the microphone 1080 .

In this embodiment, the memory 1020 of the client 1000 is used to store instructions, and the instructions are used to control the processor 1010 to operate to at least execute the method for sorting information flow according to any embodiment of the present invention. Those skilled in the art should understand that although FIG. 1 shows multiple devices of the client 1000, the present invention may only involve some of the devices. For example, the client 1000 only involves the memory 1020, the processor 1010 and the display device 1050. A skilled person can design instructions according to the solutions disclosed in the present invention. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

<method>

FIG. 2 is a schematic flowchart of a method for sorting information flows according to an embodiment of the present invention. The method may be implemented by the client 1000 .

As shown in FIG. 2 , the method for sorting information flows in this embodiment may include the following steps 2100 to 2300:

Step 2100: Obtain an information flow recommendation list according to user characteristics and current state.

The user feature refers to user preference feature information, for example, the user feature may be user interest information, user preference information, user history browsing records, and the like. The current moment status refers to the status of the user at the current moment, such as browsing the information stream list, watching a certain information stream, and so on.

The information flow recommendation list includes multiple information flows, which are calculated according to user characteristics and current state.

Specifically, when the client 1000 calculates and obtains the information flow recommendation list, the AC model in the reinforcement learning algorithm may be used as the basic model, and the model for calculating the information flow recommendation list in this embodiment is constructed by innovating on the basic model. . Among them, A in the AC model is the actor neural network, which is the Policy in Figure 3, and C in the AC model is the critic neural network, which is the Value in Figure 3.

As shown in FIG. 3 , the client 1000 can specifically calculate and obtain according to the current state S _t , the reward value r _t at the current time, the next state S _{t +1} , the user action at and the first mapping function Value Evaluation value t _d . According to the user characteristics, the current state S _t , the evaluation value t _d and the second mapping function Policy, the information flow recommendation list Environment is obtained by calculation.

Wherein, the second mapping function Policy is mainly used for decision-making behavior, and the first mapping function Value is used for evaluating the quality of the decision-making behavior, and is then fed back to the second mapping function Policy for correction. The difference of user actions at _t will lead to different sorting of information flow in the information flow recommendation list. After browsing the information flow, the user will use the click operation. Here we define a reward value, for example, the more the user clicks the reward value. higher and so on. After the user clicks or does not click on the information flow, the user's current state S _t becomes the next state S _t+1 , and the first mapping function Value will be based on the current state S _t , the current return value _rt , and the next state S t . An evaluation value t _d is calculated from the moment state S _t+1 and the user action at _t , and the evaluation value is used to evaluate the degree of conformity between the reward value _rt at the current moment and the expected reward value. In practical applications, the goal of the first mapping function Value is to minimize the evaluation value t _d , and at the same time pass the evaluation value t _d to the second mapping function Policy, so that the second mapping function Policy performs the evaluation according to the evaluation value t _d . Correction.

In this embodiment, it is defined that when a user accesses the information stream service, 8 information streams will be recommended for each refresh. The reward value here is to consider whether the user clicks on the information stream and the viewing time, and there is a position adjustment factor, that is, The higher the position of the information stream is, the larger the position adjustment factor is. In this way, the sum of the return value of each position in the 8 information streams is the overall income. Here, the overall income is defined as the original return value origin_reward . Here, the client 1000 needs to adjust the information flow with clicks and long viewing time to a more advanced position through calculation. For example, as shown in FIG. 4, the position of the information flow of R0.3 is changed from the original third position. Adjust to the first position, adjust the position of the information flow of R0.2 from the original first position to the second position, and adjust the position of the information flow of R0 from the original second position to the third position.

Specifically, the expression of the loss function critic_loss adopted by the first mapping function Value is:

critic_loss=reward+gamma*v _t+1 -v _t ; wherein, reward is the reward value at the current moment; gamma is a smoothing factor, and the value can be, for example, 0.8; v _t is the state at the current moment; v _{t+ 1} is the state at the next moment. The expression of the loss function actor_loss adopted by the second mapping function Policy is: actor_loss=reward_gain*td_error; wherein, td_error is the evaluation value; reward_gain=reward-origin_reward is the return value gain at the current moment, origin_reward is the original return value, reward is the reward value at the current moment.

Wherein, the reward value at the current moment can be calculated by the following formula:

其中，click+β*read_time为所述用户动作，用户动作包括用户点击信息流及观看时长；β为调权因子，取值例如可以是0.1；

为位置调权因子，

的取值例如可以是0.9；N为信息流数量，pos为信息流在所述信息流列表中的位置。例如，若信息流在所述信息流推荐列表中的位置为1，则对应的位置调权因子为0.9；若信息流在所述信息流推荐列表中的位置为2，则对应的位置调权因子为0.9²＝0.81。

Among them, click+β*read_time is the user action, and the user action includes the user clicks on the information flow and the viewing time; β is the adjustment factor, and the value can be, for example, 0.1;

is the position adjustment factor,

The value of , for example, may be 0.9; N is the number of information flows, and pos is the position of the information flow in the information flow list. For example, if the position of the information flow in the information flow recommendation list is 1, the corresponding position adjustment factor is 0.9; if the position of the information flow in the information flow recommendation list is 2, the corresponding position adjustment weight The factor is 0.9 ² =0.81.

Step 2200: Obtain the score of each information flow material in the information flow recommendation list according to the information flow recommendation list and the information flow feature.

The information flow feature is feature information in the information flow, for example, topic information, content information, and the like of the information flow. In this step, the client 1000 fuses the information flow recommendation list with the information flow features, and scores each information flow material in the information flow recommendation list.

Step 2300: Sort the information flows in the information flow recommendation list according to the score.

In practical applications, the information streams in the information stream recommendation list may be sorted according to the scores, and displayed on the display screen of the client 1000 .

For example, when the method of this embodiment is not used to sort the information flow, the sorting of the information flow displayed on the display screen of the client 1000 is as shown in FIG. 8 , the information flow 1 and the information flow 2 in the information flow recommendation list are , information flow 3, information flow 4 and information flow 5 are arranged and displayed in sequence. According to the method of this embodiment, after the information flow recommendation list is obtained, each information flow material in the information flow recommendation list is scored according to the information flow recommendation list and the information flow characteristics. As shown in FIG. 9 , the information flow 1 The score is 0.1, the score of information flow 2 is 0.3, the score of information flow 3 is 0.8, the score of information flow 4 is 0.2, the score of information flow 5 is 0.6, and the client 1000 recommends the information flow in the information flow list according to the score. For sorting, the order of the information flow displayed on the display screen of the client terminal 1000 is information flow 3, information flow 5, information flow 2, information flow 4, and information flow 1. It can be seen that, using the method of this embodiment, the obtained information flow ranking is of the most interest to the user, so that the user can be attracted to click and the user experience can be improved.

Further, the method for sorting information flow in this embodiment further includes: acquiring log information; according to the current state in the log information, the reward value at the current time, the state at the next time, and the user Action, update the first mapping function; update the second mapping function according to the user characteristics, the current state and the evaluation value in the log information.

As shown in FIG. 5 , the client 1000 obtains the log information Log, and after extracting, transforming, and loading (Extract, transform, load, ETL) processing the log information Log, the first mapping function and the second mapping function are updated to The AC model (AC Train) of this embodiment is updated. Provides access to the online section after completing the AC model update.

According to the information flow sorting method of this embodiment, the client obtains the information flow recommendation list according to the user characteristics and the current state of the time; according to the information flow recommendation list and the information flow characteristics, obtains the score of each information flow material in the information flow recommendation list ; Sort the information flow in the information flow recommendation list according to the score. According to the embodiments of the present invention, long-term benefits can be considered in the process of information flow sorting, and historical accumulation can be maximized, thereby improving user experience.

<device>

FIG. 6 is a schematic structural diagram of an information flow sorting apparatus 600 according to the present invention.

As shown in FIG. 6 , the information flow sorting apparatus 600 may include an obtaining module 610 , a scoring module 620 and a sorting module 630 .

Wherein, the obtaining module 610 is configured to obtain the information flow recommendation list according to the user characteristics and the current state of the time.

The scoring module 620 is configured to obtain a score for each information flow material in the information flow recommendation list according to the information flow recommendation list and the information flow feature.

The sorting module 630 is configured to sort the information flow in the information flow recommendation list according to the score.

Specifically, the obtaining module 610 can be specifically configured to: calculate and obtain the evaluation value according to the current moment state, the reward value at the current moment, the next moment state, the user action and the first mapping function; The current moment state, the evaluation value and the second mapping function are calculated to obtain the information flow recommendation list.

Wherein, the expression of the loss function critic_loss of the first mapping function is:

critic_loss=reward+gamma*v _t+1 -v _t ; wherein, reward is the reward value at the current moment; gamma is the smoothing factor; v _t is the state at the current moment; v _t+1 is the next moment state.

The expression of the loss function actor_loss of the second mapping function is:

actor_loss=reward_gain*td_error; wherein, td_error is the evaluation value; reward_gain=reward-origin_reward is the reward value gain at the current moment, origin_reward is the original reward value, and reward is the reward value at the current moment.

其中，click+β*read_time为所述用户动作，β为调权因子，

为位置调权因子；N为信息流数量。In practical applications, the reward value at the current moment can be calculated by the following formula:

Among them, click+β*read_time is the user action, β is the adjustment factor,

is the location adjustment factor; N is the number of information flows.

Further, the obtaining module 610 can also be used to: obtain log information; according to the current moment state, the reward value at the current moment, the next moment status and the user action in the log information, The first mapping function is updated; the second mapping function is updated according to the user characteristics, the current moment state and the evaluation value in the log information.

FIG. 7 is a schematic diagram of a hardware structure of an information flow sorting apparatus 700 according to another embodiment.

As shown in FIG. 7 , the apparatus 700 for sorting information flow in this embodiment may include a memory 710 and a processor 720 .

The memory 710 is used to store instructions for controlling the operation of the processor 720 to perform the method for sorting information flow according to any embodiment of the present invention.

A skilled person can design instructions according to the solutions disclosed in the present invention. How the instruction controls the processor to operate is well known in the art, so it will not be described in detail here.

The information flow sorting apparatus in this embodiment can be used to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects thereof are similar, and details are not described herein again.

<Computer storage medium>

In this embodiment, a computer storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for sorting an information flow according to any embodiment of the present invention is implemented.

Those skilled in the art should understand that in the field of electronic technology, the above method can be embodied in a product by means of software, hardware or a combination of software and hardware. Those skilled in the art can easily generate a An information processing apparatus including modules for performing respective operations in the information processing method according to the above-described embodiments.

It is well known to those skilled in the art that with the development of electronic information technology such as large-scale integrated circuit technology and the trend of software and hardware, it has become difficult to clearly demarcate the software and hardware boundaries of computer systems. Because, any operation can be implemented by software or by hardware. Execution of any instruction can be done by hardware as well as by software. The hardware implementation or software implementation for a certain machine function depends on non-technical factors such as price, speed, reliability, storage capacity, and change cycle. For the skilled person, the software implementation and hardware implementation are equivalent. Technicians can choose software or hardware to implement the above solutions as needed. Therefore, there is no specific software or hardware limitation here.

The present invention may be an apparatus, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically coded devices, such as printers with instructions stored thereon Hole cards or raised structures in grooves, and any suitable combination of the above. Computer-readable storage media, as used herein, are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (eg, light pulses through fiber optic cables), or through electrical wires transmitted electrical signals.

The computer readable program instructions described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

The computer program instructions for carrying out the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or instructions in one or more programming languages. Source or object code written in any combination, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through the Internet connect). In some embodiments, custom electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can be personalized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are all equivalent.

Various embodiments of the present invention have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or technical improvement in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims (10)

1. A method for ordering information streams, the method comprising:

obtaining an information flow recommendation list according to the user characteristics and the current time state;

obtaining the score of each information flow material in the information flow recommendation list according to the information flow recommendation list and the information flow characteristics;

and sorting the information streams in the information stream recommendation list according to the scores.

2. The method of claim 1, wherein obtaining the information flow recommendation list according to the user characteristics and the current time state comprises:

calculating to obtain an evaluation value according to the current moment state, the return value of the current moment, the next moment state, the user action and a first mapping function;

and calculating to obtain the information flow recommendation list according to the user characteristics, the current time state, the evaluation value and a second mapping function.

3. The method of claim 2, wherein the reward value reward for the current time is calculated by the following formula:

wherein click + β read _ time is the user action, β is a weighting factor,

adjusting a weight factor for the location; and N is the number of information streams.

4. The method of claim 3, wherein the loss function critic _ loss of the first mapping function is expressed as:

critic_loss＝reward+gamma*v_t+1-v_t；

wherein, reward is the return value of the current time; gamma is a smoothing factor; v. of_tThe current time state is obtained; v. of_t+1The state of the next moment.

5. The method of claim 3, wherein the loss function of the second mapping function, operator loss, is expressed as:

actor_loss＝reward_gain*td_error；

wherein td _ error is the evaluation value; the rewarded _ gain is a rewarded-origin _ rewarded value gain of the current time, the origin _ rewarded is an original rewarded value, and the rewarded is the rewarded value of the current time.

6. The method of claim 2, further comprising:

acquiring log information;

updating the first mapping function according to the current time state, the return value of the current time, the next time state and the user action in the log information;

and updating the second mapping function according to the user characteristics, the current time state and the evaluation value in the log information.

7. An apparatus for sorting a stream of information, the apparatus comprising:

the acquisition module is used for acquiring an information flow recommendation list according to the user characteristics and the current time state;

the scoring module is used for obtaining the score of each information flow material in the information flow recommendation list according to the information flow recommendation list and the information flow characteristics;

and the sorting module is used for sorting the information streams in the information stream recommendation list according to the scores.

8. The apparatus of claim 7, wherein the obtaining module is specifically configured to:

calculating to obtain an evaluation value according to the current moment state, the return value of the current moment, the next moment state, the user action and a first mapping function;

and calculating to obtain the information flow recommendation list according to the user characteristics, the current time state, the evaluation value and a second mapping function.

9. An apparatus for sorting a stream of information, the apparatus comprising: a memory for storing instructions and a processor; the instructions are for controlling the processor to operate so as to carry out the method of sorting of information streams according to any one of claims 1 to 6.

10. A computer storage medium, characterized in that a computer program is stored thereon, which computer program, when being executed by a processor, carries out the method of sorting of information flows according to any one of claims 1-6.

Images