CN115004170B - Optimized query scheduling based on data freshness requirements - Google Patents

Abstract

A method for optimizing query scheduling, comprising receiving, at an information retrieval data processing system (200), a request (110) to accelerate query execution of a specified query (120) to a time (130) before a scheduled time (190); the method includes identifying in a query a specified field (140) corresponding to data in a database (250), and retrieving a data freshness requirement (160) for the specified field and a frequency of change (150) of the data corresponding to the specified field. The method includes determining whether performing the specified query at a time prior to the scheduled time, rather than the scheduled time, violates a data freshness requirement based on a frequency of change of data corresponding to the specified field. When execution does not violate freshness, the method includes scheduling the specified query to execute at a time prior to the scheduled time.

Description

Optimized query scheduling based on data freshness requirements

technical field

The present disclosure relates to the field of query scheduling, and more particularly to the pre-scheduling of queries for execution ahead of a requested query execution time.

Background technique

A query is a request for information from an information retrieval system. There are three general approaches to formulating queries: menu-driven, query by example, and query language formulation. In the first case, a query is formulated and issued based on the selection of parameters in the menu. In the second case, the information retrieval system provides blank records and allows the end user to specify the fields and values that define the query. In the third case, end users formulate queries using stylized queries written in a query language. The latter is the most complex approach because it requires the use of a specialized language, but it is also the most powerful because it is the least constrained mode of querying an information retrieval system.

Queries are typically issued on-demand through a query interface, or programmatically during execution of a computer program. However, queries can also be issued in batch mode. That is, a query can be specified at one time, but execution of the query against the information retrieval system can be deferred until a later time. In this regard, it is common in information retrieval systems for multiple users to simultaneously submit queries to the database for execution. Therefore, if the IR system lacks sufficient computing resources to execute all submitted queries simultaneously, the IR system must defer execution of one or more of these queries and can only process a subset of the queries immediately. The process of determining which queries to defer and when to execute the deferred queries is called query scheduling.

One method of performing query scheduling is to execute incoming queries in the order in which they arrive, known as the "first-come, first-served" method. However, the first-come, first-served approach cannot distinguish between queries with different response time requirements, some queries are more time sensitive than others. If queries are simply scheduled in order of arrival, some time-sensitive queries may be forced to wait behind time-insensitive queries, which may adversely affect the availability and responsiveness of the information retrieval system.

Query scheduling can also be done with a fixed priority. In fixed-priority scheduling, each query is assigned a priority based on one or more attributes known when the query arrives, for example, the identity or type of the query requester. Thereafter, each query can be scheduled according to the assigned priority. It can be seen that fixed-priority scheduling avoids the problems of the first-come-first-served approach, as time-sensitive queries can be prioritized over less time-sensitive queries. However, fixed-priority scheduling cannot take into account "heavy" queries with relatively long execution times and "light" queries with relatively short execution times, eg, on the order of milliseconds or seconds.

Contents of the invention

Embodiments of the present disclosure address the shortcomings of the related art in query scheduling and provide a novel and non-obvious method, system and computer program product for optimized query scheduling according to data freshness requirements. In an embodiment of the present disclosure, a process for optimizing query scheduling includes receiving, in an information retrieval data processing system, a request to accelerate query execution of a specified query to a time prior to a scheduled time. The specified fields corresponding to the data in the database are then identified in the specified query. Thereafter, the data freshness requirement for the specified field and the change frequency of the data corresponding to the specified field are retrieved. Then, based on the change frequency of the data corresponding to the specified field, it is determined whether executing the specified query before the scheduled time violates the data freshness requirement. The specified query is scheduled for execution at a time prior to the scheduled time only if execution of the specified query is determined not to violate the data freshness requirement. But otherwise, keep executing the scheduled time for that specified query.

In one aspect of an embodiment, the previous time is an under-scheduled time with fewer scheduled queries consuming fewer resources of the information retrieval data processing system than the information retrieval The available resources of the data processing system at the location time or scheduling time, and there are sufficient available resources to support the execution of the specified query. In another aspect of the embodiments, the method further includes, upon determining that the specified query has an estimated execution cost below a threshold, even if the specified query is determined not to violate the data freshness requirement, still The scheduled time for executing the specified query is maintained. In yet another aspect of an embodiment, the estimated execution cost is calculated by matching at least a portion of each of the specified queries with entries in a table of query fragments and corresponding historical execution times.

In another embodiment of the present disclosure, an information retrieval data processing system is adapted for optimized query scheduling based on data freshness requirements. The system includes a host computing platform including one or more computers each having memory and at least one processor; the system also includes a query interface coupled to a database. The query interface receives a request to schedule a query against the database from a requester over a computer communication network, and in response, schedules the query for execution to return a different set of results to the requester. Finally, the system includes an optimized query scheduling module.

The module includes computer program instructions operable, when executed in the memory of the host computing platform, to receive a request to accelerate query execution of a specified query to a time prior to the scheduled time, and to identify in the specified query The named fields that correspond to data in the database. The program instructions are also capable of retrieving the data freshness requirements for the specified fields and the frequency of change of data corresponding to the specified fields, and based on the frequency of change of the data corresponding to the specified fields, determine a time before the scheduled time rather than at the scheduled time to perform the specified Query whether data freshness requirements are violated. Finally, the program instructions can schedule the specified query to be executed at a time before the scheduled time under the condition that execution of the specified query is determined not to violate the data freshness requirement by scheduling the specified query to a time before the scheduled time. But otherwise, program instructions are enabled to keep executing the specified query at said scheduled time.

Additional aspects of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosure. Aspects of the disclosure will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure, as stated.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure. The embodiments shown herein are presently preferred, however it should be understood that the disclosure is not limited to the precise arrangements and instrumentalities shown, wherein:

FIG. 1 is a schematic diagram of a process of optimizing query scheduling according to data freshness requirements;

Figure 2 is a schematic diagram of an information retrieval data processing system configured for optimized query scheduling based on data freshness requirements; and,

Figure 3 is a flowchart illustrating a process for optimizing query scheduling according to data freshness requirements.

Detailed ways

Embodiments of the present disclosure provide optimized query scheduling according to data freshness requirements of one or more queries in an information retrieval data processing system. According to an embodiment of the present disclosure, a request for accelerating query execution of a specified query from a scheduled time to a time before the scheduled time may be received in an information retrieval data processing system. In response to receipt of the request, specified fields corresponding to data in the database can be identified within the specified query. Similarly, retrieve the data freshness requirement for a specified field, and the change frequency of the data corresponding to the specified field. Thereafter, it may be determined based on the change frequency of the data corresponding to the specified field whether executing the specified query at the time before the scheduled time instead of at the scheduled time violates the data freshness requirement. In the case that execution of the specified query is determined not to violate the data freshness requirement by scheduling the specified query at a time before the scheduled time, the specified query may be scheduled for execution at a time before the scheduled time. Otherwise, however, the scheduled time for execution of that specified query can be maintained.

In a further illustration of exemplary embodiments of the present disclosure, FIG. 1 illustrates a process for optimizing query scheduling for an information retrieval data processing system according to data freshness requirements. As shown in FIG. 1, a request 110 is received requesting that the scheduling of a query 120 be accelerated to a specified time 130 earlier than the query's existing scheduled time. A field 140 of a database or data model referred to by query 120 is identified. Thereafter, a data freshness requirement 160 for a field 140 is retrieved—specifically, a previously stored indication of how recent the updated data for the field 140 must be when a query to access the data in the field 140 must be performed, whether directly part of the aggregation. Likewise, the observed volatility 150 of a field 140 is retrieved—specifically, an indication of how often the data in the field 140 has been updated in the past.

Thereafter, the freshness requirement 160 is compared to the observed volatility 150 at the specified time 130 relative to the time the request 110 has been received to determine the expected freshness of the data in the field 140 at the specified time 130 . To the extent that accelerating the scheduling of query 120 to specified time 130 would result in a violation of freshness requirement 160 , request 110 is rejected and query 120 is held in schedule 100 to execute at the previously scheduled time 190 . But otherwise, the query 120 is rescheduled in the schedule 100 to a specified time 130 . Optionally, an execution cost 170 for query 120 is determined based on at least a portion of query 120 . To the extent that the execution cost is below a threshold, no rescheduling to a specified time 130 is allowed.

The process described in connection with FIG. 1 may be implemented within information retrieval data processing system 200 . In further illustration, Figure 2 schematically illustrates an information retrieval data processing system configured to optimize query scheduling. The system includes a host computing platform 210 that includes one or more computers, each computer having memory and at least one processor. The system also includes a query interface 260 to a database 250 (or a data model modeling the data in the database 250). Query interface 260 is configured to receive queries from requesters 240 (e.g., query clients) respectively executing in different computing devices 230 via computer communication network 220, and to schedule execution of each received query in query scheduler 270, Each of these queries is assigned a specified time (day/date/time or any combination thereof) to execute. The query interface 260 is also configured to provide corresponding results of submitted and executed queries to requesters in the query client 240 .

Importantly, the system includes an optimized query scheduler module 300 . Module 300 includes computer program instructions that, when executed in host computing platform 210 , are capable of receiving respective requests from query client 240 to expedite execution of a given query to a time prior to a previously scheduled time. The computer program instructions also enable, when executed, to consult an execution cost table 290 associating different query portions with known execution costs to identify entries in the table 290 that match a portion of a specified query in order to predict the execution cost of the specified query.

The computer program instructions are further enabled during execution to the extent that the predicted execution cost exceeds a threshold to warrant an acceleration of the schedule to identify a field involved by a specified query that is included in the data freshness table 280 when executing a query involving that field Determine the known volatility of the data for the identified fields and the desired data freshness for the identified fields in . Finally, the computer program instructions are enabled during execution to accelerate the scheduling of the execution of the query to the previously scheduled time in query scheduling 270, so long as the determined volatility of the specified query does not produce data with a freshness value below the freshness requirement, For example, data may change after the requested time and before the previously scheduled time. But otherwise, computer program instructions are enabled to deny expedited scheduling requests in query scheduling 270 .

In a further illustration of the operation of the optimized query scheduler module 300, FIG. 3 is a flowchart illustrating a method of optimized query scheduling for an information retrieval data processing system. Beginning at block 310, a request to expedite scheduling of a query to a time prior to a previously scheduled time is received. In block 320, the query referred to by the request is identified along with the specified rescheduling time. In block 330, the field involved in the query is selected, and in block 340, the freshness requirements for that field are retrieved.

Then, in block 350, the observed volatility for the field is also retrieved, and in block 360, based on the observed volatility, the freshness of the data for the field at the specified time is predicted. In this regard, a data freshness table may be continuously or periodically updated as a measure of how often data is updated for different fields in a database or data model. In decision block 370, the predicted freshness is compared to the freshness requirement to determine whether rescheduling the query to the specified time would violate the freshness requirement. That is, a violation will occur if it is determined that the field's data is predicted to be updated in the time period between the specified time and the previously scheduled time such that the data is too old and not fresh at the specified time. If not, the query is rescheduled to the specified time before the previously scheduled time. But otherwise, at block 390, the request is denied.

The present disclosure can be embodied in a system, method, computer program product, or any combination thereof. A computer program product may include a computer-readable storage medium or a medium having computer-readable program instructions thereon for causing a processor to perform aspects of the present disclosure. A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.

The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing/processing device, or downloaded to an external computer or external storage device via a network. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer-readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that instructions executed by the processor of the computer or other programmable data processing apparatus create a process for implementing the flowchart and/or the means of the function/action specified in the block diagram block. These computer-readable program instructions can also be stored in a computer-readable storage medium, and the computer-readable storage medium can guide a computer, a programmable data processing device, and/or other equipment to operate in a specific manner, so that the computer with the instructions stored therein can An article of manufacture that reads a storage medium includes instructions for implementing aspects of the functions/acts specified in the flowcharts and/or block diagram blocks.

Computer-readable program instructions can also be loaded onto computers, other programmable data processing devices, or other devices, so that a series of operation steps can be executed on the computer, other programmable devices, or other devices to produce a computer-implemented process, so that the computer , other programmable devices, or instructions executed on other equipment to implement the functions/actions specified in the flowchart and/or block diagram blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a special purpose hardware-based system that performs the specified function or action, or a combination of special purpose hardware and computer instructions to implement.

Finally, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that when used in this specification, the terms "comprising" and/or "comprising of" designate the presence of stated features, integers, steps, operations, elements and/or components, but not Excludes the presence or addition of one or more other characteristics, integers, steps, operations, elements, components and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed . The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of this disclosure. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

Having thus described the disclosure of the present application in detail and by referring to the examples thereof, it will be apparent that modifications and changes may be made without departing from the scope of the disclosure as defined in the appended claims.

Claims (12)

1. A method for optimizing query scheduling, characterized in that the method comprises: At an information retrieval data processing system configured to manage queries across a plurality of different computing devices, receiving a query acceleration request to accelerate execution of a query previously scheduled for execution at a scheduled time to a time prior to the scheduled time; identifying specified fields from said query previously scheduled, said specified fields corresponding to data in a database; retrieving a data freshness requirement for said specified field from a previously scheduled query and a frequency of change of said data corresponding to said specified field from a previously scheduled query; determining whether performing the previously scheduled query at the time prior to the scheduled time instead of the scheduled time violates the data freshness requirement based on the frequency of change of the data corresponding to the specified field; as well as rescheduling the previously scheduled query on the condition that execution of the previously scheduled query is determined not to violate the data freshness requirement by scheduling the previously scheduled query at the time prior to the scheduled time The query executes at said time prior to said scheduled time, but otherwise keeps executing said scheduled time at which said query was previously scheduled. 2. The method of claim 1, wherein said preceding time is an underscheduled time with fewer scheduled queries consuming resources of said information retrieval data processing system less than the available resources of said information retrieval data processing system at the time of location and sufficient of said available resources to support a previously scheduled execution of said query. 3. The method of claim 1, further comprising, upon determining that the previously scheduled query has an estimated execution cost below a threshold, even if the previously scheduled query is determined not to violate The data freshness requirement also still maintains the scheduled time for executing the previously scheduled query. 4. The method of claim 3, wherein the estimate is computed by matching at least a portion of each previously scheduled query with entries in a table of query fragments and corresponding historical execution times Execution costs. 5. An information retrieval data processing system suitable for optimized query scheduling, characterized in that the system includes: a host computing platform comprising one or more computers each having memory and at least one processor; a query interface coupled to the database and configured to: receiving a request to schedule a query against the database from a requestor executing a plurality of different computing devices over a communications network, and scheduling the query for execution and returning a distinct set of results in response to the query to the requester; and An optimized query scheduling module, the optimized query scheduling module comprising computer program instructions, when the computer program instructions are executed in the memory of the host computing platform, performing operations comprising: receiving, in an information retrieval data processing system, a query acceleration request for accelerating execution of a query previously scheduled for execution at a scheduled time to a time prior to the scheduled time; identifying specified fields from said query previously scheduled, said specified fields corresponding to data in a database; retrieving a data freshness requirement for said specified field from a previously scheduled query and a frequency of change of said data corresponding to said specified field; determining whether performing the previously scheduled query at the time prior to the scheduled time instead of the scheduled time violates the data freshness requirement based on the frequency of change of the data corresponding to the specified field; as well as rescheduling the previously scheduled query on the condition that execution of the previously scheduled query is determined not to violate the data freshness requirement by scheduling the previously scheduled query at the time prior to the scheduled time The query executes at said time prior to said scheduled time, but otherwise keeps executing said scheduled time at which said query was previously scheduled. 6. The system of claim 5, wherein said preceding time is an underscheduled time with fewer scheduled queries consuming resources of said information retrieval data processing system less than the available resources of said information retrieval data processing system at the time of location and sufficient of said available resources to support a previously scheduled execution of said query. 7. The system of claim 5, wherein the program instructions are further such that if the previously scheduled query is determined to have an estimated execution cost below a threshold, even if the previously scheduled query is determined to In order not to violate the data freshness requirement, the scheduled time for executing the previously scheduled query is still maintained. 8. The system of claim 7, wherein the estimate is computed by matching at least a portion of each previously scheduled query with entries in a table of query fragments and corresponding historical execution times Execution costs. 9. A computer storage medium, wherein a computer program is stored on the computer storage medium, and the computer program is executed by a computer to implement the method for optimizing query scheduling according to claim 1. 10. The computer storage medium of claim 9 , wherein the preceding time is an underscheduled time with fewer scheduled queries consuming the information retrieval data processing system resources are less than the resources available to the information retrieval data processing system at the fix time and there are sufficient resources available to support the previously scheduled execution of the query. 11. The computer storage medium of claim 9 , wherein the method further comprises, upon determining that the previously scheduled query has an estimated execution cost below a threshold, even if the previously scheduled query is determined not to violate the data freshness requirement while still maintaining the scheduled time for execution of the previously scheduled query. 12. The computer storage medium of claim 11 , wherein the calculated query is calculated by matching at least a portion of each previously scheduled query with entries in a table of query fragments and corresponding historical execution times. Estimated implementation costs described above.

Images