Tags: matrixorigin/matrixone
Tags
Set the maximum number of load retries and retry time (#22633) ### **User description** ## What type of PR is this? - [ ] API-change - [x] BUG - [x] Improvement - [ ] Documentation - [ ] Feature - [ ] Test and CI - [ ] Code Refactoring ## Which issue(s) this PR fixes: issue matrixorigin/MO-Cloud#6448 ## What this PR does / why we need it: Set the maximum number of load retries and retry time ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Added retry mechanism with max 100 retries and 3-minute timeout for load operations - Enhanced logging for long-running read/write operations (>1 minute) - Improved error handling to track consecutive failures and prevent infinite loops - Refactored timing measurements to capture read/write durations more accurately ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Read packet"] --> B["Check read time"] B --> C["Write to pipe"] C --> D["Check write time"] D --> E{"Error occurred?"} E -- "Yes" --> F["Increment error counter"] F --> G{"Max retries or timeout?"} G -- "Yes" --> H["Return error"] G -- "No" --> A E -- "No" --> I["Reset error counter"] I --> A ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>mysql_cmd_executor.go</strong><dd><code>Add retry limits and enhanced error handling for load operations</code></dd></summary> <hr> pkg/frontend/mysql_cmd_executor.go <ul><li>Added retry mechanism with maximum 100 consecutive errors and 3-minute <br>timeout<br> <li> Enhanced logging for read/write operations exceeding 1 minute duration<br> <li> Improved error tracking with <code>retError</code> variable to preserve first error<br> <li> Refactored timing measurements for more accurate read/write duration <br>capture</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/22633/files#diff-af2611d5fc89704398fe09d09644efa41fec8931b395eda292f2f474f1216275">+43/-14</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___
to 3.0: enable escape and escape the JSON control characters. (#22607) Enable escape character and escape the JSON control characters. ``` create table t1 (a int primary key, b json); insert into t1 values(1, '{"key1":"你好\\t不\\r好\\f呀\\n\\\\"}'); insert into t1 values(2, '{"key2":"谢谢\\t你，\\r我非常\\f好\\n\\\\"}'); 1. select * from t1 into outfile "/tmp/gg/t1.csv" fields terminated by "," enclosed by '\''; 2. load data infile "/tmp/gg/t1.csv" into table t1 fields terminated by "," enclosed by '\'' ignore 1 lines; ``` All JSON control characters in the JSON column of t1 should be escaped when we select data out to the outfile. Approved by: @XuPeng-SH, @heni02, @daviszhen
2.2 dev convert l2sq score tp l2 score for hnsw and ivfflat index (#2… …2542) 2.2 dev convert l2sq score tp l2 score for hnsw and ivfflat index Approved by: @fengttt, @XuPeng-SH
fix runtime filter message tag (#22515) genNewTag -> genNewMsgTag Approved by: @ouyuanning, @XuPeng-SH
allow queries selecting distance to use vector index (#22425) when SELECT and ORDER BY have the same vector distance function, index should be used. Approved by: @XuPeng-SH, @heni02, @ouyuanning
fix cdc sinker and handle new table (#22390) fix cdc sinker and handle new table Approved by: @ck89119, @daviszhen, @XuPeng-SH
Fix fuzzyfilter 2.2 (#22353) ### **User description** ## What type of PR is this? - [ ] API-change - [x] BUG - [ ] Improvement - [ ] Documentation - [ ] Feature - [ ] Test and CI - [ ] Code Refactoring ## Which issue(s) this PR fixes: issue #22340 ## What this PR does / why we need it: Fix fuzzyfilter 2.2 ___ ### **PR Type** Bug fix, Tests ___ ### **Description** - Fix fuzzyfilter runtime filter vector union operation - Update merge scheduler logging format and timing - Add regression test for issue #22340 ___ ### Diagram Walkthrough ```mermaid flowchart LR A["FuzzyFilter"] -- "fix vector union" --> B["Runtime Filter"] C["Merge Scheduler"] -- "improve logging" --> D["Log Format"] E["Test Case"] -- "add regression test" --> F["Issue #22340"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>filter.go</strong><dd><code>Fix vector union operation in runtime filter</code> </dd></summary> <hr> pkg/sql/colexec/fuzzyfilter/filter.go <ul><li>Replace <code>UnionMulti</code> with <code>UnionBatch</code> method call<br> <li> Add error handling and return value to <code>appendPassToRuntimeFilter</code><br> <li> Update function signature to return error</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/22353/files#diff-1813d802e4307f38830c0ae6969859a5555ebf13463078695b93cad9a77151fc">+9/-2</a> </td> </tr> </table></td></tr><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>scheduler.go</strong><dd><code>Improve merge scheduler logging format and timing</code> </dd></summary> <hr> pkg/vm/engine/tae/db/merge/scheduler.go <ul><li>Consolidate log event names into single string format<br> <li> Rename log field keys with hyphens for consistency<br> <li> Increase heartbeat ticker interval from 10s to 60s<br> <li> Improve log message readability across multiple functions</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/22353/files#diff-e39cd8771cc815a9c7daef3e4cb9f6a7a3a91de2ed4eb91942a389452365e113">+33/-33</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>replace.result</strong><dd><code>Add regression test results for issue #22340</code> </dd></summary> <hr> test/distributed/cases/dml/replace/replace.result <ul><li>Add test case for bug #22340 with IVF index<br> <li> Include vector table creation and REPLACE operations<br> <li> Add flush operation and cleanup steps</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/22353/files#diff-9dd838f2bf6ecb5edec00fc78c2c74ca5727e691814e685a895b1fa4d309da0f">+11/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>replace.test</strong><dd><code>Add regression test case for issue #22340</code> </dd></summary> <hr> test/distributed/cases/dml/replace/replace.test <ul><li>Add test case for bug #22340 with vector table<br> <li> Include IVF index creation and REPLACE operations<br> <li> Add table separator and cleanup commands</ul> </details> </td> <td><a href="https://github.com/matrixorigin/matrixone/pull/22353/files#diff-ea066aec8bc4f48fed8e19b698009dd1238d5785ad81799c15023f9a71c66fbc">+10/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___
PreviousNext