Skip to content

运行中集群 Sync internal error 0x800009ff and stop sender 如何排查 #33240

@Sunshow

Description

@Sunshow

Bug Description
集群运行一两个月了突然无法查询,排查所有连接都异常且堆在了某个节点,由于生产系统直接重启该节点后恢复

taosd 日志:
10/14 09:13:59.538753 00000905 C VND ERROR vgId:185, msg:0xfff888053458, failed to process since Sync internal error, type:sync-snapshot-rsp, QID:0x0:0x4d381e04801941fe
10/14 09:13:59.548794 00000905 C SYN ERROR vgId:185, snapshot sender receive error:Sync internal error 0x800009ff and stop sender, sync:leader, snap-sender:0xfff8a95d9fd0 signature:(31, 3423837308), {start:348708567 end:1088581106 last-index:1088581106 last-term:31 last-cfg:-1, seq:0, ack:-2, buf:[1 0, 1], finish:0, as:0, to-dnode:1}, term:31, commit-index:1092066882, firstver:1084419370, lastver:1092066882, min-match:1092066773, snap:{last-index:1088581106, term:31}, standby:0, batch-sz:1, replicas:3, last-cfg:0, chging:0, restore:1, quorum:2, peer:{0:-1 0, 1:-1 0, 2:-1 0}, cfg:{num:3, as:2, [kdm-1:6030, kdm-3:6030, kdm-4:6030]}
10/14 09:13:59.548820 00000905 C VND ERROR vgId:185, msg:0xfffbac038c98, failed to process since Sync internal error, type:sync-snapshot-rsp, QID:0x0:0x4d381e048019420f
10/14 09:13:59.556326 00000905 C SYN ERROR vgId:185, snapshot sender receive error:Sync internal error 0x800009ff and stop sender, sync:leader, snap-sender:0xfff8a95d9fd0 signature:(31, 3423837315), {start:348708567 end:1088581106 last-index:1088581106 last-term:31 last-cfg:-1, seq:0, ack:-2, buf:[1 0, 1], finish:0, as:0, to-dnode:1}, term:31, commit-index:1092066885, firstver:1084419370, lastver:1092066885, min-match:1092066773, snap:{last-index:1088581106, term:31}, standby:0, batch-sz:1, replicas:3, last-cfg:0, chging:0, restore:1, quorum:2, peer:{0:-1 0, 1:-1 0, 2:-1 0}, cfg:{num:3, as:2, [kdm-1:6030, kdm-3:6030, kdm-4:6030]}
10/14 09:13:59.556351 00000905 C VND ERROR vgId:185, msg:0xfff870076da8, failed to process since Sync internal error, type:sync-snapshot-rsp, QID:0x0:0x4d381e0480294220

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Type in '....'
  3. See error

Expected Behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: 银河麒麟V10
  • 128G内存,鲲鹏920, 12TB free
  • TDengine Version [3.3.7.5]

Additional Context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions