Skip to content

server id lease not released when tidb exit #64191

@AmoebaProtozoa

Description

@AmoebaProtozoa

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. use tiup playground to create test cluster (latest 8.5.3)
  2. confirm server id and lease id

via tidb log:

[2025/10/30 15:27:27.322 +08:00] [INFO] [domain.go:3096] [acquireServerID] [serverID=624] ["lease id"=22319a3403996cf7]

via etcd ctl

❯ etcdctl --endpoints 127.0.0.1:2379 get "/tidb/server_id/624" --prefix -w fields
"ClusterID" : 15674009460217235928
"MemberID" : 3474484975246189105
"Revision" : 410
"RaftTerm" : 2
"Key" : "/tidb/server_id/624"
"CreateRevision" : 254
"ModRevision" : 401
"Version" : 2
"Value" : "0"
"Lease" : 2463920019337735415
"More" : false
"Count" : 1
❯ printf "%x" 2463920019337735415
22319a3403996cf7%
  1. scale-in tidb
    tiup playground scale-in --pid xxx

2. What did you expect to see? (Required)

  • key /tidb/server_id/624 deleted and lease 22319a3403996cf7 revoked

3. What did you see instead (Required)

  • key /tidb/server_id/624 deleted but lease 22319a3403996cf7 was left intact
❯ etcdctl --endpoints 127.0.0.1:2379 lease timetolive 22319a3403996cf7
lease 22319a3403996cf7 granted with TTL(43200s), remaining(42674s)
❯ etcdctl --endpoints 127.0.0.1:2379 get "/tidb/server_id/624" --prefix -w fields
"ClusterID" : 15674009460217235928
"MemberID" : 3474484975246189105
"Revision" : 494
"RaftTerm" : 2
"More" : false
"Count" : 0

this accumulate etcd stresses with each tidb restart/ new instance, and if many lease expire at the same time, it might cause pd failure.

4. What is your TiDB version? (Required)

master

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions