distributed: allow targeting extra vmagent to vmsingle/vmcluster directly

[AndrewChubatiuk]

• extra vmagent can target VMSingle/VMCluster instances directly if write and rw balancers are disabled (related issue feature: victoria-metrics-k8s-stack allow multiple vmsingle replicas (servers) #2517)

[YuryHrytsuk]

@AndrewChubatiuk could you explain the purpose of this PR? And why / how is it related to the issue linked?

[AndrewChubatiuk]

@YuryHrytsuk updated description

[YuryHrytsuk]

@YuryHrytsuk updated description

Thanks

What confuses me is word mixed balancer. Based on what I read, to write data to VM Single / Cluster, one uses vm-agent and to read data from VM Single / Cluster one uses vm-auth.

So word mixed implies that there is only one entity doing both? But if vm-agent cannot read data and vm-auth cannot write data (probably my assumption is wrong), what is actually a mixed balancer?

[AndrewChubatiuk]

this PR introduces ability to set remoteWrites for extra VMAgent (if it's enabled) to a list of VMSingle/VMCluster endpoints in global write balancer is disabled

[Haleygo]

@AndrewChubatiuk please check fix for extra vmagent -remoteWrite arg in a7fd168.

[Haleygo]

Thanks, LGTM!
Let's merge it if you're ok with my commits.

[Haleygo]

The default src_paths for vmsingle read&write look a bit weird though.

[YuryHrytsuk]

Why wouldn't it be like this by default (instead of vmauth-write-lb + 2 vm agents)?

The new architecture (judging by image) still contains vmauth-write-loadbalancer and I don't understand why this is necessary

[Haleygo]

Why wouldn't it be like this by default (instead of vmauth-write-lb + 2 vm agents)?

The new architecture (judging by image) still contains vmauth-write-loadbalancer and I don't understand why this is necessary

See

Note:
The default topology tolerates zone outages by deploying components in every availability zone and enabling minimum-downtime during outages. If not required, some components(including vmauth-global-write, vmagent(per-zone), vmauth-read-proxy(per-zone)) are optional and can be disabled based on your use case, please refer to Parameters section for details.

and #2517 (comment)

vmauth-global-write: global write entrypoint, proxies requests to one of the zone vmagent with least_loaded policy.
The main responsibility of vmagent(per-zone) is to buffer data on disk when zone is unavailable to ingest.

---

vmauth-global-write and vmagent(per-zone) can be disabled as noted in #2517 (comment). If disabled, the extra-vmagent (configured automatically in this PR) or other collectors/clients must be configured to write to both vmsingle/vmcluster.
However, that won't be the default setup; by default, we aim to provide the maximum SLA and a default buffer layer. See also https://docs.victoriametrics.com/guides/vm-architectures/#multi-cluster-and-multi-az.