Skip to content

Improve resource association #11

Open
@barkbay

Description

@barkbay

For now metrics are only associated with Pod resources:

r.metrics[metricName] = provider.CustomMetricInfo{
GroupResource: schema.GroupResource{ // TODO: infer resource from configuration
Group: "",
Resource: "pods",
},
Namespaced: true,

Also namespace and Pod's name fields are hardcoded in the Elasticsearch query:

{
	"query": {
		"bool": {
			"must": [{
				"exists": {
					"field": "%s"
				}
			}, {
				"match": {
					"kubernetes.namespace": "%s" -- here
				}
			}, {
				"match": {
					"kubernetes.pod.name": "%s" -- here
				}
			}]
		}
	}
}

Not all metrics make sense in the context of Pods. For example it does not make sense to expose system.memory.free as it is not possible to associate this metric to a specific Pod. The first approach so far has been to filter on the metric name and assume documents also have the kubernetes.namespace and kubernetes.pod.name fields:

metricSets:
  - indices: [ 'metricbeat-*' ]
    fields:
      - patterns: [ '^prometheus\.metrics\.' , '^kibana\.stats\.'  ] # only serve metrics with namespace and pod name

We then assume that we get that kind of document from Elasticsearch:

{
  "_index": "metricbeat-7.14.0-2021.09.21-000001",
  "_type": "_doc",
  "_id": "10V6DXwBJBC0gYXF1JYC",
  "fields": {
     "prometheus.metrics.go_memstats_frees_total": [
      123418991
    ],
   [ ... other metrics starting with prometheus.metrics* ...]
   "kubernetes.pod.name": [
      "kube-dns-b4f5c58c7-47twh"
    ],
   "kubernetes.namespace": [
      "kube-system"
    ],
    "@timestamp": [
      "2021-09-22T12:28:44.448Z"
    ]
  }
}

We may want to improve this association mechanism, for example by allowing other resources to be associated with a given metric. Users may also want to use other fields than kubernetes.namespace or kubernetes.pod.name to query for a given metric.

See here how associations are managed with the Prometheus adapter.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions