Monitoring Stack Demo

Wanted to share a demo/tutorial on how get started with a monitoring stack using Grafana, Loki, Mimir and Prometheus with a Promtail log sender:

https://github.com/wick02/monitoring

I also created a video demo of it working on a mac M1 along with a few of my old colleagues cloning it with no issues reported. I have around 7 years helping maintain logs and metric backends and this is my second video on Grafana which is available on Grafana's youtube channel from a meetup in 2017.

Goals of this repo:

• To trim down to the very basics of each service, to isolate them from each other so you can pick and choose what you want to use from the demo.
• I've configured it in such a way where you can scale it in a cloud environment and to give something to the developers.
• It's not dependent on keeping volumes on the machine, so you can use something like Amazon ECS without managing the volumes and use spot servers to help cut costs.
• It's not a lot of code or configuration, it uses a lot of existing tutorials already but made in such a way that I think anyone with some operational experience can use and get started with.
• It's also built in a way where the metrics are pushed to an S3 like backend using min.io so you can keep and persist all the logs and metrics.
• It uses Tenant IDs, so you can isolate offenders if you need to use this as a massive shared service for the company by rate limiting them until they stop sending you too many metrics/logs as we all are accustomed to see when we manage these type of backends.
• Since it is simple to spin up a Mimir or Loki cluster with a design like this, you could make multiple clusters and isolate components away even further

Thank you for reading and any feedback on the repo or here is appreciated. I hope to add Tempo in the future along with a terraform deployment process for this stack.