These are some of the prometheus alerts which have been useful in my (limited) experience.
This post describes how to create a server in Go, which provides data from Elasticsearch for Prometheus to consume.