In this post we are going to look at an ELK stack architecture for a small scale implementation. Keep in mind that this architecture is suitable for a small sized on-prem installation and the index capacity is determined by the hardware and disk space availability.
One of the most often asked question about the ELK stack is how can i monitor the elastic nodes itself. Monitoring the nodes here includes all indexes, all the data nodes, index size, total index size, etc. One tool that i use for my implementations is Cerebro.
ElasticSearch recommends setting up master, data and ingest nodes for a production deployment. The typical structure would be few master nodes, some data nodes and some or no ingest nodes. There is also a client node that is recommended to be installed at the same box where kibana Continue reading
How often should a new log index be created? Once a day, Once a week, Once a month? A simple search in Google would return various responses each arguing the pros and cons of creating indexes daily or weekly. Lets look at how to do that with logstash.