Whenever I’ve written about Bosun in the past I’ve always used the official Bosun Docker image provided by Stack Exchange, this has been fine for demonstrations and examples however what if we wanted to run Bosun in a production environment? Stack Exchange recommend against using the Docker image in production so this will be the first article in a series covering how to install and configure Bosun in a production environment.
We’re going to be using OpenTSDB as the backend data store for Bosun, due to the complexity of installing OpenTSDB I’m not going to be covering the installation here but if you want to get something working quickly I do have a Docker image which you can use, this image is a tweaked version of the image provided by Peter Grace with the following configuration additions which allow BosunReporter to send data to it:
I also have a tutorial showing how to setup an OpenTSDB server in Azure using HDInsight which can be found here.
Now that we have OpenTSDB setup let’s get down to installing Bosun. I am going to be using Ubuntu 18.04 LTS but the steps should work on any Linux distro.
curl -O https://dl.google.com/go/go1.11.2.linux-amd64.tar.gz
tar -xvf go1.11.2.linux-amd64.tar.gz
sudo mv go /usr/local
sudo nano ~/.profile
Add the following to the end of the file then save and exit (ctrl+o, enter, ctrl+x)
export GOPATH=$HOME/work export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
One last command now before Go should be installed:
To check everything worked correctly we can run ‘go version’ and check the response:
And we should get:
go version go1.11.2 linux/amd64
We’re going to use Supervisor to monitor and control the Bosun process so we need to install that using the following command:
sudo apt-get install supervisor
Pull Bosun from Github and configure
Now that the pre-requisites are taken care of we can finally get down to installing Bosun. We’ll start by creating a directory for Bosun and then pull the latest release from Github before setting up the configuration files.
sudo mkdir -m 777 /bosun
sudo wget -O /bosun/bosun https://github.com/bosun-monitor/bosun/releases/download/0.8.0-preview/bosun-linux-amd64
sudo chmod +x /bosun/bosun
sudo mkdir -m 777 /data
Now we need to create our bosun.toml configuration file, this is where the connection strings for OpenTSDB need to go and any other config we want to specify
sudo nano /data/bosun.toml
Here is a sample bosun.toml file, pay attention to the OpenTSDBConf section as you will have to specify the address of your OpenTSDB instance. I’d also recommend using Redis instead of Ledis in a production environment so set the RedisHost in DBConf to your Redis server. Once you are happy with your configuration save and exit.
Now we just need to create the bosunrules.conf file where our rules will be stored. We can create this as a blank file and then add configuration later through the Bosun UI.
sudo nano /data/bosunrules.conf
Just save and exit without specifying anything.
Create a supervisor file to run Bosun
Now we need to configure supervisor to run Bosun, this is just a case of creating a config file for Bosun specifying a few paths.
sudo nano /etc/supervisor/conf.d/bosun.conf
The following configuration should work if you have followed along exactly.
Now we just need to inform supervisor that we have a new configuration and hopefully Bosun will start.
sudo supervisorctl reread
sudo supervisorctl update
You can check Bosun is running by running the following command, it should respond with ‘bosun RUNNING’ along with some uptime data and the pid.
Part 2 of this series will cover installing TSDBRelay. Instead of directly sending data to Bosun we can send our data through TSDBRelay which can forward the data to opentsdb and then also to Bosun for indexing, this get’s Bosun out of the ‘critical path’ and allow us to keep collecting data in the event Bosun goes down.