Deploy NXLog Agent with Chef
Chef is a configuration management tool for automating operations and tasks on multiple nodes.
To install Chef Workstation, refer to the Chef installation guide for details.
This guide assumes that you have Chef Workstation installed on your machine. It will be automatically downloaded and installed if you have not installed Chef Client on the nodes you want to configure.
Chef architecture
Chef’s architecture consists of:
-
A workstation for creating cookbooks and managing your environment
-
A Chef server (complete with web UI)
-
A Chef client that runs on the remote server(s) for resolving and applying configurations
The Chef server is responsible for configuration management in Chef.
This guide will use a Chef server
that is already running.
According to the recipes uploaded via Chef Workstation, it keeps the Chef clients updated.
Creating the Chef file structure
You need to run a series of commands to create the correct file structure. These commands will generate a Chef repository, the cookbooks, and the recipes. Cookbooks are a collection of recipes, and there can be many recipes within a cookbook as well as many cookbooks.
-
Create the required file structure by executing the following commands:
$ chef generate repo chef-repo (1) $ cd chef-repo $ mkdir -p .chef (2) $ chef generate cookbook nxlog (3) $ cd nxlog $ chef generate recipe nxlog_install (4)
1 Initiate a Chef repository 2 Create a hidden .chef
directory containing connection information with the Chef3 Generate a cookbook 4 Create a recipe Upon completion, the following directory structure and files should be available.
├── chefignore ├── cookbooks │ ├── example │ │ ├── attributes │ │ │ └── default.rb │ │ ├── metadata.rb │ │ ├── README.md │ │ └── recipes │ │ └── default.rb │ ├── nxlog │ │ ├── CHANGELOG.md │ │ ├── chefignore │ │ ├── kitchen.yml │ │ ├── LICENSE │ │ ├── metadata.rb │ │ ├── Policyfile.rb │ │ ├── README.md │ │ ├── recipes │ │ │ ├── default.rb │ │ │ └── nxlog_install.rb │ │ ├── spec │ │ │ ├── spec_helper.rb │ │ │ └── unit │ │ │ └── recipes │ │ │ └── nxlog_install_spec.rb │ │ └── test │ │ └── integration │ │ └── default │ │ ├── default_test.rb │ │ └── nxlog_install_test.rb │ └── README.md ├── data_bags │ ├── example │ │ └── example_item.json │ └── README.md ├── LICENSE ├── policyfiles │ └── README.md └── README.md
nxlog_install.rb
is a Ruby script that we will use to describe all our tasks sent to the Chef server. -
Visit Chef Managed Server and create an account.
-
Navigate to Administration > Organizations > Create to create an organization.
-
Click on the newly created organization; from the left column, click on Starter Kit and then click on the Download Start Kit button.
-
Unzip the
chef-starter.zip
and copy the filesconfig.rb
and<your_username>.pem
to the.chef
directory. These two files contain the connection details and credentials for theChef Server
. -
Connect a node with the Chef Server by executing:
knife bootstrap NODE_IP --ssh-user root --ssh-password NODE_ROOT_PASSWORD --node-name NODE_NAME
After you execute the command, you will notice that
chef-client
automatically installs on the node. -
Execute the following command to create a directory for the NXLog Agent tar archive and configuration files. The required files are
nxlog-6.4.9634_debian10_amd64.tar.bz2
andmanaged.conf
.$ mkdir -p cookbooks/nxlog/files
The
files
directory in Chef cookbooks stores files used in your cookbook, with thecookbook_file
resource used later on. -
Test connectivity with the
Chef Server
by running:$ knife upload cookbook nxlog
When uploading cookbooks and recipes, ensure your current working directory is chef-repo/cookbooks
. -
Add the recipe to Chef’s
run list
:$ knife node run_list add NODE_NAME recipe[nxlog::nxlog_install]
NXLog Agent recipe file
The NXLog Agent recipe file resides in the cookbooks/nxlog/recipes
directory.
Copy the contents below to nxlog_install.rb
.
# Cookbook:: nxlog
# Recipe:: nxlog_install
#
# Copyright:: 2022, The Authors, All Rights Reserved.
cookbook_file "Copy NXLog archive to remote node" do
source "nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
path "/root/nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
owner "root"
group "root"
mode "0755"
end
archive_file "Extract NXLog archive" do
path "/root/nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
destination "/root/nxlog/"
owner "root"
group "root"
mode "755"
action :extract
end
execute "Install missing NXLog dependencies" do
command "apt update; apt upgrade -y; apt-get install ruby libdbi1 librdkafka1 libodbc1 libzmq5; apt install -y -f; apt autoremove -y"
end
Dir.glob("/root/nxlog/*.deb") do |nxlog_deb|
puts "installing : #{nxlog_deb}"
dpkg_package 'Install NXLog' do
package_name "#{nxlog_deb}"
source "#{nxlog_deb}"
action :install
end
end
cookbook_file "Copy NXLog archive to remote node" do
source "ubuntu.conf"
path "/opt/nxlog/etc/nxlog.d/ubuntu.conf"
owner "nxlog"
group "nxlog"
action :create
end
service 'nxlog' do
subscribes :reload, 'file[/opt/nxlog/etc/nxlog.d/ubuntu.conf]', :immediately
end
Finally, two steps remain:
-
Execute the command below to upload the recipe to the
Chef server
, which in turn, will communicate with theChef client
on the target node and run the tasks:cd ~/chef-repo/cookbooks knife upload cookbook nxlog
-
On the target node, run the command:
$ chef-client -i 300 -d
This will daemonize the Chef Infra Client and automatically run it every 300 seconds (5 minutes).