Deploy NXLog Agent with Chef

Chef is a configuration management tool for automating operations and tasks on multiple nodes.

To install Chef Workstation, refer to the Chef installation guide for details.

This guide assumes that you have Chef Workstation installed on your machine. It will be automatically downloaded and installed if you have not installed Chef Client on the nodes you want to configure.

Chef architecture

Chef’s architecture consists of:

A workstation for creating cookbooks and managing your environment
A Chef server (complete with web UI)
A Chef client that runs on the remote server(s) for resolving and applying configurations

The Chef server is responsible for configuration management in Chef. This guide will use a Chef server that is already running. According to the recipes uploaded via Chef Workstation, it keeps the Chef clients updated.

Creating the Chef file structure

You need to run a series of commands to create the correct file structure. These commands will generate a Chef repository, the cookbooks, and the recipes. Cookbooks are a collection of recipes, and there can be many recipes within a cookbook as well as many cookbooks.

Create the required file structure by executing the following commands:

$ chef generate repo chef-repo (1)
$ cd chef-repo
$ mkdir -p .chef (2)
$ chef generate cookbook nxlog (3)
$ cd nxlog
$ chef generate recipe nxlog_install (4)

1	Initiate a Chef repository
2	Create a hidden `.chef` directory containing connection information with the Chef
3	Generate a cookbook
4	Create a recipe

Upon completion, the following directory structure and files should be available.

├── chefignore
├── cookbooks
│   ├── example
│   │   ├── attributes
│   │   │   └── default.rb
│   │   ├── metadata.rb
│   │   ├── README.md
│   │   └── recipes
│   │       └── default.rb
│   ├── nxlog
│   │   ├── CHANGELOG.md
│   │   ├── chefignore
│   │   ├── kitchen.yml
│   │   ├── LICENSE
│   │   ├── metadata.rb
│   │   ├── Policyfile.rb
│   │   ├── README.md
│   │   ├── recipes
│   │   │   ├── default.rb
│   │   │   └── nxlog_install.rb
│   │   ├── spec
│   │   │   ├── spec_helper.rb
│   │   │   └── unit
│   │   │       └── recipes
│   │   │           └── nxlog_install_spec.rb
│   │   └── test
│   │       └── integration
│   │           └── default
│   │               ├── default_test.rb
│   │               └── nxlog_install_test.rb
│   └── README.md
├── data_bags
│   ├── example
│   │   └── example_item.json
│   └── README.md
├── LICENSE
├── policyfiles
│   └── README.md
└── README.md

nxlog_install.rb is a Ruby script that we will use to describe all our tasks sent to the Chef server.

Visit Chef Managed Server and create an account.
Navigate to Administration > Organizations > Create to create an organization.
Click on the newly created organization; from the left column, click on Starter Kit and then click on the Download Start Kit button.
Unzip the chef-starter.zip and copy the files config.rb and <your_username>.pem to the .chef directory. These two files contain the connection details and credentials for the Chef Server.
Connect a node with the Chef Server by executing:
```
knife bootstrap NODE_IP --ssh-user root --ssh-password NODE_ROOT_PASSWORD --node-name NODE_NAME
```
After you execute the command, you will notice that chef-client automatically installs on the node.
Execute the following command to create a directory for the NXLog Agent tar archive and configuration files. The required files are nxlog-6.5.9781_debian10_amd64.tar.bz2 and managed.conf.
$ mkdir -p cookbooks/nxlog/files
The files directory in Chef cookbooks stores files used in your cookbook, with the cookbook_file resource used later on.
Test connectivity with the Chef Server by running:
```
$ knife upload cookbook nxlog
```
When uploading cookbooks and recipes, ensure your current working directory is chef-repo/cookbooks.

Add the recipe to Chef’s run list:

$ knife node run_list add NODE_NAME recipe[nxlog::nxlog_install]

NXLog Agent recipe file

The NXLog Agent recipe file resides in the cookbooks/nxlog/recipes directory. Copy the contents below to nxlog_install.rb.

nxlog_install.rb

# Cookbook:: nxlog
# Recipe:: nxlog_install
#
# Copyright:: 2022, The Authors, All Rights Reserved.

cookbook_file "Copy NXLog archive to remote node" do
    source "nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
    path "/root/nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
    owner "root"
    group "root"
    mode "0755"
  end

archive_file "Extract NXLog archive" do
   path        "/root/nxlog-5.4.7313_ubuntu20_amd64.tar.bz2"
   destination "/root/nxlog/"
   owner       "root"
   group       "root"
   mode        "755"
   action      :extract
 end

execute "Install missing NXLog dependencies" do
  command "apt update; apt upgrade -y; apt-get install ruby libdbi1 librdkafka1 libodbc1 libzmq5; apt install -y -f; apt autoremove -y"
end

Dir.glob("/root/nxlog/*.deb") do |nxlog_deb|
   puts "installing : #{nxlog_deb}"
   dpkg_package 'Install NXLog' do
    package_name  "#{nxlog_deb}"
    source        "#{nxlog_deb}"
    action        :install
  end
 end

cookbook_file "Copy NXLog archive to remote node" do
   source "ubuntu.conf"
   path "/opt/nxlog/etc/nxlog.d/ubuntu.conf"
   owner "nxlog"
   group "nxlog"
   action :create
 end

service 'nxlog' do
  subscribes :reload, 'file[/opt/nxlog/etc/nxlog.d/ubuntu.conf]', :immediately
 end

Finally, two steps remain:

Execute the command below to upload the recipe to the Chef server, which in turn, will communicate with the Chef client on the target node and run the tasks:
```
cd ~/chef-repo/cookbooks
knife upload cookbook nxlog
```
On the target node, run the command:
$ chef-client -i 300 -d
This will daemonize the Chef Infra Client and automatically run it every 300 seconds (5 minutes).