Getting started with Chef using examples
Introduction
Chef is a configuration management system designed to allow you to automate and control vast numbers of computers in an automated, reliable, and scalable manner. This new concept is called infrastructure as code.There are plenty of tutorials on Chef and the actual chef website provides very good documentation on installing and using Chef. What i intend to do with this guide is run you through creating a cookbook showcasing some of the features of chef.
In this article, we will discuss the basics of creating a Chef cookbook. Cookbooks are the configuration units that allow us to configure and perform specific tasks within Chef on our remote nodes. We build cookbooks and then tell Chef which nodes we want to run the steps outlined in the cookbook.
Basic Cookbook Concepts
Cookbooks serve as the fundamental unit of configuration and policy details that Chef uses to bring a node into a specific state. This just means that Chef uses cookbooks to perform work and make sure things are as they should be on the node.Cookbooks are usually used to handle one specific service, application, or functionality. For instance, a cookbook can be created to use NTP to set and sync the node's time with a specific server. It may install and configure a database application. Cookbooks are basically packages for infrastructure choices.
Cookbooks are created on the workstation and then uploaded to a Chef server. From there, recipes and policies described within the cookbook can be assigned to nodes as part of the node's "run-list". A run-list is a sequential list of recipes and roles that are run on a node by chef-client in order to bring the node into compliance with the policy you set for it.
In this way, the configuration details that you write in your cookbook are applied to the nodes you want to adhere to the scenario described in the cookbook.
Cookbooks are organized in a directory structure that is completely self-contained. There are many different directories and files that are used for different purposes. Let's go over some of the more important ones now.
Recipes
A recipe is the main workhorse of the cookbook. A cookbook can contain more than one recipe, or depend on outside recipes. Recipes are used to declare the state of different resources.Chef resources describe a part of the system and its desired state. For instance, a resource could say "the package x should be installed". Another resource may say "the x service should be running".
A recipe is a list related resources that tell Chef how the system should look if it implements the recipe. When Chef runs the recipe, it checks each resource for compliance to the declared state. If the system matches, it moves on to the next resource, otherwise, it attempts to move the resource into the given state.
Resources can be of many different types. You can learn about the different resource types here. Some common ones are:
- package: Used to manage packages on a node
- service: Used to manage services on a node
- user: Manage users on the node
- group: Manage groups
- template: Manage files with embedded ruby templates
- cookbook_file: Transfer files from the files subdirectory in the cookbook to a location on the node
- file: Manage contents of a file on node
- directory: Manage directories on node
- execute: Execute a command on the node
- cron: Edit an existing cron file on the node
Attributes
Attributes in Chef are basically settings. Think of them as simple key-value pairs for anything you might want to use in your cookbook.There are several different kinds of attributes that can be applied, each with a different level of precedence over the final settings that a node operates under. At the cookbook level, we generally define the default attributes of the service or system we are configuring. These can be overridden later by more specific values for a specific node.
When creating a cookbook, we can set attributes for our service in the attributes subdirectory of our cookbook. We can then reference these values in other parts of our cookbook.
Files
The files subdirectory within the cookbook contains any static files that we will be placing on the nodes that use the cookbook.For instance, any simple configuration files that we are not likely to modify can be placed, in their entirety, in the files subdirectory. A recipe can then declare a resource that moves the files from that directory into their final location on the node.
Templates
Templates are similar to files, but they are not static. Template files end with the.erb extension, meaning that they contain embedded Ruby.
These are mainly used to substitute attribute values into the file to create the final file version that will be placed on the node.
For example, if we have an attribute that defines the default port for a service, the template file can call to insert the attribute at the point in the file where the port is declared. Using this technique, you can easily create configuration files, while keeping the actual variables that you wish to change elsewhere.
Metadata.rb
The metadata.rb file is used, not surprisingly, to manage the metadata about a package. This includes things like the name of the package, a description, etc.It also includes things like dependency information, where you can specify which cookbooks this cookbook needs to operate. This will allow the Chef server to build the run-list for the nodes correctly and ensure that all of the pieces are transfered correctly.
Create a Simple Cookbook
To demonstrate some of the work flow involved in working with cookbooks, we will create a cookbook of our own. This will be a very simple cookbook that installs and configures the Nginx web server on our node.To begin, we need to go to our ~/chef-repo directory on our workstation:
cd ~/chef-repo
Once there, we can create a cookbook by using knife. As we mentioned in previous guides, knife is a tool used to configure most interactions with the Chef system. We can use it to perform work on our workstation and also to connect with the Chef server or individual nodes.
The general syntax for creating a cookbook is:
knife cookbook create cookbook_name
Since our cookbook will deal with installing and configuring Nginx, we will name our cookbook appropriately:
knife cookbook create apache
** Creating cookbook apache
** Creating README for cookbook: apache
** Creating CHANGELOG for cookbook: apache
** Creating metadata for cookbook: apache
What knife does here is builds a simple structure within our cookbooks directory for our new cookbook. We can see our cookbook structure by navigating into the cookbooks directory, and into the directory with the cookbook name.
cd cookbooks/apache
ls
attributes CHANGELOG.md definitions files libraries metadata.rb providers README.md recipes resources templates
As you can see, this has created a folder and file structure that we can use to build our cookbook. Let's begin with the biggest chunk of the configuration, the recipe.
Create a Simple Recipe
If we go into therecipes subdirectory, we can see that there is already a file called default.rb inside:
cd recipes
ls
default.rb
Open the file with your text editor:
nano default.rb
#
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright 2014, YOUR_COMPANY_NAME
#
# All rights reserved - Do Not Redistribute
#
The only thing that is in this file currently is a comment header.
We can begin by planning the things that need to happen for our apache web server to get up and running the way that we want it to. We do this by configuring "resources". Resources do not describe how to do something; they simply describe what a part of the system should look like when it is complete.
First of all, we obviously need to make sure the software is installed. We can do this by creating a "package" resource first.
# Cookbook Name:: apache
# Recipe:: default
#
# Copyright 2016, YOUR_COMPANY_NAME
#
# All rights reserved - Do Not Redistribute
#
if node["platform"] == "ubuntu"
execute "apt-get update -y" do
end
end
package "apache2" do
package_name node["apache"]["package"] ##will install httpd on centos or apache2 on ubuntu
# action :install
end
node["apache"]["sites"].each do |sitename, data| ##for each apache sites loop though them
document_root = "/content/sites/#{sitename}" ##variable declaration that contains the path
directory document_root do ##creates the directory
mode "0755" ##recursively give the read write execute permissions
recursive true
end
if node["platform"] == "centos"
template_location = "/etc/httpd/conf.d/#{sitename}.conf"
elsif
node["platform"] == "ubuntu"
template_location = "/etc/apache2/sites-enabled/#{sitename}.conf"
end
file '/etc/apache2/sites-enabled/000-default.conf' do ##delete file 000-default
action :delete
end
template template_location do ##"/etc/apache2/sites-enabled/#{sitename}.conf" do
source "vhost.erb" ##file name that is insire /templates/default/vhost.erv
mode "0644"
variables(
:document_root => document_root,
:port => data["port"], ##will take the value from the attribute which is port 80
:domain => data["domain"] ##will take the domain name for each attribute
)
notifies :restart, "service[apache2]" ##restarts apache in order to apply changes
end
template "/content/sites/#{sitename}/index.html" do
source "index.html.erb"
mode "0644"
variables(
:site_title => data["site_title"],
:comingsoon => "Coming soon",##custom variable
:author_name => node["author"]["name"] ##this is done to test the enviroment set attribute in dev.rb
)
end
end
##executing a command example
execute "rm /etc/apache2/sites-enabled/000-default.conf" do
only_if do
File.exist?("/etc/apache2/sites-enabled/000-default.conf") ##another way of deleting a file if it exists
end
end
service "apache2" do
service_name node["apache"]["package"] ##looks at package nane to determine service to be started
action [:enable, :start]
end
Creating the Attribute file
An attribute is a specific detail about a node. Attributes are used by the chef-client to understand:- The current state of the node
- What the state of the node was at the end of the previous chef-client run
- What the state of the node should be at the end of the current chef-client run
- The state of the node itself
- Cookbooks (in attribute files and/or recipes)
- Roles
- Environments
- Data about the node collected by Ohai
- The node object that was saved to the Chef server at the end of the previous chef-client run
- The rebuilt node object from the current chef-client run, after it is updated for changes to cookbooks (attribute files and/or recipes), roles, and/or environments, and updated for any changes to the state of the node itself
So how does the chef-client determine which value should be applied? Keep reading to learn more about how attributes work, including more about the types of attributes, where attributes are saved, and how the chef-client chooses which attribute to apply.
Paste this into the file cookbooks/apache/attributes/default.rb:
default["apache"]["sites"]["Node2"] = {"site_title" => "Stelios website comung soon", "port" =>80, "domain"=> "xxx.mydomain.com"}
default["apache"]["sites"]["Node2b"] = {"site_title" => "Stelios2 website coming soon", "port" =>80, "domain"=> "yyy.mydomain.com"}
default["apache"]["sites"]["Node4"] = {"site_title" => "Stelios3 website coming soon", "port" =>80, "domain"=> "zzz.mydomain.com"}
default["author"]["name"] = "admin" ##set here but if node is set to environent dev it will be overwritten by dev.rb
case node["platform"]
when "centos"
default["apache"]["package"] = "httpd"
when "ubuntu"
default["apache"]["package"] = "apache2"
end
Save and close the file when you are finished.
Create a Template
A cookbook template is an Embedded Ruby (ERB) template that is used to dynamically generate static text files. Templates may contain Ruby expressions and statements, and are a great way to manage configuration files. Use the template resource to add cookbook templates to recipes; place the corresponding Embedded Ruby (ERB) template file in a cookbook’s /templates directory.Inside /cookbooks/apache/templates/default/ create the vhost.erb
#vhost template file
##<%if @port == 80 -%>
## NameVirtualHost *:80 ##The NameVirtualHost directive is a required to configure name-based virtual hosts, so now we can have multiple websites listening on port 80
##<% end %>
<VirtualHost *:<%= @port %> >
##will take the variable value from port passed from the recipe/default.rb
ServerName <%= @domain %>
DocumentRoot <%= @document_root %>
<Directory />
Options FollowSymLinks
AllowOverride None
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
<Directory <%= @document_root %> >
allow from all
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
</VirtualHost>
ened, we can read in the "apt" default recipe by typing:
include_recipe "apt"
package 'nginx' do
action :install
end
service 'nginx' do
action [ :enable, :start ]
end
cookbook_file "/usr/share/nginx/www/index.html" do
source "index.html"
mode "0644"
end
Save and close the file.
The other file that we need to edit is the metadata.rb file. This file is checked when the Chef server sends the run-list to the node, to see which other recipes should be added to the run-list.
Open the file now:
nano ~/chef-repo/cookbooks/nginx/metadata.rb
At the bottom of the file, you can add this line:
name 'nginx'
maintainer 'YOUR_COMPANY_NAME'
maintainer_email 'YOUR_EMAIL'
license 'All rights reserved'
description 'Installs/Configures nginx'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '0.1.0'
depends "apt"
With that finished, our Nginx cookbook now relies on our apt cookbook to take care of the package database update.
Add the Cookbook to your Node
Now that our basic cookbooks are complete, we can upload them to our chef server.We can do that individually by typing:
knife cookbook upload apt
knife cookbook upload apacheOr, we can upload everything by typing:
knife cookbook upload -a
Either way, our recipes will be uploaded to the Chef server.
Now, we can modify the run-list of our nodes. We can do this easily by typing:
knife node edit name_of_node
If you need to find the name of your available nodes, you can type:
knife node list
client1
For our purposes, when we type this, we get a file that looks like this:
knife node edit client1
{
"name": "client1",
"chef_environment": "_default",
"normal": {
"tags": [
]
},
"run_list": [
]
}
You may need to set your EDITOR environmental variable before this works. You can do this by typing:
export EDITOR=name_of_editor
As you can see, this is a simple JSON document that describes some aspects of our node. We can see a "run_list" array, which is currently empty.
We can add our Nginx cookbook to that array using the format:
"recipe[name_of_recipe]"
When we are finished, our file should look like this:
{
"name": "client1",
"chef_environment": "_default",
"normal": {
"tags": [
]
},
"run_list": [
"recipe[nginx]"
]
}