by Anil Jalela | Sep 1, 2022 | Linux
Docker was first released in March 2013 and is developed by Solomon Hykes and Sebastien Pahl
Docker is an open-source centralized platform designed to create deploy and run applications.
Docker uses containers on the host OS to run applications.
It allows applications to use the same Linux kernel as s system on the host computer, rather than creating a whole virtual OS.
We can install docker on any OS but the Docker engine runs natively on Linux distribution.
Docker is written in the “Go” language.
Docker is a tool that performs OS-level Virtualization, Also known as containerization.
Before Docker, many users face the problem that a particular code running in the developer’s code is running in the developer’s system but not in the user’s system. Docker is a set of “platform as services” that use OS-level virtualization whereas VMware uses hardware-level virtualization.
ADVANTAGE OF DOCKER
No pre-allocation of ram.
CI (continuous integration) Efficiency => docker enables you to build a container image and use that same image across every step of the deployment process.
Less cost.
It is light in weight
It can run on physical hardware virtual hardware or on the cloud.
you can reuse the image.
It took very less time to create the container.
DISADVANTAGES OF DOCKER
Docker is not a good solution for application that requires a rich GUI.
Difficult to manage a large number of containers.
Docker does not provide cross-platform compatibility means if an application is designed to run in a docker container on Windows then it can’t run on Linux or vice-versa.
Docker is suitable when the development OS and testing OS are the same if the OS is different, we should use VM.
COMPONENTS OF DOCKER
Docker Damon:- Docker demon runs on the host OS. It is responsible for running containers to manage docker services. Docker daemons can communicate with other daemons.
Docker client:- Docker users can interact with docker demons through a client (CLI). The Docker client uses commands and rest-api to communicate with the docker daemon. When a client runs any server command on the docker client terminal the client terminal sends these docker commands to the docker daemon. It is possible for the docker client to communicate with more than one daemon.
Docker Host:-Docker Host is used to provide an environment to execute and run applications it contains the docker demon, images, containers, networks, and storage.
Docker Hub/Registry:- Docker Registry manages and stores the docker images. There are two types of registry in the docker hub (1) public registry:- public registry is also called docker hub (2) Private Registry:- It is used to share images within an enterprise.
Docker images:- Docker images are the read-only binary templates used to create docker containers. Or signal file with all the dependencies and configuration required to run the program.
Ways to create an Image:- (1) Take image from docker hub (2) Create image from docker file (3) Create image from existing docker containers.
Docker Container:- Container hold the entire packages that are needed to run the application. Or in other words, We can say that the image is a template and the container is a copy of that template. It is a like virtual machine. Images become containers when they run on the docker engine.
|
(1)
|
yum install docker
|
Install docker on os
|
|
(2)
|
Service docker start
service docker stop
service docker restart
service docker status
|
Start-stop restart and status of docker service
|
|
(3)
|
docker info
|
Information about current docker
|
|
(4)
|
docker -v or docker –version
|
Check docker version
|
|
(5)
|
docker images
|
To see all images in the local machine
|
|
(6)
|
docker search image-name
docker search –no-index –no-trunc image-name
Eg:-
docker search centos
docker search –no-index –no-trunc centos
|
To find-out images in the docker hub
|
|
(7)
|
docker pull image-name
Eg:-
docker pull centos
docker pull docker.io/centos
|
Download image from docker hub to the local machine
|
|
(8)
|
Docker run –it –name docker-name imagename /bin/bash
Eg:-
docker run -it –name anil centos /bin/bash
|
To give a name to the container where -i=interactive mode and -t= terminal
|
|
(9)
|
Docker start container-name
Eg:-
docker start anil
|
To start container
|
|
(10)
|
Docker attach container-name
Eg:-
docker attach anil
|
To do inside the specific container
|
|
(11)
|
docker ps -a
|
To see all the containers
|
|
docker ps
|
To see only running containers
|
|
(12)
|
docker stop container-name
Eg:-
docker stop anil
|
To stop container
|
|
(13)
|
docker delete container-name
Eg:-
docker delete anil
|
To delete container
|
|
(14)
|
docker diff container-name
Eg:-
docker diff anil
|
Find-out difference between the original image and the container
|
|
(15)
|
docker commit container-name image-name
Eg:-
|
Create an image from the container
|
|
(16)
|
Docker build -t image-name .
|
Build an image using Dockerfile
|
|
(17)
|
Docker run -it –name newcontainer -v volume_directory_name image-name /bin/bash
Eg:-
Docker run -it –name apache1 -v /voume1 centos /bin/bash
|
Create a container with the volume
|
|
(18)
|
Docker run –it –name newcontainer name — privileged=true –volumesfrom old-container-name(which have volume) centos /bin/bash
Eg:-
Docker run –it –name apache2 — privileged=true –volumesfrom apache centos /bin/bash
|
Share volume of container one to container two
|
|
(19)
|
Docker run -it –name newcontainer -v host_dir_path:volume_directory_name image-name /bin/bash
Eg:-
Docker run -it –name apache1 -v /home/docroot:/voume1 centos /bin/bash
|
Map host directory with the container as a container volume.
|
|
(20)
|
Docker volume ls
|
List of created volumes
|
|
(21)
|
Docker volume create Volume_Name
|
Create volume with the simple command
|
| (22) |
Docker volume rm Volume_Name
|
Delete created volume
|
| (23) |
Docker volume prune
|
Delete all unused volume
|
| (24) |
Docker volume inspect Volume_Name
|
Inspect volume and find out information about the volume
|
| (25) |
Docker container inspect container_Name
|
Inspect the container and find out information about the container
|
| (26) |
Docker run -td –name container-name -p 80:80 centos
Eg:-
Docker run -td –name apache_cont -p 80:80 centos
docker exec -it container-name /bin/bash
|
Expose port for internet -t= terminal and -d for daemon
|
| (27) |
Docker port container-name
Eg:-
Docker port apache_cont
|
To check which ports are mapped (expose) for the container
|
| (28) |
Docker login
|
Connect system with hub.docker.com
|
| (29) |
Docker tag local-imagename dockerid/remote_image-name
|
Give tag name for local and remote image
|
| (30) |
Docker push dockerid/newimage
|
Push image on hub.docker.com
|
| (31) |
Docker stop $(docker ps -a -q)
|
Stop all running container
|
| (32) |
Docker rm $(docker ps -a -q)
|
Delete all stop container
|
| (33) |
Docker rmi -f $(docker images -q)
|
Delete all images
|
DOCKERFILE
A Docker file is basically a text file it contains some set of instructions it is an automation of docker image creations.
*docker file name always “Dockerfile” where D is big and “ckerfile” is small characters
Docker Components:-
* Docker Components’ name is always written in capital letters.
|
FROM
|
For the base image, the command must be on top of docker-file
|
|
RUN
|
To execute commands it will create a layer in the image
|
|
MAINTAINER
|
Author/owner/Description
|
|
COPY
|
Copy file from local system (base system) We need to provide source and destination (we can’t download files from the internet and any remote repo)
|
|
ADD
|
Add is Similar to copy but, it provides a feature to download files from the internet and also extract the file in the docker image side.
|
|
EXPOSE
|
To expose ports such as port 25 for mail and 80 for httpd
|
|
WORKDIR
|
To set the working directory for a container
|
|
CMD
|
Execute commands but during containers creation
|
|
ENTRYPOINT
|
Similar to CMD but has higher priority over CMD.!st command will be executed by ENTRYPOINT only
|
|
ENV
|
Environment Variables
|
|
ARG
|
ARG is also known as build-time variables. They are only available from the moment they are ‘announced’ in the Dockerfile with an ARG instruction up to the moment when the image is built. Running containers can’t access values of ARG variables.
|
|
VOLUME
|
Define volume directory to share files with other containers or hosts
|
Dockerfile example:-
|
FROM centos:7
MAINTAINER Anil Jalela
VOLUME [“/webdata”]
# Install Apache Php Percona Postfix Dovecot Opendkim and Opendmarc
RUN yum -y update
RUN yum install -y epel-release.noarch
RUN rpm -ivh https://repo.ius.io/ius-release-el7.rpm
RUN yum install -y epel-release.noarch
RUN yum -y remove httpd*
RUN yum -y remove php*
RUN yum install -y httpd24u
RUN yum install -y httpd24u-mod_ssl.x86_64
RUN yum -y install https://repo.percona.com/yum/percona-release-latest.noarch.rpm
RUN sudo percona-release setup ps80
RUN yum -y install percona-server-server percona-server-client percona-toolkit
RUN yum -y install certbot.noarch python2-certbot-apache.noarch
RUN yum -y install bind-utils wget zip unzip tar
RUN yum -y install https://mirror.ghettoforge.org/distributions/gf/gf-release-latest.gf.el7.noarch.rpm
RUN yum install postfix3.x86_64 postfix3-mysql.x86_64 postfix3-pcre.x86_64 postfix3-utils.x86_64
RUN yum -y remove gf-release
RUN yum install -y dovecot.x86_64 dovecot-devel.x86_64 dovecot-pigeonhole.x86_64 dovecot-mysql.x86_64
RUN yum -y remove gf-release
RUN yum install -y mod_php74.x86_64 php74-cli.x86_64 php74-gd.x86_64 php74-json.x86_64 php74-xml.x86_64 php74-xmlrpc.x86_64 php74-intl.x86_64 php74-soap.x86_64 php74-common.x86_64 php74-imap.x86_64 php74-mbstring.x86_64 php74-process.x86_64 php74-tidy.x86_64 php74-mysqlnd.x86_64 php74-pecl-zip.x86_64
CMD mkdir /home/sysadm/securefiles/secure_source/
WORKDIR /home/sysadm/securefiles/secure_source/
copy my.cnf /opt/
add site.sql /opt/
ENV mysqlpass ‘sudo grep ‘temporary password’ /var/log/mysqld.log | rev | cut -d” ” -f1 | rev | tr -d “.”‘
# Update Apache Configuration
RUN sed -E -i -e ‘/<Directory “\/var\/www\/html”>/,/<\/Directory>/s/AllowOverride None/AllowOverride All/’ /etc/httpd/conf/httpd.conf
RUN sed -E -i -e ‘s/DirectoryIndex (.*)$/DirectoryIndex index.php \1/g’ /etc/httpd/conf/httpd.conf
EXPOSE 80
# Start Apache
CMD [“/usr/sbin/httpd”,”-D”,”FOREGROUND”]
|
DOCKER VOLUME
Volume is simply a directory inside the container.
Firstly we have to declare the directory as volume and then share volume.
Even if we stop the container still we can access volume.
You can declare a directory as volume only while creating the container.
You can’t create volume from the existing container.
You can share one volume across any number of containers.
The volume will not be included when you update an image.
You can map volume in two ways.
==> container to container.
==> host to container.
Benefits of volume:-
Decoupling containers from storage
Share volume among different container
Attach the volume to containers
Only delete container volume does not delete.
DIFFERENCE BETWEEN DOCKER ATTACH AND DOCKER EXEC?
Docker exec creates a new process in the container’s environments while docker attach just connect the stranded Input/Output of the main process inside the container to the corresponding standard input/output error of the current terminal
Docker exec is specifically for running new things in the already started container.
WHAT IS THE DIFFERENCE BETWEEN EXPOSE AND PUBLISH IN A DOCKER?
Basically, we have three option
==> Neither specify expose nor -p.
==> Only specify expose.
==> Specify expose and -p.
If we neither specify expose nor -p, the service in the container will only be accessible from inside the container itself.
If you expose a port, the service in the container is not accessible from outside docker, but from inside Other docker containers, so –expose is good for internal communication.
docker run -d --expose 80 centos /bin/bash
|
If you expose and -p a port, the service in the container accessible from anywhere, even out side if container.
docker run -d --expose 80 -p 8080:8080 centos /bin/bash
|
If you do -p but not expose docker does an implicit –expose. This is because, if a port is open to the public then it is automatically open for the other container. Hence ‘-p’ includes –expose.
by Anil Jalela | Aug 27, 2022 | DevOps, Linux
Chef:- Chef is pulled base automation tool which turns your code into infrastructure and helps to manage servers with increased uptime and performance, ensure compliance, minimize cost and reduce cost.
Configuration management:- configuration management is a method through which we automate admin tasks.
Chef-Client:- tool that pulls configuration from Chef-server with help of a knife and ohai.
Workstation:-work station is a server where DevOps write code (as recipe) and store it in a cookbook.
Cookbook:- Cookbook s place(folder) where DevOps write code as a recipe for automation.
Chef-server:- The server which is managing the cookbook and is connected between the node and Workstation.
Knife:- Knife is a command line tool that uploads the cookbook to the server and connects every node with Chef-server.
Node:- The server which required configuration. Which is communicating with Chef-server using the Chef-Client using the knife.
Bootstrap:- Bootstrap is a knife process that connects nodes and Chef-server for automation.
Chef supermarket:-the place where get recipes for automation.
Ohai:- Ohai is a database that stores the current configuration of the node and supplies it to the Chef-Client before connecting to the Chef server.
Idempotency:- Tracking the state of the system resources to ensure that changes should not reapply repeatedly.
Resource
=======
Resource:- Resources are components of a recipe used to manage the infrastructure with a different type of status. There can be multiple resources in a recipe that will help in configuring t and managing the infrastructure.
|
1
|
package
|
Manage packages on node
|
package ‘tar’ do
version ‘1.16.1’
action :install
end
|
|
service
|
Manage the service on node
|
service ‘apache’ do
action [ :enable, :start ]
retries 3
retry_delay 5
end
|
|
user
|
Manage the users on the node
|
user ‘aniljalela’ do
action :create
comment ‘cron user’
uid 1234
gid ‘1234’
home ‘/home/aniljalela’
shell ‘/bin/bash’
password ‘$1$JJsvHslasdfjVEroftprNn4JHtDi’
end
|
|
group
|
Manage groups
|
Group “vmail” do
action :create
member ‘dovecot’
append true
end
dont forgot create user before create group and must use append .
|
|
template
|
Manages the files with embedded ruby template
|
|
|
Cookbook file
|
Transfer the file from the files subdirectory in the cookbook to a location of node
|
|
|
file
|
Manage content of a file on the node
|
File “systeminfo” do
content “system information”
HOSTNAME: #{node [‘hostname’]}
IPADDRESS:#{node [‘ipaddress’]}
CPU: #{node [‘cpu’][‘0’ [‘mhz’]]}
MEMORY: #{node [‘’memory][‘total’]}
owner ‘root’
group ‘root’
end
|
|
execute
|
Executes a command on the node
|
execute 'apache_configtest' do
command '/usr/sbin/apachectl configtest'
end
or
Execute “run a script” do
command <<-EIH
chown apache:apache / home / anil / jalela -R
EOH
end
(remove space from path and note that (this commands runs on every calls)
|
|
cron
|
Edits an existing cron job file on the node
|
|
|
directory
|
Manage the directory on the node
|
|
|
git
|
|
git "#{Chef::Config[:file_cache_path]}/ruby-build" do
repository 'git://github.com/sstephenson/ruby-build.git'
reference 'master'
action :sync
end
bash 'install_ruby_build' do
cwd '#{Chef::Config[:file_cache_path]}/ruby-build'
user 'rbenv'
group 'rbenv'
code <<-EOH
./install.sh
EOH
environment 'PREFIX' => '/usr/local'
end
|
|
bash
|
|
bash 'install_ruby_build' do
cwd '#{Chef::Config[:file_cache_path]}/ruby-build'
user 'rbenv'
group 'rbenv'
code <<-EOH
./install.sh
EOH
environment 'PREFIX' => '/usr/local'
end
|
|
hostname
|
|
hostname 'statically_configured_host' do
hostname 'example'
ipaddress '198.51.100.2'
end
|
|
|
|
|
|
|
|
|
Chef-Workstation:-
wget https://packages.chef.io/files/stable/chef-workstation/22.7.1006/el/7/chef-workstation-22.7.1006-1.el7.x86_64.rpm
rpm -ivh chef-workstation-22.7.1006-1.el7.x86_64.rpm or
yum localinstall chef-workstation-22.7.1006-1.el7.x86_64.rpm
[root@srv25 ~]# which chef
/usr/bin/which: no chef in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
[root@srv25 ~]#
|
1
|
chef -v
|
Check chef version information.
|
Chef Workstation version: 22.7.1006
Cook style version: 7.32.1
Chef Infra Client version: 17.10.0
Chef InSpec version: 4.56.20
Chef CLI version: 5.6.1
Chef Habitat version: 1.6.420
Test Kitchen version: 3.3.1
|
|
2
|
mkdir /home/workstation-chef.blackpost.net/chef-repo/cookbooks -p
|
cookbooks is the main directory. We will create identical cookbooks into this, and
Create recopies in the identical cookbook.
|
|
|
3
|
cd /home/workstation-chef.blackpost.net/chef-repo/
cookbooks/
chef generate cookbook common-cookbook
|
Generate cookbook common-cookbook and we add recipes into the
common-cookbook
|
.
└── common-cookbook
├── CHANGELOG.md
├── chefignore
├── compliance
│ ├── inputs
│ ├── profiles
│ ├── README.md
│ └── waivers
├── kitchen.yml
├── LICENSE
├── metadata.rb
├── Policyfile.rb
├── README.md
├── recipes
│ └── default.rb
└── test
└── integration
└── default
└── default_test.rb
|
|
4
|
Cd /home/workstation-chef.blackpost.net/chef-repo/cookbooks/common-cookbook
chef generate recipe common-recipe
|
Generate recipe name common-recipe
|
|
|
5
|
Cd /home/workstation-chef.blackpost.net/chef-repo/cookbooks/
vi common-cookbook /recipes/common-recipe.rb
|
Open generate common-recipe for add code.
|
|
|
6
|
Chef exec ruby -c common-cookbook /recipes/common-recipe.rb
|
Check code syntax of common-recipe
|
|
|
7
|
Chef-client -zr “recipe[common-cookbook::common-recipe”
|
Run recipe on local system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Attributes:- attribute is a key-value pair that represents a specific detail about a node. Which is determine
(1) current state of the node.
(2) previous chef-client run the state.
(3) what stats of the node when checking client run?
Attributes use in node. Cookbook, roles, environment, and recipes.
|
No
|
Name
|
Priority
|
|
1
|
Default
|
6
|
|
2
|
Force-default
|
5
|
|
3
|
Normal
|
4
|
|
5
|
Override
|
3
|
|
5
|
Force-override
|
2
|
|
6
|
Automatic
|
1
|
Convergence:- run chef-client to apply the recipe to bring the node into the desired state this process is known as Convergence.
Runlist:- run recipes in a sequence order that we have mentioned in the run list. Using a run list we can run multiple recipes but the condition is there must be only one recipe from one cookbook.
Chef-client -zr “recipe[common-cookbook::common-recipe], recipe[apache-cookbook::apache-recipe] ”
Include recipe:- from one recipe to call another recipes are present in the same cookbook. For include, you can use any recipe but recommend is the default.
cd /home/workstation-chef.blackpost.net/chef-repo/cookbooks/common-cookbook
vi common-recipe/default.rb
inclde_recipe “common-cookbook::common-recipe”
inclde_recipe “Apache-cookbook::Apache-recipe”
chef-client -zr “recipe[common-cookbook::default]”
Chef-Server:-
Create an account on https://manage.chef.io
Create organization via the Administration tab
We can create or manage multiple organizations on “manage.chef.io”
Each organization is for one company.
Download chef-starter.zip on your workstation and overwrite it on chef-repo
|
root@vps205204 1]# unzip chef-starter.zip
Archive: chef-starter.zip
: chef-repo/README.md
creating: chef-repo/cookbooks/
: chef-repo/cookbooks/chefignore
creating: chef-repo/cookbooks/starter/
: chef-repo/cookbooks/starter/metadata.rb
creating: chef-repo/cookbooks/starter/files/
creating: chef-repo/cookbooks/starter/files/default/
: chef-repo/cookbooks/starter/files/default/sample.txt
creating: chef-repo/cookbooks/starter/templates/
creating: chef-repo/cookbooks/starter/templates/default/
: chef-repo/cookbooks/starter/templates/default/sample.erb
creating: chef-repo/cookbooks/starter/attributes/
: chef-repo/cookbooks/starter/attributes/default.rb
creating: chef-repo/cookbooks/starter/recipes/
: chef-repo/cookbooks/starter/recipes/default.rb
: chef-repo/.gitignore
creating: chef-repo/.chef/
creating: chef-repo/roles/
: chef-repo/.chef/config.rb
: chef-repo/roles/starter.rb
: chef-repo/.chef/aniljalela.pem
[root@vps205204 1]#
|
|
cp -rpv chef-repo /home/workstation-chef.blackpost.net/chef-repo/
|
/home/workstation-chef.blackpost.net/chef-repo/.chef/config.rb is a knife file.
|
[root@vps205204 chef-repo]# cat /home/workstation-chef.blackpost.net/chef-repo/.chef/config.rb
# See http://docs.chef.io/workstation/config_rb/ for more information on knife configuration options
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name “aniljalela”
client_key “#{current_dir}/aniljalela.pem”
chef_server_url “https://api.chef.io/organizations/blackpost”
cookbook_path [“#{current_dir}/../cookbooks”]
[root@vps205204 chef-repo]#
|
Check connection with the server.
|
[root@vps205204 chef-repo]# knife ssl check
Connecting to host api.chef.io:443
Successfully verified certificates from `api.chef.io’
[root@vps205204 chef-repo]#
|
Connect node with chef-server via workstations.
knife bootstrap <IP or FQDN> -N <NODE_NAME> -x <USER> — sudo — identity-file <SSH_PEM_FILE>
or
knife bootstrap node_name -x root -P password –sudo
|
Knife bootstrap 10.01.11.1 –ssh-user ec2-user –sudo -i key.pem -N node1
|
Upload cookbook on server and list it
knife cookbook upload cookbook-name
|
knife cookbook upload common-cookbook
knife cookbook list
|
Apply recipe to specific node:-
|
Knife node run_list set node1 “recipe[common-cookbook::common-recipe]”
|
To see a list of cookbooks that are present in the chef server
|
[root@vps205204 chef-repo]# knife cookbook list
common-cookbook 0.1.0
[root@vps205204 chef-repo]#
|
To delete cookbook from chef server
|
Knife cookbook delete common-cookbook -y
|
To see the list of nodes present in the chef server.
To delete a node from the server.
|
Knife node delete node-name -y
|
To see list of clients which are present in chef-server
To delete the client from the chef-server.
|
Knife client delete client-name -y
|
To see a list of roles that are present in the chef-server.
to delete roles from chef-server
|
Knife role delete role-name -y
|
Role:-
Instead of assigning recipes using knife run_list assign a role to the server and add Recipes into the role
|
Cd /home/workstation-chef.blackpost.net/chef-repo/roles/
vi webserver.rb and add below code
name “webserver”
description “create web servers”
run_list “recipe[common-cookbook::common-recipe]”,“recipe[apache-cookbook::apache-recipe]”
|
Upload role chef-server
|
Knife role from file roles/devops.rb
|
If you want to see the role created or not on the server.
Bootstrap the node.
|
Knife bootstrap node-ip –ssh-user centos –sudo (-i) node-key.pem -N node1
|
Assign run_list to node
|
knife node run_list set node1 “role[webserver]”
|
Show which node have which roles
|
knife node show node1 (node1 is node name)
|
You need to upload the recipe to the server.
|
knife cookbook upload common-cookbook
knife cookbook upload apache-cookbook
|
We can add recipes in two ways in the role.
|
vi webserver.rb and add below code
name “webserver”
description “create web servers”
run_list “recipe[common-cookbook::common-recipe]”,“recipe[apache-cookbook::apache-recipe]”
|
Or as below which include all the recipes of common-cookbook and apache-cookbook.
|
vi webserver.rb and add below code
name “webserver”
description “create web servers”
run_list “recipe[common-cookbook”,“recipe[apache-cookbook]”
|
Loop in recipe
|
%w (tar zip mysql httpd wget vim)
.each do |p|
package p do
action :install
end
end
|
by Anil Jalela | Aug 25, 2022 | Linux
best SSH client and connection managers:
- SolarPuTTY – FREE TOOL:- A free SSH client that helps you to manage remote sessions professionally. You can control the tool from one console with a tabbed interface.
- PuTTY:- A client program for SSH that allows you to run secure remote sessions over a network. Made initially for Windows, the tool also runs on Linux and Mac machines.
- WinSCP:- A popular, secure file transfer software (SCP, SFTP, etc.), but it also has an SSH client that helps remote connections over the network.
- Bitvise:- An SSH client tool that works only for Windows. It supports all versions up to the latest Windows 10. It is free and supports an unlimited number of user connections.
- SecureCRT:- Designed for use with Windows, Linux, and Mac, SecureCRT is a commercial product that provides terminal emulation for computers.
- AbsoluteTelnet:- SSH, dial-up, Telnet, and many more in a single session or tabbed multi-session interface.
- DropBear:- An open-source application and is comparatively a smaller SSH server and client.
- Terminus:- A paid SSH client tool that works on Windows, Linux, and Mac. With Terminus, you can organize host groups.
- KiTTY:- Α fork edition of PuTTY, which is considered the best SSH and Telnet client in the world. It is designed for Windows and has all the features of the original PuTTY application.
- mRemoteNG:- Α fork edition of mRemote, and it is an open-source, multi-protocol, tabbed remote connection manager for Windows operating system.
- MobaXterm:- Α toolbox for a remote computing system. It is a Windows application with functions used by webmasters, programmers, IT administrators, and anybody who needs to manage remote jobs.
- SmarTTY:- A multi-tabbed free SSH client that allows you to copy directories and files with SCP besides SSH connections.
- ZOC Terminal:- A terminal emulation software for both Windows and macOS. The application is professional in approach and has many features that make it reliable and secure.
- Xshell:- One of the most potent SSH clients. The tool allows users to easily create, launch, and edit sessions with Session Manager and Inheritable Session Properties.
- ShellNGN:- A web-based SSH client that offers all-in-one server management. The application includes SFTP, RDP, VNC, and many more.
- Puttycm:- PuTTY Connection Manager If the PuTTY Connection Manager opens the original PuTTY in a separate window, instead of opening as a TAB,
by Anil Jalela | Aug 3, 2022 | DevOps, Linux
Git is free and open source software for distributed version control: tracking changes in any set of files, usually used for coordinating work among programmers collaboratively developing source code during software development. Its goals include speed, data integrity, and support for distributed, non-linear workflows (thousands of parallel branches running on different systems).
Git was originally authored by Linus Torvalds in 2005 for the development of the Linux kernel, with other kernel developers contributing to its initial development. Since 2005, Junio Hamano has been the core maintainer. As with most other distributed version control systems, and unlike most client-server systems, every Git directory on every computer is a full-fledged repository with complete history and full version-tracking abilities, independent of network access or a central server. Git is free and open-source software distributed under the GPL-2.0-only license.
Characteristics:-
Strong support for non-linear development
Distributed development
Compatibility with existing systems and protocols
Efficient handling of large projects
Cryptographic authentication of history
Toolkit-based design
Pluggable merge strategies
Garbage accumulates until collected
Periodic explicit object packing
* remote server = Linux system in the cloud
* client = local computer(window, Linux, Mac)
|
1
|
yum -y install git
|
For install Git
|
Install git on the remote server and On the client.
|
|
2
|
git –version
|
check the currently installed git version
|
|
|
3
|
git config –global user.name ‘Nitwings Server’
git config –global user.email ‘[email protected]’
|
Set a repository username and email to identify where to push or pull code from.
|
Set on remote Server.
|
|
4
|
git config –global user.name ‘Anil Jalela’
git config –global user.email ‘[email protected]’
|
Set a repository username and email to identify where to push or pull code from.
|
On client.
|
|
5
|
git config -l
|
list of information about git configuration including user name and email
|
On client and server.
|
|
6
|
mkdir /home/git-client.blackpost.net/public_html/repos/project-client
|
Creating Git repository Folder
|
On client.
|
|
7
|
git init
|
Initialize Git repository
|
On client.
|
|
8
|
mkdir /home/git-server.blackpost.net/public_html/repos/project-server
|
Initialize Git repository
|
On Server.
|
|
9
|
git init –bare
|
Initialize Git repository
|
On Server.
|
|
10
|
echo “Hello Nitwings” > /home/git-client.blackpost.net/public_html/repos/project-client/index.html
|
create 1st file in an empty repository
|
On client.
|
|
11
|
git status
git status –short
|
to check git status use the command “git status”
|
On client.
|
|
12
|
git add index.html
|
Add index.html in the current directory to the Staging Environment
|
On client.
|
|
13
|
git add –all
|
Add all files in the current directory to the Staging Environment
|
On client.
|
|
14
|
git add -A
git add .
|
Add all files in the current directory to the Staging Environment
|
On client.
|
|
15
|
git commit -m “this is our 1st project release”
|
Move from Staging to commit for our repo(master repo) when work is done.
-m = message
|
On client.
|
|
16
|
git commit -a
|
commit file directly in a master repo without adding a stage repo
|
|
|
17
|
git log
|
view master repo history for commits
|
|
|
18
|
git –help
|
For git help
–help instead of -help to open the relevant Git manual page
|
|
|
19
|
git help –all
or
git help –a
|
To list all possible commands
|
|
|
20
|
git add –help
|
git command -help
|
|
|
21
|
git branch branch-name
|
Create git branch
(brach create in same repo dir)
|
|
|
22
|
git branch
|
Show branch in the repo
|
|
|
23
|
Git checkout ‘branch-name’
|
Switched to branch
in branch working dir is same.
|
|
|
24
|
git checkout -b ‘fastfix’
|
Create fast-fix branch and use it for changes
|
|
|
25
|
git checkout master
git branch
|
Switched to branch ‘master’
and show all branch
|
|
|
26
|
git merge ‘fastfix’
|
merge the selected branch (master) with not selected branch name
|
|
|
27
|
git branch -d fastfix
|
Delete fast-fix branch
|
|
|
28
|
Git log
|
View the history of commits for the repository
|
|
|
29
|
touch .gitignore
git add .gitignore
git commit .gitignore
|
Create .gitignore file
and add and commit to ignoring some files and folders from the workspace
|
|
|
Vi .gitignore
*.txt
|
In .gitignore add a line to ignore all .txt files
|
|
|
30
|
Vi .gitignore
rndcode/
|
In .gitignore add a line to ignore all files in any directory named rndcode
|
|
|
31
|
Vi .gitignore
*.bat
!server.bat
|
In .gitignore, ignore all .bat files, except the server.bat
|
|
|
32
|
git log –oneline
|
Show the log of the repository, showing just 1 line per commit
|
|
|
33
|
git show commit-id
|
|
|
|
34
|
git log -10
|
|
|
|
35
|
git log –grep “word”
|
|
|
|
36
|
git stash
|
|
|
|
37
|
git stash list
|
|
|
|
38
|
git stash apply staash@{file number}
|
|
|
|
39
|
git stash clear
|
|
|
|
40
|
git reset file-name
|
|
|
|
41
|
git reset .
|
|
|
|
42
|
git reset –hard
|
|
|
|
43
|
git clean -n
|
|
|
|
44
|
git clean -f
|
|
|
|
45
|
git tag -a tagname -m “message” commit-id
|
|
|
|
46
|
git tag
|
|
|
|
git show tag-name
|
|
|
|
47
|
git -d tag tag-name
|
|
|
|
|
|
|
GitHub or Your Server Repo:-
|
git remote add origin github-url
|
Add a remote repository as an origin
|
|
|
Git fetch origin
|
Get all the change history of the origin for this branch
|
|
|
git merge origin/master
|
Merge the current branch with the branch master, on the origin
|
|
|
git pull origin
|
Update the current branch from its origin using a single command
|
|
|
git pull origin master
|
Update the current branch from the origin master using a single command
|
|
|
git push origin
|
push the current branch to its default remote origin
|
|
|
git push origin master
|
push the current branch to its default remote maser
|
|
|
Git branch -a
|
List all local and remote branches of the current Git
|
|
|
Git branch -r
|
List only remote branches of the current Git
|
|
|
git clone url
|
Clone the remote repository
|
|
|
git clon url project-client
|
Clone the repository https://blackpost.net/wings.git to a folder named “project-client“:
|
|
|
Git remote rename origin upstream
|
Rename the origin remote to upstream
|
|
|
git remote add ssh-origin user@blackpostnet:/git-reo-path/
|
Add a remote repository via SSH as an origin
|
|
|
Git remote set-url origin user@blackpostnet:/git-reo-path/
|
Replace remote origin URL
|
|
|
|
|
|
What needs to learn in Git?
Introduction
Understanding version control
The history of Git
About distributed version control
Who should use Git?
Installing Git on Windows
Installing Git on Linux
Configuring Git
Exploring Git auto-completion
Using Git help
Initializing a repository
Understanding where Git files are stored
Performing your first commit
Writing commit messages
Viewing the commit log
Exploring the three-trees architecture
The Git workflow
Using hash values (SHA-1)
Working with the HEAD pointer
Adding files
Editing files
Viewing changes with diff
Viewing only staged changes
Deleting files
Moving and renaming files
Undoing working directory changes
Unstaging files
Amending commits
Retrieving old versions
Reverting a commit
Using reset to undo commits
Demonstrating a soft reset
Demonstrating a mixed reset
Demonstrating a hard reset
Removing untracked files
Using gitignore
Understanding what to ignore
Ignoring files globally
Ignoring tracked files
Tracking empty directories
Referencing commits
Exploring tree listings
Getting more from the commit log
Viewing commits
Comparing commits
Branching overview
Viewing and creating branches
Switching branches
Creating and switching branches
Switching branches with uncommitted changes
Comparing branches
Renaming branches
Deleting branches
Configuring the command prompt to show the branch
Merging code
Using fast-forward merge vs true merge
Merging conflicts
Resolving merge conflicts
Exploring strategies to reduce merge conflicts
Saving changes in the stash
Viewing stashed changes
Retrieving stashed changes
Deleting stashed changes
Working with GitHub
Setting up a GitHub account
Adding a remote repository
Creating a remote branch
Cloning a remote repository
Tracking remote branches
Pushing changes to a remote repository
Fetching changes from a remote repository
Merging in fetched changes
Checking out remote branches
Pushing to an updated remote branch
Deleting a remote branch
Enabling collaboration
A collaboration workflow
Using SSH keys for remote login
Managing repo in GitHub
Managing users in GitHub
Managing keys in GitHub
Webhook in GitHub
Next update soon…
by Anil Jalela | Jul 19, 2022 | Linux
Advanced Policy Firewall (APF) v1.7.5
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Contents:
1 ............. Introduction
1.1 ........... Introduction: Supported Systems & Requirements
2 ............. Installation
2.1 ........... Installation: Boot Loading
3 ............. Configuration
3.1 ........... Configuration: Basic Options
3.2 ........... Configuration: Advanced Options
3.3 ........... Configuration: Reactive Address Blocking
3.4 ........... Configuration: Virtual Network Files
3.5 ........... Configuration: Global Variables & Custom Rules
4 ............. General Usage
4.1 ........... General Usage: Trust System
4.2 ........... General Usage: Global Trust System
4.3 ........... General Usage: Advanced Trust Syntax
4.4 ........... General Usage: Dynamic Trust Files
5 ............. License
6 ............. Support Information
1) Introduction:
Advanced Policy Firewall (APF) is an iptables(netfilter) based firewall system
designed around the essential needs of today's Internet deployed servers and the
unique needs of custom deployed Linux installations. The configuration of APF
is designed to be very informative and present the user with an easy to follow
process, from top to bottom of the configuration file. The management of APF on
a day-to-day basis is conducted from the command line with the 'apf' command,
which includes detailed usage information and all the features one would expect
from a current and forward thinking firewall solution.
The technical side of APF is such that it embraces the latest stable features
put forward by the iptables(netfilter) project to provide a very robust and
powerful firewall. The filtering performed by APF is three fold:
1) Static rule based policies (not to be confused with a "static firewall")
2) Connection based stateful policies
3) Sanity based policies
The first, static rule based policies, is the most traditional method of
firewalling. This is when the firewall has an unchanging set of instructions
(rules) on how traffic should be handled in certain conditions. An example of
a static rule based policy would be when you allow/deny an address access to the
server with the trust system or open a new port with conf.apf. So the short of
it is rules that infrequently or never change while the firewall is running.
The second, connection based stateful policies, is a means to distinguish
legitimate packets for different types of connections. Only packets matching a
known connection will be allowed by the firewall; others will be rejected. An
example of this would be FTP data transfers, in an older era of firewalling
you would have to define a complex set of static policies to allow FTA data
transfers to flow without a problem. That is not so with stateful policies,
the firewall can see that an address has established a connection to port 21
then "relate" that address to the data transfer portion of the connection and
dynamically alter the firewall to allow the traffic.
The third, sanity based policies, is the ability of the firewall to match
various traffic patterns to known attack methods or scrutinize traffic to
conform to Internet standards. An example of this would be when a would-be
attacker attempts to forge the source IP address of data they are sending to
you, APF can simply discard this traffic or optionally log it then discard it.
To the same extent another example would be when a broken router on the
Internet begins to relay malformed packets to you, APF can simply discard them
or in other situations reply to the router and have it stop sending you new
packets (TCP Reset).
These three key filtering methods employed by APF are simply a generalization
of how the firewall is constructed on a technical design level, there are a
great many more features in APF that can be put to use. For a detailed
description of all APF features you should review the configuration file
/etc/apf/conf.apf which has well outlined captions above all options. Below is
a point form summary of most APF features for reference and review:
- detailed and well commented configuration file
- granular inbound and outbound network filtering
- user id based outbound network filtering
- application based network filtering
- trust based rule files with an optional advanced syntax
- global trust system where rules can be downloaded from a central management
server
- reactive address blocking (RAB), next generation in-line intrusion prevention
- debug mode provided for testing new features and configuration setups
- fast load feature that allows for 1000+ rules to load in under 1 second
- inbound and outbound network interfaces can be independently configured
- global tcp/udp port & icmp type filtering with multiple methods of executing
filters (drop, reject, prohibit)
- configurable policies for each ip on the system with convenience variables to
import settings
- packet flow rate limiting that prevents abuse on the most widely abused
protocol, icmp
- prerouting and postrouting rules for optimal network performance
- dshield.org block list support to ban networks exhibiting suspicious activity
- spamhaus Don't Route Or Peer List support to ban known "hijacked zombie" IP
blocks
- any number of additional interfaces may be configured as firewalled
(untrusted) or trusted (not firewalled)
- additional firewalled interfaces can have there own unique firewall policies
applied
- intelligent route verification to prevent embarrassing configuration errors
- advanced packet sanity checks to make sure traffic coming and going meets
the strictest of standards
- filter attacks such as fragmented UDP, port zero floods, stuffed routing,
arp poisoning and more
- configurable type of service options to dictate the priority of different types
of network traffic
- intelligent default settings to meet every day server setups
- dynamic configuration of your servers local DNS revolvers into the firewall
- optional filtering of common p2p applications
- optional filtering of private & reserved IP address space
- optional implicit blocks of the ident service
- configurable connection tracking settings to scale the firewall to the size of
your network
- configurable kernel hooks (ties) to harden the system further to syn-flood
attacks & routing abuses
- advanced network control such as explicit congestion notification and overflow
control
- special chains that are aware of the state of FTP DATA and SSH connections to
prevent client side issues
- control over the rate of logged events, want only 30 filter events a minute?
300 a minute? - you are the boss
- logging subsystem that allows for logging data to user space programs or
standard syslog files
- logging that details every rule added and a comprehensive set of error checks
to prevent config errors
- if you are familiar with netfilter you can create your own rules in any of
the policy files
- pluggable and ready advanced use of QoS algorithms provided by the Linux
- 3rd party add-on projects that compliment APF features
Still on the feature todo list is:
- full support for NAT/MASQ including port forwarding
- cluster oriented round-robin packet or port forwarding
- in-line firewall reactive address blocking of connction floods
- and much more...
1.1) Introduction: Supported Systems & Requirements
The APF package is designed to run on Linux based operating systems that have
an operational version of the iptables (netfilter) package installed. The
iptables (netfilter) package is supported on Linux kernels 2.4 and above,
you can find out more details on the netfilter project at:
http://www.netfilter.org/
If the version of Linux you are using already has an included copy of
iptables then chances are very high it has all the iptables modules that APF
will need. If you are configuring iptables in your own custom kernel then you
should be sure that the following modules are compiled with the kernel for
modular support:
ip_tables
iptable_filter
iptable_mangle
ip_conntrack
ip_conntrack_irc
ip_conntrack_ftp
ipt_state
ipt_multiport
ipt_limit
ipt_recent
ipt_LOG
ipt_REJECT
ipt_ecn
ipt_length
ipt_mac
ipt_multiport
ipt_owner
ipt_state
ipt_ttl
ipt_TOS
ipt_TCPMSS
ipt_ULOG
If you would like to make sure you support these modules then you can take a
look inside of /lib/modules/kernelver/kernel/net/ipv4/netfilter/ directory.
# ls /lib/modules/$(uname -r)/kernel/net/ipv4/netfilter/
The known Linux platforms that APF will run on are very diverse and it is hard
to keep track but here is a short summary:
Redhat Enterprise AS/ES 2+
CentOS Any
Fedora Core Any
Slackware 8.0+
Debian GNU/Linux 3.0+
Suse Linux 8.1+
Unbuntu Any
TurboLinux Server 9+
TurboLinux Fuji (Desktop)
RedHat Linux 7.3,8,9
The base system specs for APF operating as intended are not set in stone and
you can easily scale the package into almost any situation that has a Linux
2.4+ kernel, iptables and bash shell with standard set of gnu-utils (grep,
awk, sed and the like). Below is a short table of what is recommended:
DEVICE MIN RECOMMENDED
CPU: 300Mhz 600Mhz
MEM: 64MB 96MB
DISK: OS OS
NETWORK: Any Any
2) Installation
The installation setup of APF is very straight forward, there is an included
install.sh script that will perform all the tasks of installing APF for you.
Begin Install:
# sh install.sh
or
# INSTALL_PATH=/etc/yourpath sh install.sh
If one so desires they may customize the setup of APF by editing the variables
inside the install.sh script followed by also editing the path variables in
the conf.apf and internals.conf files. This is however not recommends and the
default paths should meet all user needs, they are:
Install Path: /etc/apf
Bin Path: /usr/local/sbin/apf
The package includes two convenience scripts, the first is importconf which will
import all the variable settings from your previous version of APF into the
new installation. The second is get_ports, a script which will output the
systems currently in use 'server' ports for the user during the installation
process in an effort to aid in configuring port settings.
All previous versions of APF are saved upon the installation of newer
versions and stored in /etc/apf.bkDDMMYY-UTIME format. In addition, there is a
/etc/apf.bk.last sym-link created to the last version of APF you had installed.
After installation is completed the documentation and convenience scripts are
copied to /etc/apf/docs and /etc/apf/extras respective.
2.1) Installation: Boot Loading
On installation APF will install an init script to /etc/init.d/apf
and configure it to load on boot. If you are setting up APF in a more
custom situation then you may follow the below instructions.
There is really 3 modes of operation for having APF firewall our system
and each has no real benifit except tailoring itself to your needs.
The first is to setup APF in the init system with chkconfig (done by
default during install), as detailed below:
chkconfig --add apf
chkconfig --level 345 apf on
Secondly, you can add the following string too the bottom of the
/etc/rc.local file:
sh -c "/etc/apf/apf -s" &
It is NOT recommended that you use both of these startup methods together,
for most systems the init script via chkconfig should be fine.
The third and final approuch is to simply run APF in an on-demand fashion. That
is, enable it with the 'apf -s' command when desired and disable it with the
'apf -f' when desired.
3) Configuration:
On your first installation of APF it will come pretty bare in the way of
preconfigured options, this is intentional. The most common issue with many
firewalls is that they come configured with so many options that a user may
never use or disable, that it leaves systems riddled with firewall holes.
Now with that said, APF comes configured with only a single incoming port
enabled by default and that is port 22 SSH. Along with a set of common practice
filtering options preset in the most compatible fashion for all users. All the
real advanced options APF has to offer are by default disabled including
outbound (egress) port filtering, reactive address blocking (rab) and the
virtual network subsystem to name a few.
The main APF configuration file is located at /etc/apf/conf.apf and has
detailed usage information above all configuration variables. The file uses
integer based values for setting configuration options and they are
0 = disabled
1 = enabled
All configuration options use this integer value system unless otherwise
indicated in the description of that option.
You should put aside 5 minutes and review the configuration file from top to
bottom taking the time to read all the captions for the options that are
provided. This may seem like a daunting task but a firewall is only as good
as it is configured and that requires you, the administrator, to take a few
minutes to understand what it is you are setting up.
APF is a very powerful firewall that when setup to make use of all the advanced
features, will provide a sophisticated and robust level of protection. Please
continue reading further along this file for more information or see the
support options at the bottom of this file for further assistance if you find
yourself lost in the configuration process.
3.1) Configuration: Basic Options
This section will cover some of the basic configuration options found inside
of the conf.apf configuration file. These options, despite how basic, are the
most vital in the proper operation of your firewall.
Option: DEVEL_MODE
Description: This tells APF to run in a development mode which in short means
that the firewall will shut itself off every 5 minutes from a cronjob. When
you install any version of APF, upgrade or new install, this feature is by
default enabled to make sure the user does not lock themself out of the
system with configuration errors. Once you are satisfied that you have the
firewall configured and operating as intended then you must disable it.
Option: INSTALL_PATH
Description: As it implies, this is the installation path for APF and unless
you have become a brave surgeon it is unlikely you will ever need to reconfigure
this option - on we go.
Option: IFACE_UNTRUSTED
Description: This variable controls the interface that firewall rules are applied
against. This interface is commonly the Internet facing interface or any interface
that faces the main network of untrusted communication (WAN).
Option: IFACE_TRUSTED
Description: It is common that you may want to set a specific interface as
trusted to be excluded from the firewall, these may be administrative private
links, virtualized VPN interfaces or a local area network that is contains
trusted resources. This feature is similar to what some term as demilitarized
zone or DMZ for short, any interfaces set in this option will be excempt from
all firewall rules with an implicit trust rule set early in the firewall load.
Option: SET_VERBOSE
Description: This option tells the apf script to print very detailed event logs
to the screen as you are conducting firewall operations from the command line.
This will allow for easier trouble shooting of firewall issues or to assist the
user in better understanding what the firewall is doing rule-by-rule. Although
the SET_VERBOSE option is new to APF, this level of logging has long been
provided in the /var/log/apf_log file and still remains as such.
Option: SET_FASTLOAD
Description: This tells APF to use a special feature to take saved snap shots
of the running firewall. Instead of regenerating every single firewall rule
when we stop/start the firewall, APF will use these snap shots to "fast load"
the rules in bulk. There are internal features in APF that will detect when
configuration has changed and then expire the snap shot forcing a full reload
of the firewall.
Option: SET_VNET
Description: The ever curious option called SET_VNET, to put it brief this
option controls the virtual network subsystem of APF also known as VNET. This
is a subsystem that generates policy files for all aliased addresses on the
IFACE_IN/OUT interfaces. In general this option is not needed for the normal
operation of APF but is provided should you want to easily configured unique
policies for the aliased addresses on an Interface. Please see topic 3.4 of
this document for more advanced details related to this option.
Option: SET_ADDIFACE
Description: This allows you to have additional untrusted interfaces firewalled
by APF and this is done through the VNET system. So for example let assume you
have a datacenter provided eth2 interface for local network backups but you
know hundreds of other Internet facing servers are also on this network. In
such a situation it would be the best course to enable this option (along with
SET_VNET) so that the interface is firewalled. Please see topic 3.4 of this
document for more advanced details related to this option.
Option: IG_TCP_CPORTS
Description: This controls what TCP ports are allowed for incoming traffic, this
is also known as the "server" or "listening services" ports. You would for
example configure here the ports 21,25,80,110,443 if you were operating the
FTP, SMTP, HTTP, POP3 & HTTPS services from this host. This is a global context
rule and will apply to all addresses on this host unless virtual net rules are
set to operate in another fashion.
Option: IG_UDP_CPORTS
Description: This controls what UDP ports are allowed for incoming traffic, this
is also known as the "server" or "listening services" ports. You would for
example configure here the ports 20,53 if you were operating the FTP & DNS
services from this host. This is a global context rule and will apply to all
addresses on this host unless virtual net rules are set to operate in another
fashion.
Option: IG_ICMP_TYPES
Description: This controls what ICMP types are allowed for incoming traffic,
these are control messages that the Internet uses to communicate any number of
error messages during communication between hosts and networks. The default
options should meet most needs however if you wish to filter a specific set
of ICMP types you should review the 'internals/icmp.types' file. This is a
global context rule and will apply to all addresses on this host unless virtual
net rules are set to operate in another fashion.
Option: EGF
Description: This is a top level control feature for enabling or disabling all
the outbound (egress) filtering features of the firewall. In the most basic
setup of the firewall from install, this will be set to disabled and we will be
operating in a mostly inbound (ingress) only filtering fashion. It is however
recommended that you enable the outbound (egress) filtering as it provides a
very robust level of protection and is a common practice to filtering outbound
traffic.
Option: EG_TCP_CPORTS
Description: This controls what TCP ports are allowed for outgoing traffic, this
is also known as the "client side" communication on a host. Here we would set
any ports we wish to communicate with on the Internet, for example if you use
many remote RSS feeds on websites then you will want to make sure port 80,443
is defined here so you can access the HTTP/HTTPS service on Internet servers.
This is a global context rule and will apply to all addresses on this host
unless virtual net rules are set to operate in another fashion.
Option: EG_UDP_CPORTS
Description: This controls what UDP ports are allowed for outgoing traffic, this
is also known as the "client side" communication on a host. Here we would set
any ports we wish to communicate with on the Internet, for example if you use
many remote RSYNC servers then you will want to make sure port 873 is defined
here so you can properly access the RSYNC service on Internet servers. This is
a global context rule and will apply to all addresses on this host unless
virtual net rules are set to operate in another fashion.
Option: EG_ICMP_TYPES
Description: This controls what ICMP types are allowed for outgoing traffic,
these are control messages that the Internet uses to communicate any number of
error messages during communication between hosts and networks. The default
options should meet most needs however if you wish to filter a specific set
of ICMP types you should review the 'internals/icmp.types' file. This is a
global context rule and will apply to all addresses on this host unless virtual
net rules are set to operate in another fashion.
Option: LOG_DROP
Description: The use of this option allows to firewall to perform very detailed
firewall logging of packets as they are filtered by the firewall. This can help
identify issues with the firewall or provide insightful information on who is
taking pokes at the host. Typically however this option is left disabled on
production systems as it can get very noisy in the log files which also can
increase i/o wait loads to the disk from the heavy logging.
3.2) Configuration: Advanced Options
The advanced options, although not required, are those which afford the firewall
the ability to be a more robust and encompassing solution in protecting a host.
These options should be reviewed on a case-by-case basis and enabled only as
you determine there merit to meet a particular need on a host or network.
Option: SET_MONOKERN
Description: This option tells the system that instead of looking for iptables
modules, that we should expect them to be compiled directly into the kernel. So
unless you have a custom compiled kernel on your system where modular support
is disabled or iptables (netfilter) is compiled in directly, you should not
enable this option. There are also exceptions here if you have a unique system
setup and APF is unable to find certain iptables modules but you know for a
fact they are there, then enable this option.
Option: VF_ROUTE
Description: This option will make sure that the IP addressess associated to
the IFACE_* variables do actually have route entries. If a route entry can not
be found then APF will not load as it is likely a configuration error has been
made with possible results being a locked-up server.
Option: VF_LGATE
Description: This option will make sure that all traffic coming into this host
is going through this defined MAC address. This is not something you will want
enabled in most situations but it is something certain people will desire with
servers residing behind a NAT/MASQ gateway for example.
Option: RAB
Description: This is a top level toggle for the reactive address blocking in
APF and does nothing more than either enable or disable it.
Option: RAB_SANITY
Description: This enables RAB for sanity violations, which is when an address
breaks a strict conformity standard such as trying to spoof an address or modify
packet flags. When addresses are found to have made such violations they are
temporarily banned for the duration of RAB_TIMER value in seconds.
Option: RAB_PSCAN_LEVEL
Description: This enables RAB for port scan violations, which is when an
address attempts to connect to a port that has been classifed as malicious.
These types of are those which are not commonly used in today's Internet but are
the subject of scrutiny by attackers, such as ports 1,7,9,11. The values for
this option are broken into 4 intergers and they are 0 for disabled, 1 for
low security, 2 for medium security and 3 for high security.
Option: RAB_HITCOUNT
Description: This controls the amount of violation hits an address must have
before it is blocked. It is a good idea to keep this very low to prevent
evasive measures. The default is 0 or 1, meaning instant block on first hit.
Option: RAB_TIMER
Description: This is the amount of time (in seconds) that an address gets
blocked for if a violation is triggered, the default is 300s (5 minutes). This
option has a max accepted value of 43200 seconds or 12 hours.
Option: RAB_TRIP
Description: This allows RAB to 'trip' the block timer back to 0 seconds if an
address attempts ANY subsiquent communication while still on the inital block
period. This option really is one of the more exciting features of the RAB
system as it can cut off an attack at the legs before it ever mounts into
something tangible against the system.
Option: RAB_LOG_HIT
Description: This controls if the firewall should log all violation hits from
an address. It is recommended that this be enabled to provide insightful log
data on addresses which are attempting to probe or conduct questionable actions
against this host. The use of LOG_DROP variable set to 1 will override this to
force logging.
Option: RAB_LOG_TRIP
Description: This controls if the firewall should log all subsiquent traffic
from an address that is already blocked for a violation hit, this can generate
allot of logs. However, the use of this option despite the depth of log data it
may generate could provide valuble information as to the intents of an attacker.
The use of LOG_DROP variable set to 1 will override this to force logging.
Option: TCP_STOP, UDP_STOP, ALL_STOP
Description: These options tell the firewall in which way to go about filtering
traffic, the supported values are DROP, RESET, REJECT and PROHIBIT. We will
review these options below in short and provide the pro/con's of their uses.
- The default is DROP which tells the firewall silently discard packets
and not reply to them at all, which some consider to be "stealth" firewall
behavoir. The direct benifit is that it saves system resources, especially
during a DoS attack in not having to reply to every discarded packet. However
the problem is experienced attackers know the way TCP/IP works and it is such
that when you try to connect to a service that is unavailable, your server or
local router replies with an "icmp-port/host-unreachable" message. So when an
attacker probing your IP address receives no reply from the server or local
router to the scans, they will instantly know you are running a firewall,
possibly peaking curiosity more.
- Then we have RESET which allows the firewall to reply to discarded packets
in such a way that it trys to make the remote host "reset/terminate" the
connection attempts to you. This option is more in-line with TCP/IP
standards however in most situations will provide no real benifits or
drawbacks. In some really isolated situations you may find that using RESET
during DoS attacks will help terminate connections more promptly but in
general this does not serve to counter the system resources expended to
send back replies to every single packet filtered.
- Then we have the REJECT value which is a more common alternative to DROP as
it allows the firewall to reply to packets with an error message. This
acomplishes the goal of filtering a packet while at the same time not
allowing the remote host to know that we are running a firewall, they just
think the port/service is closed/unavailable.
- Finally we have the PROHIBIT value which is specific for UDP_STOP but can
be used as other *_STOP values with similar effect. When we set PROHIBIT we
are telling the firewall to reply to the sender of packets with only ICMP
error messages instead of like the case with RESET, TCP packets. This is a
good alternative to reply to packets with as it does not load the system
as "much" during aggressive attacks. This is also the default expected
reply for UDP packets that are not accepted by a host, however APF will by
default use a DROP value on UDP packets.
Option: PKT_SANITY
Description: This option controls the way packets are scrutinized as they flow
through the firewall. The main PKT_SANITY option is a top level toggle for all
SANITY options and provides general packet flag sanity as a pre-scrub for the
other sanity options. In short, this makes sure that all packets coming and
going conform to strict TCP/IP standards. In doing so we make it very difficult
for attackers to inject raw/custom packets into this host.
Now onto the sanity filters, these are options that allow APF to scrutinize
traffic coming into and out of the server so it conforms to TCP/IP standards
and also filters common attack characteristics. There are a number of sanity
options and each one has a well detailed captain in hte configuration file. In
addition, these options comes preconfigured to suite most situation needs and
provide the best protection possible. With that, I will defer the PKT_SANITY
details to the conf.apf file where you can find ample information on each
option.
Moving forward we now have the Type of Service (TOS) settings which provide a
simple classification system to dictate traffic priority based on port numbers.
The use of TOS in it respective capacities can have a wide ranging impact on the
performance of your services, both positive and negative depending on settings.
That is why it is very important that you understand and study the impact of any
changes to TOS values and then act accordingly, as no two networks are alike. A
very good rule of thumb with TOS configuration is to look at the name of the TOS
value and apply some good judgement to how that name applies to certain service
based traffic on your network. For example the TOS value Minimize-Cost designed
to minimize data transmission generally not be a good setting to improve the
responce time or throughput of HTTP connections. A more fitting setting for
this would be "Maximum Throughput - Minimum Delay", as set to default for HTTP.
The default TOS settings are designed to improve throughput and reliability for
FTP,HTTP,SMTP,POP3 and IMAP, please review conf.apf under the TOS_ settings for
further details on Type of Service (TOS).
Following the TOS settings we find the traceroute settings TCR_ which tell the
firewall if and how we should handle traceroute traffic. This is by default
enabled in APF, mostly cause of popular demand but really there is no reason
to have it enabled or disabled other than personal preference. The TCR_PASS
option tells the firwall if we want to accept traceroutes and on the TCR_PORTS
3.3) Configuration: Reactive Address Blocking
3.4) Configuration: Virtual Network Files
3.5) Configuration: Global Variables & Custom Rules
4) General Usage:
The /usr/local/sbin/apf command has a number of options that will ease the
day-to-day use of your firewall. Here is a quick snap-shot of the options:
usage /usr/local/sbin/apf [OPTION]
-s|--start ......................... load the firewall rules
-r|--restart ....................... stop (flush) & reload firewall rules
-f|--stop .......................... stop (flush) all firewall rules
-l|--list .......................... list chain rules
-t|--status ........................ firewall status
-e|--refresh ....................... refresh & resolve dns names in trust rules
-a HOST CMT|--allow HOST COMMENT ... add host (IP/FQDN) to allow_hosts.rules and
immediately load new rule into firewall
-d HOST CMT|--deny HOST COMMENT .... add host (IP/FQDN) to deny_hosts.rules and
immediately load new rule into firewall
-u|--remove HOST ................... remove host from [glob_]deny_hosts.rules
and immediately remove rule from firewall
-o|--ovars ......................... output all configuration options
These options explain themselves very clearly such as the start/stop/restart
operations.
The -l|--list option will list all the firewall rules you currently have loaded,
this is more of a feature intended for experienced users but nevertheless can be
insightful for any administrator to peak at.
As for the -t|--status option, this will simply show you page-by-page the APF
status log that tracks any operations you perform with APF - if something is
not working properly, this is what you want to run.
The -e|--refresh option will flush the trust system chains and reload them from
the rule files, this will also cause any dns names in the rules to re-resolve.
This feature is ideal if you have dynamic dns names in the trust system, apart
from that it has few other uses.
If you need to quickly allow or deny someone access on the system then the
-a|--allow and -d|--deny options are your champions. If you need to quickly
remove an allow or deny entry from the firewall then the -u|--remove option
is there for it. These options are immediate in action and do NOT require
the firewall to be restarted. Please the below sections of this file for
more information on the trust system.
Finally the -o|--ovars options is a debug feature, if something is not working
the way it was intended and you need help them please send me an email to
[email protected] and be sure to include the output of this option with your email.
4.1) General Usage: Trust System:
The trust system in APF is a very traditional setup with two basic trust levels;
allow and deny. These two basic trust levels are also extended with two global
trust levels that can be imported from a remote server to assist with central
trust management in a large scale deployment. We will first look at the basic
trust levels then have a look at the extended global trust system in the
following section 4.2 then the advanced trust syntax in 4.3.
The two basic trust level files are located at:
/etc/apf/allow_hosts.rules
/etc/apf/deny_hosts.rules
These files by nature are static, meaning that once you add an entry to them,
they will remain in the files till you remove them yourself. The trust files
accept both FQDN (fully qualified domain names) and IP addresses with optional
bit masking. Examples of these formats are:
yourhost.you.com (FQDN)
192.168.2.102 (IP Address)
192.168.1.0/24 (IP Address with 24 bit mask)
The definition of IP bit masking is slightly out of the scope of this document
but some common bit masks that are used would be:
/24 (192.168.1.0 to 192.168.1.255)
/16 (192.168.0.0 to 192.168.255.255)
If you have common abuse from a network of addresses you can whois that address
then determine the network operators assigned address space and ban the network
with bit masking.
There are two methods for adding entries to the trust files and they are first
and formost by using an editor or interface of some type to edit the two files
manually, such as nano (pico clone) or vi (old school editor).
The second is by using the 'apf' command with the options --allow (-a for
short), --deny (-d for short) and --remove (-u for short). The --allow|-a and
--deny|-d flags both accept a comment option which is simply a string at the
end of the command that you would like added to the trust rule files for
reference. Here are some operating examples of these commands:
Trust an address:
apf -a ryanm.dynip.org "my home dynamic-ip"
Deny an address:
apf -d 192.168.3.111 "keeps trying to bruteforce"
Remove an address:
apf -u ryanm.dynip.org
Please take note that the --remove|-u option does not accept a comment string
for obvious reason and that it will remove entries that match from
allow_hosts.rules, deny_hosts.rules and the global extensions of these files.
4.2) General Usage: Global Trust System
4.3) General Usage: Advanced Trust Syntax
Advanced trust usage; The trust rules can be made in advanced format with 4
options (proto:flow:port:ip);
1) protocol: [packet protocol tcp/udp]
2) flow in/out: [packet direction, inbound or outbound]
3) s/d=port: [packet source or destination port]
4) s/d=ip(/xx) [packet source or destination address, masking supported]
Flow assumed as Input if not defined. Protocol assumed as TCP if not defined.
When defining rules with protocol, flow is required.
Syntax:
proto:flow:[s/d]=port:[s/d]=ip(/mask)
s - source , d - destination , flow - packet flow in/out
Examples:
inbound to destination port 22 from 24.202.16.11
tcp:in:d=22:s=24.202.16.11
outbound to destination port 23 to destination host 24.2.11.9
out:d=23:d=24.2.11.9
inbound to destination port 3306 from 24.202.11.0/24
d=3306:s=24.202.11.0/24
4.4) General Usage: Dynamic Trust Files
dyn_allow_hosts.rules
dyn_deny_hosts.rules
5) License:
APF is developed and supported on a volunteer basis by Ryan MacDonald
[[email protected]]
APF (Advanced policy firewall) is distributed under the GNU General Public
License (GPL) without restrictions on usage or redistribution. The APF
copyright statement, and GNU GPL, "COPYING.GPL" are included in the top-level
directory of the distribution. Credit must be given for derivative works as
required under GNU GPL.
6) Support Information:
If you require any assistance with APF you may refer to the R-fx Networks
community forums located at http://forums.rfxnetworks.com. You may also send
an e-mail to [email protected].
The offical home page for APF is located at:
http://www.rfxnetworks.com/apf.php
All bugs or feature requests should be sent to [email protected] and please be sure
to include as much information as possible or conceptual ideas of how you think
a new feature should work.
