Select Page
Linux readlink and realpath Command Tutorial for Beginners (with Examples)

Linux readlink and realpath Command Tutorial for Beginners (with Examples)

We’ve already discussed the ln command, which lets you create links between files. But there are also tools that allow you to resolve these links.

In this tutorial, we will discuss the basics of realpath and readlink commands that are similar (if not same) in the sense that they display resolved symbolic links in output. But before we do that, it’s worth mentioning that all examples here have been tested on Ubuntu 18.04 LTS.

As already mentioned in the beginning, both realpath and readlink commands display resolved path for symlinks in the output.

Following is their syntax:

realpath [OPTION]… FILE…

readlink [OPTION]… FILE…

And here’s what their respective man pages say:

Print the resolved absolute file name
print resolved symbolic links or canonical file names

Following are some Q&A-styled examples that should give you a better idea on how these tools work.

Simple, just provide the file or path as input. Following example shows both these commands resolving a symbolic link.

resolve link using realpath and readlink

And here’s how they work with paths:

resolve path using realpath and readlink

So you can see both commands successfully resolved symlinks in the two cases.

Q2. How to print resolved path relative to a directory?

The realpath command lets you easily do this. For example, suppose this is what you are trying to do:

realpath /var/local/Downloadslink

However, an additional requirement is to have the output of this command relative to the /home/himanshu directory. Then following is the command that you need to execute:

realpath /var/local/Downloadslink –relative-to=/home/himanshu

Here’s the output:


So you can see that the ‘–relative-to’ option allowed you to have output relative to the /home/himanshu directory.

Q3. How to just get rid of .. in paths?

There may be times when expanding/resolving symlinks isn’t the aim, but getting rid of .. in the path is. The realpath command allows you to do this.

All you have to do is to use the -s command line option. Here’s an example:

realpath -s ../../var/local/Downloadslink

And following is the output of this command:


Q4. How to change the delimiting character?

By default, newline is used as the delimiting character. However, both commands provide way to have NUL as delimiter instead.

The option you need to use in both cases is -z.

realpath -z [FILE/PATH]

readlink -z [FILE/PATH]


What we’ve discussed here are just a handful of options/features these commands provide. Once you are done practicing these, head to readlink and realpath man pages (here and here) to learn more about these tools.

Himanshu Arora has been working on Linux since 2007. He carries professional experience in system level programming, networking protocols, and command line. In addition to HowtoForge, Himanshu’s work has also been featured in some of world’s other leading publications including Computerworld, IBM DeveloperWorks, and Linux Journal.

Install Observium Network Monitoring Software on Debian 9

Install Observium Network Monitoring Software on Debian 9

In this guide, we’ll show you how to install and configure the latest version of Observium Network Monitoring software (Community Edition) on Debian 9 release, in order to monitor the network equipment at your premises.

Observium is a powerful and flexible free and open source auto-discovery network monitoring software written mostly in PHP programming language and deployed in Linux under Apache/Nginx web servers, PHP and MySQL/MariaDB database management system, also known as LAMP or LEMP stack. Ovservium uses SNMP protocol to query the state of network hosts, servers, routers, switches, and other network devices, supporting a large array of network hardware and operating systems, such as Linux, Windows, Cisco, HP, FreeBSD, Juniper, Brocade, Dell, and other important network devices vendors. The process of managing the application can be easily achieved via a simple and intuitive web based interface.


  • Debian 9 minimal installation on a bare-metal server machine or on a virtual private server
  • sudo root privileges for a local or remote account or direct access to root account
  • A static IP address configured for one of your system network interfaces cards
  • A domain name, private or public, depending on your deployment, with proper DNS records configured for web services. If you don’t have a valid or a registered domain name you can perform the installation and access the website via your server IP address
  • If you want to use Observium e-mail notifications, you should have a running mail server properly configured at your premises with remote access to its IMAP and SMTP services. For the same task you can also use a public email service, such as Gmail or Yahoo! Mail.

Install Apache, PHP, and MySQL

Before starting with the installation and configuration of Observium from sources in your own server, first assure the system meets all the software requirements for compiling and installing the application. In the first step, update your system repositories and software packages by issuing the following command.

apt update

apt upgrade

Next, execute the following command in order to install some necessary utilities that will be used to further manage your system from command line.

apt install wget bash-completion curl

After the system has been fully updated and the required utilities to manage your server have been installed, setup the name of your system by executing the following command. Replace your hostname variable accordingly.

hostnamectl set-hostname

Verify machine hostname and hosts file by issuing the following commands.


cat /etc/hostname

hostname –s

hostname –f

Finally, reboot Debian server in order to apply kernel updates and the hostname changes properly.

systemctl reboot

Observium is a web-based network monitoring platform written in PHP server-side programming language. In order to execute the PHP file scripts of the application, a web server, such as Apache HTTP server, and a PHP processing gateway must be installed and made operational in the system.  In order to install Apache web server and the PHP interpreter alongside with all required PHP modules needed by the application to run properly, issue the following command in your server console.

apt install apache2 libapache2-mod-php7.0 php7.0 php7.0-gd php7.0-opcache php7.0-json php7.0-mbstring php7.0-mcrypt php-pear php7.0-cli php7.0-snmp

Issue the following command in order to verify if all the installed PHP modules are enabled in your system

php –m

Also, make sure you install the following utilities required by Observium to query and monitor network devices via SNMP protocol, detect and insert other network parameters and display system resources graphs.

apt install fping rrdtool graphviz ipmitool snmp whois mtr-tiny imagemagick python-mysqldb

After Apache and PHP has been installed, test if the web server is up and running and listening for network connections on port 80 by issuing the following command with root privileges.

netstat –tlpn

In case netstat network utility is not installed by default in your Debian system, execute the following command to install it.

apt install net-tools

By inspecting the netstat command output, you can see that apache web server is listening for incoming network connections on port 80. For the same task you can also use the ss command, which is automatically installed, by default, in Debian 9.

ss- tulpn

In case you have a firewall enabled in your system, such as UFW firewall application, you should add a new rule to allow HTTP traffic to pass through firewall by issuing the following command.

ufw allow WWW


ufw allow 80/tcp

If you’re using iptables raw rules to manage Firewall rules in your Debian server, add the following rule to allow port 80 and 22 (for SSH) inbound traffic on the firewall so that other network administrators can browse the online application.

apt-get install -y iptables-persistent

iptables -I INPUT -p tcp –destination-port 80 -j ACCEPT

iptables -I INPUT -p tcp –destination-port 22 -j ACCEPT

netfilter-persistent save

systemctl restart netfilter-persistent

systemctl status netfilter-persistent

systemctl enable netfilter-persistent.service

Next, enable and apply the following Apache modules required by the application to run properly, by issuing the following commands.

a2enmod rewrite

systemctl restart apache2

Finally, test if Apache web server default web page can be displayed in your client’s browsers by visiting your Debian machine IP address or your domain name or server FQDN via HTTP protocol, as shown in the below image. If you don’t know your machine IP address, execute ifconfig or ip a commands to reveal the IP address of your server.


In order to install and access Observium web admin dashboard via HTTPS protocol (that will secure the traffic for your clients) issue the following command to enable Apache web server SSL module and SSL site configuration file.

a2enmod ssl

a2ensite default-ssl.conf

Next, open Apache default SSL site configuration file with a text editor and enable URL rewrite rules by adding the following lines of code after DocumentRoot directive, as shown in the below sample:

nano /etc/apache2/sites-enabled/default-ssl.conf

SSL site configuration file excerpt:

  Options +FollowSymlinks
  AllowOverride All
  Require all granted

Also, make a change to VirtualHost line to look like the following:


Change Virtual host

Close the SSL Apache file and open /etc/apache2/sites-enabled/000-default.conf file for editing and add the same URL rewrite rules as for SSL configuration file. Insert the following lines of code after DocumentRoot statement:

  Options +FollowSymlinks
  AllowOverride All
  Require all granted

Apache options

Finally, restart the Apache daemon to apply all rules configured so far and visit your domain via HTTP protocol. Because you’re using the automatically Self-Signed certificates pairs issued by Apache at installation, certificate that is untrusted by the browser, an error warning should be displayed in the browser.

systemctl restart apache2


Accept the warning in order to accept the untrusted certificate and continue to be redirected to Apache default web page, as illustrated in the following image.

Apache default page

In case the UFW firewall application blocks incoming network connections to HTTPS port, you should add a new rule to allow HTTPS traffic to pass through firewall by issuing the following command.

ufw allow ‘WWW Full’


ufw allow 443/tcp

If iptables is the default firewall application installed to protect your Debian system at network level, add the following rule to allow port 443 inbound traffic in the firewall so that visitors can browse your domain name.

iptables -I INPUT -p tcp –destination-port 443 -j ACCEPT

netfilter-persistent save

systemctl restart netfilter-persistent

systemctl status netfilter-persistent

In the next step, we need to make some further changes to PHP default configuration file in order to assure that the following PHP variables are enabled and the PHP timezone setting is correctly configured and matches your system’s geographical location. Open the /etc/php/7.0/apache2/php.ini file for editing and assure that the following lines are setup as shown. Also, initially, make a backup of PHP configuration file.

cp /etc/php/7.0/apache2/php.ini{,.backup}

nano /etc/php/7.0/apache2/php.ini

Search, edit and change the following variables in the php.ini configuration file:

file_uploads = On
default_charset               = UTF-8
error_reporting = E_ALL & ~E_NOTICE
date.timezone = Europe/London

Replace the date.timezone variable according to your server geographical location by consulting the list of time zones provided by PHP docs at the following link

If you want to increase the load speed of your website pages via OPCache plugin available for PHP7, append the following OPCache settings at the bottom of the PHP interpreter configuration file, below the [opcache] statement, as detailed below:

nano /etc/php/7.0/apache2/php.ini


Close the php.ini configuration file and verify the end of PHP configuration file to check if the OPCache variables have been correctly added by issuing the below command.

grep opcache /etc/php/7.0/apache2/php.ini

After you’ve made all changes explained above, restart apache daemon to apply the new changes by issuing the following command.

systemctl restart apache2

Finally, create a PHP info file by executing the following command and check if the PHP time zone has been correctly configured by visiting the PHP info script file from a browser at the following URL, as illustrated in the below image. Scroll down to date setting to check PHP time zone configuration.

echo ‘‘| tee /var/www/html/info.php


PHP configuration

Observium web application stores different configurations, such as users, sessions, contacts, network devices, IP addresses, network interfaces, and other data, in an RDBMS database.  In this guide, we’ll configure the Observium application to use MariaDB database as backend. Issue the following command to install MariaDB database and the PHP module needed to access the MySQL database.

apt install mariadb-server php7.0-mysql mariadb-client

After you’ve installed MariaDB, check by running the netstat command if the daemon is running and listening for connections on localhost, port 3306.

netstat –tlpn | grep mysql

Then, log in to MySQL console and secure MariaDB root account by issuing the following commands.

mysql -h localhost

Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MariaDB connection id is 2
Server version: 10.1.26-MariaDB-0+deb9u1 Debian 9.1

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

MariaDB [(none)]> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [mysql]> update user set plugin='' where user='root';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

MariaDB [mysql]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [mysql]> exit

On the next step, secure MariaDB by executing the script mysql_secure_installation provided by the installation packages from Debian stretch repositories. While running the script will ask a series of questions designed to secure MariaDB database, such as: to change MySQL root password, to remove anonymous users, to disable remote root logins and delete the test database. Execute the script by issuing the following command and assure you type yes to all questions asked in order to fully secure MySQL daemon. Use the following script output excerpt as a guide.




In order to log into MariaDB to secure it, we'll need the current

password for the root user. If you've just installed MariaDB, and

you haven't set the root password yet, the password will be blank,

so you should just press enter here.

Enter current password for root (enter for none):

OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB

root user without the proper authorisation.

You already have a root password set, so you can safely answer 'n'.

Change the root password? [Y/n] y

New password:

Re-enter new password:

Password updated successfully!

Reloading privilege tables..

... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone

to log into MariaDB without having to have a user account created for

them. This is intended only for testing, and to make the installation

go a bit smoother. You should remove them before moving into a

production environment.

Remove anonymous users? [Y/n] y

... Success!

Normally, root should only be allowed to connect from 'localhost'. This

ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] y

... Success!

By default, MariaDB comes with a database named 'test' that anyone can

access. This is also intended only for testing, and should be removed

before moving into a production environment.

Remove test database and access to it? [Y/n] y

- Dropping test database...

... Success!

- Removing privileges on test database...

... Success!

Reloading the privilege tables will ensure that all changes made so far

will take effect immediately.

Reload privilege tables now? [Y/n] y

... Success!

Cleaning up...

All done! If you've completed all of the above steps, your MariaDB

installation should now be secure.

Thanks for using MariaDB!

In order to test MariaDB security, try logging into the database from console with no root password. The access to the database should be denied if no password is provided for the root account, as illustrated in the below command excerpt:

mysql -h localhost -u root

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

If the password is supplied, the login process should be granted to MySQL console, as shown in the command following sample:

mysql -h localhost -u root -p

Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MariaDB connection id is 15
Server version: 10.1.26-MariaDB-0+deb9u1 Debian 9.1
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
MariaDB [(none)]> exit

Next, log in to the MariaDB database console and create by issuing the following commands a database for Observium application and a user with a password that will be used to manage the application database. Replace the database name, user, and password accordingly.

mysql –u root -p

Welcome to the MariaDB monitor.  Commands end with ; or g.
Your MariaDB connection id is 2
Server version: 10.1.26-MariaDB-0+deb9u1 Debian 9.1

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

MariaDB [(none)]> create database observium_db;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> grant all privileges on observium_db.* to 'observium_user'@'localhost' identified by 'password1234';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit

In order to apply all changes made so far, restart MySQL and Apache daemons and verify if daemons are running by issuing the following commands.

systemctl restart mysql apache2

systemctl status mysql apache2

Install Observium Network Monitoring Platform

After all system requirements are met to install the application, visit Observium official page at and grab the latest gzip compressed archive by issuing the following command.


After the gzip archive download finishes, extract Observium archive file to your current working directory and list the extracted files by issuing the following commands. Also, remove the default index.html file installed by Apache web server to webroot path and also delete the info.php file created earlier.

tar xfz observium-community-latest.tar.gz


rm /var/www/html/index.html

rm /var/www/html/info.php

The installation files of Observium are located in your current working directory in the observium/ directory. Issue ls command to list this directory files. Copy all the content of the extracted directory to your web server document root path one level up (/var/www/ directory) by issuing the following commands. Also, make sure you copy the hidden files to installation directory and create the logs and rrd directories in this location.

cp -rf observium/* /var/www/

cp observium/.scrutinizer.yml /var/www/

mkdir /var/www/logs

mkdir /var/www/rrd

ls -al /var/www/

Next, execute the following commands in order to grant Apache runtime user with full write permissions to the web root path and application’s rrd and logs directories. Use ls command to list permissions for application’s installed files, located in the /var/www/html/ and rrd and logs directories.

chown -R www-data:www-data /var/www/rrd/

chown -R www-data:www-data /var/www/logs/

chown -R www-data:www-data /var/www/html/

ls -al /var/www/html/

In the next step, create a new application configuration file based on Observium default template configuration file by issuing the following commands.

cd /var/www/

cp config.php.default  config.php

Next, start editing the Observium configuration file and replace the following MySQL connection variables accordingly.

nano /var/www/ config.php

On top of the config.php file, search and update MySQL database name and credentials according to your own settings, as shown in the following file excerpt:

$config['db_extension'] = 'mysqli';
$config['db_host']      = 'localhost';
$config['db_user']      = 'observium_user';
$config['db_pass']      = 'password1234';
$config['db_name']      = 'observium_db';

Configure MySQL connection

Next, save and close the config.php file and import Observium MySQL database schema by executing the discover.php script with the –u flag as shown in the following screenshot. The PHP script is located in the /var/www/ directory.

/var/www/discovery.php –u

Observium discovery

Next, add the first Observium admin account with the highest privileges level (10) by executing the adduser.php script located in the /var/www/ directory. Choose a strong password for Observium admin account.

/var/www/adduser.php observium_admin strongpass123 10

To log in to Observium web admin panel, open a browser and navigate your server’s IP address or domain name or server FQDN via HTTPS protocol. Log in to Observium dashboard with the username and password configured for admin account in the above step.


Observium login

After you log in to the application with the admin credentials, you further tune-up the application settings by navigating to /settings/ URL or click on Global Settings -> Edit menu, as illustrated in the following screenshot. The Observium parameters configured via web interface can also be hardcoded into the application configuration file: /var/www/config.php

Observioum setup

In order to monitor a network device via the SNMP protocol, hit on the top left menu and select Add Device from the pop-down menu, as shown in the image below. You can also add a new device via the command line.

/var/www/add_device.php  hostname_or_IP   SNMP_community_string   v2c

Observioum dashboard

Finally, to force visitors to securely browse the Observium interface via HTTPS protocol that encrypts the traffic between the server and clients browsers, return to your server’s terminal and edit the .htaccess file located in your website document root path, by issuing the below command.

nano /var/www/html/.htaccess

Add the following rules in .htaccess file after RewriteEngine on line, in order to automatically redirect domain traffic to HTTPS.

.htaccess file excerpt:

# Redirect to HTTPS
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]

You can also disable Apache HTTP server web document listing by adding the following line after redirect to HTTPS lines:

Options -Indexes

At the bottom of the file you can change the native PHP server settings with the following configuration examples. Modify the PHP settings to match your own server resources and configurations.

# Modify PHP settings
php_value session.use_trans_sid 0
php_value register_globals 1

Rewrite setup

In order for the application to automatically discover and add new network devices into the database and check and update their status every five minutes, add the following crontab jobs to be executed under Apache runtime user privileges:

crontab –u www-data –e

 crontab file excerpt:

# Run a complete discovery of all devices once every 6 hours
33 */6 * * * /var/www/discovery.php -h all >> /dev/null 2>&1

# Run automated discovery of newly added devices every 5 minutes
*/5 * * * * /var/www/discovery.php -h new >> /dev/null 2>&1

# Run multithreaded poller wrapper every 5 minutes
*/5 * * * * /var/www/ 4 >> /dev/null 2>&1

That’s all! You have successfully installed and configured Observium network monitoring application in Debian 9. However, because Apache HTTP server uses Self-Signed certificates to encrypt the traffic between the server and visitor browser, a warning message will always be generated and displayed in visitor’s browser. This warning can be annoying by network administrators that visit the web application via a new browser or IP address. In this case you should buy a certificate issued by a trusted Certificate Authority or get a free certificate pair from Let’s Encrypt CA.

For other custom configurations regarding Observium application, visit the documentation page at the following address:

Linux printenv Command Tutorial for Beginners (with Examples)

Linux printenv Command Tutorial for Beginners (with Examples)

Environment variables play a major role in Linux. Most programs, whether command line based or GUI based, deal with environment variables in one way or the other. So naturally, there are tools that let you access environment variables from the command line.

One such tool is printenv. In this tutorial, we will discuss printenv using some easy to understand examples. But before we do that, it’s worth mentioning that all examples here have been tested on an Ubuntu 18.04 LTS machine.

Linux printenv command

The printenv command in Linux provides you with the ability to view all or a part of the environment. Following is its syntax:

printenv [OPTION]… [VARIABLE]…

And here’s how the tool’s man page defines it:

Print the values of the specified environment VARIABLE(s). If no VARIABLE is specified, print 
name and value pairs for them all.

Following are some Q&A-styled examples that should give you a better idea on how the printenv command works.

Q1. How to use printenv?

Basic usage is pretty easy. All you have to do is to run printenv sans any arguments.


Following is the output produced on my system:

LESSCLOSE=/usr/bin/lesspipe %s %s
[email protected]=ibus
LESSOPEN=| /usr/bin/lesspipe %s

So you can see that almost all environment variables set in my system were produced in output.

Q2. How to make printenv display only select info?

Suppose you want printenv to display details related to only a specific environment variable in output, then all you have to do is the pass the name of that variable as input to the command.

Here’s an example:

printenv SHELL

And here’s the output produced by this command:


Of course, you can add more variables to the input list. For example, the following command:

printenv SHELL HOME

produced the output shown below:


Q3. How to make printenv use NUL instead of newline?

By default, the printenv command uses newline as the terminating character between output entries. However, if you want, you can make the tool use NUL as the terminator instead.

For this, you’ll have to use the –null command line option. Following is an example:

printenv –null SHELL HOME

And here’s a screenshot of the output produced:

How to make printenv use NUL instead of newline


If your Linux command line work involves dealing with environment variables, the printenv command is an important tool to remember. Here, in this tutorial, we have discussed majority of the options this tool offers. Once you are done practicing these (the learning curve isn’t steep, so it shouldn’t take time), you can head to the printenv man page to learn more about this command line utility.

Himanshu Arora has been working on Linux since 2007. He carries professional experience in system level programming, networking protocols, and command line. In addition to HowtoForge, Himanshu’s work has also been featured in some of world’s other leading publications including Computerworld, IBM DeveloperWorks, and Linux Journal.

How to Install Nginx with RTMP Module on CentOS 7

How to Install Nginx with RTMP Module on CentOS 7

RTMP (Real Time Messaging Protocol) is a high-performance protocol for transmission of data, audio, and video over the internet between flash-based platform technologies. Originally developed by Macromedia, and now owned by Adobe.

Nginx RTMP is a nginx-based media streamer with RTMP, HLS, and MPEG-DASH live streaming. Nginx RTMP module has lot of features, including H264/AAC support, HTTP callback support, HTTP control module for recording the audio/video etc.

In this tutorial, I will show you how to install and configure the Nginx web server with the RTMP module. We will be using the latest CentOS 7 server, and this guide will include the configuration of nginx and rtmp module and creating RTMP live stream and video on demand.


  • CentOS 7 Server
  • Root privileges

What we will do?

  1. Install Dependencies
  2. Download Nginx with Additional Package and RTMP Module
  3. Compile Nginx and Install from Source
  4. Configure Nginx as a Service
  5. Configure Nginx RTMP Module
  6. Setup First RTMP Live Stream
  7. Testing

Step 1 – Install Dependencies

In this tutorial, we will build the Nginx web server from source. We need to install all packages dependencies needed, including development tools, EPEL repository, and other packages.

Install CentOS ‘Development Tools’.

sudo yum -y groupinstall ‘Development Tools’

Add the EPEL repository.

sudo yum -y install epel-release

Install Nginx dependencies.

sudo yum install -y  wget git unzip perl perl-devel perl-ExtUtils-Embed libxslt libxslt-devel libxml2 libxml2-devel gd gd-devel pcre-devel GeoIP GeoIP-devel

Wait for all packages installed.

Install Nginx and RTMP dependencies

Step 2 – Download Nginx with Additional Package and RTMP Module

In this step, we will download nginx source code with the additional dependencies including pcre, zlib, and the OpenSSL.

Go to the ‘/usr/local/src’ directory.

cd /usr/local/src

Download Nginx 1.14.0 and extract it.

tar -xzvf nginx-1.14.0.tar.gz

Download the pcre package and extract it.


Download the zlib package and extract it.

tar -xzvf zlib-1.2.11.tar.gz

Download the OpenSSL package and extract it.

tar -xzvf openssl-1.1.0h.tar.gz

Next, clone the Nginx RTMP Module source code using git command.

git clone

And remove all compressed tar.gz and .zip files.

rm -f *.tar.gz *.zip

Following are the directory details we have.

ls -lah

Nginx sources downloaded

And we’re ready to compile and install.

Step 3 – Compile Nginx and Install from Source

Go to the ‘nginx-1.14.0’ directory.

cd nginx-1.14.0/

Configure the nginx 1.14.0 using those parameters below.

./configure –prefix=/etc/nginx

Compile nginx

Compile and install Nginx with RTMP module by running following commands.

sudo make
sudo make install

When the installation is complete, we need to create a new symlink module directory, create a new nginx user and group, and create a new nginx cache directory.

Create nginx symlink module to the ‘/etc/nginx’ configuration directory.

sudo ln -s /usr/lib64/nginx/modules /etc/nginx/modules

Create a new ‘nginx’ system user and group.

sudo useradd -r -d /var/cache/nginx/ -s /sbin/nologin -U nginx

Now create a new Nginx cache directory ‘/var/cache/nginx’ and change the owner of the directory to ‘nginx’ user and group.

mkdir -p /var/cache/nginx/
chown -R nginx:nginx /var/cache/nginx/

Test nginx configuration and the installed nginx version.

nginx -t
nginx -V

And the following is the result.

Nginx compiled successfully

The Nginx web server has been installed on CentOS 7 with the RTMP Module enabled.

Step 4 – Configure Nginx as a Service

In this tutorial, we will be running nginx as a service and we need to create a new nginx service file to the systemd service directory.

Go to the ‘/lib/systemd/system’ directory and create a new ‘nginx.service’ file using vim.

cd /lib/systemd/system/
vim nginx.service

paste the configuration below.

Description=nginx - high performance web server

ExecStartPre=/usr/sbin/nginx -t -c /etc/nginx/nginx.conf
ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID


Save and exit.

Now reload the systemd system.

systemctl daemon-reload

Start the nginx service and enable it to launch everytime at system boot.

systemctl start nginx
systemctl enable nginx

The Nginx web server is up and running as a service on CentOS 7 system.

Nginx systemd service

Step 5 – Configure Nginx RTMP Module

In this step, we will create a new custom Nginx configuration for RTMP module.

Go to the ‘/etc/nginx’ configuration directory and backup the original ‘nginx.conf’ file.

cd /etc/nginx/
mv nginx.conf nginx.conf.asli

Now create a custom configuration ‘nginx.conf’.

vim nginx.conf

Paste Nginx RTMP configuration below.

worker_processes  auto;
events {
    worker_connections  1024;

# RTMP configuration
rtmp {
    server {
        listen 1935; # Listen on standard RTMP port
        chunk_size 4000;

# Define the Application
        application show {
            live on;
            # Turn on HLS
            hls on;
            hls_path /mnt/hls/;
            hls_fragment 3;
            hls_playlist_length 60;
            # disable consuming the stream from nginx as rtmp
            deny play all;


http {
    sendfile off;
    tcp_nopush on;
    aio on;
    directio 512;
    default_type application/octet-stream;

    server {
        listen 8080;

        location / {
            # Disable cache
            add_header 'Cache-Control' 'no-cache';

            # CORS setup
            add_header 'Access-Control-Allow-Origin' '*' always;
            add_header 'Access-Control-Expose-Headers' 'Content-Length';

            # allow CORS preflight requests
            if ($request_method = 'OPTIONS') {
                add_header 'Access-Control-Allow-Origin' '*';
                add_header 'Access-Control-Max-Age' 1728000;
                add_header 'Content-Type' 'text/plain charset=UTF-8';
                add_header 'Content-Length' 0;
                return 204;

            types {
                application/dash+xml mpd;
                application/ m3u8;
                video/mp2t ts;

            root /mnt/;

Save and exit.

Next, we need to create a new directory for the HLS configuration and we’ve defined the web root directory is on the ‘/mnt’ directory.

Create the ‘hls’ directory under the ‘/mnt’ directory and change the owner of the directory to the nginx user and group.

mkdir -p /mnt/hls
chown -R nginx:nginx /mnt/hls

Test the configuration and restart the nginx service.

nginx -t
systemctl restart nginx

Configure Nginx for RTMP

Step 6 – Setup First RTMP Live Stream

In this tutorial, we will create new RTMP stream video on demand using the mp4 videos on the server, and create a new live RTMP stream that will be broadcasted from the local computer using the OBS software.

Go to the ‘/etc/nginx’ configuration directory and edit the ‘nginx.conf’ file.

cd /etc/nginx/
vim nginx.conf

Paste configurations below in to the ‘rtmp { … }’ bracket.

        # RTMP video on demand for mp4 files
        application vod {
            play /mnt/mp4s;

        # RTMP stream using OBS
        application stream {
            live on;

Save and exit.

Now create a new directory ‘mp4s’ for storing all vod videos, and change the owner to the nginx user group.

mkdir -p /mnt/mp4s
chown -R nginx:nginx /mnt/mp4s

Test nginx configuration and make sure there is no error, then restart the nginx service.

nginx -t
systemctl restart nginx

Nginx configuration for the RTMP live stream and the vod stream has been completed.

Setup RTMP live stream

Step 7 – Testing

Test RTMP live stream and vod stream using the VLC player.

Video On Demand Stream

Open the VLC app on your computer.

Click the ‘File’ menu choose the ‘Open Network’ option.

Now type the RTMP URL for our vod stream.


Click the ‘Open’ button.

RTMP client setup

And the following are the results of the video stream.

RTMP Video Stream

Live Stream using OBS (Open Broadcaster Software)

In this tutorial, we will test the RTMP live stream using the obs software, and the vlc player.

We will stream only the terminal window using the open broadcaster software, and try to watch the RTMP live stream using the VLC player.

Following is my configuration for the obs software.

Live Stream using OBS (Open Broadcaster Software)

Open the VLC app on your computer.

Click the ‘File’ menu, choose the ‘Open Network’ option.

Now type the RTMP URL for our live stream.


Click the ‘Open’ button.

Live stream configuration

And the following is the result of the live stream using open broadcaster through the nginx rtmp module.

Live stream sows up in Media client

Installation of the Nginx web server with RTMP module on CentOS 7 has been completed successfully.


About Muhammad Arul

Muhammad Arul is a freelance system administrator and technical writer. He is working with Linux Environments for more than 5 years, an Open Source enthusiast and highly motivated on Linux installation and troubleshooting. Mostly working with RedHat/CentOS Linux and Ubuntu/Debian, Nginx and Apache web server, Proxmox, Zimbra Administration, and Website Optimization. Currently learning about OpenStack and Container Technology.

Building and flashing a secured AOSP build with verified boot and separate lockscreen password for the Nexus 5X

Disclaimer and License

All data and information provided in this tutorial is for informational purposes only. The author makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this tutorial and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.

In no event, the author or howtoforge will be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this tutorial.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License.


The Nexus 5X and 6P were the first devices that supported verified boot based on user-supplied (and not vendor-enforced) signing keys. Before its “implosion” [1], CopperheadOS (a security-enhanced AOSP variant) used to provide good documenation and scripts for building and flashing a secure AOSP version [2]. However, the project stopped providing any updates in the recent months, so most former users are looking for viable alternatives.

My impression is that many users agree that running a self-built AOSP ROM is a much better alternative to other options such as switching to e.g. LineageOS. There are a number of good reasons for this:

  • AOSP is plain stock and has only few possibly unwanted features
  • AOSP can be built as “user” instead of “userdebug” build vartiant and, thus, is expected to be more secure (I tried compiling user builds of LineageOS, but they seem to be broken due to the invasive changes LineageOS made to the AOSP sources)
  • Once sources are fetched, AOSP can be simply built. Unlike LineageOS, it does not start downloading additional sources during the build.

Since the Nexus 5X is believed to be a well-supported developer device, I expected the process of building AOSP to be easy and well documented. However, it turns out that there are a number of caveats:

  • including the vendor binaries by following the official documentation results in incomplete builds that cannot be used for incremental updates without unlocking and wiping each time (the vendor partition, radio ROM etc. is not included in the build)
  • the official documentation does not describe how to use verified build in a way that is directly applicable. There is better documentation in the CopperheadOS docs [05], but the instructions rely on outdated scripts that are not applicable for AOSP.
  • there is no documentation on how to use a “weak” PIN as passphrase but a strong password as disk encryption key (unlike newer Pixel devices, the Nexus 5X is based on the older FDE approach). The methods that work for LineageOS devices are not applicable as they assume the device is rooted which is not the case for regular AOSP user builds.

This tutorial aims to provide detailed instructions on how to solve these caveats, building and flashing AOSP for the Nexus 5X with verified boot and using separate lockscreen/encryption secrets. It should also apply for the Nexus 6P with small changes, but I was unable to test it since I didn’t have a Nexus 6P at hand.

Except for a small script collection (required for properly extracting the vendor blobs from Binaries supplied by Google) and its dependency “oatdump” (that is downloaded as binary from on a public share) the instructions do not make use of any “unofficial” (in the sense of “non-Google-provided”) third-party resources.

Be aware of the following freedom issues:

  • The AOSP source tree contains a number of prebuilt binaries (e.g. toolchain, Linux kernel, …). While these binaries could be rebuilt from source, the needed steps are not covered in this tutorial.
  • The source code for the vendor blobs required for using many hardware components of the Nexus 5X is not publically available!
  • The tool “android-prepare-vendor” used to extract the proprietary vendor files uses prebuilt binaries itself (some even externally hosted).

Requirements and assumptions

The tutorial assumes that you have the following prerequisites (other versions/distributions might work as well but could require other or additional packages):

– a Nexus 5X with an unlocked bootloader (unlocking is not covered in this tutorial)

– (virtual) machine running Debian9 in the x86_64 variant, used exclusively for our purpose (we assume you use sudo, if not, adapt the commands)

– at least 5 GB of RAM (more is better)

– aprox 200 MB of disk space

– fast internet connection (we need to download around 30G of data)

Installing dependencies

First, install dependencies as described in the LineageOS build instructions [3] (AOSP build instructions do not provide this list):

sudo apt install bc bison build-essential ccache curl flex g++-multilib gcc-multilib git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev

Now, install additional dependencies:

sudo apt install cmake zip unzip openjdk-8-jdk-headless

Then, setup a bin path in your home directory (in Debian 9 this path is automatically configured in the bash profile):

mkdir -p ~/bin

Install the repo command:

curl > ~/bin/repo
chmod a+x ~/bin/repo

Verify the repo binary’s checksum. It should be e147f0392686c40cfd7d5e6f332c6ee74c4eab4d24e2694b3b0a0c037bf51dc5 for the current version 1.23. for later versions check the AOSP build instructions page [4]. Use the following command to compute the checksum:

sha256sum ~/bin/repo

Next, provide a git identity by running the following commands (you can leave the example data in if you prefer to remain anonymous):

git config –global “[email protected]

git config –global “Your Name”

Unfortunately, the brotli package (required to pack the builds) in Debian 9 is too old, so we need to build the current version ourselves. First get the source code and change in its directory:

git clone

Run the build (replace -j15 by the number of your cpu threads):

cd ~/brotli


make -j15

Finally, copy the resulting binary to our bin path:

cp brotli ~/bin/

Finally, logout and login again so your bash profile is re-read.

Getting the vendor blobs

There are several issues with using the vendors blobs from the binary driver packages provided by Google (see [5]). To solve them, we use the set of external scripts “android-prepare-vendor” by “anestisb” that extracts the vendor blobs them from the factory images instead.

First, clone the repository:

git clone

Using Google’s site [6], find out the latest Build tag for your Nexus 5X (currently it is OPM6.171019.030.K1).

Change to the repository, create an output directory and run the script (we run the script as root due to issues with fuse in Debian 9):

cd android-prepare-vendor

mkdir bullhead-blobs

sudo ./ -k -d bullhead -a bullhead -b OPM6.171019.030.K1 -o bullhead-blobs

Downloading AOSP sources

Note: The following steps lack instructions for verifying the downloaded source.

First, create a directory where the sources will be stored:

mkdir -p ~/aosp

Knowing the current build tag for the Nexus 5X, find out what the corresponding Android tag is using the overview available at [6]. Then, checkout the Android manifest for the corresponding branch (in this example, we use android-8.1.0_r46):

cd ~/aosp

repo init -u -b android-8.1.0_r46

Now, you can edit the ~/aosp.repo/manifest.xml to exclude certain repositories or to include others (note: this is purely optional). Use as usual in XML to comment out repositories. I recommend excluding/replacing the following:

  • – the QuickSearchBox is mostly broken in AOSP anyways
  • – replace this with a repository that contains a patched device tree where the two faulty cpu cores are disabled. This comes with a ~30% performance penalty. Yet, recommended, even if you have Nexus 5X that is not affected by the bootloop, as it is likely it will be affected in the future.
  • – there are better alternatives to the stock AOSP calendar you can install later (such as Etar)
  • – is a better alternative for the AOSP messenging app
  • – OpenCamera is a better alternative for the stock camera

Now, fetch all the repositores (can take a long time, mostly depending on your internet connection):

repo sync

Finally, copy the previously generated vendor blobs as root (this is required, or qmus and other blobs will be missing and cause later compilation failure) into the vendor directory of your AOSP tree (replace the factory build number by the current one):

sudo cp -av ~/android-prepare-vendor/bullhead-blobs/bullhead/opm6.171019.030.k1/vendor .

sudo cp -av ~/android-prepare-vendor/bullhead-blobs/bullhead/opm6.171019.030.k1/vendor_overlay .

Make your user the owner of the vendor directories (or the build will fail later). Replace yourusername with your actual username:

sudo chown -R yourusername:yourusername ~/aosp/vendor

sudo chown -R yourusername:yourusername ~/aosp/vendor_overlay

Generating Keys

Set the build variables:

source build/

Build the tool needed for generating the verity key:

make generate_verity_key

Create a directory for storing your keys (CopperheadOS docs [2] recommend using a separate key for each device, in this case bullhead):

mkdir -p keys/bullhead

Now it's time to generate the keys (do not set passwords on your keys):

cd keys/bullhead

../../development/tools/make_key releasekey '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key platform '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key shared '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key media '/C=DE/ST=Hometown/L=XX/O=yournameher /OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key verity '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

cd ~/aosp

Convert the verity key to the format required by AOSP:

out/host/linux-x86/bin/generate_verity_key -convert keys/bullhead/verity.x509.pem keys/bullhead/verity_key


Make sure your build variables are set:

cd ~/aosp

source build/

Create a user-lunchconfig for the bullhead device (replace user by userdebug if you want a userdebug configuration instead):

lunch aosp_bullhead-user

Disable Jack (it often causes compile trouble and has been deprecated in Android 9 anyways):


Compile the target-files-package (replace -j15 by the number of your cpu threads):

make target-files-package -j15

Packaging and signing

Create a directory to store the output files (outside of the usual build directory named out):

mkdir dist

Run the dist target (replace -j15 by the number of your cpu threads):

make dist -j15

Create a signed-target-files package, replacing the default test keys with your keys (replace yourusername with your actual username on the system):

build/tools/releasetools/sign_target_files_apks -o -d keys/bullhead –replace_verity_public_key keys/bullhead/ –replace_verity_private_key keys/bullhead/verity –replace_verity_keyid keys/bullhead/verity.x509.pem out/dist/ dist/

Create a signed OTA package:

build/tools/releasetools/ota_from_target_files -k keys/bullhead/releasekey dist/ dist/


Boot your Nexus 5X device into bootloader (hold volume down, then hit power).

Connect your device via USB to your machine (and make it available to the VM in case you build in a VM). You can also copy the contents from the dist directory to another machine and flash from there, but we assume you flash from using the fastboot/adb binaries built from AOSP sources (if you flash from outside, make sure your fastboot binary is recent).

Unpack the images from the

cd ~/aosp/dist

unzip IMAGES/*

Now, flash all the images:

../out/host/linux-x86/bin/fastboot flash boot boot.img

../out/host/linux-x86/bin/fastboot flash recovery recovery.img

../out/host/linux-x86/bin/fastboot flash vendor vendor.img

../out/host/linux-x86/bin/fastboot flash system system.img

Select “reboot system” using your device’s physical buttons and make sure your new system works.

Finally, reboot back to the bootloader, and re-lock it (will erase all data):

out/host/linux-x86/bin/fastboot flash oem locking

That’s it!

Setting separate boot/lockscreen secrets

Setting a separate boot/lockscreen password can be done with a small trick:

  • Unlock the bootloader (wipes all data)
  • compile and flash a userdebug build (see above)
  • lock the bootloader
  • set a lockscreen pin/password using the Android UI. Make sure you choose the right one because you won’t be able to change it again without wiping your data once you switch to the user build.
  • connect to the device via adb
  • as root, run the following command: vdc cryptfs changepw password your-new-password
  • reboot, and make sure it works
  • compile a regular user build (do not unlock the bootloader!)
  • flash the user build

Note: Without additional steps, the recovery does not allow to flash older builds. Thus, you need to flash a user build that is newer than your userdebug build!

Handling updates

If you want to upgrade to a newer AOSP release, you first need to find out the new release number.

Then, wipe the old manifest (make a backup, in case you made changes you want to redo on the updated one):

cd ~/aosp

rm -rf .repo/manifests.git

rm -rf .repo/manifest.xml

Also, wipe the (then outdated) vendor blobs and the prepare-vendor repository:

rm -rf ~/android-prepare-vendor

rm -rf vendor

rm -rf vendor_overlay

Also, clean the build tree and the build artifacts:

rm -rf out

rm -rf dist

Then, redo ONLY the following steps:

  • Getting the vendor blobs
  • Downloading AOSP sources (this will be much quicker, because only the changes will be pulled in)
  • Compiling
  • Packaging and signing

In theory, you should be able to sideload updates as new OTA packages from recovery without wiping (since it will be signed signed with the same – your – keys). In practice, it does not work yet (see the next section). There, you will also have to proceed as follows:

  • Backup all your data
  • Unlock the bootloader (your data will be wiped)
  • Flash the updated images and re-lock your bootloader as described in the section “Flashing”

WIP: Signed OTA updates

This section is WIP, the described instructions do not work yet!

In theory, it should be possible to create and flash signed OTA updates from recovery. However, all my attempts to do this resulted in a “Signature verification failed” error. Since this works when using the Google-supplied vendor files directly instead of using android-prepare-vendor, I assume it is related to the vendor files or other files (like the bootloader or radio images) not being properly signed.

Create the signed OTA package as follows:

build/tools/releasetools/ota_from_target_files -k keys/bullhead/releasekey dist/ dist/

Reboot to recovery using your device’s physical buttons.

In recovery,you will see a small android symbol. Hold down the power button and press volume up to get into the recovery menu.

Now, select “update from adb” using your device’s physical buttons.

Sideload your signed OTA package:

out/host/linux-x86/bin/adb sideload dist/