April 11, 2019

How to stream encrypted data into amazon glacier deep archive

This will be very short tutorial. Imagine you want to archive your old raw files (.nef) from directory 2013 into amazon glacier deep archive using amazon s3 official linux client and use also openssl encryption. You need working aws cli. I used expected size of archive as 1TB. Change text in red to applicable to your environment. All this was tested on fully updated debian 9 and fedora 29.


find "2013" -type f -regextype posix-egrep -regex ".*\.(NEF|nef)$" -print0|tar -cvf - --null -T - |openssl aes-256-cbc -a -salt -pass pass:password | aws s3 cp - s3://yours3backupbucket/2013.archive --storage-class DEEP_ARCHIVE --expected-size 1000000000000


you need to inicialize restore of your archive and wait about 48hours, then issue command:
aws s3 cp s3://yours3backupbucket/2013.archive - | openssl enc -aes-256-cbc -a -d|tar xvf -

only list restore:

aws s3 cp s3://yours3backupbucket/2013.archive - | openssl enc -aes-256-cbc -a -d|tar tvf -

- use xz in pipe after tar step if you want compression (also unxz in restore step before tar)
- use --expected-size parameter (bytes) of aws s3 cp command, if you need larger archive to put into glacier (bigger than 5GB). glacier supports archive up to 40TB
- you can change s3 storage class, but if you want to keep costs to minimum, you should use DEEP_ARCHIVE option

May 5, 2017

U2F + sudo + Fedora 25

This article is for owners of U2F security keys (Universal 2nd Factor), which would like to harden their sudo command. We will setup Fedora 25 system, so every sudo command will require 2nd factor authentication via usage of U2F security token.

Heavy lifting for implementing pam-u2f protocol was done by Yubico, you can see details on webpage

Fortunately for us, major linux distributions are including necessary libraries already, we will show setup on Fedora 25.

WARNING: during setup, be sure to have opened also terminal with root privileges, otherwise you can lock yourself out

1. Instalation of packages

# dnf install pamu2fcfg pam-u2f

2. Setup of /etc/u2f_mappings

We will use mapping between u2f tokens and users configured in file /etc/u2f_mappings

Login as user under which should you use your token

$ pamu2fcfg -u `whoami` -opam://`hostname` -ipam://`hostname`

Repeat of every U2F token you own. You will be combining these lines into one line in next step.

Now in separate window login as root and create file /etc/u2f_mappings with this format


Example for username "user" with two U2F keys:

# cat /etc/u2f_mappings

Make no mistake, for one username everything should be on ONE line

3. Setup /etc/sudoers

Important thing here is to set "timestamp_timeout=0" as default, so each subsequent sudo command still asks for authentication. You should edit this file via "visudo" command.

# egrep -v '^#|^$' /etc/sudoers
Defaults   !visiblepw
Defaults    env_reset,timestamp_timeout=0
Defaults    env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_C
Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin
root    ALL=(ALL)       ALL
%wheel  ALL=(ALL)       ALL

Dont forget to setup desired user as part of "wheel" system group (in /etc/group or via command usermod).

4. Setup /etc/pam.d/sudo

In configs below replace $HOSTNAME with output of command `hostname`

We will setup system, so each sudo command will require U2F authentication (for example via touch method).


If you setup pam_u2f module only as "sufficient" for authentication, you will be still able to execute sudo commands with user password only! I dont recommend this as you will have same situation as without U2F security token, for example if somebody will obtain your password (phising?), he can remotely execute commands as root via sudo.

Much better is to force every sudo command to require U2F authentication (via touch for example).

Entering sudo password could be optional, because you are already logged in as user (which uses same password as sudo command).

So here are 2 scenarios for finishing your work:

4.a) If you WANT to have U2F confirmation AND also want to enter SUDO password, then

# setup these auth lines in /etc/pam.d/sudo
auth       required origin=pam://$HOSTNAME appid=pam://$HOSTNAME authfile=/etc/u2f_mappings cue
auth       include      system-auth

This is recommended method and true two factor authentication.

Result will looks like this:

[host ~]$ sudo -i
Please touch the device.
[sudo] password for user: 
[host ~]#

4.b) If you WANT to have only U2F confirmation before executing sudo command, then

# setup these auth lines in /etc/pam.d/sudo
auth       [success=done        new_authtok_reqd=done   default=die] origin=pam://$HOSTNAME appid=pam://$HOSTNAME authfile=/etc/u2f_mappings cue
auth       include      system-auth

This method is weaker than first one (=not true 2nd factor, as you only input user password during login) but still safe from attacks on root with phished user password. It all depends on your threat model. One could argue that user data are more important than root data, but that is only true on workstation with one user. In multi-user environment, this could be damage-limiting lifesaver, as it prevents malware/attacker running random root commands via sudo.

Result will looks like this:

[host ~]$ sudo -i
Please touch the device.
[host ~]# 

PS: if you have strange problems with pam auth, add "debug" parameter after in /etc/pam.d/sudo, it will help you identifying cause.

November 14, 2016

Serial console in RHEL 7

This was tested on RHEL7.3

If you DONT WANT to make serial console as default console:

Enable getty on your serial port

# systemctl enable getty@ttyS0
Created symlink from /etc/systemd/system/ to /usr/lib/systemd/system/getty@.service.

And reboot (or use systemctl start for same service)

If you WANT to make serial console as default console

Dont enable getty like in previous step! (you will have duplicate gettys running on serial port, and they will collide).

Modify /etc/sysconfig/grub to contain entries for your serial port (ttyS0 for in this example)

GRUB_TERMINAL_INPUT="serial console"
GRUB_TERMINAL_OUTPUT="serial console"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1"
GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet console=tty0 console=ttyS0,9600"

Recreate grub file:

# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-514.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-514.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-5aa1bafc68ad4d229e06111f3ad495b3
Found initrd image: /boot/initramfs-0-rescue-5aa1bafc68ad4d229e06111f3ad495b3.img

After reboot, you can see getty running on serial port

# systemctl |grep ttyS0
sys-devices-pnp0-00:04-tty-ttyS0.device                                             loaded active plugged   /sys/devices/pnp0/00:04/tty/ttyS0
serial-getty@ttyS0.service                                                          loaded active running   Serial Getty on ttyS0

BTW, I recommend to delete words "rhgb quiet" from GRUB_CMDLINE_LINUX line in /etc/sysconfig/grub. It helps in various debugging sessions, but can sometimes create problem, if there is a LOT of output (in which case just leave it enabled - default).

April 25, 2016

Fedora 23 (KDE): Using GPG + YubiKey4 + PCSCD

For those who have problems with integration of gpg and yubikey 4, this post is for you. Most problems come from idea, that gpg agent runs scdaemon, which prevents other processes to read yubikey usb device. Solution is to run pcscd and let only that handle all communication to yubikey. Now gpg commands like "gpg2 --card-status" can be run right after yubioath tool exited and there is no need to manually cleaning of scdaemon processes.

Here are configs, which are needed to run:

Modify file /etc/X11/xinit/xinitrc-common to contain last 2 lines:
$ tail /etc/X11/xinit/xinitrc-common 
    if [ "x$TMPDIR" != "x" ]; then
        SSH_AGENT="/usr/bin/ssh-agent /bin/env TMPDIR=$TMPDIR"

SSH_AGENT="/usr/bin/gpg-agent --daemon --enable-ssh-support --homedir $HOME/.gnupg"; export SSH_AGENT;
SSH_AUTH_SOCK=$HOME/.gnupg/S.gpg-agent.ssh; export SSH_AUTH_SOCK;

Make sure file $HOME/.gnupg/gpg.conf contains line use-agent:
$ tail -5  $HOME/.gnupg/gpg.conf 
keyid-format 0xlong


Create file .gnupg/scdaemon.conf (I have 2 readers in notebook):
$ cat $HOME/.gnupg/scdaemon.conf 
reader-port "Yubico Yubikey 4 OTP+U2F+CCID 00 00"
reader-port "Yubico Yubikey 4 OTP+U2F+CCID 01 00"
pcsc-driver /usr/lib64/
card-timeout 5

Create file .gnupg/scd-event and make it executable. This will trigger when yubikey is removed from computer.
$ cat $HOME/.gnupg/scd-event 

if [ "$state" = "NOCARD" ]; then
  pkill -9 scdaemon

Modify .gnupg/gpg-agent.conf to include ssh-support:
$ cat .gnupg/gpg-agent.conf
###+++--- GPGConf ---+++###
###+++--- GPGConf ---+++### Thu Mar 12 10:53:37 2015 CET
# GPGConf edited this configuration file.
# It will disable options before this marked block, but it will
# never change anything below these lines.
default-cache-ttl-ssh 1209600
default-cache-ttl 1209600
max-cache-ttl 1209600
pinentry-program /usr/bin/pinentry-qt

Now udev rules for local users access are defined in 2 files:
$ cat /etc/udev/rules.d/69-yubikey.rules 
ACTION!="add|change", GOTO="yubico_end"
ATTRS{idVendor}=="1050", ATTRS{idProduct}=="0010|0110|0111|0114|0116|0401|0403|0405|0407|0410", \

$ cat /etc/udev/rules.d/70-u2f.rules 
ACTION!="add|change", GOTO="u2f_end"
# Yubico YubiKey
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="1050", ATTRS{idProduct}=="0113|0114|0115|0116|0120|0402|0403|0406|0407|0410", TAG+="uaccess"
# Alcor Micro Corp. AU9540 Smartcard Reader
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="058f", ATTRS{idProduct}=="9540", TAG+="uaccess"
# Happlink (formerly Plug-Up) Security KEY
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="2581", ATTRS{idProduct}=="f1d0", TAG+="uaccess"
#  Neowave Keydo and Keydo AES
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="1e0d", ATTRS{idProduct}=="f1d0|f1ae", TAG+="uaccess"
# HyperSecu HyperFIDO
KERNEL=="hidraw*", SUBSYSTEM=="hidraw", ATTRS{idVendor}=="096e", ATTRS{idProduct}=="0880", TAG+="uaccess"

Create file /etc/polkit-1/rules.d/99-smartcard.rules and substitute username with loginname of desired user:
# cat /etc/polkit-1/rules.d/99-smartcard.rules
polkit.addRule(function(action, subject) {
    if ( == "org.debian.pcsc-lite.access_pcsc" &&
        subject.user == "username") {
            return polkit.Result.YES;

polkit.addRule(function(action, subject) {
    if ( == "org.debian.pcsc-lite.access_card" &&
        action.lookup("reader") == 'Yubico Yubikey 4 OTP+U2F+CCID 00 00' &&
        subject.user == "username") {
            return polkit.Result.YES;    }

polkit.addRule(function(action, subject) {
    if ( == "org.debian.pcsc-lite.access_card" &&
        action.lookup("reader") == 'Yubico Yubikey 4 OTP+U2F+CCID 01 00' &&
        subject.user == "username") {
            return polkit.Result.YES;    }

Make sure environment variables are set correctly for local user (in .bashrc)
export GPG_SOCKET_FILE=$HOME/.gnupg/S.gpg-agent
export GPG_TTY=$(tty)
export SSH_AUTH_SOCK="$HOME/.gnupg/S.gpg-agent.ssh"

Enable pcscd:
systemctl enable pcscd.socket
systemctl enable pcscd.service

And reboot. Now you should have in KDE everything ready to run oathtool and gpg agent with yubikey 4 smartcard support. If there is error that yubikey is already in use, just reinsert it.

January 27, 2015

RHEL 6: Using linux cgroups as universal start/stop script

Have you ever had to deal with programs, which dont have correct start/stop scripts written and they are spawning many processes?

Maybe this universal start/stop script will help you. First enable cgroups support in OS, this is example for RHEL 6:

chkconfig cgconfig on 
chkconfig cgred on

and reboot.

Now comes script.

What you have to specify is STARTFILE - name of program, which doesnt have correct start/stop script and does spawn many other programs as children. Next specify unique name for CGROUP, under which we will track PIDs of spawned children. It can be any name. (#exec line I use for debugging, is not really needed).

In start part, I send pid of current (this) script running to prepared cgroup, which does tracking of pids for us. Then I run script STARTFILE as user "username".

In stop part script first send kill to all pids in CGROUP, after 1 second checks if there are any pids still running and if yes, after 10 seconds it sends kill -9. You can increase this delay, or even make a loop to periodicaly check for pids existence, its up to you.

# cat /etc/init.d/impex-onl
#exec >> /tmp/testing 2>&1


#### dont edit after this line ##### 

existing_pids () {
 ( ps --no-header -e -o pid | tr -d ' ' ; cat $PIDFILE ) | sort | uniq -d

cgcreate -g cpu:$CGROUP

start() {
 echo "echo $$ > $PIDFILE"
 echo $$ > $PIDFILE
 echo "starting $STARTFILE" >> /dev/stderr
 su -l -c "$STARTFILE" username

stop() {
 existing_pids | while read i; do
  kill $i
 sleep 1

 lines=`existing_pids|wc -l`
 if test "$lines" -gt 0; then
  sleep 10;
  existing_pids | while read i; do
   kill -9 $i

# See how we were called.
case "$1" in
	echo -n "Pids of running $CGROUP processes: " >> /dev/stderr
	existing_pids | xargs echo >> /dev/stderr
	lines=`existing_pids|wc -l`
	if test "$lines" -gt 0; then
	 exit 0
         exit 3
	echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
	exit 1

exit 0

November 10, 2014

RHEL 7 + selinux + sendmail greylisting

If you have combination RHEL 7 + selinux + sendmail greylisting (smf-grey), maybe this will help you resolve problems.

First identify from /var/log/audit/audit.log lines, which are related to your case, for example:

type=AVC msg=audit(1415573843.208:2609): avc:  denied  { getattr } for  pid=12969 comm="sendmail" path="/run/smfs/smf-grey.sock" dev="tmpfs" ino=65392 scontext=system_u:system_r:sendmail_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file
type=AVC msg=audit(1415575051.524:3068): avc:  denied  { write } for  pid=13609 comm="sendmail" name="smf-grey.sock" dev="tmpfs" ino=65392 scontext=system_u:system_r:sendmail_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=sock_file
type=AVC msg=audit(1415575527.915:3082): avc:  denied  { connectto } for  pid=13759 comm="sendmail" path="/run/smfs/smf-grey.sock" scontext=system_u:system_r:sendmail_t:s0 tcontext=unconfined_u:system_r:initrc_t:s0 tclass=unix_stream_socket

Install packages policycoreutils, policycoreutils-python, checkpolicy and setroubleshoot. Now execute:
cat file_with_needed_audit_lines |audit2allow -m greylist > greylist.te
checkmodule -M -m -o greylist.mod greylist.te
semodule_package -m greylist.mod -o greylist.pp
semodule -i greylist.pp

Oh, and if you need to enable other ports to some services, use semanage, example:

semanage port -l | grep smtp
semanage port -a -t smtp_port_t -p tcp 2525

September 5, 2012

Identify WWN of FC card interface in RHEL6

If you have many FC cards in your server like in this example, how to identify what are WWNs of dual FC interface in first ISP2532 card on bus?

# lspci |grep Fibre
05:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
05:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
08:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
08:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
0b:00.6 Fibre Channel: QLogic Corp. 8200 Series 10GbE Converged Network Adapter (FCoE) (rev 54)
0b:00.7 Fibre Channel: QLogic Corp. 8200 Series 10GbE Converged Network Adapter (FCoE) (rev 54)
11:00.6 Fibre Channel: QLogic Corp. 8200 Series 10GbE Converged Network Adapter (FCoE) (rev 54)
11:00.7 Fibre Channel: QLogic Corp. 8200 Series 10GbE Converged Network Adapter (FCoE) (rev 54)

You can run something like this:

# for i in /sys/class/fc_host/host*; do (cd $i; pwd -P; cat port_name) ; done

Now you see pairs of WWN and location on bus.

August 28, 2012

Online resize of multipath device in RHEL 6

Task: On RHEL 6 64-bit, perform online resize of filesystem /journal from 50GB to 100GB. That filesystem is part of multipath device brc001_journal_pv.

Serial console in RHEL 6

There are changes between RHEL5 and RHEL6 about how to setup serial console. In RHEL 6 you no longer specify it in /etc/inittab, but leave it on upstart daemon work.

1. If you DONT WANT to have serial console as your default console:

# cat /boot/grub/menu.lst
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal --timeout=10 serial console
title Red Hat Enterprise Linux (2.6.32-131.0.15.el6.x86_64)
 root (hd0,0)
 kernel /vmlinuz-2.6.32-131.0.15.el6.x86_64 ro root=/dev/mapper/rootvg-rootvol rd_LVM_LV=rootvg/rootvol rd_LVM_LV=rootvg/swapvol rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto quiet console=ttyS0,9600n8 console=tty0
 initrd /initramfs-2.6.32-131.0.15.el6.x86_64.img

# grep ttyS0 /etc/securetty 

# cat /etc/init/ttyS0.conf
start on runlevel [345]
stop on runlevel [S016]

instance /dev/ttyS0
exec /sbin/agetty ttyS0 9600 vt100-nav

And initialize console with:

# initctl start ttyS0

2. If you DO WANT to have serial console as default console:

Add serial and terminal lines to menu.lst as above plus append "console=tty0 console=ttyS0,9600" to kernel line in /boot/grub/menu.lst

Thats all, as upstart will automatically start serial login process on last specified console parameter extracted from kernel line (in this case ttyS0).

July 30, 2012

How to create ultra small Linux machine on Amazon EC2

If you are using amazon aws services, you know that by default Amazon will let you use virtual machine with 8GB root filesystem. But sometimes you are using under 2GB of actual space, so why not to shrink this machine and pay only for 2GB instead of 8GB? This mini-tutorial is for that.

June 22, 2011

Set outgoing IP address in Solaris 10

root@sol3 # route get default
   route to: default
destination: default
       mask: default
  interface: e1000g0:1
 recvpipe  sendpipe  ssthresh    rtt,ms rttvar,ms  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0

root@sol3 # ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
        inet netmask ff000000
e1000g0: flags=9040843 mtu 1500 index 2
        inet netmask ffffff00 broadcast
        groupname prodgr
        ether 0:c:29:20:e6:9
e1000g0:1: flags=1000843 mtu 1500 index 2
        inet netmask ffffff00 broadcast
e1000g0:2: flags=1000843 mtu 1500 index 2
        inet netmask ffffff00 broadcast
e1000g1: flags=69040843 mtu 1500 index 3
        inet netmask ffffff00 broadcast
        groupname prodgr
        ether 0:c:29:20:e6:13

root@sol3 # route delete default
delete net default: gateway
root@sol3 #
root@sol3 # route add default -setsrc
add net default: gateway

root@sol3 # route get default
   route to: default
destination: default
       mask: default
  interface: e1000g0:1
 recvpipe  sendpipe  ssthresh    rtt,ms rttvar,ms  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0

February 10, 2010

rTorrent patch - almost linear downloading

Are you rTorrent user? Want to play .avi files as soon as possible, while downloading? Then maybe this patch will you find handy. Info: tested on rtorrent/libtorrent 0.8.5/0.12.5  All changes are affecting only one file: libtorrent-0.12.5/src/download/

Update, May 2010: Patch applicable to rtorrent 0.8.6/0.12.6

November 23, 2009

IP multipathing with RHEL 5

In this page you can find sample working configs, which do following:

- setup two bond interfaces (bond0, bond1) with 802.3ad support
- setup two aliases for bond0
- setup two VLANs on bond1
- setup custom routes for these VLANs
- setup default gw trought bond0

All this we make on RedHat Linux 5.4, server is SunFire x4150 with 4x onboard ethernet devices.

IO multipathing with RHEL 5

In this post, we describe working IO multipathing configuration. Used technologies: RedHat Linux 5.4 64bit, 2x SunFire x4150, Sun StorageTek 2540, 2x Qlogic FC HBA.

November 22, 2009

Turn on word wrapping in "pre" tag

On Blogger, you need to edit template to turn on wrapping of tag "pre", otherwise long lines are not showing correctly. So paste this to your template:
pre {
white-space: -moz-pre-wrap; /* Mozilla, supported since 1999 */
white-space: -pre-wrap; /* Opera 4 - 6 */
white-space: -o-pre-wrap; /* Opera 7 */
white-space: pre-wrap; /* CSS3 */
word-wrap: break-word; /* IE 5.5+ */
 background: #dddddd;
 display: block;
 padding: 0.5em 1em;
 border: 1px solid #bebab0;