Linux

From Aftenstorm Intelligence Unit
Jump to: navigation, search

Contents

This is a collection of everything I had to do read and do to get things working

Linux on the Desktop Debate

Linux does not have a shot at the desktop and never will.
Define desktop. If it is the principle UI that you use to communicate with the Internet and run applications, then...
There are 7 billion people. ~2 billion PCs and ~5 billion phones worldwide. The growth UI will be in phones and other low cost devices. *That*, is the new desktop and it will be Linux-based.
My TiVo and my Blu-ray player run Linux. That could be considered a media desktop. Or media UI. For some, sadly, TV is their principle app.
Linux has won the desktop OS wars. It's just that nobody knows it yet.
As for desktop UI apps, the future is HTML5.

source: http://linux.slashdot.org/story/11/08/10/1555219/Old-Arguments-May-Cost-Linux-the-Desktop


Linux, and open source in general, will never be that popular, simply because of cognitive load. It's software designed by engineers, with no clear understanding of style or ergonomics.

To use a car example, it's like a car with high torque and excellent gas mileage, but ugly to look at and the instruments are labelled differently and in the back seat.
Many companies hire artists and usability experts to look at the final product and make tweaks and recommendations. Some even take the trouble to  engage focus groups of customers to find out what features are confusing, what aspects are uncomfortable, what looks ugly. They take this information  and change their product for the better.
For the most part, the success of Apple products is for this reason: the iPod was not the first MP3 player on the market, but it's usability and  aesthetic appeal and robustness made it highly popular.
Open source, on the other hand, is usually done by a single engineer putting in most of the effort. The results usually have the following pattern:
1) Documentation: Writing documentation is boring. Put up a wiki and let the users fill in the details.
2) Aesthetic looks: This is not important. Give the user a panel to change the environment to suit their tastes.
3) Compatibility: Not important. "Search for text" is different in every application, it's impossible for your fingers to memorize the action.
4) Simplicity: More features is better! Try viewing the man page for "ls" some time. Or preferences in VLC.
5) Descriptives: Don't choose descriptive names for anything. Instead of "Internet Explorer", "Paint Shop Pro" and "Media Player", use terms like  "Gimp, Firefox, and VLC".
This last is one reason why old folks have a tough time using the new technology. They have to learn a completely new language: Every random word  that they *thought* they knew ("pidgin", "handbrake", "calibre") means something different in the new system. 

Gimme a break. The top five or so open source projects try to deal with these issues, but the overwhelming majority are robust, strong, functional, and totally enigmatic.

Where are the open source tech writers? The ones who take that part of the problem and work alongside the engineers to ensure quality documentation?  Where are the open source ergonomic experts, the usability analysts, the aesthetic artists? Who ever does usability studies, or consistency between  apps?
Until the engineers get a clue, open source projects will never be more than a closet of hobbyist projects.
Making good software is more than robust coding.

Things I had to learn when returning to Linux in 2012

I had to learn the following new modern Linux approaches to booting, mounting, partitioning, encrypting and ip-security:

gpt partitions vs mbr
ncat
mtr
curl
initramfs
initrd
parted
dm_crypt
cryptsetup
LUKS
UUIDs instead of classical device names
LVM

unsure of the suitability, stability and recoverability

GRUB2

no guide available, very poorly documented not mastering grub2 at all

IPV6 and IPTABLES
I had to re-learn how to write conditional scripts.
How to boot a system from an install cd
How to export system variables
To regain the wizard-of-bash feeling again

Apple Products - iPad, iPhone, iPod

Transferring photos off device

gphoto2 --port=usb: -R -P

This will dump all photos and movies to current directory

Finding open ports

Finding ports running

lsof -iTCP -sTCP:LISTEN

lsof -iUDP -sUDP:LISTEN

netstat --listen

Proxmox

LXC

Making containers super privelieged - Running Docker

place this into the bottom of an 11x.conf in /etc/pve/lxc folder

  lxc.aa_profile: unconfined
  lxc.cgroup.devices.allow: a
  lxc.cap.drop:
  lxc.mount.auto: cgroup


Too many open files

    sysctl fs.inotify.max_user_instances=512

https://forum.proxmox.com/threads/failed-to-allocate-directory-watch-too-many-open-files.28700/

Rescanning hardware

USB

  sudo service udev restart

lsblk lshw


SCSI

http://www.unixarena.com/2013/06/how-to-scan-new-fc-luns-and-scsi-disks.html

[root@mylinz1 ~]# ls /sys/class/scsi_host/host host0 host1 host2 In this case,you need to scan host0,host1 & host2.

3.Scan the SCSI disks using below command.

  [root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host0/scan
  [root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host1/scan
  [root@mylinz1 ~]# echo "- - -" > /sys/class/scsi_host/host2/scan


SSH

Private Keys Public Keys

https://www.certdepot.net/rhel7-configure-ssh-key-based-authentication/

Generate privatekey and publickey

    ssh-keygen -b 2048 -t rsa

Copy the public key to the server

    ssh-copy-id -i .ssh/id_rsa.pub user01@server2.example.com

Mosh

How to fix LOCALE issues with connecting to Mosh servers

Run the following on the server

export LANG="en_US.UTF-8"
export LC_COLLATE="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
export LC_MESSAGES="en_US.UTF-8"
export LC_MONETARY="en_US.UTF-8"
export LC_NUMERIC="en_US.UTF-8"
export LC_TIME="en_US.UTF-8"
export LC_ALL="en_US.UTF-8"

source: https://github.com/keithw/mosh/issues/224

UPS

https://wiki.archlinux.org/index.php/APC_UPS

Install the apcupsd package. Configure APC UPS

The main configuration file for the APC UPS daemon can be found here: /etc/apcupsd/apcupsd.conf In the following example, the lines of text are changed to support a USB style cable: Before: UPSCABLE smart

UPSTYPE smartups

DEVICE /dev/ttyS0 After: UPSCABLE usb

UPSTYPE usb

DEVICE /dev/usb/hiddev0-15

If apcupsd does not start, in /etc/defaults/apcupsd change ISCONFIGURED=no to ISCONFIGURED=yes Test

First, enable and start the systemd service, apcupsd.service. Next, wait about a minute and confirm the daemon is running and properly monitoring the battery:

  1. apcaccess status


Iptables

Simple port forwarding

#!/bin/bash
# -i eth0 is the INCOMING network device
# -dport is the external port
# -to is the inside ip-address
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 8000 -j DNAT --to 128.39.115.4:443
iptables -A INPUT -p tcp -m state --state NEW --dport 8000 -i eth0 -j ACCEPT


source: http://www.debian-administration.org/articles/73

Benchmarking

sysbench method

Needs sysbench package installed.

 for i in cpu threads mutex memory; do
        sysbench --test=$i run
 done

Apache2

Install and configure

Great article from DigitalOcean: https://www.digitalocean.com/community/articles/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu

After install configuring

Enable Mod_Rewrite

http://askubuntu.com/questions/48362/how-to-enable-mod-rewrite-in-apache


VLC - Videolan

Changing VLC Language in Linux

Run this from the terminal:

env LANGUAGE=en vlc

source: http://www.linuxformat.com/forums/viewtopic.php?p=91260

Running VLC as root

Run this command, to hack the vlc binary

sed -i 's/geteuid/getppid/g' `which vlc`

Benchmarking Linux

sysbench
for i in cpu threads mutex memory; do
        sysbench --test=$i run
done

source with other suggestions also: http://serverfault.com/questions/294985/simple-linux-server-benchmark

Networking

Bandwidth monitoring

Bandwidth in computer networking refers to the data rate supported by a network connection or interface. One most commonly expresses bandwidth in terms of bits per second (bps). The term comes from the field of electrical engineering, where bandwidth represents the total distance or range between the highest and lowest signals on the communication channel (band).

Bandwidth represents the capacity of the connection. The greater the capacity, the more likely that greater performance will follow, though overall performance also depends on other factors, such as latency.

Here is the list of bandwidth monitoring tools for your network bandwidth

bmon bwbar bwm bwm-ng iftop iperf ipfm speedometer cbm ibmonitor pktstat mactrack MRTG Cacti

Now we will see each tool separately

bmon

bmon is a portable bandwidth monitor and rate estimator running on various operating systems. It supports various input methods for different architectures.

Various output modes exist including an interactive curses interface, lightweight HTML output but also formatable ASCII output

Current Stable Version :- 2.1.0

Install bmon in Ubuntu

sudo apt-get install bmon

This will complete the installation.

If you want to open the application you need to enter the following command

bmon

you should see the following output

If i select the eth0 i should see all the traffic details

If you want to know more available options check man page

bwbar

This program will output a PNG and a text file that can be used in scripts or be included in web pages to show current bandwidth usage. The amount of total bandwidth can be customized. The PNG output appears as a bar graph showing maximum possible usage with the current inbound or outbound usage shown as a differently colored bar.

Current Stable Version :- 1.2.3

Install bwbar in ubuntu

sudo apt-get install bwbar

This will complete the installation

There is a /etc/default/bwbar file to configure before this package will work and here is my config

  1. Directory to put files into

DIR=/var/www

  1. Who to run as

RUNAS=www-data

  1. What are the options (eth0, scale of 1.5mbps)

OPTIONS="eth0 1.5″

Now you need to chnage the permissions using the following command

sudo chown -R www-data:www-data /var/www/bwbar/

Start the bwbar using the following command

sudo /etc/init.d/bwbar start

You can also launch bwbar using the following command

bwbar eth0 100 -x 200 -y 7 -t 10 -p network.png -f network.txt -d /var/www/bwbar &

Now you need to go to http://serverip/bwbar you should see the bandwidth bar as follows

If you want to know about available option check man page

bwm

BandWidth Monitor This is a very tiny bandwidth monitor (not X11). Can monitor up to 16 interfaces in the in the same time, and shows totals too.

Current Stable Version :- 1.1.0

Install bwm ubuntu

sudo apt-get install bwm

This will complete the installation now

If you want to run the application use the following command

bwm

Output looks like below

If you want more details and available options check man page

bwm-ng

small and simple console-based bandwidth monitor.Bandwidth Monitor NG is a small and simple console-based live bandwidth monitor.

Current Stable Version :- 0.5

features

supports /proc/net/dev, netstat, getifaddr, sysctl, kstat and libstatgrab

unlimited number of interfaces supported

interfaces are added or removed dynamically from list

white-/blacklist of interfaces

output of KB/s, Kb/s, packets, errors, average, max and total sum

output in curses, plain console, CSV or HTML

configfile

Install bwm-ng in ubuntu

sudo apt-get install bwm-ng

This will complete the installation

If you want to run the application you need to use the following command

bwm-ng

Output looks like below

If you want more details and available options check man page

iftop

iftop does for network usage what top does for CPU usage. It listens to network traffic on a named interface and displays a table of current bandwidth usage by pairs of hosts.

Current Stable Version :- 0.17

Install iftop in ubuntu

sudo apt-get install iftop

This will complete the installation

If you want to run the application you need to use the following command

iftop

Output looks like below

If you want more details and available options check man page

ipfm

IP Flow Meter (IPFM) is a bandwidth analysis tool, that measures how much bandwidth specified hosts use on their Internet link.

Current Stable Version :- 0.11.5

Install ipfm in Ubuntu

sudo apt-get install ipfm

This will complete the installation

Now if you want to configure this for your network you need to copy example configuration file from /usr/share/doc/ipfm/examples if you want to check this file click here

once you configure this file you need to start the service using the following command

sudo /etc/init.d/ipfm start

If you want more details and available options check man page

Speedometer

Measure and display the rate of data across a network connection or data being stored in a file.

Current Stable Version :- 2.4

Install speedometer in ubuntu

Check which version of python is the default by running

python -V

Then issue the following commands as root to install speedometer (choose the correct Urwid package for your python version, ie. if python -V reports version 2.3.X then install python2.3-urwid)

apt-get install python2.4-urwid

Download the speedometer.py source file.

As user issue the following commands in the directory that you downloaded the source file

sudo cp speedometer.py /usr/local/bin/speedometer

sudo chown root: /usr/local/bin/speedometer

sudo chmod 755 /usr/local/bin/speedometer

Now you can run the speedometer application using the following

/usr/local/bin/speedometer

Now you should see the following screen output

Speedometer Usage

Usage: speedometer [options] tap [[-c] tap]

Available options

speedometer -h

Usage: speedometer [options] tap [[-c] tap]...

Monitor network traffic or speed/progress of a file transfer. At least one tap must be entered. -c starts a new column, otherwise taps are piled vertically.

Taps:

[-f] filename [size] display download speed [with progress bar]

-f must be used if directly following another

file tap without an expected size specified

-rx network-interface display bytes received on network-interface

-tx network-interface display bytes transmitted on network-interface

Options:

-i interval-in-seconds eg. "5″ or "0.25″ default: "1″

-p use plain-text display (one tap only)

-b use old blocky display instead of smoothed display even when UTF-8 encoding is detected

-z report zero size on files that don't exist instead of waiting for them to be created

Usage Examples

How long it will take for my 38MB transfer to finish?

speedometer favorite_episode.rm $((38*1024*1024))

How quickly is another transfer going?

speedometer dl/big.avi

How fast is this LAN?

$ cat /dev/zero | nc -l -p 12345

$ nc host-a 12345 > /dev/null

$ speedometer -rx eth0

How fast is the upstream on this ADSL line?

speedometer -tx ppp0

How fast can I write data to my filesystem? (with at least 1GB free)

dd bs=1000000 count=1000 if=/dev/zero of=big_nothing & speedometer big_nothing

cbm

cbm — the Color Bandwidth Meter — displays the current traffic on all network devices.

Current Stable Version :- 0.1-1

Install cbm in ubuntu

First you need to download the .deb package from here

once you have the .deb package you need to install using the following comamnd

sudo dpkg -i cbm_0.1-1_i386.deb

this will complete the installation now if you want to use the appltcation use the following command

cbm

Output looks like below

pktstat

pktstat listens to the network and shows the bandwidth being consumed by packets of various kinds in realtime. It understands some protocols (including FTP,HTTP, and X11) and adds a descriptive name next to the entry (e.g., ‘RETR cd8.iso', ‘GET http://slashdot.org/' or ‘xclock -fg blue').

Current Stable Version :- 1.8.3

Install pktstat in Ubuntu

First you need to download .rpm package from here once you have the .rpm package you need to convert this .rpm file to .deb file using alien

Install alien

sudo apt-get install alien

Now you need to use the follwoing command to convert .rpm to .deb

sudo alien -k pktstat-1.7.2q-0.i386.rpm

Now you should be having pktstat_1.7.2q-0_i386.deb package

Install pktstat in Ubuntu

sudo dpkg -i pktstat_1.7.2q-0_i386.deb

This will complete the installation now you can open the application using the following command

pktstat

output looks like below


If you want more available options for pktstat check man page

ibmonitor

ibmonitor is an interactive linux console application which shows bandwidth consumed and total data transferred on all interfaces.

Current Stable Version :- 1.4

Its main features are:

Shows received, transmitted and total bandwidth of each interface

Calculates and displays the combined value of all interfaces

Displays total data transferred per interface in KB/MB/GB

Values can be displayed in Kbits/sec(Kbps) and/or KBytes/sec(KBps)

Can show maximum bandwidth consumed on each interface since start of utility

Can show average bandwidth consumption on each interface since start of utility

The output with all features (max, avg and display in Kbps and KBps) easily fits on a 80×24 console or xterm

Can interactively change its output display format depending on key pressed by user.

Install ibmonitor in Ubuntu

First you need to download the latest version from here

wget http://ovh.dl.sourceforge.net/sourceforge/ibmonitor/ibmonitor-1.4.tar.gz

Now you have ibmonitor-1.4.tar.gz

Extract this file using the following commands

tar xvfz ibmonitor-1.4.tar.gz

cd ibmonitor

If you want to run the application use the following command

Once you are in ibmonitor folder use

./ibmonitor

Output looks like the following screen

iperf

While tools to measure network performance, such as ttcp, exist, most are very old and have confusing options. Iperf was developed as a modern alternative for measuring TCP and UDP bandwidth performance.

Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss.

Current Stable Version :- 2.0.2

Install iperf in ubuntu

sudo apt-get install iperf

iperf Syntax

iperf [-s|-c host] [options]

Example

iperf -c server address -F file-name

iperf -c server address -I

The -F option is for file input.

The -I option is for input from stdin.

If you want more details and available options check man page

tcptrack

tcptrack is a sniffer which displays information about TCP connections it sees on a network interface. It passively watches for connections on the network interface, keeps track of their state and displays a list of connections in a manner similar to the unix ‘top' command. It displays source and destination addresses and ports, connection state, idle time, and bandwidth usage.

Current Stable Version :- 1.1.5

Install tcptrack in Ubuntu

sudo apt-get install tcptrack

this will complete the installation

tcptrack Syntax

tcptrack [-dfhvp] [-r ] -i []

Examples

tcptrack requires only one parameter to run: the -i flag followed by an interface name that you want tcptrack to monitor. This is the most basic way to run tcptrack

tcptrack -i eth0

tcptrack can also take a pcap filter expression as an argument. The format of this filter expression is the same as that of tcpdump and other libpcap-based sniffers. The following example will only show connections from host 10.45.165.2

tcptrack -i eth0 src or dst 10.45.165.2

The next example will only show web traffic (ie, traffic on port 80)

tcptrack -i eth0 port 80

The following output screen will show you more details

MRTG

The Multi Router Traffic Grapher or just simply MRTG is free software for monitoring the traffic load on network links. It allows the user to see traffic load on a network over time in graphical form.

Current Stable Version :- 2.15.0

Project Homepage

http://oss.oetiker.ch/mrtg/

Cacti

Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive,easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.

Current Stable Version :- 0.8.6i

Project Homepage

http://www.cacti.net

Regarding MRTG and Cacti i am going write detailed articles in future

Incoming search terms:

   ubuntu network monitor
   iperf ubuntu
   ubuntu bandwidth monitor
   bandwidth monitor ubuntu
   network monitor ubuntu
   bwbar
   ubuntu iperf
   network monitoring ubuntu
   ubuntu network monitoring
   iperf for ubuntu

GIT

SEE the dedicated git page

IPV6

Disable IPV6

How to turn off IPv6

from shell:

sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1
sysctl -w net.ipv6.conf.lo.disable_ipv6=1

permanent at bootup [debian]: edit /etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1

WGET

Scrape website for images

wget -nd -r -l 2 -A jpg,jpeg,png,gif http://t.co
# -nd = --no-directories (all files are saved in the current directory or -P)
# -r -l 2 = --recursive --level=2
# -A jpg,jpeg = --accept jpg,jpeg

source: http://stackoverflow.com/questions/4602153/how-do-i-use-wget-to-download-all-images-into-a-single-folder

CURL

Wget functionality with CURL

curl -OL h ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.30.tar.gz > /usr/local/src/pcre-8.30.tar.gz

source: http://about.piratereverse.info/proxy/

Initrd

I never had to create initrd files before. So I had to turn to Cyberciti.biz's great linux wiki guide. http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html

cd /boot
mkinitrd -o initrd.img-2.6.25 2.6.25
(adjust the filename to the kernel version number)

More initrd knowledge source: http://www.dark.ca/2009/06/10/initrd-modules-and-tools/

Shell stuff

Show Unmounted Devices

for DISKLABEL in `find /dev/disk/by-label/ -type l`; do RES=`readlink -f $DISKLABEL`; grep -q "^$RES" /proc/mounts ||echo "$RES (${DISKLABEL//*\//})"; done

source: http://www.linuxquestions.org/questions/linux-hardware-18/show-all-unmounted-disk-labels-722515/

Show the most used commands (from bash_history)

sed -e 's/^sudo //' -e 's/^man *//' .bash_history |awk '{ print  }'|sort|uniq -c|sort -n


Finding the extensions in a directory

find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -g

Compare two files

http://www.cyberciti.biz/faq/how-do-i-compare-two-files-under-linux-or-unix/

Extract Unrar Unzip files recursively

Run this oneliner: 
find -iname \*.zip -exec unzip {} \;

source: http://superuser.com/questions/248287/7-zip-extract-recursively


or this

find . -name '*.rar' -execdir 7z x {} \; 
find . -name '*.rar' -execdir unrar x -y {} \; 


source: http://superuser.com/questions/410972/recursive-unrar-of-several-folders

tmux

How to run a process as a psuedo-daemon with tmux

For those times when you just have to get it to run at boot, no matter what SystemD tells you.

#!/bin/bash
# this script is called "sess"
tmux new-session -d -s sess1
# this statement is a life-saver for tmux detached sessions
tmux set-option -t sess1 set-remain-on-exit on
# In my case running the script in a new window worked
tmux new-window -d -n 'nameofWindow' -t sess1:1 'sudo /home/pi/bin/script.py'
exit 0

Now the following script was called from the rc.local and the Pi was rebooted. Eventually on reboot when you attach the session using sudo tmux a Once gets a tmux session with 2 windows

Initial one is just an empty session triggered due to tmux new-session -d -s sess1 and the another one from the tmux new-window command which can be opened using CTRL+B + 1 since it was mentioned as sess1:1 (note: Hot keys may vary for user, the default tmux hotkey (bindkeys) are CTRL+B) Inference

If the script ends with an error, the Window will show you where the error was in my case errors in my Python script and at the Bottom it will show Pane is Dead. Hence due to errors in the script the tmux session was exited without giving any relevant log(feedback) hence no output was logged in the above mentioned /tmp/tmux.log

Hence it is always recommended using the set-remain-on-exit on when running scripts with tmux in case if there are faults in the the script in detached mode

Source: https://superuser.com/questions/1091887/tmux-does-not-trigger-when-placed-in-rc-local

Tmux Hotkeys

CTRL+B-D to detach the program
tmux attach to attach again
Nice howto with a display of hotkeys
http://net.tutsplus.com/tutorials/tools-and-tips/intro-to-tmux/

snagged from url above: Creating Panes Or, in other words, splitting the main window. First of all, I must say that each tmux command is prefixed using the following key combination: <Ctrl-b>. This can be changed, but we will learn how to configure and customize tmux later on.

So, in order to split a window vertically (or in right and left panes) the following command should be used:

1 <Ctrl-b>% and to split the window in horizontal panes you can use:

1 <Ctrl-b>" And the result should look like following:

splitting windows Moving From One Pane to Another and Positioning Panes In order to move the cursor from one pane to the other (activating panes), the arrow keys are used. The command looks like this:

1 <Ctrl-b>[Up, Down, Right, Left] If you want to go to the previously active pane, you can use the following command:

1 <Ctrl-b>; Also, if you are not satisfied with the position of a pane, you can rotate the panes using the command:

1 <Ctrl-b><Ctrl-o> Resizing Panes Once created, you can change each panes size, in one cell step, using:

1 <Ctrl-b><Ctrl-Up[Down][Left][Right]> or in five cells step using:

1 <Ctrl-b><Meta-Up[Down][Left][Right]> Closing a Pane When you want to close the current pane you can use:

1 <Ctrl-b>x Create a New Window Sometimes you may want to create another window, for example, to work on another project. This window might contain a completely different set of panes with different programs in each of them. To do so, issue the following command:

1 <Ctrl-b>c Then if you want to switch to the next window you can use:

1 <Ctrl-b>n And you can switch to the previous window by using:

1 <Ctrl-b>p Or you might select the window interactively with:

1 <Ctrl-b>w Closing a Window In order to close the currently opened window, you use:

1 <Ctrl-b>& Copy Mode Suppose you have issued a command on the terminal and the output of the command does not fit in one screen, so you'll need to scroll up in order to see the entire output. If you try pressing the Up key, this won't scroll you up, as it will only show you your command history. To scroll up the screen, use the following command:

1 <Ctrl-b>[ And then hit one of the following keys: Up, Down, PgUp or PgDn to scroll up or down.

Also, when in this mode you can copy text from the history and then paste it with:

1 <Ctrl-b>] In order to exit this insert mode, just hit esc.

Now there are a lot of other commands bound to various keys. You can list all of the key bindings by issuing:

1 <Ctrl-b>? Configuring Tmux

tmux is highly configurable. The configuration file is either /etc/tmux.conf for system wide settings or (recommended) ~/.tmux.conf for user specific settings.

Change the Prefix Key One of the first things that most users change is the mapping of the prefix key (since <Ctrl-b> doesn't seem to be so handy). Most users change it to <Ctrl-a>. This can be done like so:

1 2 3 set -g prefix C-a unbind C-b bind C-a send-prefix The -g option in the first command tells tmux that this is a global option, meaning this is set for all windows and sessions.

Change the Key Bindings Some users may prefer Vi or Emacs like bindings for the key actions. This is done using:

1 2 set -g status-keys vi setw -g mode-keys vi The setw command, sets the option for the window (affects all the panes in a window).

Status Line You can perform various configurations of the status line: you can turn it on or off, you can change its background and foreground color, you can change what information is displayed inside it, etc.

To turn the status bar off, issue the following command:

1 set -g status off Or you may try something like this:

1 2 3 set -g status-bg blue set -g status-fg white setw -g status-left #H:#S at #W:#T ... which changes the status line background to blue, the text color to white and displays to the left of the status bar the hostname of localhost, followed by a colon and the session name followed by the 'at' string and the window name, a colon, and lastly the pane title.

You can also display the status line at the bottom or at the top of the window:

1 set -g status-position [bottom | top] For further information on configuration and other configuration options you can check the options section of the manual.


Further tmux description
http://lukaszwrobel.pl/blog/tmux-tutorial-split-terminal-windows-easily
Arch Linux Tmux article
https://wiki.archlinux.org/index.php/tmux#Installation

Init scripts

Understanding PID 1

http://0pointer.de/blog/projects/systemd.html
http://felipec.wordpress.com/2013/11/04/init/
http://www.reddit.com/r/linux/comments/1pvrhe/demystifying_the_init_system_pid_1/

Systemd

https://wiki.archlinux.org/index.php/Systemd
https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530

Upstart

Sysvinit

Init scripts

Initramfs

setting a static ip in initramfs

boot time argument --append=IP=192.168.0.10::192.168.0.1:255.255.255.0:eugenemdavis.com:eth0:off


http://www.eugenemdavis.com/comment/1533

SSH to decrypt encrypted LVM during headless server boot?

http://unix.stackexchange.com/questions/5017/ssh-to-decrypt-encrypted-lvm-during-headless-server-boot

http://blog.nguyenvq.com/2011/09/13/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu/'

https://wiki.recompile.se/wiki/Mandos

Adding more TTYs

mknod /dev/tty13 c 4 13" will create tty13.
"the devices are majorblock 4, minor block is whatever tty number it is"
my inittab coding is
1:2345:respawn:/sbin/getty 38400 tty1
2:23:respawn:/sbin/getty 38400 tty2
3:23:respawn:/sbin/getty 38400 tty3
4:23:respawn:/sbin/getty 38400 tty4
5:23:respawn:/sbin/getty 38400 tty5
6:23:respawn:/sbin/getty 38400 tty6
8:23:respawn:/sbin/getty 38400 tty8
9:23:respawn:/sbin/getty 38400 tty9
10:23:respawn:/sbin/getty 38400 tty10
11:23:respawn:/sbin/getty 38400 tty11
12:23:respawn:/sbin/getty 38400 tty12
13:23:respawn:/sbin/getty 38400 tty1

source: http://www.linuxquestions.org/questions/debian-26/more-then-12-tty-console-292911/

Fine tuning Linux for desktop use

Add the following to /etc/sysctl.conf

#System booster
vm.swappiness=1
vm.vfs_cache_pressure=50

Source: http://rudd-o.com/linux-and-free-software/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that

Re-initialising USB devices

http://unix.stackexchange.com/questions/7412/how-to-reconnect-a-logically-disconnected-usb-device

Detecting and setting up Nvidia and ATI cards

Envy 
EnvyNG 
Jockey

EnvyNG - http://albertomilone.com/projects.html Jockey drivermanager - https://launchpad.net/jockey

Changing brightness

/proc/acpi is being deprecated. Find the equivalents of what you want to do in /sys. Backlight stuff is in /sys/class/backlight. Or you compile your own  kernel, it's still possible to activate the deprecated /proc/acpi stuff. 
For instance,
Code:
echo $BRIGHTNESS > /sys/class/backlight/acpi_video0/brightness

EDIT: I have not succeeded with this technique yet. (Boosting a crappy LCD panel above 255 levels)

Chroot - changing root partition

cd /mnt/pioneer-system # or where you are preparing the chroot dir
mount -t proc proc proc/
mount -t sysfs sys sys/
mount -o bind /dev dev/
then chroot!

source: http://superuser.com/questions/165116/mount-dev-proc-sys-in-a-chroot-environment

http://www.intertwingly.net/blog/2012/01/04/Bootstrapping-Debian-Unstable
https://wiki.archlinux.org/index.php/Change_Root
http://www.goudkov.com/public/articles/changing_distro.jsp
http://en.gentoo-wiki.com/wiki/Chroot_from_a_livecd
http://unixwiz.net/techtips/chroot-practices.html

Difference between chroot and pivot_root

Re: Difference between pivot_root and chroot?
   To: debian-handheld@lists.debian.org
   Subject: Re: Difference between pivot_root and chroot?
   From: Patrick Albuquerque <patrick@albuquerque.ca>
   Date: Sat, 25 Dec 2004 16:02:34 -0600
   Message-id: <20041225220234.GI4281@albuquerque.ca>
   In-reply-to: <20041225113509.GB6104@w-m-p.com>
   References: <1103627278.13105.267.camel@trixter> <20041221195320.GE29947@w-m-p.com> <1103660077.13107.345.camel@trixter> <20041221203034.GG29947@w-m-p.com> <20041224195415.GH4281@albuquerque.ca> <20041225113509.GB6104@w-m-p.com>

On Sat, Dec 25, 2004 at 05:35:10AM -0600, Klaus Weidner wrote: > On Fri, Dec 24, 2004 at 01:54:15PM -0600, Patrick Albuquerque wrote: > > Would you be able to explain the difference between pivot_root and chroot? > > > > Sorry, it is not clear to me, from reading the man pages, or google. > > The main difference is that pivot_root is intended to switch the complete > system over to a new root directory and remove dependencies on the old > one, so that you would be able to unmount the original root directory and > proceed as if it had never been in use. chroot is intended to apply for > the lifetime of a single process, with the rest of the system continuing > to run in the old root dir, and the system being unchanged when the > chrooted process exits. > > The difference is shrinking now that Linux has per-process namespaces > anyway. If you "chroot" a single init-type process that never exits and > don't care about unmounting the original root, it's the same for > practical purposes. > > -Klaus >

Thanks, that is very helpful.

Happy Holidays, Patrick. source: http://lists.debian.org/debian-handheld/2004/12/msg00029.html

Making Broadcom B43xx Wireless Cards work Offline Version

Download this package: http://packages.ubuntu.com/precise/utils/b43-fwcutter
And these firmware files:
http://mirror2.openwrt.org/sources/broadcom-wl-5.10.56.27.3_mipsel.tar.bz2
http://downloads.openwrt.org/sources/wl_apsta-3.130.20.0.o
Then do this on the laptop
cd ~/Desktop/b43
sudo dpkg -i b43-fwcutter*
sudo b43-fwcutter -w /lib/firmware broadcom-wl-5.10.56.27.3/driver/wl_apsta/wl_prebuilt.o
sudo b43-fwcutter -w /lib/firmware wl_apsta-3.130.20.0.o
sudo chmod 755 /lib/firmware/b43
sudo chmod 755 /lib/firmware/b43legacy
sudo modprobe -r b43
sudo modprobe b43
sudo modprobe b43legacy

Making Broadcom B43xx Wireless Cards work Online Version

ON LINUX MINT, UBUNTU & DEBIAN: STEP01 -

   sudo apt-get install firmware-b43-installer

STEP02 -

   modprobe b43

profit!


Source: http://forums.linuxmint.com/viewtopic.php?f=194&t=85474

https://help.ubuntu.com/community/WifiDocs/Driver/bcm43xx#b43%20-%20Internet%20access

http://wireless.kernel.org/en/users/Drivers/b43#Supported_devices

I solved the problem! Wireless works! This may sound ridiculous, but I had never heard of the wireless.kernel.org website. I stumbled across it when I caught an error message during the bootup from a live USB stick of Linux Mint 7 (LTS) that I thought I'd try to see if I got better luck.

That error message led me to this page about the b43 driver.

I followed the instructions for installing Device firmware installation: Ubuntu/Debian ... with the caveat that I needed a variant installer package for this machine: firmware-b43-lpphy-installer instead of the stock firmware-b43-installer.

So, in the end, here are the exact steps I used to get wireless working. (Note that all internet access until wireless functioned was through an ethernet cable, of course.)

1. Installed fresh copy of LMDE 201109

2. Fully upgraded the system and restarted.

3. Installed the b43 driver firmware per the instructions at wireless.kernel.org and restarted.

STEP01 -

   sudo apt-get install firmware-b43-installer

STEP02 -

   modprobe b43


4. Installed the debian broadcom-sta driver using module assistant per the instructions here on the Linux Mint forums.

4a. Blacklisted all of the following modules (i.e., added entries to /etc/modprobe.d/blacklist.conf): b44, b43, b43legacy, ssb, brcm80211, acer_wmi.

5. Restarted.

6. Profit!

I hope this guide proves helpful for others caught in this particularly painful trap.

FILESYSTEMS

PROC

snagged in its entirety from Petur.EU blog: http://www.petur.eu/blog/?p=320

Using the /proc filesystem
// February 28th, 2011 // linux




The proc filesystem is a special filesystem found on most UNIX-based systems.
It holds a great deal of information, in ASCII format, most of which is not very friendly to the average user.

It is important that you keep in mind that the files under /proc are not kept on a physical storage, meaning they are subject to change after reboot. Also, they should not really be called files as they are pseudo-files, as they exist only in memory.
I break that rule on regular basis and intend to do that also in this article.

I’ve made a list of some of the files i find to be of most use.

/proc/[pid]/

/proc contains a directory named after the PID (process identification number) of each excising process on the system.
Lets take a look at some of the files found there.

/proc/[pid]/cmdline

Contains the command line used to launch the process.

/proc/[pid]/cwd

This is a symbolic link to the current working directory of the process.
If you have a process with the PID 1234, then you can find out it’s current working directory by using the command “cd /proc/1234/cwd; /bin/pwd”

/proc/[pid]/status

This file contains information about the processes status, such as it’s name, state, pid, parent pid, owner.

/proc/cmdline

Contains all the arguments passed to the kernel at boot time.
/proc/cpuinfo

Perhaps the most known one, it contains processor related information, such as the architecture, frequency and amount of cache found on the cpu.
/proc/filesystems

A list of all the file systems supported by the current kernel.
Lines beginning with ‘nodev’ indicate non-physical filesystems such as network filesystems and proc.
/proc/loadavg

Holds information regarding the load average of the system.

The first three fields are the same ones you get from ‘uptime’.

The fourth field consists of two numbers seperated by a slash, the first one represents the number of currently executing processes/threads. This number will not exceed the number of processors cores the system has.
The second number (the one after the slash) represents the number of processes/threads currently existing on the system.

The fifth field is the PID of the process most recently created.Now, this is where you need to be careful.If you execute ‘cat /proc/loadav’, then this number will represent the PID of the cat command you just executed!

/proc/free

Contains statistics about memory usage.
The command ‘free’ makes use of this file to build its output.

/proc/net/

This directory holds alot of files rated to the networking layer.
All the files are ASCII structured and can be read.

/proc/net/arp

Holds the arp table

/proc/net/dev

Information such as the total number of received and transmitted packets and bytes by each network interface.

/proc/net/route

Holds the routing table, in hexademical format.

/proc/net/wireless

Holds information related to the current wireless connection, such as thequality and number of discarded packets.

/proc/swaps

Shows the amount of swap in use and the priority of the defined swap partitions.

/proc/sys/kernel/hostname

Contains the current hostname of the system.
You can change this by executing “echo ‘newHostname’ > /proc/sys/kernel/hostname”

/proc/sys/kernel/threads-max

Specifies the maximum number of processess/threads that can excist at any given time on the system.

Compare this to the current number of processes/threads from the fourth field in /proc/loadavg

/proc/sys/vm/swappiness

The value in this file controls how willing the kernel will be to swap memory.
If you raise this number, the kernel will want to swap more often, while lowering it will decrease his tendency to swap.
The default value is 60.

/proc/uptime

Contains two numbers, the first one tells you how long the system has been up (in seconds), while the second one tells you for how long it has been idle.
You can use something like:echo `cut -d’ ‘ -f2 /proc/uptime` / `cut -d’ ‘ -f1 /proc/uptime` | bc -l to get the percentage of idle time on your computer.

/proc/vmstat

Contains virtual memory statistics

/proc/sys/net/ipv4/conf/default/forwarding

Controls whether the kernel will allow tcp forwarding.The default value is 0 which means forwarding is OFF.You can set this to 1 if you with to enable it…

Think: Internet connection sharing without password protection.

EXT4

Linux Encryption - LUKS

My own wrapper script, after learning to do all of these things below: https://github.com/thomasfrivold/luksus

A very good howto http://sleepyhead.de/howto/?href=cryptpart

LUKS on Debian http://wiki.drewhess.com/wiki/Creating_an_encrypted_filesystem_on_a_partition

A very good faq: http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions#6._Issues_with_Specific_Versions_of_cryptsetup

main webpage http://www.saout.de/misc/dm-crypt/

cryptsetup luks main google code page http://code.google.com/p/cryptsetup/

cryptsetup faq http://code.google.com/p/cryptsetup/wiki/FrequentlyAskedQuestions#3._Common_Problems

simple cryptsetup luks howto using keyfiles (lacking encryption security comments) http://zeroone.homeunix.net/~blip/computer/mbwe/encrypt-disk.html

https://wiki.archlinux.org/index.php/System_Encryption_with_LUKS#Introduction

madduck's howto using keyfiles http://madduck.net/docs/cryptdisk/

ryan finnie's howto using keyfiles http://www.finnie.org/2009/07/26/keyfile-based-luks-encryption-in-debian/

fedoraproject wiki LUKS (very clean) http://fedoraproject.org/wiki/Implementing_LUKS_Disk_Encryption

arch linux's great wiki https://wiki.archlinux.org/index.php/System_Encryption_with_LUKS

ubuntu's chaotic howto https://help.ubuntu.com/community/EncryptedFilesystemHowto5

an old debian etch howto http://wejn.org/how-to-make-passwordless-cryptsetup.html

Adding keys to partition and booting

http://www.oxygenimpaired.com/ubuntu-with-grub2-luks-encrypted-lvm-root-hidden-usb-keyfile
https://help.ubuntu.com/community/EncryptedFilesystemLVMHowto


PRINTERS

Canon Pixma MP560

Canon Pixma MP560 printer and Linux When I bought this printer, I failed to find any Linux drivers for it. I must admit I didn't try hard to find them either since I was very much in a hurry at that time. Since then I had just assumed that drivers for Linux just don't exist.

But today I decided to search/try harder to get it working against my fedora laptop. The result was that I found the drivers very easily and after several minutes of efforts, I finally got it working! So I thought it blog about it and provide some pointers so process gets easier for others:

Ensure you have 'DefaultLanguage en_GB' line in your /etc/cups/cupsd.conf and your firewall isn't blocking Port 8611 for TCP/UDP. Get the drivers from here. The drivers are available as rpm and deb packages. Once you have the packages downloaded and installed on your system, find out the mac address of your printer somehow. The method I used was to watch for packets in wireshark and pinging the broadcast address of the network. Once you have the mac address, Run this command as root:

/usr/sbin/lpadmin -p MP560LAN -m canonmp560.ppd -v cnijnet:/${MAC_ADDRESS} -E

Note that mac address bytes must be separated by '-' here rather than ':'. Your printer should now be installed and working! source: http://zee-nix.blogspot.com/2011/05/canon-pixma-mp560-printer-and-linux.html

Fixing grub2

finding disk uuid numbers

run blkid

http://eaglegordon.hubpages.com/hub/Finding-UUIDs-numbers

grub2 rescue

http://www.cyberciti.biz/tips/restore-debian-linux-grub-boot-loader.html


Oneliners and Unix Shell Magic zsh bash shell

Automatic screen on login

Automatic 'screen' on remote logins

If you routinely log into one or more remote systems using SSH, and have a flaky internet connection or an incompetent ISP, you probably already know about screen's ability to detach and reattach sessions.

However, you still have to manually type screen -r to resume a detached session, each time -- and sometimes you'll forget, start working in an SSH session, get logged out, and lose your state.

Here's the next step -- automatic screen-sessions for any remote logins.

Bonus features in the screenrc:

color terminal-window support logging of the entire session, to a datestamped logfile under $HOME/lib/screen-logs .bashrc

Add these lines at the top of ~/.bashrc on the target host:


# Auto-screen invocation. see: http://taint.org/wk/RemoteLoginAutoScreen
# if we're coming from a remote SSH connection, in an interactive session
# then automatically put us into a screen(1) session.   Only try once
# -- if $STARTED_SCREEN is set, don't try it again, to avoid looping
# if screen fails for some reason.
if [ "$PS1" != "" -a "${STARTED_SCREEN:-x}" = x -a "${SSH_TTY:-x}" != x ]
then
  STARTED_SCREEN=1 ; export STARTED_SCREEN
  [ -d $HOME/lib/screen-logs ] || mkdir -p $HOME/lib/screen-logs
  sleep 1
  screen -RR && exit 0
  # normally, execution of this rc script ends here...
  echo "Screen failed! continuing with normal bash startup"
fi
# [end of auto-screen snippet]
.screenrc

Create ~/.screenrc on the target host, containing:


# see http://www4.informatik.uni-erlangen.de/~jnweiger/screen-faq.html
# support color X terminals
termcap  xterm 'XT:AF=\E[3%dm:AB=\E[4%dm:AX'
terminfo xterm 'XT:AF=\E[3%p1%dm:AB=\E[4%p1%dm:AX'
termcapinfo xterm 'XT:AF=\E[3%p1%dm:AB=\E[4%p1%dm:AX:hs:ts=\E]2;:fs=\007:ds=\E]2;screen\007'
termcap  xtermc 'XT:AF=\E[3%dm:AB=\E[4%dm:AX'
terminfo xtermc 'XT:AF=\E[3%p1%dm:AB=\E[4%p1%dm:AX'
termcapinfo xtermc 'XT:AF=\E[3%p1%dm:AB=\E[4%p1%dm:AX:hs:ts=\E]2;:fs=\007:ds=\E]2;screen\007'
# auto-screen support; see http://taint.org/wk/RemoteLoginAutoScreen
# detach on hangup
autodetach on
# no startup msg
startup_message off
# always use a login shell
shell -$SHELL
# auto-log
logfile $HOME/lib/screen-logs/%Y%m%d-%n.log
deflog on

Note: if you just want the auto-screen feature, the middle 7 lines are the important bit; you can probably omit the "color X terminals" and "auto-log" stanzas if you like.

Source: http://taint.org/wk/RemoteLoginAutoScreen

Daemons

NFSd

My own mini howto

server side

write which drives and to what IPs to export nfs share /etc/exports like this. Notice the first export is the root of the virtual filesystem:

/usr/nfsexport 128.39.115.3(ro,nohide,fsid=0,insecure,no_subtree_check,async,no_root_squash)
/usr/nfsexport/garota 128.39.115.3(ro,nohide,no_subtree_check,async,no_root_squash)  
/usr/nfsexport/machinegun1 128.39.115.3(ro,nohide,no_subtree_check,async,no_root_squash) 

specifically declare who to allow to connect to nfs /etc/hosts.allow restart ntfs /etc/init.d/nfs-kernel-server restart

mount --bind /mnt/6tbdisk /nfsexports/mainarchive

rpcbind mountd nfsd statd lockd rquotad : 128.39.115.3 128.39.115.4



client side
create mountpoints mkdir -p
add mounts in /etc/fstab
mount -a -t nfs4
mount

ls mountpoint

source: https://help.ubuntu.com/community/SettingUpNFSHowTo

other guides

This guide is farily exhaustive and provides every little detail to get started

https://help.ubuntu.com/community/SettingUpNFSHowTo

except the nohide point, which can be read about here:

http://www.troubleshooters.com/linux/nfs.htm

note: nohide must be added to the root nfs export, at least for an secure installation

IPSec OpenSWAN VPN

source: http://www.byteme.org.uk/howtos/ipsec-vpn-server-howto.html

This page describes my VPN setup that allows "roadwarriors" (employees with laptops roaming around the country) to securely connect back in to my network. I have recently simplified the network greatly by removing the old gateway server (i broke it!) and replacing this with a Netgear ADSL router-modem. The Netgear unit is supplying IP addresses to my network (DHCP server enabled) and is the gateway on the system. This setup should be fairly generic and many router-modems will support this. Try to look out for VPN passthrough on the router-modem it should help a lot.

Basic router/network setup

As the router cannot deal with IPSEC it is necessary to have a computer inside the network that can. I am currently using a old laptop with a failed LCD display, it is only 450MHz and seems to cope fine. Firstly I installed Debian testing from the sarge-net install CD image on Debian's website. This was a painless process as long as you have a supported network card all you need to do is answer some simple questions. Once Debian is installed and running it is necessary to modify the network settings to give the VPN server a static IP address. Below are the entries I made to /etc/network/interfaces

# The primary network interface
#iface eth0 inet dhcp
iface eth0 inet static
   address 192.168.0.2
   netmask 255.255.255.0
   gateway 192.168.0.1
   mtu 1200

This is all fairly straight forward except perhaps the MTU parameter which will be explained later. Next it is necessary to add a few rules to the router. This Will vary from router to router but the concepts are the same. It is necessary to forward any incoming UDP 500 from the router to 192.168.0.2 and any incoming UDP 4500 from the router to 192.168.0.2. UDP-4500 was not a pre defined service on the Netgear so I had to add that on the services option and i called it ESPINUDP. Note the UDP-4500 is only needed for NAT-T (but that is a very nice thing to have). There were NO options to do anything with the ESP protocol that just seems to work with the Netgear (maybe because it specifices IPSEC pass through?).

Setting up the VPN server

Openswan seems to be the preferred "swan" in Debian so a simple

apt-get install openswan will grab everything you need. I will assume that all the required certificates have been generated see X509 Certificate Generation. You should have 3 files from the certificate generation stage, a Certificate Authority certificate "cacert.pem", A private key "vpnserver.key", and a public key "vpnserver.pem" copy these files to the following locations :-

cacert.pem -----> /etc/ipsec.d/cacerts vpnserver.key -----> /etc/ipsec.d/private vpnserver.pem -----> /etc/ipsec.d/certs There should also be a CRL (that lists revoked certificates) but it is not strictly necessary just good practise. Next add a line to /etc/ipsec.secrets to allow openswan to access the private key. Using your password as necessary

RSA vpnserver.key "my_password_for_the_private_key"

Finally we come to the main configuration file, this is what I am currently using on the server

version 2.0

Config setup
   nat_traversal=yes
   virtual_private=%v4:10.0.0.0/8,%v4:174.16.0.0/12,%v4:192.168.0.0/16,%v4:!192.168.0.0/24
conn roadwarrior
   dpdaction=clear
   authby=rsasig
   leftrsasigkey=%cert
   rightrsasigkey=%cert
   right=%any
   rightnexthop=192.168.0.1
   rightsubnet=vhost:%no,%priv
   left=%defaultroute
   leftcert=vpnserver.pem
   auto=add
   pfs=yes
   leftsubnet=192.168.0.0/24
   
include /etc/ipsec.d/examples/no_oe.conf

IP Forwarding and masquerading With this setup it will currently be impossible to access any machine other that the vpnserver as there is a fundamental routing problem, a packet from roadwarrior x.x.x.x destined for 192.168.0.20 will be received by the vpnserver (192.168.0.2) decrypted and dropped. If ip_forward is enabled then the packet will not be dropped but sent to 192.168.0.20. 192.168.0.20 will reply to x.x.x.x which means the reply will go directly to the gateway 192.168.0.1 and back to x.x.x.x UNENCRYPTED. Any sensible firewalls will not then drop this packet. To solve this it is necessary to do two things:-

  • Enable IP Forwarding
  • Enable Masquerading

To enable IP Forwarding add/change the line in /etc/network/options so it reads :-

ip_forward=yes

To enable masquerading it is necessary to run the iptables command

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

This will now masquerade any incoming packets from x.x.x.x to 192.168.0.2 so the routing works and the encryption is intact. (** TODO **) It SHOULD be possible to automatically run the above commands. I have created a script with the above commands and place it in /etc/network/if-up.d/ but i don't think it works. I am not sure if the script should have a special name such as eth0-iptables.sh I need to check this when I can down the VPN server for 5 minutes.

MTU

MTU is a problem with this setup. MTU stands for maximum transmission size and it sets how large a chunk of data can be sent from a network interface. A typical Ethernet interface has a MTU of 1500 while a PPP connection has a slightly lower size due to the overhead of the ppp layer. If we then add an IPSEC layer then overall data size reduces even further. What happens if the MTU is left at 1500 is that servers on the internal LAN will send data in 1500 chunks, the IPSEC wrapping will then be forced to fragment the data in to two packets, a full one and a partial one. This cases chaos and packet loss. To avoid this setting the ethernet to a MTU of 1200 ensures that there is always sufficeint overhead to ensure the ESP wrapper (IPSEC) doesn't cause the data to overflow a single IP frame. In fact 1200 is very consertivative, i believe nornal ipsec requires 56 bytes and NAT-T sequires a few more ao 1400 should be OK for many situations.

Next version of server and more MTU problems!

I decided to upgrade my servers to Debian Sarge and build mail, filesharing and ipsec all on one system. I no longer had a direct connection to the internet (through a modem) but via a network with a dedicated router. This caused by chaos!. The problem is one of MTU. As the router connected to the internet via a ppp link (adsl) and then connected to the ipsec server via eth0 there are mtu issues. For me the ipsec tunnel came up but it was impossible to access things such as pop3 mail etc on the server. If i tried to access other systems (ie NOT the endpoint then providing the MTU rules as discussed above were followed then it this worked.

So access to the endpoint has a mtu issue. There are no network devices to set the mtu on so what do you do? I discovered a new command the "ip" command, i have never used this before. In fact "ip route" shows intresting routing stuff and infact you can set the mtu on a specific route :-). So I found i could change the mtu of the ipsec route no problems, and setting to 1300 fixed the pop3 etc probelsm :-)

Automating the process:- setting the route by hand is a non starter for a production situation, so how to do it automaticly. Well it seems that the updown scrips are the answer. In /usr/lib/ipsec/ there are scripts incluidng _updown, that get run by the openswan system. To change the MTU of a new ipsec connection open the /usr/lib/ipsec/_updown file and search for the line :-

case "$PLUTO_PEER_CLIENT_NET/$PLUTO_PEER_CLIENT_MASK" in
       "0.0.0.0/0.0.0.0")
               # opportunistic encryption work around
               # need to provide route that eclipses default, without
               # replacing it.
               it="ip route  0.0.0.0/1 $parms2 &&
                       ip route  128.0.0.0/1 $parms2"
               ;;
       *)      it="ip route  $parms $parms2 $parms3"
               ;;

and change it to :-

case "$PLUTO_PEER_CLIENT_NET/$PLUTO_PEER_CLIENT_MASK" in
        "0.0.0.0/0.0.0.0")
                # opportunistic encryption work around
                # need to provide route that eclipses default, without
                # replacing it.
                it="ip route  0.0.0.0/1 $parms2 &&
                        ip route  128.0.0.0/1 $parms2 mtu 1300"
                ;;
        *)      it="ip route  $parms $parms2 $parms3 mtu 1300"
                ;;

Now all new ipsec connections will get a MTU of 1300. Jobs done!

Postgresql

pgqueryspy 2000 db0

You have tuned your PostgreSQL database, and you, of course, turned off query logging, but a problem has cropped up. How can you see the activity on your database at any slice of time?

Like me, you have probably turned on command stats to view pg_stat_activity, but that gives you only moment in time. You could use ethereal to capture the Postgres queriy packets, but then what? What if you don’t have X installed? tcpdump you say? What do you do with the dump?

Enter pgqueryspy.

Simply type pgqueryspy 2000 db0 to capture the next 2000 packets to the server named ‘db0′. pgqueryspy then prints out the queries that it intercepts in the packet stream for your consumption on the command line. It even tosses those pesky “BEGIN” and “END” queries for you.

Not too terribly invasive, no load on your database, and you find out NOW what is happening with your database. Code is available here: pgqueryspy.c.

I accept NO RESPONSIBILITY FOR YOUR USE OF THIS CODE. It could toss your database/network/server/grandma out the window. YMMV, caveat emptor, etc, etc.

I am putting this code out here for two reasons:

Like me, Google has not led you to any other solution to sniff packets of Postgres queries without making the database log (which requires a database restart). As soon as I post this code, someone is going to come along and tell me that I could do it in a single command line, and that my 2 hours building this were for nothing. I say bring it on, prove me wrong. You know why? a) my C skills are non-existent, and b) I wish I knew how to do this on the command line UPDATE: Forgot to give credit for the help. Thanks to Kris, as well as the libpcap packet capture tutorial. http://dotnot.org/blog/archives/category/how-do-i/

Named

Adding a domain to named

vi /etc/named.conf (adding master zone for newdomain.com) cp /var/named/example.org.hosts /var/named/newdomain.com.hosts vi /var/named/newdomain.com.hosts kill -HUP pid-of-named

source: http://dotnot.org/blog/archives/category/how-do-i/

Detecting Mouse Movement on Linux

#!/bin/bash
while :
do
cat /dev/input/mice | read -n 1
date
sleep 1
done

Linux Kernel

Compiling the kernel

Install some necessary packages: 
apt-get install fakeroot kernel-package

Get the latest kernel sources from http://kernel.org
unpack to /usr/src
Use the generic .config file that ships 
with the kernel and copy to  /usr/src/linuxkernelsourcedir/.config 
cp /usr/src/linuxfolder/arch/ia64/configs/generic_defconfig /usr/src/linuxfolder/.config
Add some more things you might want in the menuconfig
make menuconfig
make-kpkg clean

Set concurrency_level so that we use all CPU cores for compiling. It's Cores+1, so
my quadcore cpu is then set to 5
export CONCURRENCY_LEVEL=3
fakeroot make-kpkg --append-to-version "-customkernel" --revision "1" --initrd kernel_image kernel_headers

This will greatly speed up your compilation time. The rest of the compilation command is pretty self-explanatory. With fakeroot, we are making kernel packages (make-kpkg), appending a string to name our kernel (“customkernel”), giving it a revision number (“1”) and we are telling make-kpkg to build both an image package and a header package. Once the compilation is finished, and depending on your machine, and number of modules you are compiling, it can take quite a long time, change directories to one back from the Linux source directory, and you should see two new *.deb files – one linux-image file and one linux-headers file.

Now simply install the two new packages that fakeroot and make-kpkg created
dpkg -i linux-image-3.12.0-customkernel_1_i386.deb linux-headers-3.12.0 customkernel_1_i386.deb

source: http://www.tecmint.com/kernel-compilation-in-debian-linux/

Updating System Kernel Modules

depmod -a

Update system on new modules edit: What? I wrote this myself? This is Soo old knowledge

Online Linux Resources

Many fantastic tips from cb.vu

http://cb.vu/unixtoolbox.xhtml

Linux commandline minireference

http://www.pixelbeat.org/cmdline.html

Compiling FFMPEG X264 and Friends

https://ffmpeg.org/trac/ffmpeg/wiki/UbuntuCompilationGuide

Commandline-fu

http://www.commandlinefu.com/commands/browse

Shell-fu

http://www.shell-fu.org/

Many fantastic tips from Cyberciti

http://www.cyberciti.biz/tips/gnulinux-advanced-administration-pdf-book.html

On hardening of Linux and Administrators Guide



Distribution specific

New Debian Install / Ubuntu Install

Any Debian / Ubuntu install

Packages I like to install on a fresh Debian mini-install:

   apt-get install ne emacs mosh openssh-server mosh screen tmux mc links lynx wget curl zsh fish atop htop ethtool screen tmux most jed joe iptraf cryptsetup
   # oh my zsh
   sh -c "$(curl -fsSL https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh)"
   # oh my fish
   curl -L https://github.com/oh-my-fish/oh-my-fish/raw/master/bin/install | fish
   # sealion
   curl -s https://agent.sealion.com | sudo bash /dev/stdin -o 805ffc2c-85ac-4876-9d77-315304e98366
   # docker
   wget -qO- https://get.docker.com/ | sh


apt-get install mosh openssh-server screen tmux mc links lynx wget curl zsh fish atop htop ethtool screen tmux most jed joe iptraf cryptsetup
#webmin
wget http://prdownloads.sourceforge.net/webadmin/webmin_1.650_all.deb
# Need to have Webmin, so handy, connect at https://localhost:10000
dpkg -i webmin_1.650_all.deb
apt-get install -f -y
dpkg -i webmin_1.650_all.deb

LAMP STACK SERVER

#!/bin/bash
apt-get update
apt-get install mc jed nano wget screen iptraf nmap zsh fish git curl ruby rails lynx mysql-server libapache2-mod-auth-mysql apache2 php5 libapache2-mod-php5 php5-mcrypt php5-mysql php5-gd php5-xcache
apachectl restart
a2enmod rewrite
service apache2 restart
mysql_install_db
/usr/bin/mysql_secure_installation
cd /usr/local
wget http://prdownloads.sourceforge.net/webadmin/webmin_1.650_all.deb
# Need to have Webmin, so handy, connect at https://localhost:10000
dpkg -i webmin_1.650_all.deb
apt-get install -f -y
dpkg -i webmin_1.650_all.deb
# Spice up ZSH
wget --no-check-certificate https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | sh

Distro specific

Ubuntu

Kernel PPAs - Get new kernels

For v3.12
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-saucy/linux-headers-3.12.0-031200-generic_3.12.0-031200.201311031935_amd64.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-saucy/linux-headers-3.12.0-031200_3.12.0-031200.201311031935_all.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.12-saucy/linux-image-3.12.0-031200-generic_3.12.0-031200.201311031935_amd64.deb
sudo dpkg -i linux-headers-3.12.0-*.deb linux-image-3.12.0-*.deb
wget http://kernel.ubuntu.com/~kernel-ppa/mainline/*
bleeding edge http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/
follow the guide here: http://ubuntuhandbook.org/index.php/2013/11/linux-kernel-3-12-released-install-ubuntu-or-linux-mint/

Command-not-found malfunctioned

export LANGUAGE=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
locale-gen en_US.UTF-8
sudo dpkg-reconfigure locales
source: http://ivaniliev.com/sorry-command-not-found-has-crashed/

Change GCC Version

The best way to correctly install gcc-4.9 and set it as your default gcc version use:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test sudo apt-get update sudo apt-get install gcc-4.9 g++-4.9 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9 The --slave, with g++, will cause g++ to be switched along with gcc, to the same version. But, at this point gcc-4.9 will be your only version configured in update-alternatives, so add 4.8 to update-alternatives, so there actually is an alternative, by using:

sudo apt-get install gcc-4.8 g++-4.8 sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.8 Then you can check which one that is set, and change back and forth using:

sudo update-alternatives --config gcc NOTE: You could skip installing the PPA Repository and just use /usr/bin/gcc-4.9-base but I prefer using the fresh updated toolchains. source: http://askubuntu.com/questions/466651/how-do-i-use-the-latest-gcc-4-9-on-ubuntu-14-04

Debian

APT and Aptitude

APT reference card

http://www.cyberciti.biz/ref/apt-dpkg-ref.html

APT reconfigure package

reconfigure a package

dpkg-reconfigure packagename

APT Adding an APT source to add foreign packages

example below:

Install Howto
Add the following line to /etc/apt/sources.list: 
deb http://www.debian-multimedia.org squeeze main
Update the package index: 
apt-get update
Install GPG key of the repository: 
apt-get install debian-multimedia-keyring
Install libfaac-dev deb package: 
apt-get install libfaac-dev

source: http://pkgs.org/debian-squeeze/multimedia-main-i386/libfaac-dev_1.28-0.3_i386.deb.html

APT Adding free and non-free music&video packages

Install Howto
Add the following line to /etc/apt/sources.list: 
deb http://www.debian-multimedia.org squeeze main
Update the package index: 
apt-get update
Install GPG key of the repository: 
apt-get install debian-multimedia-keyring
Install libfaac-dev deb package: 
apt-get install libfaac-dev

WARNING: The following packages cannot be authenticated!

løsning1 - userfault: svarte du y og trykket enter når du fikk spørsmålet om du ville fortsette, eller trykket du bare enter? Hvis du bare trykker enter så svarer den nemlig den bokstaven som er uppercase, altså N. Hvis det er tilfellet, prøv igjen også trykk y og trykk enter.

apt-get install sshfs

løsning2 - forfriske apt listene (men ikke oppgradere):

apt-get update
apt-get install sshfs


løsning3 - løsning for å fikse signaturproblemet:

"looks like the signatures are not downloading"

UBUNTU DEBIAN

apt-get install ubuntu-archive-keyring
# apt-get install debian-archive-keyring # for debian obviously
apt-get update
apt-get install sshfs


source: http://www.cyberciti.biz/faq/linux-aptget-warning-following-packages-cannot-authenticated/ source: http://changelog.complete.org/archives/496-how-to-solve-the-following-packages-cannot-be-authenticated

Messing around with UDEV and ethX names

http://www.debianhelp.co.uk/udev.htm http://www.debianadmin.com/rename-network-interface-using-udev-in-linux.html

Initramfs network does not work tg3

great resource http://blather.michaelwlucas.com/archives/665

Fixing circular error with libc6 package

sudo apt-get remove libc6-dev # might not work
sudo apt-get -d install libc6 # might not work, but keep going
cd /var/cache/apt/archives
sudo dpkg --force-depends --install libc6_2.8~20080505-0ubuntu7_i386.deb findutils_4.4.0-2ubuntu3_i386.deb
sudo apt-get -f install
sudo apt-get dist-upgrade

source: http://ubuntuforums.org/showthread.php?t=1097955 source: http://ubuntuforums.org/showthread.php?t=1779799

APPLICATION SPECIFIC

Installing Spotify

Debian
# 1. Add this line to your list of repositories by
#    editing your /etc/apt/sources.list
deb http://repository.spotify.com stable non-free
# 2. If you want to verify the downloaded packages,
#    you will need to add our public key
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4E9CFF4E
# 3. Run apt-get update
sudo apt-get update
# 4. Install spotify!
sudo apt-get install spotify-client