Practical Linux Tricks for the Aspiring Hacker

This is a collection of commands I've picked up over the last few years, which I've found immensely useful. My favourite ones are probably:

less: search/filter on a file or long text
^r: reverse search
!$: last argument of previous command

By "favourite", I mean I've used these commands a lot, and they've drastically increased my productivity.

Cool Stuff

Control (^) Commands

^c # Duh. https://xkcd.com/416/
^d # Exit / EOF.

Reverse/Forward Search: for those long commands stashed in history. Works in PowerShell and REPLs too!

^r
^s

^{Note: to make ^s work in bash/zsh, you may need to run stty -ixon, which disables software control flow.}

Ternary Expression

[ 1 -eq 1 ] && echo 'true' || echo 'false'
true

Clear screen. Useful for graphical hiccups.

reset

Run shell script without chmod +x.

. ~/.zshrc
source ~/.zshrc

Tree view of files.

tree

Strings

Double-Quotes vs. Single-Quotes

Double-quotes allow variable expansion and command substitution.
Single-quotes don't. Prefer single-quotes for simple strings.

echo "$((1+1))" "$SHELL"
2 /bin/zsh
echo '$((1+1))' '$SHELL'
$((1+1)) $SHELL

Multi-Line / Escape
Prefix the string with $.

echo $'...'

Escape Single-Quotes

Example

Multi-Line Strings.

echo $'1\n2\n3'
1
2
3

Find words containing 't in comma-separated line.

echo -n $'can\'t,don\'t,I\'ll,I\'m,won\'t' | awk -vRS=, $'$0 ~ /\'t/'
can't
don't
won't

Previous-Command Tricks

$?: exit code of previous command
- By convention, 0 means no error. Non-0 implies an error occurred.
!!: previous command
!$ or $_: last argument of previous command

Examples

Retry with sudo.

mkdir /var/tmp/lol
Permission denied.
sudo !!
→ sudo mkdir /var/tmp/lol
Success!

Found an interesting directory, but forgot to cd.

ls long/path
cd !$
→ cd long/path

Rename file in folder from file.txt to booyah.md.

cat long/path/file.txt
mv "!$" "$(dirname !$)/booyah.md"
→ mv long/path/file.txt long/path/booyah.md

Other Useful Commands (stolen from here)

!!:n - nth argument from previous command
!^ - first argument (after the program/built-in/script) from previous command
!* - all arguments from previous command
!n - command number n from history
!pattern - most recent command matching pattern
!!:s/find/replace - last command, substitute find with replace

Redirection

< out.txt # Read from file (to stdin).
> out.txt # Write to file (from stdout).
>> out.txt # Append to file (from stdout).
2> out.txt # Write to file (from stderr).
&> out.txt # Redirect all output.
&> /dev/null # Redirect everything into the void.
2>&1 # Redirect stderr to stdout.

>& # Same as `&>`.

Powerful Utilities

awk: filter lines, filter columns, math, scripting, etc.
sed: filter/replace text
grep: filter lines
cut: filter columns
tr: replace/remove characters
wc: count characters/bytes/words
find: find files in folder, execute command for each file with -exec
xargs: feed arguments into commands, simple cmdline multi-processing

I won't cover too much of these commands here, as tons of articles already cover them. And you can browse examples online or in their man pages.

awkward things

awk - Cut

Cut third field.
awk '$0=$3'

Print third field. (Pretty much same as the command above.)
awk '{print $3}'

Use ',' as field delimiter, e.g. for CSVs.
awk -F, '{print $3}'

Or use the script variable `FS` (Field Separator).
awk -v FS=, '{print $3}

awk - Filtering

Without entering the scripting environment {...}, awk will run filters against each line.

seq 1 4 | awk '$1 % 2 == 1'
1
3
echo $'foo1\nbar1\nfoo2' | awk '$0 ~ /^foo/'
foo1
foo2

awk - Math

Add 5 to the first arg, then print the line.
awk '{$1 += 5}1'

seq 1 3 | awk '{$1 += 5}1'
6
7
8

awk - Scripting

awk '{print "booyah",$1,"yahoo"}'
awk also has variables, if, for, while, arrays, etc.

Script variables. (Useful for configuring row/column delimiters.)

RS: Record Separator (rows)
FS: Field Separator (columns)
ORS: Output Row Separator
OFS: Output Field Separator
NR: Record Number (current row, 1-indexed) [read-only]
NF: Number of Fields [read-only]

grep

Useful Flags

-i # case-insensitive
-E # regex
-v # non-match (inVert)

grep – Find String in Files

grep -rnw /path/to/somewhere/ -e 'pattern'

-r or -R is recursive,
-n is line number, and
-w stands for match the whole word.
-l can be added to just give the file name of matching files.
-e is the pattern used during the search

Ref: https://stackoverflow.com/a/16957078/10239789

xargs

xargs is a versatile command-line utility that allows efficient execution of commands, making it a powerful tool for automation and batch processing.

Interesting options:

-P <n> # max procs
-n <n> # num args
-I {}  # pattern to insert into command

Examples

Combine multiple lines into 1 line.

echo $'1\n2\n3'
1
2
3
echo $'1\n2\n3' | xargs 
1 2 3

Multi-Processing: Execute ./do-something-to-file.sh <file> on multiple files, with at most 4 processes.

cat files.txt | xargs -P 4 -n1 ./do-something-to-file.sh

Multi-Processing: Port Scan with Ports 1-1000 through proxychains.

seq 1 1000 | xargs -P 50 -I{} proxychains4 -q nmap -p {} -sT -Pn --open -n -T4 --oN nmap.txt --append-output 192.168.101.10

Other Utilities

basename ~/.bashrc
.bashrc
dirname ~/.bashrc
/home/trebledj

Directory Stack

pushd # Push current directory, for easier returning.
popd  # Return to directory on top of stack.
dirs  # List history of dirs.

pushd/popd Example

cd ~/a/b/c
pushd deep/nested/directory
Jump to `deep/nested/directory`, push `~/a/b/c` into the stack.
cd ../../jump/around/some/more
cd ../../and/a/little/more
popd  # Return to `~/a/b/c`.

less

less is a powerful text viewer (read-only), with capabilities to navigate, search, and filter lines in a file or long text.

Get some help. See all commands:

less - Nice Options

less file.txt

Renders ANSI colors.
less -R file.txt

Pager search becomes case insensitive.
less -I file.txt

Line numbers.
less -N file.txt

You can turn on/off these options inside less by typing -I<Enter>, -R<Enter>, or -N<Enter>. This is useful if you forget to turn them on beforehand (e.g. after curling a web request).

j # Line down.
k # Line up.
f # Page down.
b # Page up.
d # Half-page down.
u # Half-page up.

g # Go to start of file.
G # Go to end of file.

<n> g # Go to nth line.

# Go to the n% line of the file.
<n> p
20p 40p 50p 80p

# What's the current line?
^g

less - Search / Filtering

# Search (regex enabled).
/ <pattern>
# For case-insensitive search, use `less -I`.

# Navigate search results: next/prev result.
nN

# Filter lines by search.
& <pattern>
# Filter NON-MATCHING lines
& ! <pattern>
# Clear filter.
& <enter>

less - Scrolling

Personally, I prefer less+F over tail -f.
Use ^c to exit the feed.

# Continuous feed (e.g. for streams of data)
F

less - Working with Multiple Files

less also works with multiple files passed in the command line, e.g. less *.txt.

# Next file.
:n
# Previous file.
:p

More commands in man less.

Processes

fg/bg - "I'll be back."

Shells allow you to move processes between the foreground (which accepts interactive input) and background (to run things which don't require input).

^z # Push process to background (and pause it).
bg # Start background process.
fg # Bring most recent background process into foreground.
fg 2 # Bring job 2 into foreground.
jobs # View background jobs.

^c # Good ol' ctrl-c stops the process in fg.
kill <pid> # Kill process with given process ID.

Start a command in the background.
<cmd> &

Example

Start an HTTP server on port 8080.

python -m http.server 8080 &
[1] 17999

The process is started in the background with job number 1, PID 17999.

To kill the process:

fg
^c

or...

kill 17999

Process ID (PID) and Job Number are two different things.

PIDs apply to all users in the entire system, and are assigned by the kernel.
Job Numbers apply to the current shell, and are numbered linearly from 1 onwards.

View Running Procs

ps aux

Combine with grep/less for filtered results.

Networking

IP and Ports

IP Addresses and Networks

ifconfig
ifconfig tun0

Get Our Public IP

curl ifconfig.me
X.X.X.X

Open Ports/Sockets

netstat -anp

-a: all sockets
-n: numeric addresses
-p: associated processes

Listen/Connect

Initiate a connection.
nc 192.168.1.1 8080

Listen for a connection.
nc -nlvp 4444

Listen persistently (keep listening after disconnect).
nc -nklvp 4444

Download Files

Download and save to a local file.
curl <url> -O
wget <url>

Download with a custom filename.
curl <url> -o filename.txt
wget <url> -O filename.txt

Download silently and display in `less`.
curl <url> -s | less
wget <url> -s | less
curl some.api.site/api/v1/users/ -s | jq | less

Upload Files

python -m uploadserver

By default, uploadserver starts a server at port 8000.
Get our IP from ifconfig.

curl -F files=@file1.txt -F files=@file2.txt 192.168.45.179:8000/upload

Files

Check File Sizes

Get available disk space.
df -h
Get file size of current directory, in human readable format.
du -sh .
Get file size of txt files, in human readable format.
du -sh *.txt

Find/Operate on Files

Find files in operating system, and ignore errors.
find / -name '*needle*' 2>/dev/null

Find [f]iles or [d]irectories.
find . -type f -name '*needle*'
find . -type d -name '*needle*'

Find and run command on files. \; is needed for `find` to know where to terminate.
find . -type f -name '*complex*' -exec echo {} \;
find . -type f \( -name '*complex*' -or -name 'query' \) -exec du -h {} \;

git gud

Git commands for completeness.

Make new branch.
git checkout -b <name>

Checkout commits in tree before HEAD.
git checkout HEAD~1  # 1 commit before.
git checkout HEAD~10 # 10 commits before.

Checkout commit from parent.
git checkout HEAD^  # 1 commit before (from parent 1, base).
git checkout HEAD^2 # 1 commit before (from parent 2, target).

Store changes locally.
git stash
git stash pop

Clean edited files.
git reset [--hard]
--hard removes unstaged files.

View changes.
git diff | less
git diff <file> # See change in specific file.

Jump through commits (to find, say, the cause of a bug).
git bisect [start|reset|good|bad|skip]

git tree

Command-line git tree from git log.
git log --graph --abbrev-commit --decorate --format=format:'%C(bold blue)%h%C(reset) - %C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an%C(reset)%C(bold yellow)%d%C(reset)' --all

More detailed git-tree 
git log --graph --abbrev-commit --decorate --format=format:'%C(bold blue)%h%C(reset) - %C(bold cyan)%aD%C(reset) %C(bold green)(%ar)%C(reset)%C(bold yellow)%d%C(reset)%n''          %C(white)%s%C(reset) %C(dim white)- %an%C(reset)' --all

Add them as git aliases in ~/.gitconfig or script aliases in ~/.bashrc.
See https://stackoverflow.com/a/9074343/10239789.

Fun watch: So You Think You Know Git?

vim

Haha. Nope.

Not covering that here.

How to Exit Vim

Obligatory.

:wq  # Write to file + exit.
:q!  # Force exit.

Okay, that's enough vim.

Useful Things

Set line numbers.

:set number

Hacky Hack Hack

Generate Bytes

Buffer overflow for fun and profit.

echo

echo -n '\x01\x02'

echo -n '\x41' | xxd
00000000: 41                     A

perl (good for repetitive sequences)

perl -e 'print "\x41"x4 . "\x42\x43"' | xxd
00000000: 4141 4141 4243         AAAABC

I've mentioned this elsewhere, but I'll repeat it here: I don't recommend using Python 3 to generate strings on-the-fly, as its string/byte-string mechanics are unintuitive. Prefer perl or echo instead.

For example: python -c 'print("\xc0")' prints \xc3\x80 (À) instead of \xc0. Why? Because the Python string "\xc0" is interpreted as U+00C0, which is \xc3\x80 in UTF-8.

assert '\xc0'.encode() == b'\xc3\x80'

Printing bytes in Python is difficult to do concisely.

Simple Binary Analysis

Look for strings.

strings file
strings -n <numchars> file

Look for strings and print addresses (in hex)!

od -A x -S 4 file

Tracing

strace - trace system calls (open, read, write, etc.)
ltrace - trace library (glibc) calls