SBC Setup

From FreeSWITCH Wiki
Jump to: navigation, search

Contents

Introduction

Below you'll find a step by step setup for installing FS as a SBC. The LCR engine is provided by Kamailio and its module carrierroute. Kamailio is an opensource SIP Proxy (not a B2BUA).
In this setup, the dialplan is detailed only for inbound to outbound traffic, but it could be easily extended for outbound to inbound traffic (or DID).
The design is the following:
FS is configured with an internal and an external profile, each profile listening on a different network interface.
Kamailio is listening on the loopback interface, and is not used by any other process than FS.

Internal--------FS--------External
                |
                |
              Kamailio


When FS receives an INVITE, FS fires this event in the default context, where it always matches the extension named LOOKUP_ROUTE. In this extension, FS sends the INVITE to Kamailio, that will replies with a 302 Redirect SIP message that contains the route FS has to use to reach the number dialed. The call is then transfered in the context ROUTING where FS will match the associated Gateway (in this setup it is an associated code) that will route the call.
If Kamailio has no route for the dialed number, it replies with a "604 unable to route the call" that FS will relay.
So Kamailio is used only for route lookup by FS, and FS is the only equipment that is seen in the SIP transaction.


This design has been choosen cause I needed a way to store big LCR table (it's now more than 450 000 routes on the production server) and LCR module was not available when I began playing with FS (version 1.0.1).
I extensively tested this setup on an Intel Quad Core server (Q9550 running at 2.83Ghz with 8GB RAM. 4GB should have been far enough).
I ran some tests using SIPP with 750 simultaneous calls and between 50 to 100 calls per second (CPS) for more than 10 days with 25%-30% CPU ressource free. FS version was 1.2 at this time.


Advantages
Stable with quite huge routing table: Kamailio was using less than 3% CPU when doing the SIPP tests with 160 000 entries in LCR table.
Scalable: some people are using carrierroute module with LCR table up to 1 million entries.
Correct call per second rate: up to 100cps.


Drawbacks
Each INVITE you send to FS will first send an INVITE to Kamailio, and once FS receives the Kamailio answer, FS will bridge the call to the Peer that will process the call.
I'm feeling that if you want to process more call per seconds (more than 100 CPS), the fact that the Sofia library is monothreaded and that it has to deal with one more session (FS to Kamailio) before processing the call to the Peer could prevent FS to reach a higher CPS rate. (For everybody that has a better internal knowledge of FS than me, feel free to correct this assumption).


Other ways to go to achieve the same (not tested)
Try mod_lcr that has been introduced in FS 1.0.3 and report on the list and Wiki.
Use the XML CURL module to process the call routing using your preferred script language running on yout HTTP server.

Preliminaries

Install Debian stable netinstall (64 bits version, take care you need a 64bit processor)
http://cdimage.debian.org/debian-cd/5.0.2/amd64/iso-cd/debian-502-amd64-netinst.iso

Upgrade debian distribution
apt-get update
apt-get dist-upgrade

Install SSH Server for remote administration
apt-get install openssh-server

Install Vlan packages
apt-get install vlan

Install Bridge packages
apt-get install bridge-utils

Install Packet Capture packages
apt-get install ngrep
apt-get install tshark
apt-get install tcpdump

Install text editor
apt-get install emacs22-nox

Install screen
apt-get install screen

Disable useless services
/etc/init.d/exim4 stop
update-rc.d -f exim4 remove
/etc/init.d/portmap stop
update-rc.d -f portmap remove
/etc/init.d/nfs-common stop
update-rc.d -f nfs-common remove

Restart server
reboot

Check TCP or UDP services running
netstat -tnlp
only SSH (port 22) should be listening
netstat -unlp
only dhclient3 (port 68) should be listening (I need dhcp for my tests and to get an IP for administration of the test server)

Install utils for compilation
apt-get install build-essential

Installation of FreeSWITCH

Install dependencies
apt-get install subversion subversion-tools automake1.9 gcc-4.1 autoconf make wget libtool g++ libncurses5 libncurses5-dev
apt-get install debhelper automake1.9 autoconf libtool unixodbc-dev libasound2-dev libcurl4-openssl-dev libssl-dev libogg-dev libvorbis-dev libperl-dev libgdbm-dev libdb-dev libgnutls-dev libspandsp-dev libtiff4-dev

Download FreeSWITCH latest Git and put them in /opt/sources/freewitch
cd /opt/sources/
git clone git://git.freeswitch.org/freeswitch.git freeswitch-git
cd freeswitch-git
./bootstrap.sh
Edit modules.conf depending on your needs
./configure -prefix=/opt/freeswitch-git --enable-core-odbc-support
make
make install
make sounds-install
make moh-install

To upgrade to latest Git
cd /opt/sources/freeswitch-git
make current
and that's it you get the latest Git in /opt/freeswitch-git


Prepare Network interface (I'm using a switch with vlans, useful for network isolation)
* Load linux vlan module
modprobe 8021q


* add 2 vlans (one for internal profile ex: vlan 100, the other for external profile ex: vlan 200)
vconfig add eth0 100
vconfig add eth0 200


* bring these interfaces up
ip link set up dev eth0.100
ip link set up dev eth0.200


* bring them IP
ip addr add 10.10.10.254/24 dev eth0.100
ip addr add 10.10.20.254/24 dev eth0.200


Edit sip_profiles to use these IP
cd /opt/freeswitch/conf/sip_profiles
edit internal.xml and replace $${local_ip_v4} with internal IP (10.10.10.254)
edit external.xml and replace $${local_ip_v4} and $${external_rtp_ip} with external IP (10.10.20.254)

Optimizations before launching FS

ulimit -c unlimited
ulimit -d unlimited
ulimit -f unlimited
ulimit -i unlimited
ulimit -n 999999
ulimit -q unlimited
ulimit -u unlimited
ulimit -v unlimited
ulimit -x unlimited
ulimit -s 240
ulimit -l unlimited
ulimit -a

Start FreeSWITCH
/opt/freeswitch/bin/freeswitch -nc -hp
-nc stands for no console
-hp stands for high priority

Connect on FreeSWITCH via fs_cli
/opt/freeswitch/bin/fs_cli
with this tool you'll be able to get a console connection to FS
the log level has not been changed so it could be very verbose
to disable log, type /nolog in console

Check if FS is running
netstat -unlp
you should have this

udp        0      0 10.10.10.254:5060       0.0.0.0:*                           9240/freeswitch 
udp        0      0 10.10.20.254:5060       0.0.0.0:*                           9240/freeswitch 

Installation of Kamailio

Installation of Bison, a parser generator
apt-get install bison

Installation of Flex, a fast lexical analyzer generator
apt-get install flex

Installation of MySQL and MySQL dev libraries
apt-get install mysql-server-5.0
You'll be prompt to set the password of your database, let's test with password "1234" or whatever you want

apt-get install libmysql++-dev

Installation of libconfuse, needed for carrierroute module compilation
apt-get install libconfuse-dev


Get sources of Kamailio and put them in /opt/sources
cd /opt/sources
wget http://www.kamailio.org/pub/kamailio/1.5.0/src/kamailio-1.5.0-notls_src.tar.gz
tar -xvzf kamailio-1.5.0-notls_src.tar.gz
cd kamailio-1.5.0-notls

If you need a LCR route table of approximatively 50 000 routes, then bypass this step.
For those who'd like to load a bigger LCR route table in carrierroute (for my needs: 160 000 routes)
emacs /opt/sources/kamailio-1.5.0-notls/config.h

#define PKG_MEM_POOL_SIZE 32*1024*1024          /*!< Used only if PKG_MALLOC is defined*/
#define SHM_MEM_SIZE 4*32                       /*!< Used if SH_MEM is defined*/


Note: these values have been set by empirical means and are maybe too high even for 160 000 routes

Compile Kamailio in /opt/kamailio-1.5.0
make prefix=/opt/kamailio-1.5.0
make prefix=/opt/kamailio-1.5.0 modules=modules/db_mysql modules
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute modules
make prefix=/opt/kamailio-1.5.0 install
make prefix=/opt/kamailio-1.5.0 modules=modules/carrierroute install
make prefix=/opt/kamailio-1.5.0 modules=modules/db_mysql install

Add a symbolic link for /opt/kamailio-1.5.0
ln -sf /opt/kamailio-1.5.0/ /opt/kamailio

Edit Kamailio Database Settings
cd /opt/kamailio/etc/kamailio/
emacs kamctlrc

# this parameter...
 DBENGINE=MYSQL
## database host
 DBHOST=localhost
## database name (for ORACLE this is TNS name)
 DBNAME=openser
# database path used by dbtext or db_berkeley
# DB_PATH="/usr/local/etc/kamailio/dbtext"
## database read/write user
 DBRWUSER=openser
## password for database read/write user
 DBRWPW="1234"
## database read only user
 DBROUSER=openserro
## password for database read only user
 DBROPW="1234"
## database super user (for ORACLE this is 'scheme-creator' user)
 DBROOTUSER="root"
# user name column
 USERCOL="username"

Then launch Kamailio utils to create database
/opt/kamailio/sbin/kamdbctl create
You'll be prompted for root sql password (remember you used 1234 or ...) and don't forget to set yes for carrierroute table creation

Populate carrierroute database with some routes
Connect to mysql
mysql -u root -p
Enter your root password
In mysql console:

use openser;
INSERT INTO carrier_name (carrier) VALUES ("default");
INSERT INTO domain_name (domain) VALUES ("default");
INSERT INTO carrierroute (carrier,domain,scan_prefix,flags,mask,prob,strip,rewrite_host,rewrite_prefix,rewrite_suffix,description) VALUES("1","1","1000","0","0","1","0","PEER-01","","","FRANCE");
exit


You have inserted your first route, that will match prefix 1000 and route this to PEER_01 with a description, FRANCE in this example. PEER_01 will be defined later in FS dialplan. The description parameter could be exported, for billing purpose for example. In this setup, I'll export the description to an AREA variable in FS.

Configuration of Kamailio
cd /opt/kamailio/etc/kamailio
mv kamailio.cfg kamailio.cfg.default
emacs kamailio.cfg


and put the config below:

# KAMAILIO basic configuration script for use with carrierroute and FS

####### Global Parameters #########

debug=1
log_stderror=no
log_facility=LOG_LOCAL0

fork=yes
children=8

/* uncomment the next line to disable TCP (default on) */
disable_tcp=yes

/* uncomment the next line to disable the auto discovery of local aliases
   based on revers DNS on IPs (default on) */
auto_aliases=no


/* uncomment and configure the following line if you want opensips to 
   bind on a specific interface/port/proto (default bind on all available) */
listen=udp:127.0.0.1:5062


####### Modules Section ########

#set module path
mpath="/opt/kamailio/lib64/kamailio/modules/"

/* uncomment next line for MySQL DB support */
loadmodule "db_mysql.so"
loadmodule "sl.so"
loadmodule "tm.so"
loadmodule "rr.so"
loadmodule "maxfwd.so"
loadmodule "textops.so"
loadmodule "mi_fifo.so"
loadmodule "xlog.so"
loadmodule "carrierroute.so"
loadmodule "pv.so"
loadmodule "avpops.so"

#----------------- setting module-specific parameters ---------------


# ----- mi_fifo params -----
modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo")

# ------ Carrierroute ------
modparam("carrierroute", "config_source", "db")
modparam("carrierroute", "db_url", "mysql://openser:1234@localhost/openser")
modparam("carrierroute", "carrierroute_table", "carrierroute")
modparam("carrierroute", "default_tree", "default")
modparam("carrierroute", "fetch_rows", 2000)

####### Routing Logic ########


# main request routing logic

route{

        if (!mf_process_maxfwd_header("10")) {
                sl_send_reply("483","Too Many Hops");
                exit;
        }

        t_check_trans();

        if ($rU==NULL) {
                # request with no Username in RURI
                sl_send_reply("484","Address Incomplete");
                exit;
        }

        # LOOKUP ROUTE TABLE WHEN ASKED BY HEADER: X-ROUTE:LOOKUP
        if (is_method("INVITE") && $hdr(X-ROUTE)=="LOOKUP"){
                if(!cr_route("default", "default", "$rU", "$rU", "call_id","$avp(s:route_desc)")){
                     #xlog("ROUTING FAILED: no route found for $rU");
                     sl_send_reply("604", "Unable to route this call");
                     exit;
                } else {
                     #xlog("LOOKUP FOUND: $rd $avp(s:route_desc)");
		     avp_pushto("$ru/username", "$avp(s:route_desc)");			       
                     sl_send_reply("302", "$rd");
                     exit;
                }
        }
}

Start Kamailio
/opt/kamailio/sbin/kamctl start

Check Kamailio is running
netstat -unlp
You should see something like this

udp        0      0 127.0.0.1:5062          0.0.0.0:*                           16269/kamailio


Check your route table
Seems broken in Kamailio 1.5.0, check SQL table
/opt/kamailio/sbin/kamctl cr show
You should see a route with PEER_01

Configuration of FreeSWITCH

Dialplan configuration for FS
cd /opt/freeswitch/conf/dialplan
emacs default.xml


Put this

<?xml version="1.0" encoding="utf-8"?>
<!-- http://wiki.freeswitch.org/wiki/Dialplan_XML -->
<include>
  <context name="default">

     <extension name="LOOKUP_ROUTE">
      <condition field="destination_number" expression="(\d+)$">
         <action application="set" data="hangup_after_bridge=true"/>
         <action application="set" data="continue_on_fail=true"/>
         <action application="export" data="sip_h_X-ROUTE=LOOKUP"/>
         <action application="bridge" data="sofia/internal/${destination_number}@127.0.0.1:5062"/>
         <action application="set" data="ROUTE_GW=${sip_redirect_contact_host_0}"/>
         <action application="set" data="AREA=${sip_redirect_contact_user_0}"/>
         <action application="transfer" data="${destination_number} XML ROUTING"/>
      </condition>
    </extension>

   </context>

  <context name="ROUTING">
    
    <extension name="PEER_01">
      <condition field="${ROUTE_GW}" expression="PEER_01">
	<action application="set" data="hangup_after_bridge=true"/>
        <action application="set" data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION"/>
        <action application="set" data="PEER=1.1.1.1"/>
        <action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
        <action application="set" data="PEER=2.2.2.2"/>
        <action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
        <action application="set" data="PEER=3.3.3.3"/>
        <action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
      </condition>
    </extension>

  </context>  

</include>


In fact PEER_01 is used as an index for a combination of peers. I do this to have a failover mechanism to a backup peer if the 1st peer is down.


You can load balance to different PEER combinations by using probability in carrierroute table. Ex: using 2 routes pointing to PEER_01 and PEER_02 with a probability of 0.5 and you will share the load between these 2 peers combination.

Test connectivity between FreeSWITCH and Kamailio

Test that FS and Kamailio are talking to each others
Register an IP phone using credentials in /opt/freeswitch/conf/directory/default/1001 for example
to dump traffic on the loopback interface for SIP messages between FS and Kamailio
ngrep -d lo -nn -W byline


When your phone is registered, dial 1000
You should see this in the ngrep trace

U 10.10.10.254:5060 -> 127.0.0.1:5062
INVITE sip:1000@127.0.0.1:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: <sip:mod_sofia@10.10.10.254:5060>.
User-Agent: FreeSWITCH-mod_sofia/1.0.trunk-11592M.
Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, PRACK, MESSAGE, SUBSCRIBE, NOTIFY, REFER, UPDATE, REGISTER, INFO.
Supported: timer, precondition, path, replaces.
Allow-Events: talk, refer.
Min-SE: 120.
Content-Type: application/sdp.
Content-Disposition: session.
Content-Length: 333.
X-ROUTE: LOOKUP.
Remote-Party-ID: "1001" <sip:1001@10.10.10.254>;screen=yes;privacy=off.
.
v=0.
o=FreeSWITCH 3223313286554167397 4405097820346134292 IN IP4 10.10.10.254.
s=FreeSWITCH.
c=IN IP4 10.10.10.254.
t=0 0.
m=audio 31686 RTP/AVP 9 0 8 3 101 13.
a=rtpmap:9 G722/8000.
a=rtpmap:0 PCMU/8000.
a=rtpmap:8 PCMA/8000.
a=rtpmap:3 GSM/8000.
a=rtpmap:101 telephone-event/8000.
a=fmtp:101 0-16.
a=rtpmap:13 CN/8000.
a=ptime:20.

#
U 127.0.0.1:5062 -> 10.10.10.254:5060
SIP/2.0 302 PEER_01.
Via: SIP/2.0/UDP 10.10.10.254;rport=5060;branch=z9hG4bKZjpQ3tQ8SN7gc.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 INVITE.
Contact: sip:1000@PEER_01.
Server: Kamailio (1.4.3-notls (x86_64/linux)).
Content-Length: 0.
.

#
U 10.10.10.254:5060 -> 127.0.0.1:5062
ACK sip:1000@127.0.0.1:5062 SIP/2.0.
Via: SIP/2.0/UDP 10.10.10.254;rport;branch=z9hG4bKZjpQ3tQ8SN7gc.
Max-Forwards: 69.
From: "1001" <sip:1001@10.10.10.254>;tag=pZ4egZ0m43Q3c.
To: <sip:1000@127.0.0.1:5062>;tag=458fb4012080e656b6742c09466dabcd.1740.
Call-ID: 7015d499-6c86-122c-80be-001b21009ccd.
CSeq: 110711449 ACK.
Content-Length: 0.
.

Optimizations

Stop FreeSWITCH
/opt/freeswitch/bin/freeswitch -stop

Lowering FS Log Level
emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml

    
    <!-- Default Global Log Level - value is one of debug,info,notice,warning,err,crit,alert -->
    <param name="loglevel" value="err"/>

Set Call Admission Control
emacs /opt/freeswitch/conf/autoload_configs/switch.conf.xml

    <!--Most channels to allow at once -->
    <param name="max-sessions" value="1000"/>
    <!--Most channels to create per second -->
    <param name="sessions-per-second" value="100"/>


With these settings, we have defined a maximum of 500 bridged calls (2*500 channels) and a maximum of 100 new sessions per second. You should set these parameters according to your hardware

Creation of a ramdisk for the FS database
mv /opt/freeswitch/db /opt/freeswitch/db_old
mkdir /opt/freeswitch/db
mount -t tmpfs tmpfs /opt/freeswitch/db

Use Proxy Media to keep FS in the RTP media path (topology hiding) but without analyzing RTP or DTMF
emacs /opt/freeswitch/conf/sip_profiles/internal.xml

<param name="inbound-late-negotiation" value="true"/>


Add directive proxy media in dialplan

    <extension name="PEER_01">
      <condition field="${sip_h_X-ROUTE}" expression="PEER_01">
        <action application="set" data="proxy_media=true"/>
	<action application="set" data="hangup_after_bridge=true"/>
        <action application="set" data="continue_on_fail=NORMAL_TEMPORARY_FAILURE,TIMEOUT,NO_ROUTE_DESTINATION"/>
        <action application="set" data="PEER=1.1.1.1"/>
        <action application="bridge" data="sofia/external/${destination_number}@1.1.1.1"/>
        <action application="set" data="PEER=2.2.2.2"/>
        <action application="bridge" data="sofia/external/${destination_number}@2.2.2.2"/>
        <action application="set" data="PEER=3.3.3.3"/>
        <action application="bridge" data="sofia/external/${destination_number}@3.3.3.3"/>
      </condition>
    </extension>

Disable Presence Support in SIP_Profiles
emacs /opt/freeswitch/conf/sip_profiles/internal.xml

<param name="manage-presence" value="false"/>


emacs /opt/freeswitch/conf/sip_profiles/external.xml

<param name="manage-presence" value="false"/>

Check that your Network Interfaces Cards (NIC) are not sharing IRQ
cat /proc/interrupts
ideally, your ethernet cards should be put on different IRQ, and if you have more than one CPU, you can force affinity of a NIC with a specific CPU (I will add a note later on this)

Check CPU usage

Install some useful utils for checking CPU consumptions
apt-get install sysstat
You can check the load of your CPUs using
mpstat -P ALL 1

13:13:13     CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
13:13:14     all    0,00    0,00    0,00    0,00    0,00    0,00    0,00  100,00    162,00
13:13:14       0    0,00    0,00    0,00    0,00    0,00    0,00    0,00  100,00    162,00

The %idle represents the idle percentage of the CPU, higher is better as it means that you have free CPU ressources


vmstat 1

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0     80 234312  93496 1273928    0    0     1    32   55   65  2  3 95  0
 0  0     80 234296  93496 1273928    0    0     0     0  106 1299  0  0 100  0

The last 2 columns are interesting:
id: idle, same as before for mpstat, the higher is better
wa: waiting IO, when this number is high it means that your CPU is waiting for IO (ex: HDD with too much access) and it will have a negative impact on performance


top
I think that I do not have to introduce this tool


Links to Kamailio and carrierroute