Mod xml curl

From FreeSWITCH Wiki

(Redirected from Xml curl)
Jump to: navigation, search

Contents

Documentation

Overview

mod_xml_curl can be used to dynamically control the behavior of FreeSWITCH™. FreeSWITCH™ will call mod_xml_curl when it needs to get information that it would normally read from the XML configuration files. By using mod_xml_curl, you can provide the same configuration to multiple instances of FreeSWITCH™ without having to duplicate the configuration. As you can imagine, this is the way to go when you want to scale up your system with multiple switches.

The module can be used to get any information that would be in freeswitch.xml.

Section names are passed to the script(s) on the webserver; currently, section names include:

Once you load the mod_xml_curl module, all future attempts to access XML will be sent to the webserver for any of the subscribed sections ("bindings"). To process only partial parts of a section (the public context of the dialplan, for example), just return nothing for all other requests, and FreeSWITCH will attempt to find the rest of the section in the static XML files, in /dialplan/default.xml. At a minimum you must have a modules.conf section which loads the mod_xml_curl module, a mod_xml_curl configuration section, and it is recommended that you have a logger module installed so you can see if anything goes wrong. Additional modules are loaded via the post_load_modules.conf file, which has basically the same format as modules.conf.

There is a PHP example that does everything from a webserver, with per switch configuration capability in FreeSWITCH™ SVN (scripts/contrib/trixter/xml-curl). This can be used for total switch provisioning from a webserver, providing a template for a web based management and control suite for FreeSWITCH™.

Activate

First of all, please make sure you build FreeSWITCH with this feature: 

make mod_xml_curl
make mod_xml_curl-install

Next, do not forget to activate mod_xml_curl in modules.conf.xml. mod_xml_curl is disabled by default on fresh install. 

<load module="mod_xml_curl"/>

Please put the mod_xml_curl entry more to the top of your modules.conf.xml file, do not put it at the end. Otherwise most of the configurations will not be requested from the external server. In setting the mod_xml_curl entry more to the end of the file you may thus control, for which module a request is being sent to the external server and thus reduce the number of external requests (and save time).

Configuring

The file conf/autoload_configs/xml_curl.conf.xml is used in the default FreeSWITCH™ setup.

Here is a minimum configuration file, it will fetch a dialplan from localhost, port 8080. 

<configuration name="xml_curl.conf" description="cURL XML Gateway">
  <bindings>
    <binding name="dialplan fetcher">
      <param name="gateway-url" value="http://localhost:8080" bindings="dialplan"/>
    </binding>
  </bindings>
</configuration>

Here is a slightly more advanced configuration, with a backup webserver that will be used if the primary server is non-responsive. The binding blocks are processed in sequential order, in a failover manner. 

<configuration name="xml_curl.conf" description="cURL XML Gateway">
  <bindings>
    <binding name="dialplan fetcher">
      <param name="gateway-url" value="http://localhost:8080" bindings="dialplan"/>
    </binding>
    <binding name="backup dialplan fetcher">
      <param name="gateway-url" value="http://anotherhost:8080" bindings="dialplan"/>
    </binding>
  </bindings>
</configuration>

If you're using lighttpd, nginx, thin and probably a few others as your webserver, you might receive some errors regarding "Expect: 100" or have performance problems. You can turn this off by adding this string to your binding: 

<param name="disable-100-continue" value="true"/>


Bindings

You may have more than one 'bindings' section, each fetching one type of information from somewhere. mod_xml_curl will POST a query string to the webserver with information about what it wants to fetch. More about this in the section below.

You may also have multiple bindings of the same type, FreeSWITCH™ will then visit each binding until it receives a match back from the server.

Binding

A binding provides information about the webserver that mod_xml_curl will visit, and what type of information to fetch.
You can also provide authentication options. HTTP-Basic will be used by default, but HTTP-Digest is supported. You should not use any sensitive passwords in HTTP-Basic mode (since they are sent in plain text). You may be able to get around this issue by either using HTTPS on the webserver or switching to HTTP-Digest.

Here is a binding example where you provide authentication (should be placed in the <bindings> tag) 

    <binding name="dialplan fetcher with authentication">
      <param name="gateway-url" value="http://localhost:8080" bindings="dialplan"/>
      <param name="gateway-credentials" value="muser:mypass"/>
      <param name="auth-scheme" value="basic"/>
    </binding>
  • binding name - The binding name is only added for your convenience, used only to describe the binding.
  • gateway-url - The URL that should be visited to fetch information
  • bindings - The bindings that this URL can give you information about, separate bindings with a 'pipe' character (|), example: dialplan|directory
  • gateway-credentials - Optional. The username and password to use to access the webserver. Only required if authentication is required on the webserver.
  • auth-scheme - Optional. The authentication scheme to use. Either basic or digest. Only basic is used by default, omitting this parameter while using digest on the web server will cause all requests to be rejected with HTTP 401 Unauthorized.

You may also create 3 binding structures, one each for dialplan, directory and configuration

Examples

bindings="configuration"

Request

FreeSWITCH™ will make a series of POST requests to your webserver to retrieve configuration information for each module.

The POST will contain a query string with the following values:

  • key_value = iax.conf|event_socket.conf|sofia.conf|...
  • key_name = name
  • section = configuration
  • tag_name = configuration

The format of the query string is: 

key_value=<value>&key_name=<name>&section=configuration&tag_name=configuration

Response

The response must look like this: 

<document type="freeswitch/xml">
  <section name="configuration">

    <!--The content of the tag SECTION is identical to what you find in the default config files.-->

    <configuration name="SECTIONNAME.conf" description="SECTIONDESCRIPTION">
      <settings>
         <!--ADD your parameters here-->
      </settings>
    </configuration>

  </section>
</document>

At least for sofia.conf you should provide the whole XML structure between and including the <configuration> tags, not only the settings part. You can get a template for this section from log/freeswitch.xml.fsxml .

bindings="directory"

User directory: used to provide information about user agents, or prevent them from logging in (See: XML User Directory Guide)

Note that the parameter "accept-blind-reg" must be set to "false" in sofia.conf.xml, your web page will not get called otherwise.

Order of Requests

As mod_sofia starts, it makes a number of different requests to help you determine what exactly is necessary in your response. Responding with as little as necessary is advisable, in order to avoid CPU spikes related to processing large XML documents. In other words, if you have a large number of users you can adversely affect voice quality by processing more XML than necessary. A small number of users with an adequate CPU do not always require this consideration, and the entire directory may be returned upon each request, for convenience sake.

Startup

When mod_sofia loads, for each profile it will need to scan the directory to retrieve any gateways and alias any domains. You do not have to return information about any of your users at this time. This same action will be taken for each rescan or restart of a profile. Below is an example request for the default internal profile: 

[hostname] => testmachine
[section] => directory
[tag_name] => 
[key_name] => 
[key_value] => 
[Event-Name] => REQUEST_PARAMS
[Core-UUID] => c5c8cbf4-60c3-45a2-b110-933da620cfd2
[FreeSWITCH-Hostname] => testmachine
[FreeSWITCH-IPv4] => 192.168.1.10
[FreeSWITCH-IPv6] => ::1
[Event-Date-Local] => 2009-10-27 00:52:52
[Event-Date-GMT] => Tue, 27 Oct 2009 07:52:52 GMT
[Event-Date-Timestamp] => 1256629972839876
[Event-Calling-File] => sofia.c
[Event-Calling-Function] => config_sofia
[Event-Calling-Line-Number] => 3056
[purpose] => gateways
[profile] => internal

A sample minimal response to this request for a multi-user and multi-domain setup is given below. For addition information on domain params and variables, see the default domain in the example configuration. 

<document type="freeswitch/xml">
  <section name="directory">	
    <domain name="domain1.awesomevoipdomain.faketld">
      <params>
        <param name="dial-string" value="{presence_id=${dialed_user}@${dialed_domain}}${sofia_contact(${dialed_user}@${dialed_domain})}"/>
      </params>
      <variables>
        <variable name="example_var" value="example_value_1"/>
      </variables>
      <user id="default" />
    </domain>
    <domain name="domain2.awesomevoipdomain.faketld">
      <params>
        <param name="dial-string" value="{presence_id=${dialed_user}@${dialed_domain}}${sofia_contact(${dialed_user}@${dialed_domain})}"/>
      </params>
      <variables>
        <variable name="example_var" value="example_value_2"/>
      </variables>
      <user id="default" />
    </domain>
  </section>
</document>

ACL

   [hostname] => black
   [section] => directory
   [tag_name] => domain
   [key_name] => name
   [key_value] => 192.168.0.2
   [domain] => 192.168.0.2
   [purpose] => network-list

This last post is regarding mod_sofia asking for users with cidr = attributes for adding them to the ACLs.

Authorization

An example registration is shown below. The request will be almost identical for a registration as for an unregistration. The user 1004@domain1.awesomevoipdomain.faketld (a polycom 320 addressed at 192.168.1.100) is attempting to authenticate to a server (192.168.1.10): 

[hostname] => testmachine
[section] => directory
[tag_name] => domain
[key_name] => name
[key_value] => domain1.awesomevoipdomain.faketld
[Event-Name] => REQUEST_PARAMS
[Core-UUID] => c5c8cbf4-60c3-45a2-b110-933da620cfd2
[FreeSWITCH-Hostname] => 25515_1_36308_177178
[FreeSWITCH-IPv4] => 192.168.1.10
[FreeSWITCH-IPv6] => ::1
[Event-Date-Local] => 2009-10-27 00:47:10
[Event-Date-GMT] => Tue, 27 Oct 2009 07:47:10 GMT
[Event-Date-Timestamp] => 1256629630733916
[Event-Calling-File] => sofia_reg.c
[Event-Calling-Function] => sofia_reg_parse_auth
[Event-Calling-Line-Number] => 1671
[action] => sip_auth
[sip_profile] => internal
[sip_user_agent] => PolycomSoundPointIP-SPIP_320-UA/3.1.0.0084
[sip_auth_username] => 1004
[sip_auth_realm] => domain1.awesomevoipdomain.faketld
[sip_auth_nonce] => 533c5264-12cb-4f8b-bcdb-5ecabe5e540f
[sip_auth_uri] => sip:domain1.awesomevoipdomain.faketld:5060
[sip_contact_user] => 1004
[sip_contact_host] => 192.168.1.100
[sip_to_user] => 1004
[sip_to_host] => domain1.awesomevoipdomain.faketld
[sip_from_user] => 1004
[sip_from_host] => domain1.awesomevoipdomain.faketld
[sip_request_host] => domain1.awesomevoipdomain.faketld
[sip_request_port] => 5060
[sip_auth_qop] => auth
[sip_auth_cnonce] => hSVnPb32nA/OtkY
[sip_auth_nc] => 00000001
[sip_auth_response] => 6e4e611d7593d52e02451b70900071d8
[sip_auth_method] => REGISTER
[key] => id
[user] => 1004
[domain] => domain1.awesomevoipdomain.faketld
[ip] => 192.168.1.100

This can be a fairly large and ominous looking POST at first glance, so let us break it down into only what is critical to generating a response for mod_xml_curl: 

[hostname] => testmachine
[section] => directory
[tag_name] => domain
[key_name] => name
[key_value] => domain1.awesomevoipdomain.faketld
[key] => id
[user] => 1004
[domain] => domain1.awesomevoipdomain.faketld

Now things are easier to see. The information above tells us that mod_xml_curl from the host "testmachine" is asking for directory information, and it wants to look up a user by id (1004) in a specific domain (domain1.awesomevoipdomain.faketld). The logic in our script checks the section type, and that tag_name and key_name are set to "domain" and "name" respectively. Furthermore, we can check to ensure that domain matches key_value, before attempting to select our specific user from a database. A minimum example response is shown below: 

<document type="freeswitch/xml">
  <section name="directory">
    <domain name="domain1.awesomevoipdomain.faketld">
      <params>
        <param name="dial-string" value="{presence_id=${dialed_user}@${dialed_domain}}${sofia_contact(${dialed_user}@${dialed_domain})}"/>
      </params>
      <groups>
        <group name="default">
         <users>
          <user id="1004">
            <params>
              <param name="password" value="some_password"/>
            </params>
          </user>
         </users>
        </group>
      </groups>
    </domain>
  </section>
</document>

If you don't want to communicate passwords in plain text over your network, you can use the `a1-hash` param instead. It's value should be an md5sum containing `user:domain:password`.

For example, if you have a user 1002 on domain 127.0.0.1 with password 1234, you can generate the proper hash by: 

echo -n 1002:127.0.0.1:1234 | md5sum

The proper param tag for this example is: 

<param name="a1-hash" value="50046ba744759aa83e045ba0b996e7a9"/>

Voicemail request

Mod_voicemail occasionally needs to look up a user's id. The request example is below and can be responded to in the same way as an authorization request for simplicity: 

[hostname] => testmachine
[section] => directory
[tag_name] => domain
[key_name] => name
[key_value] => domain1.awesomevoipdomain.faketld
[Event-Name] => GENERAL
[Core-UUID] => c5c8cbf4-60c3-45a2-b110-933da620cfd2
[FreeSWITCH-Hostname] => testmachine
[FreeSWITCH-IPv4] => 192.168.1.10
[FreeSWITCH-IPv6] => ::1
[Event-Date-Local] => 2009-10-27 00:47:40
[Event-Date-GMT] => Tue, 27 Oct 2009 07:47:40 GMT
[Event-Date-Timestamp] => 1256629660158410
[Event-Calling-File] => mod_voicemail.c
[Event-Calling-Function] => resolve_id
[Event-Calling-Line-Number] => 1278
[action] => message-count
[key] => id
[user] => 1004
[domain] => domain1.awesomevoipdomain.faketld

Dial by user/<username> Request

If you try to dial a user in the directory via the user/<username> syntax the request looks like this: 

   [as_channel] => true
   [key] => id
   [user] => username ; directory username you tried to dial
   [domain] => 192.168.0.2

You can respond like: 

<document type="freeswitch/xml">
  <section name="directory">
    <domain name="192.168.0.2">
      <user id="username">
        <params>
          <param name="dial-string" value="${sofia_contact(${dialed_user}@${dialed_domain})}"/>
        </params>
      </user>
    </domain>
  </section>
</document>

Which tells FreeSWITCH™ how to dial that particular user - it can be via sofia, openzap or anything that isn't a user/ dialstring itself.

bindings="dialplan"

See also: Dialplan

This option retrieves call routing details from the webserver in a similar manner to other binding methods.

Request

FreeSWITCH™ will POST a request to your webserver with information about the incoming call.

The query string will look like something like this (everything is sent in one line, it is wrapped for readability): 

section=dialplan&tag_name=&key_name=&key_value=&context=default&destination_number=556
&caller_id_name=FreeSwitch&caller_id_number=5555551212&network_addr=&ani=&aniii=&rdnis=
&source=mod_portaudio&chan_name=PortAudio/556&uuid=b7f0b117-351f-9448-b60a-18ff91cbe183
&endpoint_disposition=ANSWER
  • section - What kind of information that is requested (dialplan, configuration or directory)
  • tag_name -
  • key_name -
  • key_value -
  • context - What configuration context to fetch, normally "default" unless you have multiple contexts.
  • destination_number - Number called
  • caller_id_name - Name of the calling user
  • caller_id_number - Number of the calling user
  • network_addr - IP address of the called user.
  • ani - Phone number that the user called from (if calling from PSTN)
  • aniii -
  • rdnis -
  • source - source technology, ie. mod_sofia, mod_portaudio, mod_dingaling, etc.
  • chan_name - parsable channel name
  • uuid - Unique identifier for this call, will follow the call as long as it's in FreeSWITCH™
  • endpoint_disposition - Call state when mod_xml_curl requests information

Sample xml_curl dialplan POST

Note: it seems the most interesting fields for basic (destination-inside-context-based) call routing are Hunt-Context and Hunt-Destination-Number.

Reply

Here is a sample reply. You will only need to send one extension back, and a condition is required to parse properly.
Your condition may be elaborate or simply '<condition>'. 

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<document type="freeswitch/xml">
  <section name="dialplan" description="RE Dial Plan For FreeSwitch">
    <context name="default">
      <extension name="test9">
        <condition field="destination_number" expression="^83789$">
          <action application="bridge" data="iax/guest@conference.freeswitch.org/888"/>
        </condition>
      </extension>
    </context>
  </section>
</document>

General "Not found" reply

If you do not have a match AND you would like FreeSWITCH™ to stop looking in the current binding, you should return a "not found" result. 

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<document type="freeswitch/xml">
  <section name="result">
    <result status="not found" />
  </section>
</document>
Important note: If you return an empty response instead of the not found, you may see 
[ERR] switch_xml.c:1534 switch_xml_locate() Error[[error near line 1]: root tag missing]
when issuing 'reloadxml'. Making sure you return a valid response or not found eliminates the ERR.

Debugging

You can use the following freeswitch command to help with debugging your configuration. It will return a filename with the resulting XML for each xml_curl query. 

xml_curl debug_on

Really Large CURL Transactions

If you get an error like this: 

mod_xml_curl.c:121 Oversized file detected [1056100 bytes]

Then that means you're really serving up some big-time configs. No worries, just edit mod_xml_curl.c (line 64 as of this writing) and adjust XML_CURL_MAX_BYTES as needed.

How XML CURL co-exists with static files

Two types of data sources

The XML configuration data for FreeSWITCH™ can come from many sources, but those sources can be grouped into two types:

Static Files
These are the files that live under your conf directory, and are included by the conf/freeswitch.xml file. This data is read into memory when FreeSWITCH™ starts and does not change unless the process is restarted, or the reloadxml API is used.

NOTE: You can "stack" dialplans if necessary. For example, look in conf/sip_profiles/internal.xml and locate this line: 

<param name="dialplan" value="XML"/>

It is possible to put a comma-separated list of dialplans in this parameter. Examples: 

<param name="dialplan" value="XML,XML:/path/to/custom/dialplan.xml"/>
<param name="dialplan" value="ENUM,CUSTOM,XML"/>


Dynamic Bindings
It is possible to write code that will provide configuration data to FreeSWITCH™ upon request. This happens in real-time at the moment a particular module requests a piece of configuration data, and therefore has the potential to always be the most recent data.

The dynamic mechanism is what allows mod_xml_curl to work. It is the same mechanism that allows you for example to generate configuration using Lua, Perl or Python.

Searching for data

When a module needs a piece of configuration data, it will ask FreeSWITCH™ for that data using a common API. FreeSWITCH™ then searches for that data in the following order:

  1. Any dynamic sources that were registered, in the order they were registered. Typically, if you are using mod_xml_curl, then it will be the only dynamic source you need to worry about.
  2. The in-memory configuration data built from your local XML files in the conf directory.

FreeSWITCH™ will stop looking as soon as it finds the requested data. In this way, it is possible to either provide new configuration data that is not present in your static files, or to override your static data with dynamic data.

Pre-processor directives and variables

Note: Right now, this section is based more on my evaluation of the FreeSWITCH™ source code than on actually using the mechanism. I'm putting it here now while it is fresh in my mind because I believe it to be accurate.

When mod_xml_curl retrieves an XML document, it saves that document to a temporary file, and then tells FreeSWITCH™ to process that file like it would process any other XML configuration file. This means that any X-PRE-PROCESS tags you place in your dynamic configuration will be processed before handing the resulting XML to the main configuration parser, just like your conf/* files.

Some care must be taken when setting variables using X-PRE-PROCESS tags. Setting a variable using set in a X-PRE-PROCESS tag sets a "core global" variable. In all XML configuration these variables are referenced by the form $${varname}, and are actually treated as preprocessor variables. That is, the same parser pass that processes X-PRE-PROCESS tags also substitutes these variables for their value at that very instant in time, and the resulting xml document is passed to the main configuration parser, and that is the configuration that is used. Simply put, if you do something like this somewhat forced example: 

...
<X-PRE-PROCESS cmd="set" data="global_codec_prefs=PCMU"/>
...
<param name="codec-prefs" value="$${global_codec_prefs}"/>
...
<X-PRE-PROCESS cmd="set" data="global_codec_prefs=PCMA"/>
...
<param name="codec-prefs" value="$${global_codec_prefs}"/>
...

Just pretend that is happening across two different SIP profiles so that it seems somewhat plausible. The first reference to $${global_codec_prefs} will expand to mu-law, and the second one (and all subsequent references in time) to $${global_codec_prefs} will expand to a-law. The first setting is unchanged.

This point deserves some discussion because it's important to know that, as truly global variables, values of variables set in conf/vars.xml or in any X-PRE-PROCESS tag are accessible to the XML you generate dynamically, which makes it OK to reference $${local_ip_v4} from your generated XML and the Right thing will happen.

It's also possible to overwrite variables that were previously set in vars.xml or any X-PRE-PROCESS tag using your generated XML. Doing this, however will have absolutely no effect on any previously parsed XML that used that variable. Also, executing a reloadxml will cause the original vars.xml to be re-parsed, thus overwriting your overwrite. If this sounds confusing that is because you can probably create some confusing and difficult to track bugs in your configuration by doing this.

Having said all of this, we can distill the information down to the simple caveat that, while it is safe and good to reference static variables in your dynamic configuration, it is probably not safe to try to overwrite them. Treat them with the respect you'd treat any global variable and you should be fine.

Notes

tags: mysql database realtime schema.

Retrieved from "http://wiki.freeswitch.org/wiki/Mod_xml_curl"
Personal tools
Community
Support FreeSWITCH