Saturday, 27 July 2013

Two accepted papers

Our Poster Paper titled "Bayesian Regularized Neural Network for Peer-to-Peer Botnet Detection in Quasi Real-Time" has been accepted at the 13th annual IEEE conference on P2P computing (University of Trento, Italy). Congrats to the authors!
Authors: Pratik Narang, Sharath Chandra G, Chittaranjan Hota


Another work of ours, titled "Feature Selection for Detection of Peer-to-Peer Botnet Traffic" has been accepted at the ACM Compute conference (Vellore, India). Kudos to the authors!
Authors: Pratik Narang, Jagan Mohan Reddy, Chittaranjan Hota

Tuesday, 9 July 2013

Campus Wi-Fi with CoovaChilli, FreeRADIUS and Active Directory

Setting up an authenticated Wi-fi with CoovaChilli & FreeRADIUS over Windows Active Directory
Please note:
  • This is NOT a technical guide or a tutorial for CoovaChilli/FreeRADIUS/Active directory. It is a documentation of our own experience with these technologies, shared on the web for our own future reference and in the hope that someone in a similar situation may find it useful. Accept all advice with a grain of salt! Your mileage may vary.
  • The content of this post has been liberally borrowed from the online sources listed in the references. The reader is advised to refer to them for more details. 

The task at hand
We need to set-up our Wi-fi network which will be authenticated over the Active Directory provided by our Institute. This Wi-fi network will serve as a data collection network for our network-related experiments.
The equipments used-
  • A 64bit machine with Ubuntu 12.04 having two NIC cards
  • A Belkin Basic N150 router
  • CoovaChilli
  • FreeRADIUS


Getting started
Configure the Belkin router to not to act as a router or a firewall and only provide the Access point functionality. It is a fairly simple procedure and not elaborated here. For the Belkin Basic N150 being used by us, we just had to choose the 'enable as Access point' (or something similar) option from the router's configurations, and assign it a static IP address. Just be careful to NOT to plug in the WAN wire into the router's WAN interface. It must go in ANY of the LAN interfaces.

Open the /etc/network/interfaces file and make sure that you have one NIC with a static IP. This is the IP coovachilli will run-off as a server. Another thing you need to do is enable packet forwarding and NAT between the interfaces.
After doing this, our /etc/network/interfaces file looked like this-

    auto lo
    iface lo inet loopback

    auto eth0
    iface eth0 inet dhcp

    auto eth1
    iface eth1 inet static
        address 172.28.8.2
        netmask 255.255.248.0
        post-up iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward

To make sure packet forwarding is enabled, enable it from /etc/sysctrl.conf as well. To do that, you just need to run this command-
    sed --in-place=.old 's/^#\(net.ipv4.ip_forward=1\)/\1/' /etc/sysctl.conf

Install FreeRADIUS and related stuff used by CoovaChilli-
    sudo apt-get install freeradius freeradius-utils libtool libssl-dev libcurl4-openssl-dev
Install Samba if you don't have it.
    sudo apt-get install samba
Install winbind
    sudo apt-get install winbind

We will configure them later.

The following is based on reference [1].

The first thing to do is join the Ubuntu server to the domain controller (DC).
Linux must be configured to join a Windows domain. This is done by using the Samba file server which offers several interesting tools. The goal is not to create a Samba file server but only to use some tools which come with this server.
Let Samba know what domain it belongs to. Open /etc/samba/smb.conf and set the following-
    workgroup = MYDOMAIN ##whatever it is...
    realm = realm.your-company.com
    #In authentication section-  
    security= ads 
    #In 'Share definitions' section, uncomment-  
    comment = Home Directories 
    browseable = no  
    writable= yes

Edit the file /etc/nsswitch.conf and add winbind at the end of each line shown below
    passwd:     files winbind 
    shadow:     files winbind 
    group:      files winbind 
    protocols:  files winbind 
    services:   files winbind 
    netgroup:   files winbind 
    automount:  files winbind 

Restart the machine.
Verify if the Samba service is running by typing
    ps –ef | grep nmbd 
    ps –ef | grep smbd 

Do- net join -U <authenticated_AD_user> 
And it will join your machine (not the user) to the 'realm'. 
Our output was-
    net join -U compnet2012
    Enter compnet2012's password:
    Using short domain name -- BITS-XYZ
    Joined 'WLC-OPTIPLEX' to realm 'bits...'
    No DNS domain configured for wlc-optiplex. Unable to perform DNS Update.
    DNS update failed!
Even though DNS update failed, there was no hindrance in any of the future tasks. The failure was obvious since we are not admin for AD!

Next, try with wbinfo command-
    root@wlc-OptiPlex:/etc/samba# wbinfo -a compnet2012%<password>
    plaintext password authentication failed
    Could not authenticate user compnet2012%mnb0567 with plaintext password
    challenge/response password authentication failed
    error code was NT_STATUS_WRONG_PASSWORD (0xc000006a)
    error message was: Wrong Password
    Could not authenticate user compnet2012 with challenge/response

If it fails, try again without supplying password in plain text-
    root@wlc-OptiPlex:/etc/samba# wbinfo -a compnet2012
    Enter compnet2012's password:
    plaintext password authentication succeeded
    Enter compnet2012's password:
    challenge/response password authentication succeeded

 
Let’s try to authenticate with NTLM, which is necessary for using FREERADIUS with Active Directory.
Type the following line
    ntlm_auth –-request-nt-key –-domain=MYDOMAIN –-username=<your username>
You will be prompted for your password.
The command line returns 'NT_STATUS_OK : Success (0x0)' if the username and password are the same as those stored in Active Directory.

Note that this mechanism is based on a challenge/response of the nt-key, a character string that has been encrypted with information taken from the username and password.
During this operation, no exchange of user information takes place. Everything is based upon a comparison of encrypted strings.

The following were taken from [3], [5]

Configuring FreeRADIUS to use ntlm_auth
    sudo -i #become root user
    cd /etc/freeradius
    vim  clients.conf
At the end of this file, you need to add your Access Point as a client for RADIUS-
    client 172.25.ab.xyz {
        secret = <enter this>
        shortname = <choose anything>
        nastype = other
    }
Save and exit from the file.

Now, create a file (if it doesn't exist) ntlm_auth in the /modules folder of freeradius, and the put the following text in it-

     exec ntlm_auth {
           wait = yes
           program = "/path/to/ntlm_auth --request-nt-key --domain=MYDOMAIN --username=%{mschap:User-Name} --password=%{User-Password}"
            }

 
This configuration tells the server to run the ntlm_auth program with the user name and password obtained from the Access-Request. You will also have to list ntlm_auth in the authenticate sections of each the /sites-enabled/default file, and of the /sites-enabled/inner-tunnel file:
     authenticate {
        ...
        ntlm_auth
        ...
    } 

and add the following text for testing purposes only to the top of the users file.
    DEFAULT     Auth-Type = ntlm_auth
This configuration says "for all users, if the authenticate method has not been set, set it to use the ntlm_auth program".

Let us add a test user to freeradius. Add the following to the /etc/freeradius/users file-

    testuser Cleartext-Password := "userpass1"    
    Simultaneous-Use = 999999,    
    Idle-Timeout = 86400

Start the server using freeradius -X, and wait for the debugging text to stop scrolling by. If all goes well, you should see the following text:
      Ready to process requests
In another terminal window on the same machine, type the following command:
   radtest testuser userpass1 localhost 0 testing123
If all goes well, you should see the server returning an Access-Accept message, and the window with radtest should print text similar to the following:
   rad_recv: Access-Accept packet from host 127.0.0.1 port 1812, length=20
This text means that authentication succeeded. A few lines above this text, the debug output will also show the exact command line used to run ntlm_auth.

Configuring FreeRADIUS to use ntlm_auth for MS-CHAP
First, delete the testing entry used above from the users file, as leaving it in will break other authentication types. 
Then, find the mschap module in /modules/mschap file, and look for the line containing ntlm_auth = . It is commented out by default, and should be uncommented, and edited to be as follows. Update the fields in bold to match your local configuration.
    ntlm_auth = "/path/to/ntlm_auth --request-nt-key --username=%{mschap:User-Name:-None} --domain=%{%{mschap:NT-Domain}:-MYDOMAIN} --challenge=%{mschap:Challenge:-00} --nt-response=%{mschap:NT-Response:-00}" 
Look for the line containing the following text and uncomment it-
    with_ntdomain_hack = yes

The following section is based on [5], [6] 

Install & Configure CoovaChilli  
Download CoovaChilli's source code from their website- http://coova.org/Download. Note that [4] says that CoovaChilli doesn't work with 64bit OS and RADIUS authentication is flawed in those setups. But we were able to successfully do our installation and set-up on a 64bit Ubuntu 12.04 :)

    tar -zxvf coova-chilli-1.3.0.tar.gz
    cd coova-chilli-1.3.0/
    ./configure --enable-dhcpopt --enable-debug2 --enable-chilliproxy --enable-chilliradsec --enable-miniportal --enable-miniconfig --enable-libjson --with-openssl --with-curl
    make
    sudo make install

Our initial run of CoovaChilli was simply with './confgure'. But we were facing some errors with it and moreover we needed some of those '--enable' options. Hence we decided to go ahead with the above said '--enable' and '--with' build options.

Install haserl. Get the tar from http://sourceforge.net/projects/haserl/files/. Untar and build in the same manner as CoovaChilli (no configure options, of course).

By default, Chilli is installed in /usr/local/etc/ folder. Let's check-
    cd /usr/local/etc/
    ls -lt
    total 12
    drwxr-xr-x 2 root root 4096 Jul  9 11:56 init.d
    drwxr-xr-x 3 root root 4096 Jul  9 11:56 chilli
    -rw-r--r-- 1 root root  861 Jul  9 11:56 chilli.conf

Now that its installed, we need to configure it before we enable it. To do this, copy the defaults to the config file
cd /usr/local/etc/chilli cp ./defaults ./config
From here, edit /etc/chilli/config. Only a few changes are required-

    HS_WANIF=eth0            # WAN Interface toward the Internet
    HS_LANIF=eth1              # Subscriber Interface for client devices
    HS_NETWORK=172.25.9.0      # HotSpot Network (must include HS_UAMLISTEN)
    HS_NETMASK=255.255.248.0   # HotSpot Network Netmask
    HS_UAMLISTEN=172.25.9.1    # HotSpot IP Address (on subscriber network)
    HS_UAMPORT=3990            # HotSpot UAM Port (on subscriber network)
    HS_UAMUIPORT=4990          # HotSpot UAM "UI" Port (on subscriber network, for embedded portal)

    # OpenDNS Servers           ##set your own here, like-
    HS_DNS1-172.16.ab.xyz  ##Ours didn't work for us, so we resorted to defaults here
    #uncomment this line-
    HS_RAD_PROTO="mschapv2" 

    ##For better security, it is better to change these from default values-
    HS_RADSECRET=testing123    # Set to be your RADIUS shared secret
    HS_UAMSECRET=change-me     # Set to be your UAM secret
    ##change the name to what you want-
    HS_LOC_NAME="My HotSpot"           # WISPr Location Name and used in portal

Specify 'dhcpstart <number>' in chilli.conf (in /usr/local/etc/) if you want the DHCP IP addresses to begin from a certain number, say .20, and not immediately from .2 on your network. We used 20 here, to reserve the IPs from .2 to .20 for, say, our APs, WLCs, etc. 

Reboot the machine.

Testing the set-up
Start RADIUS-  
    freeradius -X
Start Chilli
    /usr/local/etc/init.d/chilli ##will create some files such as  main.conf
    sudo chilli
Starting Chilli gives an error saying that user chilli is not found. We never created user chilli but all things went fine.
 
Use any Wi-fi client to connect to your AP. As you try to connect to the Internet, CoovaChilli's Captive portal will capture the page and redirect to a login page. Any user who is authenticated with the Active Directory should be able to authenticate. 

You can customize CoovaChilli's captive portal by editing the files in /usr/local/etc/chilli/www/ folder.

Saturday, 6 July 2013

Riding the elephant, virtually!

The NetClique team of Prof. Abhishek and Pratik, along with our latest member Ashwini Patil (student, M.E. CS) worked together to create a Hadoop cluster for our experiments. The cluster was deployed using Virtual Machines on a 24 core machine . We decided to document the steps here, just in case someone dealing with similar experiments finds it helpful. Thanks to Ashwini for her documentation efforts. 


Let us get started!

Basics steps of the Hadoop setup covered here are:

  • install virtual machine manager for GUI interface (optional)
  • install the required number of Ubuntu virtual machines using ubuntu-vm-builder
  • choose one of the VMs as manager and install cloudera manager on it

Detailed installation procedure goes like this-

Step 1: Installing virtual machine manager
Use the following command to install KVM and supporting packages. 
Virt-Manager is a graphical application for managing your virtual machines. You can use the kvm command directly, but libvirt and Virt-Manager simplify the process.
sudo apt-get install qemu-kvm libvirt-bin bridge-utils virt-manager

Only the root user and users in the libvirtd group have permission to use KVM virtual machines. 
Run the following command to add your user account to the libvirtd group:
sudo adduser <name> libvirtd

After running this command, log out and log back in. 

Run the following command after logging back in and you should see an empty list of virtual machines. 
This indicates that everything is working correctly.
virsh -c qemu:///system list

->Follow this link for help on installing VMs using it. 

Step2: Installing VMs using ubuntu-vm-builder
2.1-> We wrote a Shell script to install 30 Virtual Machines at a stretch. 
For instance,the following bash script will install 30 VMs (IP range is from .41 to .70)

     #!/bin/bash
     for i in $(seq 41 1 70)
     do
     echo <password>| `sudo -S ubuntu-vm-builder kvm precise
     --arch 'amd64'  --mem '1024'  --rootsize '15000'
     --kernel-flavour 'generic' --hostname "vm1$i"  
     --domain 'ss1.netclique.in'
     --mirror 'http://archive.ubuntu.com/ubuntu'
     --components 'main,universe,restricted'
     --addpkg openssh-server --addpkg vim  --addpkg expect 
     --name "vm_1$i"  --user "vm1$i"  --pass '<password>'
     --ip "172.25.3.1$i"  --mask '255.255.255.0'
     --net '172.25.3.0'  --bcast '172.25.3.255'
     --gw '172.25.3.1'  --dns '172.16.100.221' 
     --bridge 'br0'  --libvirt 'qemu:///system'
     --dest /media/.../vm14Si
     --firstboot /home/bits/firstboot.sh`

           done

   ->Follow this link here which gives you a 'parameter generator' for ubuntu-vm-builder.

2.2->To list the VMs created, use virsh 'list --all'

2.3->We need to do certain boot time settings to make things work. [These are there in the file firstboot.sh, which is supplied as a 'firstboot' argument with ubuntu-vm-builder]
->edit /etc/apt/apt.conf file, to configure our Internet proxy.
->we would need passwordless sudo permissions for the user for Cloudera installation on that system.This can done by adding following line to /etc/sudoers file 
  <username> ALL=NOPASSWD: ALL
->edit /etc/hosts file to add the hosts IP and name.
     
Our firstboot script, firstboot.sh, looks like this:

    #!/bin/bash
    touch /etc/apt/apt.conf 

    #adding proxy configurations to the file
    cat>> /etc/apt/apt.conf <<END
    Acquire::http::proxy "http://172.xx.y.zz:3128/";
    Acquire::ftp::proxy "ftp://172.xx.y.zz:3128/";
    Acquire::https::proxy "https://172.xx.zz.23:3128/";
    END    

    #adding password less sudo user called testuser
    cat>> /etc/sudoers <<EOF
    testuser ALL=NOPASSWD: ALL
    EOF   

    `apt-get update`
    `apt-get upgrade`
    #Adding the hosts IP and hostname to its /etc/hosts file 
    #and  deleting the entry which maps the hostname to loopback address
    sed -i '/127.0.1.1/d' /etc/hosts
    ip=`ifconfig eth0|grep 'inet addr'|awk -F : '{print $2}'|awk '{print $1}'`
    hostname=`ifconfig eth0|grep 'inet addr'|awk -F : '{print $2}'|awk '{print $1}'|awk -F .     '{print $4}'`
    echo "$ip vm$hostname.ss1.netclique.in vm $hostname" >>  /etc/hosts

#add the Cloudera Manager's entry in /etc/hosts. The IP is hard-coded here. 
echo "172.xx.y.171 cloudera-manager1.ss1.netclique.in cloudera-manager1">>/etc/hosts

# In the /etc/hosts file for the Cloudera Manager, a similar entry for the name node must be added.

It is recommended that the name node be a powerful machine (RAM ~8GB, Hard disk ~30GB). You will need to create it separately, although all parameters remain same as above. 
The node being used as the Cloudera-manager will also have similar configurations as above, but it will need to have the name node's entry in its /etc/hosts file.

Step 3: Install Cloudera manager on a Manager machine (it can be any of VMs or physical machines).