linux: network manager keeps removing the nameserver i set in resolv.conf!

I want to always have Google's public DNS servers (8.8.8.8, 8.8.4.4) in my /etc/resolv.conf

However my WiFi settings are set to DHCP (for both office and home) and whenever I get a DHCP address, I also get the DNS servers configured on the DHCP server. These server settings overwrite my /etc/resolv.conf and I lose the

Having to set the nameservers manually in /etc/resolv.conf on each DHCP connection is too painful, so I found a simple and effective solution to this problem:

$ sudo vim /etc/dhcp3/dhclient.conf

# ensure the following line is uncommented in dhclient.conf:

# prepend domain-name-servers 8.8.8.8,8.8.4.4;

$ sudo service network-manager restart

This configures the DHCP client to always prepend the list of DNS servers obtained from DHCP with "8.8.8.8, 8.8.4.4" - and it works like a charm!

how to add multiple ip addresses to loopback and ping an AVD's host machine

From my Android Virtual Device I was trying to figure out how to connect to a service running on the same machine running the emulator.

For example, let's say I'm running Apache on my laptop, and also running the AVD on my laptop. In a browser on my laptop I can type "http://localhost/index.html" to test the index.html present in my Apache webroot.

But if I put "http://localhost/index.html" in the Android browser in the AVD, I don't get anything. That's because 'localhost' refers to the AVD itself, not the host outside the AVD.

This stackoverflow question saved the day: it turns out that from within the AVD you can ping the host machine at 10.0.2.2 - this is a kind of hard-coded virtual IP address in Android emulators for the host box.


Next, I wanted to make 10.0.2.2 go to localhost so that when I play a URL from the logs like http://10.0.2.2:8080/sms/srs?merch_txn_id=AK0AqgCkAMIAwwB5AOMAeA%3D%3D&i3p_db=ptsqa_fortumo3&i3p_host=10.0.2.2%3A5432&country=US
I don't want to have to replace 10.0.2.2 with localhost every time in every URL.

This part of the puzzle was solved by this very helpful link : I simply had to create a virtual IP address for the loopback device via:
ip -4 addr add 10.0.2.2/32 dev lo

After this we can see that 10.0.2.2 is an alias for the loopback interface, "lo":

[02:24:26] ~ $ ip addr
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 10.0.2.2/32 scope global lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

linux: vnstat rocks!

vnstat is a superb tool to monitor monthly bandwidth usage.

I get notified by my ISP when I hit my monthly download quota (25 GB) that my speed will now be reduced to 256 kbps. Sometimes I'm left wondering how the heck I hit 25 gigs of downloading. So I started looking for a tool on Ubuntu that would monitor my bandwidth usage and show me daily/weekly/monthly download totals.

Requirements: It should work silently and unobtrusively in the background, and it should start automatically on reboot like a daemon. It would be nice to have it display graphical graphs. Also nice to have it display realtime stats (e.g. current rx and tx speeds).

There are quite a few tools that I found: bandwidthd, bwmon, bwbar etc (complete list here: http://www.ubuntugeek.com/bandwidth-monitoring-tools-for-ubuntu-users.html).

I didn't try all of them, but the most popular (or the one that appears at the top of the google search results for "bandwidth monitor tool ubuntu") is bandwidthd, and I just didn't have the patience to get it to work. There's no user manual, no real how-to and I gave up quickly on it.

vnstat saved the day. It exactly meets my requirements. You don't need to install it from the tarball either (in fact, that didn't work too well for me, I kept getting post-install configuration errors), and if you're on ubuntu, vnstat is in the standard apt repositories.

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources.

 

[08:17:59] ~ $ vnstat -d

eth1  /  daily
     day         rx      |     tx      |    total    |   avg. rate
 ------------------------+-------------+-------------+---------------
  08/24/11    134.81 MiB |    7.24 MiB |  142.05 MiB |   13.47 kbit/s
  08/25/11     84.21 MiB |    5.63 MiB |   89.85 MiB |   24.82 kbit/s
 ------------------------+-------------+-------------+---------------
 estimated       244 MiB |      14 MiB |     258 MiB |
[08:18:04] ~ $ vnstat -s
                  rx      /      tx      /     total    /   estimated
eth1:
 Aug '11    219.02 MiB  /   12.88 MiB  /  231.90 MiB  /  293.00 MiB
 yesterday    134.81 MiB  /    7.24 MiB  /  142.05 MiB
 today     84.21 MiB  /    5.63 MiB  /   89.85 MiB  /     258 MiB

 

network monitoring tools on linux

netactview: a network activity viewer for linux

I was trying to find a linux equivalent for "diamondcs port explorer"/tcpview/cports. It was surprisingly hard to get this information: took well over an hour, but I finally got exactly what I needed: netactview

netactview dynamically updates what ports are in use by what application, just like tcpview/cports/etc

It's available here:
http://netactview.sourceforge.net/download.html



iftop: display bandwidth usage on an interface

iftop needs libpcap and libcurses, and helps identify which connection is sucking out all your bandwidth. Quoting the home page, "Handy for answering the question "why is our ADSL link so slow?"

iftop can also show source and destination ports in each network connection (press ? for the help screen, just like you would in top).

iftop home page

linux: which process is listening on port X?

Discovered a new tool, ss, to view "socket statistics. From the man page:

Name
ss - another utility to investigate sockets

Synopsis
ss [options] [ FILTER ]

Description
ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP information than state than other tools.

[root@g2aqa3br1.qai ~]# ss -t
State      Recv-Q Send-Q      Local Address:Port          Peer Address:Port
ESTAB      0      0               127.0.0.1:56227            127.0.0.1:6802
ESTAB      0      0               127.0.0.1:56228            127.0.0.1:6802
ESTAB      0      0            172.29.8.131:38140          10.230.6.27:ldaps
ESTAB      0      0            172.29.8.131:38142          10.230.6.27:ldaps

reference: http://linux.die.net/man/8/ss

 

[root@g2aqa3br1.qai ~]# netstat -plunt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 ::ffff:172.29.8.131:1098    :::*                        LISTEN      18572/java
tcp        0      0 :::1099                     :::*                        LISTEN      18572/java
tcp        0      0 :::80                       :::*                        LISTEN      26695/httpd
tcp        0      0 :::22                       :::*                        LISTEN      7327/sshd
tcp        0      0 :::443                      :::*                        LISTEN      26695/httpd
udp     2616      0 0.0.0.0:514                 0.0.0.0:*                               6898/syslogd
[root@g2aqa3br1.qai ~]# ps 26695
PID TTY      STAT   TIME COMMAND
26695 ?        SNs    0:00 /opt/ec/apache2/bin/httpd -d /opt/ec/apache2 
-f /opt/ec/broker/conf/httpd.conf -k start -DSSL

reference:
http://www.cyberciti.biz/faq/find-out-which-service-listening-specific-port

windows: powertools that replace the plain old netstat command

• currports: powerful, easy-to-use and free! Can filter processes. A perfect replacement for port explorer.
  
• tcpview (sysinternals)
  
• procmon (sysinternals)
  
• port explorer (Trialware, old favorite. Development has long stopped since the parent company seems to be dead. Also redundant now, thanks to the above free options)

network basics: how NAT works

source: http://en.wikipedia.org/wiki/Port_address_translation
 

In A Nutshell : Example

• A host at private IP address 192.168.0.2 on the private network may ask for a connection to a remote host on the public network.
• The initial packet has the address 192.168.0.2:15345.
• The PAT device (which we assume has a public IP of 1.2.3.4) may arbitrarily translate this source address:port pair to 1.2.3.4:16529, making an entry in its internal table that port 16529 being used for a connection by 192.168.0.2 on the private network, with port 15345.
• When a packet is received from the public network by the PAT device for address 1.2.3.4:16529 the packet is forwarded to 192.168.0.2:15345.

Port Address Translation (PAT)

Port Address Translation (PAT) is a feature of a network device that translates TCP or UDP communications made between hosts on a private network and hosts on a public network. It allows a single public IP address to be used by many hosts on a private network, which is usually a Local Area Network or LAN.
A PAT device transparently modifies IP packets as they pass through it. The modifications make all the packets which it sends to the public network from the multiple hosts on the private network appear to originate from a single host, (the PAT device) on the public network.

Translation of the Endpoint

With PAT, all communication sent to external hosts actually contain the external IP address and port information of the PAT device instead of internal host IPs or port numbers.

• When a computer on the private (internal) network sends a packet to the external network, the PAT device replaces the internal IP address in the source field of the packet header (sender's address) with the external IP address of the PAT device. It then assigns the connection a port number from a pool of available ports, inserting this port number in the source port field (much like the post office box number), and forwards the packet to the external network. The PAT device then makes an entry in a translation table containing the internal IP address, original source port, and the translated source port. Subsequent packets from the same connection are translated to the same port number.

• The computer receiving a packet that has undergone PAT establishes a connection to the port and IP address specified in the altered packet, oblivious to the fact that the supplied address is being translated (analogous to using a post office box number).

• A packet coming from the external network is mapped to a corresponding internal IP address and port number from the translation table, replacing the external IP address and port number in the incoming packet header (similar to the translation from post office box number to street address). The packet is then forwarded over the inside network. Otherwise, if the destination port number of the incoming packet is not found in the translation table, the packet is dropped or rejected because the PAT device doesn't know where to send it.

PAT will only translate IP addresses and ports of its internal hosts, hiding the true endpoint of an internal host on a private network.

perl: simple socket programming

Here's a telnet kinda program in perl (i.e. a generic TCP client):

#!/usr/bin/perl
use IO::Socket;

my $dest = shift;
my $port = shift;
my $message;
my $line;

my $sock = IO::Socket::INET -> new ( PeerAddr => $dest, PeerPort => $port, Proto => "tcp" ) or die "Could not establish TCP connection: $!";

$sock->autoflush(1);

while (1)
{
    $message = <stdin>;
    print $sock $message;

    while ($line = <$sock>)
    {
        print $line;
    }
}

close $sock;

smtp: sending an email from the telnet prompt

This mail relay must accept SMTP connection from your host and must accept relaying. To check if the mail relay is working try



telnet mailrelay.domain 25
.... answer from mail relay .....

helo 
mail from: root@
rcpt to: @

data

mail test from unix
.



mail server should answer something like mail sent. If this work you can try with a normal mail client like



mailx -s "subject" @
mail test from unix
.



To check if this has work look at /var/adm/syslog/mail.log

you should see a couple of lines stating the mail has been accepted locally and sent to the relay and accepted.

linux: loading modules causes problems in iptables

You may run into a few problems with loading modules. For example, you could get errors claiming that there is no module by such a name and so on. This may, for example look like the following.

insmod: iptable_filter: no module by that name found

This is no reason for concern yet. This or these modules may possibly have been statically compiled into your kernel. This is the first thing you should look at when trying to solve this problem. The simplest way to see if these modules have been loaded already or if they are statically compiled into the kernel, is to simply try and run a command that uses the specific functionality. In the above case, we could not load the filter table. If this functionality is not there, we should be unable to use the filter table at all. To check if the filter table is there, we do the following.

iptables -t filter -L
   

This should either output all of the chains in the filter table properly, or it should fail. If everything is o.k., then it should look something like this depending on if you have rules inserted or not.

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
    

If you do not have the filter table loaded, you would get an error that looks something like this instead.

iptables v1.2.5: can't initialize iptables table `filter': Table \
     does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
    

This is a bit more serious since it points out that first of all, we do not have the functionality compiled into the kernel, and second, that the module is not possible to find in our normal module paths. This may either mean that:

• you have forgotten to install your modules,
• you have forgotten to run depmod -a to update your module databases or
• you have not compiled the functionality as either module or statically into kernel.

There may of course be other reasons for the module not to be loaded, but these are the main reasons. Most of these problems are easily solved.

• The first problem would simply be solved by running make modules_install in the kernel source directory (if the source has already been compiled and the modules have already been built).
• The second problem is solved by simply running depmod -a once and see if it works afterward.
• The third problem is a bit out of the league for this explanation, and you are more or less left to your own wits here. You will most probably find more information about this on the Linux Documentation Project homepage.

Source

netperf design

Netperf Always Uses Two Separate Connections

One connection for control, one connection for data.

When you execute netperf, the first thing that will happen is the establishment of a control connection to the remote system. This connection will be used to pass test configuration information and results to and from the remote system. Regardless of the type of test being run, the control connection will be a TCP connection using BSD sockets.

Once the control connection is up and the configuration information has been passed, a separate connection will be opened for the measurement itself using the APIs and protocols appropriate for the test. The test will be performed, and the results will be displayed.

Netperf places no traffic on the control connection while a test is in progress. Certain TCP options, such as SO_KEEPALIVE, if set as your system's default, may put packets out on the control connection.

Source

networking: tso (tcp segmentation offload) overview

TSO, a feature found in the OS, improves performance by offloading packet formation from the host processor to the Ethernet network controller.



TSO does not require special end-to-end enablement because it uses IEEE® standards-based 1500-byte frames. TSO allows the host to send 64 KB blocks of data to the network adapter, instead of 1518-byte packets. The network adapter's controller parses the 64 KB block into standard Ethernet packets, thus reducing the host processor utilization. By increasing network throughput and reducing CPU utilization, TSO increases network efficiencies.







IT managers considering Linux can benefit from choosing a server with a network connection that includes support for the new TCP Segmentation Offload (TSO) technology (also known as Large Send Offload [LSO] in Microsoft operating systems). TSO, originally designed for the Microsoft Windows® 2000 OS, is now also available for Windows XP and Windows .NET Server 2003. The performance gains offered by TSO were so substantial in the Microsoft operating systems that Intel took advantage of TSO in the Linux environment. Today, the Intel 82545EM, 82546EB, and 82540EM Gigabit Ethernet controllers support TSO. Intel continues to work closely with Linux kernel developers to make TSO a robust Linux offering.



Jumbo frames (frames larger than the standard 1518 bytes) is another technology that improves performance by reducing the amount of processing that the operating system must complete (see Figure 3). When a large block of data is requested-64 KB or more-the OS must break the block into packet-sized pieces, and then further process these pieces into packets that can ultimately be driven out onto the network.











When Jumbo frames is enabled on the network, packets going out on the network are simply oversized and the OS has fewer packets to create and transmit. However, when block sizes are smaller, Jumbo frames performance begins to diminish as latencies are introduced. Both TSO and Jumbo frames achieve performance gains. However, there is a disadvantage with Jumbo frames. With Jumbo frames, every node on the network and all of the switching infrastructure in between must be specifically enabled.





Source: http://www.dell.com/content/topics/global.aspx/power/en/

networking: vlans, vlan ids, vlan trunks

Networking FAQ: VLAN
Wikipedia

VLANs:
are independent logical LANs within the same physical network. They help in reducing the broadcast domain and aids in network administration by separating logical segments of a LAN (like company departments) that should not exchange data using a LAN (they still can exchange data by routing).

VLANs use medium to high range switches that enable software partitioning of the available ports, based on certain criteria. This set of ports is called a Virtual LAN or is abbreviated to VLAN. As you can imagine, the switch fabric could forward the Ethernet frames to the ports belonging to the same VLAN, while it would prevent any communication among distinct Virtual LANs.

VLAN Trunks:
When a single switch is not sufficient for a company, but the LAN extends over a set of them, the need arises to create Virtual LANs on each and enable communication between them. The first solution could be to use a port dedicated to the uplink for each VLAN. This would however lead to waste in terms of ports and cables; if the Virtual LANs common to two switches are n you must use n uplink cables.

A better solution is to create a trunk or trunking: in other words, both switches are attributed a common port (trunk port) to all the VLANs that need to be transported. Such trunks must run between these "tagged ports" of VLAN-aware devices, so they are often switch-to-switch or switch-to-router links rather than links to hosts.The switches tag each packet outbound of the trunk with a VLAN ID and each packet entering via trunking is forwarded on the right VLAN based on the VLAN ID. It is obvious that the two switches must use the same trunking protocol to communicate correctly via the trunk. There are different types of these protocols, which are often proprietary, and this could lead to inter-operational problems among different brands of switch that use the Virtual LANs. Yet, the most used trunking protocol is IEEE 802.1Q. The latter, for each Ethernet frame exiting the trunk configured port, adds 4 bytes and only 12 bits of which are used to identify the VLAN. The VLAN ID is therefore between 1 and 4094, considering 0 and 4095 are reserved values.