python: how to fix 'fatal error: Python.h: No such file or directory'

Was trying to install PyYAML. Got the latest tarball from the PyYAML website, and ran the usual:

sudo python setup.py install

which resulted in:

running install
running build
running buildpy
creating build
creating build/lib.linux-i686-2.6
creating build/lib.linux-i686-2.6/yaml
copying lib/yaml/constructor.py -> build/lib.linux-i686-2.6/yaml
.
.
.
running buildext
creating build/temp.linux-i686-2.6
checking if libyaml is compilable
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c build/temp.linux-i686-2.6/checklibyaml.c -o build/temp.linux-i686-2.6/checklibyaml.o
checking if libyaml is linkable
gcc -pthread build/temp.linux-i686-2.6/checklibyaml.o -lyaml -o build/temp.linux-i686-2.6/checklibyaml
building 'yaml' extension
creating build/temp.linux-i686-2.6/ext
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c ext/yaml.c -o build/temp.linux-i686-2.6/ext/yaml.o
ext/yaml.c:4: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1


Google quickly yielded the solution. The missing Python.h was in the python-dev package. Since I was running python2.6.6, I simply needed to do:

sudo apt-get install python2.6-dev

after which compilation and installation was successful:

running install
running build
running buildpy
running buildext
building '_yaml' extension
.
.
.
running installegginfo
Writing /usr/local/lib/python2.6/dist-packages/PyYAML-3.10.egg-info


git: how to ignore certain file types globally

I wanted to tell git not to ever include .pyc files in any project, anywhere on my system, instead of having to specify a separate .gitignore file in every python project directory. A simple google search solved the problem; here's one way to do this:

Create a ~/.gitignore file in your home directory:

.DS_Store
*.pyc

Then run:
git config --global core.excludesfile ~/.gitignore

That's it!

 

thanks to [source]

linux: vnstat rocks!

vnstat is a superb tool to monitor monthly bandwidth usage.

I get notified by my ISP when I hit my monthly download quota (25 GB) that my speed will now be reduced to 256 kbps. Sometimes I'm left wondering how the heck I hit 25 gigs of downloading. So I started looking for a tool on Ubuntu that would monitor my bandwidth usage and show me daily/weekly/monthly download totals.

Requirements: It should work silently and unobtrusively in the background, and it should start automatically on reboot like a daemon. It would be nice to have it display graphical graphs. Also nice to have it display realtime stats (e.g. current rx and tx speeds).

There are quite a few tools that I found: bandwidthd, bwmon, bwbar etc (complete list here: http://www.ubuntugeek.com/bandwidth-monitoring-tools-for-ubuntu-users.html).

I didn't try all of them, but the most popular (or the one that appears at the top of the google search results for "bandwidth monitor tool ubuntu") is bandwidthd, and I just didn't have the patience to get it to work. There's no user manual, no real how-to and I gave up quickly on it.

vnstat saved the day. It exactly meets my requirements. You don't need to install it from the tarball either (in fact, that didn't work too well for me, I kept getting post-install configuration errors), and if you're on ubuntu, vnstat is in the standard apt repositories.

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources.

 

[08:17:59] ~ $ vnstat -d

eth1  /  daily
     day         rx      |     tx      |    total    |   avg. rate
 ------------------------+-------------+-------------+---------------
  08/24/11    134.81 MiB |    7.24 MiB |  142.05 MiB |   13.47 kbit/s
  08/25/11     84.21 MiB |    5.63 MiB |   89.85 MiB |   24.82 kbit/s
 ------------------------+-------------+-------------+---------------
 estimated       244 MiB |      14 MiB |     258 MiB |
[08:18:04] ~ $ vnstat -s
                  rx      /      tx      /     total    /   estimated
eth1:
 Aug '11    219.02 MiB  /   12.88 MiB  /  231.90 MiB  /  293.00 MiB
 yesterday    134.81 MiB  /    7.24 MiB  /  142.05 MiB
 today     84.21 MiB  /    5.63 MiB  /   89.85 MiB  /     258 MiB

 

python interview questions - part 1

1. Q:What is PYTHONPATH?
   A: Python uses the directories listed in PYTHONPATH to find a module.
2. Q: How can I see the PYTHONPATH on my system?
   A: Several ways:          

    

   • From the shell prompt: echo $PYTHONPATH
   • Programmatically in python itself:
     

     >>> import sys
     >>> sys.path
     ['', '', '', '/usr/lib/python2.6/dist-packages/pygame/tests', '/usr/lib/python2.6/dist-packages/pygame', '', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.6/gtk-2.0']
     

3. Q: How can I add a directory to the PYTHONPATH?
   A: Programmatically, in python itself:          

    

   #!/usr/bin/python
   
   import sys
   sys.path.append('path/to/my/modules/')
   
   # now import a module/.py file that resides in "path/to/my/modules"
   import my_module
   

4. Q: What are some standard python modules?
   A: Take a look at the Python Standard Library Reference Manual: http://docs.python.org/library/index.html
   Some standard modules are: os, sys, string, re, socket, datetime, math, commands [a useful wrapper around ps.popen() for running unix commands], httplib, urllib, json, ssl, thread, getopt, optparse, random
5. Q: What are packages in python?
   A: Packages are used to organize and group modules. Packages are just folders of modules with a special init.py file that indicates to Python that this folder is special because it contains Python modules. Packages are just a convenience to hierarchically organize modules.
6. Q: What is init.py used for?
   A: Source: http://effbot.org/pyfaq/what-is-init-py-used-for.htm
   It is used in marking a directory as a package directory. If you have the files:
   
mydir/mypackage/init.py
mydir/mypackage/mymodule.py

   and mydir is on the PYTHONPATH, you can import the code in mymodule.py via from mypackage import mymodule
   or via import mypackage.mymodule          

    

   If you remove the init.py file from the mypackage directory, Python will NOT look for modules inside the mypackage directory, and you will not be able to import the mymodule module.

   The init.py file is usually empty. Furthermore, init.py is the first file to be loaded in a package, so you can use it to execute initialization code that you want to run each time a module is loaded, or specify the submodules to be exported.

7. Q: What is the init function for?
   A: init is similar to a constructor for a class in Python. It is a way to specify default values for an object's data members when you create an object. These values are set in the class definition.          

    

   The init() method is called immediately after an instance of the class is created. It would be tempting — but technically incorrect — to call this the “constructor” of the class. It’s tempting, because it looks like a C++ constructor (by convention, the init() method is the first method defined for the class), acts like one (it’s the first piece of code executed in a newly created instance of the class), and even sounds like one. Incorrect, because the object has already been constructed by the time the init() method is called, and you already have a valid reference to the new instance of the class.

   The first argument of every class method, including the init() method, is always a reference to the current instance of the class, called "self".

   Example:

   class Point:
       def __init__ (self, arg1=0, arg2=0):
           self.x = arg1
           self.y = arg2
   
       def display(self):
           print "Point::display(): x=" + str(self.x) + " and y=" + str(self.y)
   
   p1 = Point(5, 8);
   p2 = Point();
   p1.display();
   p2.display();
   

   results in:
   
Point::display(): x=5 and y=8
Point::display(): x=0 and y=0


8. Q: What is main used for?
   A: source: http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm
   The if name == "main": ... trick exists in Python so that our Python files can act as either reusable modules, or as standalone programs.         

    

   Example:
   square.py :

   def square(x):
       return x*x
   
   if __name__ == '__main__' ;
       print "square.py | the square of 10 is: " , square(10)
   

   test.py:

   import square
   
   print "test.py | the square of 20 is: " , square(20)
   

   execution:

   $ python square.py
   square.py | the square of 10 is: 100
   $ python test.py
   test.py | the square of 20 is: 400
   

9. Q: What is a lambda function? Explain with an example. Typical usage?
   A: A lambda function is a shorthand for an anonymous function.
   The form is lambda <arguments>: <resultant-expression>
    
10. Q: What is the map function? Explain with an example. Typical usage?
    A: map applies a function to every element of a list and returns the resultant list.
    PNEMONIC: Hence it "maps" a function to every element of a list.
    Example:         

     

    >>> map ( lambda x: x*2, [1,2,3] )
    [2, 4, 6]
    

11. Q: What is the reduce function? 
    A: reduce takes as input a function and a list. The function must take exactly two arguments. reduce applies the function to the first two successive elements of the list from left to right, and then replaces the first two elements with the result of the function.
    PNEMONIC: Hence it "reduces" the number of elements in the list by one on each operation, until eventually only one element is left.Example:         

     

    >>> # this effectively sums the elements in the list by doing ((((1+2)+3)+4)+5)
    >>> reduce( lambda x,y : x+y, [1,2,3,4,5] )
    15
    

12. Q: What is the filter function?
    A: filter applies a function to every element of a list. Unlike, map, however, the function must return either True or False for each check. filter returns a list of all elements that pass this check.
    PNEMONIC: Hence it "filters" those elements that pass the test of the function.         

     

    Example:

    >>> # get those elements in the list that are divisible by 3
    >>> filter ( lambda x : x%3==0 , [1,2,3,4,5,6])
    [3,6]
    

13. Q: What is a list comprehension? Explain with an example. Typical usage?
14. Q: What is pass used for?
    A: The pass statement in Python is like a empty set of curly braces ({}) in Java or C. 
15. Q: How do you create an empty tuple?
    A: t = ()
16. Q: How do you create a tuple containing one element?
    A: t = (something, ) #note the comma at the end - this is the only way to specify a single-element tuple 
17. Q: How can you use the and-or trick safely (to simulate a C-style ternary expression)?
    A: <exprtotest> and [a] or [b])[0] 
18. Q: How can you use a much-nicer and more-readable ternary style expression in Python?
    A: Example: map(lambda x:x*2, range(0,20)) if (2 > 1) else ""
19. Q: How can you delete an element or a slice of elements from a list?
    A: Using the del statement
    
    >>> a = [-1, 1, 66.25, 333, 333, 1234.5]
    >>> del a[0]
    >>> a
    [1, 66.25, 333, 333, 1234.5]
    >>> del a[2:4]
    >>> a
    [1, 66.25, 1234.5]
    >>> del a[:]
    >>> a
    [] 

     

     

20. Q: What are ".pyc" files and when are they generated?
    A: Importing a module is a relatively costly affair, so Python does some tricks to make it faster. This .pyc file is useful when you import the module the next time from a different program - it will be much faster since a portion of the processing required in importing a module is already done. Also, these byte-compiled files are platform-independent.
    source
21. Q: What is the use of the inbuilt dir function?
    A: You can use the built-in dir function to list the identifiers that an object defines. When you supply a module name to the dir() function, it returns the list of the names defined in that module. When no argument is applied to it, it returns the list of names defined in the current module.
    source
22. Q: Explaing duck-typing wrt Python.
    A: Duck-typing can be simply explained as, "As long as you get the job done, we don't care who your parents are." 

     

    In other words, as long as an object can perform these actions:

    • quackLikeADuck()
    • walkLikeADuck()
    • swimLikeADuck()

    then that object is a Duck for Python. It doesn't matter which class the object is derived from (who its parents are). This greatly improves polymorphism and thus flexibility. Duck-typing avoids tests using type() or isinstance()

    Example: The canonical examples of duck typing in Python are file-like classes. You can use cStringIO to handle a string like a file, or GzipFile to handle a gzipped-object like a file. Similarly you can treats sockets like files too. However, sockets lack the 'tell()' method and so cannot be used everywhere that a file can be used. This shows the flexibility of duck typing: a file-like object needs to implement methods only that it is able to. Thus it can only be used in situations where it makes sense. Where the object is expected to support the 'tell()' method, and it doesn't, it can throw a "TellException" instead, which makes it easy to handle the unsupported-method-exception in the calling code and then move on. This is called as EAFP: "Easier to Ask Forgiveness than to ask for Permission" style programming.

network monitoring tools on linux

netactview: a network activity viewer for linux

I was trying to find a linux equivalent for "diamondcs port explorer"/tcpview/cports. It was surprisingly hard to get this information: took well over an hour, but I finally got exactly what I needed: netactview

netactview dynamically updates what ports are in use by what application, just like tcpview/cports/etc

It's available here:
http://netactview.sourceforge.net/download.html



iftop: display bandwidth usage on an interface

iftop needs libpcap and libcurses, and helps identify which connection is sucking out all your bandwidth. Quoting the home page, "Handy for answering the question "why is our ADSL link so slow?"

iftop can also show source and destination ports in each network connection (press ? for the help screen, just like you would in top).

iftop home page

Ubuntu!!!

A person with ubuntu is open and available to others, affirming of others, does not feel threatened that others are able and good, for he or she has a proper self-assurance that comes from knowing that he or she belongs in a greater whole and is diminished when others are humiliated or diminished, when others are tortured or oppressed.

-- Archbishop Desmond Tutu, 1999

phone screen questions

Phone screen technical questions

What is the difference between client-side and server-side scripting? Under what situation will you use which type?

Give me some examples of each type of scripting technology.

Describe the company to him, what kind of work is done here – brief intro to the project/product.

Instructions

1. Go through those questions with the candidate on the phone, and fill in the gathered answers in the space provided after each question
2. Pick one experience on their resume and ask the candidate a question or two to verify their participation and role in that experience.

General

1. What is the difference between a process and a thread?
   A process has its own address space; a thread is part of a process and uses the address space of its parent process.
   
   
2. What is the difference between a heap and a stack?
   A heap is memory used to store non-local data. A stack can be used to only store local data (ie, data that can be accessed only by the function that is executing, or any function that it calls).
   
   
3. What is an access violation?
   Illegal access to a memory location, or accessing a memory location that is not mapped.
   
   
4. How do you determine the code location of an access violation that occurs in the field?
   Configure the process to generate an image dump file and then load this in the debugger to view the call stack.

MultiThreaded Experience

1. What is the difference between a semaphore and a lock?
   A lock allows exactly one entity (thread, process) to access a protected resource; a semaphore allows a specific number (controlled number) of entities (processes, threads) to allow a resource.
   
   
2. What is a deadlock?
   A situation where there are two resources protected by two locks, A and B. One process (or thread) comes along, locks A, and wants to lock B. But another process (or thread) has locked B and is waiting to lock A. Neither one can proceed and this situation is called a deadlock.

C/C++

1. What are the two arguments to the function “main”?
   argc, and argv
   Count of arguments and an array (or vector) of string arguments.
   
   
2. What is a C++ class?
   It is a type of data structure that allows for the code and data associated with a logical object to be abstracted or packaged together.
   
   
3. What is a virtual function?
   A function of a class that can be overridden by derived classes.
   
   
4. What is the difference between a private member and a protected member of a C++ class?
   Private allows access by methods of the class (or by friends of the class).  Protected allows access by derived classes as well.
   
   
5. What is a Reference in C++?
   Like a pointer, it is the address of an object, but syntactically, it behaves like the object itself.
   
   
6. Why are templates important to C++?
   No root base class exists for creation of generic containers as in Java or other rooted OO languages, and templates allow enforcement of type safety.

COM/COM+

1. What are the three methods on the IUnknown interface?
   AddRef, Release, QueryInterface.
   
   
2. How do you create a new com object?
   CoCreateInstance
   
   
3. What are the different apartment threading models in COM?
   STA (single threaded), MTA (multi threaded), TNA (thread neutral) and Both/Any (allows STA, MTA or TNA to match creator apartment)
   
   
4. What language is used to define COM interfaces?
   IDL.

Java

1. What is a Java Class?
   It is a type of data structure that allows for the code and data associated with a logical object to be abstracted or packaged together.
   
   
2. What package do you need to import for simple TCP/IP communications?
   java.net.*
   
   
3. What is the Java VM?
   Java programs are compiled to bytecode that is then executed by the Java Virtual Machine.
   
   
4. How do you create a pointer in Java?
   You don’t!
   
   
5. What is the java facility that allows you to inspect classes at runtime called?
   reflection

Networking:

1. What is a DMZ?
   De-Militarized Zone, or the network zone between two firewalls.
   
   
2. What is NAT?
   Network Address Translation - or a device that maps one IP address to another.
   
   
3. Why do you need technology like NAT?
   To map internal IP addresses/Ports (which are typically in the non-routable range) to external IP addresses/Ports.
   
   
4. What is the purpose of calling bind when listening on a TCP socket?

To specify a local end point (IP address and port) to accept connections on

IIS/ASP:

1. How are ASP pages different from HTML?
   ASP pages contain server side scripting.
   
   
2. What is a CSS file used for?
   CSS = cascading style sheet, used to specify formatting separately from the HTML content.

3. How do you perform client side validation on a form element?
   Typically, associate a client side validation script with an event on the HTML element.   .NET supports some declarative client side validation.

SQL Server

1. What is a join?
   A SQL operation that produces a single (logical) table from two or more tables, by matching one column in one table with a column in the other table.
   
   
2. How are inner and outer joins different?
   An inner join is where the result contains rows that have a match in the column values in both tables.   An outer join allows data from one of the tables to be included even if the join column data is null.
   
   
3. What is normalization?
   The design process that eliminates repetitive data.
   
   
4. Assume you have a SQL table with 30 million rows and you want to fetch a few records.  How do you ensure the query returns quickly?
   Ensure that the where and order by criteria are supported with appropriate indexes.

XML / XSLT

1. What is a DOM
   Document Object Model is a parsed, in memory XML document.

2. What is XSLT
   Extensible Style Language Transformations.
   
   
3. What is XSLT useful for?
   For converting XML data to any other format.
   
   
4. What techniques are normally used to embed an arbitrary XML document inside another as a text entity?
   Normally either XML entity encoding (e.g. “&” -> “&&”) or enclosing the data in a CDATA section.