python: how to fix 'fatal error: Python.h: No such file or directory'

Was trying to install PyYAML. Got the latest tarball from the PyYAML website, and ran the usual:

sudo python setup.py install

which resulted in:

running install
running build
running buildpy
creating build
creating build/lib.linux-i686-2.6
creating build/lib.linux-i686-2.6/yaml
copying lib/yaml/constructor.py -> build/lib.linux-i686-2.6/yaml
.
.
.
running buildext
creating build/temp.linux-i686-2.6
checking if libyaml is compilable
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c build/temp.linux-i686-2.6/checklibyaml.c -o build/temp.linux-i686-2.6/checklibyaml.o
checking if libyaml is linkable
gcc -pthread build/temp.linux-i686-2.6/checklibyaml.o -lyaml -o build/temp.linux-i686-2.6/checklibyaml
building 'yaml' extension
creating build/temp.linux-i686-2.6/ext
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.6 -c ext/yaml.c -o build/temp.linux-i686-2.6/ext/yaml.o
ext/yaml.c:4: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'gcc' failed with exit status 1


Google quickly yielded the solution. The missing Python.h was in the python-dev package. Since I was running python2.6.6, I simply needed to do:

sudo apt-get install python2.6-dev

after which compilation and installation was successful:

running install
running build
running buildpy
running buildext
building '_yaml' extension
.
.
.
running installegginfo
Writing /usr/local/lib/python2.6/dist-packages/PyYAML-3.10.egg-info


git: how to ignore certain file types globally

I wanted to tell git not to ever include .pyc files in any project, anywhere on my system, instead of having to specify a separate .gitignore file in every python project directory. A simple google search solved the problem; here's one way to do this:

Create a ~/.gitignore file in your home directory:

.DS_Store
*.pyc

Then run:
git config --global core.excludesfile ~/.gitignore

That's it!

 

thanks to [source]

linux: vnstat rocks!

vnstat is a superb tool to monitor monthly bandwidth usage.

I get notified by my ISP when I hit my monthly download quota (25 GB) that my speed will now be reduced to 256 kbps. Sometimes I'm left wondering how the heck I hit 25 gigs of downloading. So I started looking for a tool on Ubuntu that would monitor my bandwidth usage and show me daily/weekly/monthly download totals.

Requirements: It should work silently and unobtrusively in the background, and it should start automatically on reboot like a daemon. It would be nice to have it display graphical graphs. Also nice to have it display realtime stats (e.g. current rx and tx speeds).

There are quite a few tools that I found: bandwidthd, bwmon, bwbar etc (complete list here: http://www.ubuntugeek.com/bandwidth-monitoring-tools-for-ubuntu-users.html).

I didn't try all of them, but the most popular (or the one that appears at the top of the google search results for "bandwidth monitor tool ubuntu") is bandwidthd, and I just didn't have the patience to get it to work. There's no user manual, no real how-to and I gave up quickly on it.

vnstat saved the day. It exactly meets my requirements. You don't need to install it from the tarball either (in fact, that didn't work too well for me, I kept getting post-install configuration errors), and if you're on ubuntu, vnstat is in the standard apt repositories.

vnStat is a console-based network traffic monitor for Linux and BSD that keeps a log of network traffic for the selected interface(s). It uses the network interface statistics provided by the kernel as information source. This means that vnStat won't actually be sniffing any traffic and also ensures light use of system resources.

 

[08:17:59] ~ $ vnstat -d

eth1  /  daily
     day         rx      |     tx      |    total    |   avg. rate
 ------------------------+-------------+-------------+---------------
  08/24/11    134.81 MiB |    7.24 MiB |  142.05 MiB |   13.47 kbit/s
  08/25/11     84.21 MiB |    5.63 MiB |   89.85 MiB |   24.82 kbit/s
 ------------------------+-------------+-------------+---------------
 estimated       244 MiB |      14 MiB |     258 MiB |
[08:18:04] ~ $ vnstat -s
                  rx      /      tx      /     total    /   estimated
eth1:
 Aug '11    219.02 MiB  /   12.88 MiB  /  231.90 MiB  /  293.00 MiB
 yesterday    134.81 MiB  /    7.24 MiB  /  142.05 MiB
 today     84.21 MiB  /    5.63 MiB  /   89.85 MiB  /     258 MiB

 

python interview questions - part 1

1. Q:What is PYTHONPATH?
   A: Python uses the directories listed in PYTHONPATH to find a module.
2. Q: How can I see the PYTHONPATH on my system?
   A: Several ways:          

    

   • From the shell prompt: echo $PYTHONPATH
   • Programmatically in python itself:
     

     >>> import sys
     >>> sys.path
     ['', '', '', '/usr/lib/python2.6/dist-packages/pygame/tests', '/usr/lib/python2.6/dist-packages/pygame', '', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.6/gtk-2.0']
     

3. Q: How can I add a directory to the PYTHONPATH?
   A: Programmatically, in python itself:          

    

   #!/usr/bin/python
   
   import sys
   sys.path.append('path/to/my/modules/')
   
   # now import a module/.py file that resides in "path/to/my/modules"
   import my_module
   

4. Q: What are some standard python modules?
   A: Take a look at the Python Standard Library Reference Manual: http://docs.python.org/library/index.html
   Some standard modules are: os, sys, string, re, socket, datetime, math, commands [a useful wrapper around ps.popen() for running unix commands], httplib, urllib, json, ssl, thread, getopt, optparse, random
5. Q: What are packages in python?
   A: Packages are used to organize and group modules. Packages are just folders of modules with a special init.py file that indicates to Python that this folder is special because it contains Python modules. Packages are just a convenience to hierarchically organize modules.
6. Q: What is init.py used for?
   A: Source: http://effbot.org/pyfaq/what-is-init-py-used-for.htm
   It is used in marking a directory as a package directory. If you have the files:
   
mydir/mypackage/init.py
mydir/mypackage/mymodule.py

   and mydir is on the PYTHONPATH, you can import the code in mymodule.py via from mypackage import mymodule
   or via import mypackage.mymodule          

    

   If you remove the init.py file from the mypackage directory, Python will NOT look for modules inside the mypackage directory, and you will not be able to import the mymodule module.

   The init.py file is usually empty. Furthermore, init.py is the first file to be loaded in a package, so you can use it to execute initialization code that you want to run each time a module is loaded, or specify the submodules to be exported.

7. Q: What is the init function for?
   A: init is similar to a constructor for a class in Python. It is a way to specify default values for an object's data members when you create an object. These values are set in the class definition.          

    

   The init() method is called immediately after an instance of the class is created. It would be tempting — but technically incorrect — to call this the “constructor” of the class. It’s tempting, because it looks like a C++ constructor (by convention, the init() method is the first method defined for the class), acts like one (it’s the first piece of code executed in a newly created instance of the class), and even sounds like one. Incorrect, because the object has already been constructed by the time the init() method is called, and you already have a valid reference to the new instance of the class.

   The first argument of every class method, including the init() method, is always a reference to the current instance of the class, called "self".

   Example:

   class Point:
       def __init__ (self, arg1=0, arg2=0):
           self.x = arg1
           self.y = arg2
   
       def display(self):
           print "Point::display(): x=" + str(self.x) + " and y=" + str(self.y)
   
   p1 = Point(5, 8);
   p2 = Point();
   p1.display();
   p2.display();
   

   results in:
   
Point::display(): x=5 and y=8
Point::display(): x=0 and y=0


8. Q: What is main used for?
   A: source: http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm
   The if name == "main": ... trick exists in Python so that our Python files can act as either reusable modules, or as standalone programs.         

    

   Example:
   square.py :

   def square(x):
       return x*x
   
   if __name__ == '__main__' ;
       print "square.py | the square of 10 is: " , square(10)
   

   test.py:

   import square
   
   print "test.py | the square of 20 is: " , square(20)
   

   execution:

   $ python square.py
   square.py | the square of 10 is: 100
   $ python test.py
   test.py | the square of 20 is: 400
   

9. Q: What is a lambda function? Explain with an example. Typical usage?
   A: A lambda function is a shorthand for an anonymous function.
   The form is lambda <arguments>: <resultant-expression>
    
10. Q: What is the map function? Explain with an example. Typical usage?
    A: map applies a function to every element of a list and returns the resultant list.
    PNEMONIC: Hence it "maps" a function to every element of a list.
    Example:         

     

    >>> map ( lambda x: x*2, [1,2,3] )
    [2, 4, 6]
    

11. Q: What is the reduce function? 
    A: reduce takes as input a function and a list. The function must take exactly two arguments. reduce applies the function to the first two successive elements of the list from left to right, and then replaces the first two elements with the result of the function.
    PNEMONIC: Hence it "reduces" the number of elements in the list by one on each operation, until eventually only one element is left.Example:         

     

    >>> # this effectively sums the elements in the list by doing ((((1+2)+3)+4)+5)
    >>> reduce( lambda x,y : x+y, [1,2,3,4,5] )
    15
    

12. Q: What is the filter function?
    A: filter applies a function to every element of a list. Unlike, map, however, the function must return either True or False for each check. filter returns a list of all elements that pass this check.
    PNEMONIC: Hence it "filters" those elements that pass the test of the function.         

     

    Example:

    >>> # get those elements in the list that are divisible by 3
    >>> filter ( lambda x : x%3==0 , [1,2,3,4,5,6])
    [3,6]
    

13. Q: What is a list comprehension? Explain with an example. Typical usage?
14. Q: What is pass used for?
    A: The pass statement in Python is like a empty set of curly braces ({}) in Java or C. 
15. Q: How do you create an empty tuple?
    A: t = ()
16. Q: How do you create a tuple containing one element?
    A: t = (something, ) #note the comma at the end - this is the only way to specify a single-element tuple 
17. Q: How can you use the and-or trick safely (to simulate a C-style ternary expression)?
    A: <exprtotest> and [a] or [b])[0] 
18. Q: How can you use a much-nicer and more-readable ternary style expression in Python?
    A: Example: map(lambda x:x*2, range(0,20)) if (2 > 1) else ""
19. Q: How can you delete an element or a slice of elements from a list?
    A: Using the del statement
    
    >>> a = [-1, 1, 66.25, 333, 333, 1234.5]
    >>> del a[0]
    >>> a
    [1, 66.25, 333, 333, 1234.5]
    >>> del a[2:4]
    >>> a
    [1, 66.25, 1234.5]
    >>> del a[:]
    >>> a
    [] 

     

     

20. Q: What are ".pyc" files and when are they generated?
    A: Importing a module is a relatively costly affair, so Python does some tricks to make it faster. This .pyc file is useful when you import the module the next time from a different program - it will be much faster since a portion of the processing required in importing a module is already done. Also, these byte-compiled files are platform-independent.
    source
21. Q: What is the use of the inbuilt dir function?
    A: You can use the built-in dir function to list the identifiers that an object defines. When you supply a module name to the dir() function, it returns the list of the names defined in that module. When no argument is applied to it, it returns the list of names defined in the current module.
    source
22. Q: Explaing duck-typing wrt Python.
    A: Duck-typing can be simply explained as, "As long as you get the job done, we don't care who your parents are." 

     

    In other words, as long as an object can perform these actions:

    • quackLikeADuck()
    • walkLikeADuck()
    • swimLikeADuck()

    then that object is a Duck for Python. It doesn't matter which class the object is derived from (who its parents are). This greatly improves polymorphism and thus flexibility. Duck-typing avoids tests using type() or isinstance()

    Example: The canonical examples of duck typing in Python are file-like classes. You can use cStringIO to handle a string like a file, or GzipFile to handle a gzipped-object like a file. Similarly you can treats sockets like files too. However, sockets lack the 'tell()' method and so cannot be used everywhere that a file can be used. This shows the flexibility of duck typing: a file-like object needs to implement methods only that it is able to. Thus it can only be used in situations where it makes sense. Where the object is expected to support the 'tell()' method, and it doesn't, it can throw a "TellException" instead, which makes it easy to handle the unsupported-method-exception in the calling code and then move on. This is called as EAFP: "Easier to Ask Forgiveness than to ask for Permission" style programming.