How to iterate over the file in python

You should learn about EAFP vs LBYL.

from sys import stdin, stdout
def main(infile=stdin, outfile=stdout):
    if isinstance(infile, basestring):
        infile=open(infile,'r')
    if isinstance(outfile, basestring):
        outfile=open(outfile,'w')
    for lineno, line in enumerate(infile, 1):
        line = line.strip()
         try:
             print >>outfile, int(line,16)
         except ValueError:
             return "Bad value at line %i: %r" % (lineno, line)

if __name__ == "__main__":
    from sys import argv, exit
    exit(main(*argv[1:]))

with open('test.txt', 'r') as inf, open('test1.txt', 'w') as outf:
    for line in inf:
        line = line.strip()
        if line:
            try:
                outf.write(str(int(line, 16)))
                outf.write('\n')
            except ValueError:
                print("Could not parse '{0}'".format(line))

The traceback indicates that probably you have an empty line at the end of the file. You can fix it like this:

f = open('test.txt','r')
g = open('test1.txt','w') 
while True:
    x = f.readline()
    x = x.rstrip()
    if not x: break
    print >> g, int(x, 16)

On the other hand it would be better to use for x in f instead of readline. Do not forget to close your files or better to use with that close them for you:

with open('test.txt','r') as f:
    with open('test1.txt','w') as g: 
        for x in f:
            x = x.rstrip()
            if not x: continue
            print >> g, int(x, 16)

Just use for x in f: ..., this gives you line after line, is much shorter and readable (partly because it automatically stops when the file ends) and also saves you the rstrip call because the trailing newline is already stipped.

The error is caused by the exit condition, which can never be true: Even if the file is exhausted, readline will return an empty string, not None. Also note that you could still run into trouble with empty lines, e.g. at the end of the file. Adding if line.strip() == "": continue makes the code ignore blank lines, which is propably a good thing anyway.