Concatenating text files in Python

We can use the built-in inputfile module to concatenate files while optionally applying filters and transformations to each line, without abusing nested with blocks and for loops.

It combines nicely with glob.glob(), to merge and transform files with a given name pattern:

import fileinput
import glob

def process(line):
    # do something with line
    return line

fnames = glob.glob('*.txt')
with fileinput.input(fnames) as f_in, open('out.txt', 'w+') as f_out:
    for line in f_in:
        f_out.write(process(line))

Note that the naive way of implementing the merge-transform operation would look something like:

with open('out.txt', 'w+') as f_out:
    for fname in fnames:
        with open(fname, 'r') as f_in:
            for line in f_in:
                process(line)

We observe the latter introduces at least an additional for loop, and a nested with block.