Python | Write data of multiple files to main file

_files | File handling | Python functions

Michael Zippo 18.07.2021

The source directory contains n files, and the structure is the same for all files. The purpose of this code is — read all files one at a time, and then append the output to a single master file that is structured similar to the original files.

Taking three input files named emp_1.txt as an example, emp_2.txt , emp_3.txt , the output will contain data from all input files.

  Input:         Output:

Method # 1: Using the os module

import os

# list of files in the directory

lis = os.listdir ( ’D: python’

’ data_files data_files’ )

print (lis)

tgt = os.listdir ( ’ D: python’

’ data_files target_file’ )

file_dir = ’D: python data_files data_files ’

out_file = r ’D: python data_files target_file master.txt’

ct = 0

print ( ’target file:’ , tgt)

try :

# check if the fa yl

# if yes, delete the file

# otherwise the data will be added to the existing file

if len (tgt)" 0 :

os.remove ( ’ D: python’

’ data_files target_file master.txt’ )

open (tgt, ’a’ ). close ()

else :

# create empty file

open ( tgt, ’a’ ). close ()

except :

head = open ( ’D: python’

’ data_files target_file master.txt’ , ’a +’ )

line = ’empno, ename , sal’

# write title to output

print (head, line)

head.close ()

# below the loop to write data to the output file

for line1 in lis:

f_dir = file_dir + ’’ + line1

# open files in read mode < / p>

in_file = open (f_dir, ’ r + ’ )

# open output in append mode

w = open (out_file, ’a +’ )

d = in_file.readline ()

d = in_file.readlines ()

w.write ( "" )

for line2 in d:

print (line2)

w.write (line2)

ct = ct + 1

w.close ()

Output:

Method # 2: Using pandas

import pandas as pd

# pd.read_csv creates data frames

df1 = pd.read_csv ( ’ D: pythondata_filesdata_filesemp_1.txt’ )

df2 = pd.read_csv ( ’D: pythondata_filesdata_filesemp_2.txt’ )

df3 = pd.read_csv ( ’ D: pythondata_filesdata_filesemp_3.txt’ )

frames = [df1, df2, df3]

# concat function concatenates frames

result = pd.concat ( frames)

# to_csv function writes output to file

result.to_csv ( ’D: python data_files’

’ target_file master.txt’ , encoding = ’utf-8’ , index = False )

Exit :