1. Python
  2. Python | Write data of multiple files to main file

Python | Write data of multiple files to main file



The source directory contains n files, and the structure is the same for all files. The purpose of this code is — read all files one at a time, and then append the output to a single master file that is structured similar to the original files.

Taking three input files named emp_1.txt as an example, emp_2.txt , emp_3.txt , the output will contain data from all input files. 

  Input:         Output:

Method # 1: Using the os module

import os

# list of files in the directory

lis = os.listdir ( `D: python`

` data_files data_files` )

print (lis)

tgt = os.listdir ( ` D: python`

` data_files target_file` )

  

file_dir = `D: python data_files data_files `

out_file = r `D: python data_files target_file master.txt`

ct = 0

 

print ( `target file:` , tgt)

try :

# check if the fa yl

# if yes, delete the file

# otherwise the data will be added to the existing file

  if len (tgt) & gt; 0 :

os.remove ( ` D: python`

` data_files target_file master.txt` )

  

open (tgt, `a` ). close ()

  else :

# create empty file

open ( tgt, `a` ). close ()

except :

  head = open ( `D: python`

  ` data_files target_file master.txt` , `a +` )

 

line = `empno, ename , sal`

# write title to output

print (head, line)

head.close ()

  # below the loop to write data to the output file

for line1 in lis:

f_dir = file_dir + `` + line1

  # open files in read mode < / p>

in_file = open (f_dir, ` r + ` )

 

# open output in append mode

w = open (out_file, `a +` )

d = in_file.readline ()

  d = in_file.readlines ()

  w.write ( "" )

for line2 in d:

print (line2)

w.write (line2)

 

ct = ct + 1  

w.close () 

Output:

Method # 2: Using pandas

import pandas as pd

# pd.read_csv creates data frames

df1 = pd.read_csv ( ` D: pythondata_filesdata_filesemp_1.txt` )

df2 = pd.read_csv ( `D: pythondata_filesdata_filesemp_2.txt` )

df3 = pd.read_csv ( ` D: pythondata_filesdata_filesemp_3.txt` )

 

frames = [df1, df2, df3]

  
# concat function concatenates frames

result = pd.concat ( frames)

# to_csv function writes output to file

result.to_csv ( `D: python data_files`

  ` target_file master.txt` , encoding = `utf-8` , index = False )

Exit :