Shell script for file handling by Date range

Introduction:

All the files exist in UNIX have got three timestamps associated with them which capture the file creation time, access time and modification time. If we need to identify a set of files fall under a date range, and perform specific actions on the selected files, we can make use of any of these timestamps.

The script files_by_date_range.sh can be used to list, copy, move and concatenate a set of files whose file modification timestamp fall under the two date parameters passed while executing the script.

This script can be reused by modifying the filenames and paths specific for different applications or environments. Also the scripts can be called from applications like Informatica where the date parameters can be passed dynamically.

Usage:

Open the script files_by_date_range.txt (in notepad/textpad).

• Modify the source/Target file paths and file name (Which has to be copied/moved etc) as per requirement.
• Comment (add a ‘#’ before) or remove the functionalities (list/copy/move/concat) from the script which are not required.
• Save file with .sh extension (Choose File type as “All files”)
• Ftp the file to the appropriate directory in UNIX box.
• Give execute permission for the script. (chmod 755 files_by_date_range.sh)

Note: 755 for -rwxr-xr-x , 777 for rwxrwxrwx

Execute the script as below:

files_by_date_range.sh YYYYMMDDHHMM YYYYMMDDHHMM

Note: First date parameter for begin data and second parameter is for end date.

Logic Description:

The script takes two date parameters of format YYYYMMDDHHMM.

This shell scripts creates two reference date files ‘beg_date’ and ‘end_date’ with

File-modified-dates as passed parameters. Then the script finds the files modified in between the date ranges by comparing each file with the reference date files (‘File- modified-date’> beg_date & ‘File-modified-date’<end_date). The identified files will undergo the corresponding action associated with the ‘exec’ command in the script.

SAMPLE SCENARIOS FOR USING THIS SHELL SCRIPT

1) Archival of files for a date range

Suppose we need to archive a set of files from one directory to another based on date range (From 2009 Jan 1^st to 2009 Jul 31^st), comment the list/move/concatenate steps in the script and execute the script as:

files_by_date_range.sh 200901010000 200907315959

2) copy/move of Files for a date range

Comment the list/concatenate steps in the script and then execute as above.

3) File Consolidation based on date range

Suppose we need to consolidate files for a month/year into a single file (e.g for month of Jul 2009), execute the script as:

files_by_date_range.sh 200907010000 200907315959

4) Automated script execution.

The script can be called from application like Informatica where the date parameters can be passed dynamically. These date parameters can be prepared according to the business requirements and can be written into parameter files (Same Informatica job can be used for Daily/Monthly/Yearly files consolidation depends upon the parameter file values).

Conclusion:

The shell scripts can be modified according to your requirements. Also the shell script can be scheduled from crontab or from other applications (e.g. Informatica)