How to write ABAP program to Merge multiple files in one file in Application server

 In some scenarios, it may happen that business has loaded multiple files in a period for sales for example. All these files should be merged into one consolidation file and loaded once to target instead of loading each file separately.

Also the consolidation file will have a standard name which the infopackage can recognize and pick up rather than changing the infopackage to load multiple files with different names.

Sample Scenario:

Client XYZ loads sales files in the application server. These files are loaded by many sales persons belonging to diverse sales areas. The format of the file is same but the names may differ like:

sales_actual_02012017_California.csv

sales_actual_02012017_Texas.csv

sales_actual_02032017_Denver.csv

All these files should be loaded to North America cube target at the end of Feb month.

Here we can use the ABAP program in the process chain to consolidate above three files into one consolidation file and load it to the north america cube.

Below is the sample code to help you in creating the program in SE38:

REPORT zsample_consolidation.
" ---------------------------------------------"
*------Description-------------------------------*
*This program create consolidation file based on the file    *
*selected using program variant with proper naming convention *
*in the the XYZ Reports file directory
*---------------------------------------------*
*AUTHOR            : AK Verma
*Date              : 17 FEB 2017
*---------Changes-----------------------------*
* 1) Done by: 
* 2) Date: 
* 3) Change description: 
*------------------------------------------------*
**Declaration
TYPES: BEGIN OF itab1 ,
rec(1000) TYPE c,
END OF itab1.
DATA: itab TYPE TABLE OF itab1,
wa TYPE  itab1.
DATA : f_fname TYPE localfile . " Final file name.
DATA: p_file TYPE localfile.  "  To capture all file name
DATA: ifile TYPE TABLE OF  salfldir WITH HEADER LINE.
DATA: p_path TYPE salfile-longname." VALUE '/usr/sap/DEV/XYZ/Reports'.

PARAMETERS: p_filetp TYPE C LENGTH 3.

IF sy-sysid EQ 'DEV'.
p_path  = '/usr/sap/DEV/XYZ/Reports'.
ELSEIF sy-sysid EQ 'QA'.
p_path  = '/usr/sap/QA/XYZ/Reports'.
ELSEIF sy-sysid EQ 'PROD'.
p_path  = '/usr/sap/PROD/XYZ/Reports'.
ENDIF.

* To get all file in application file folder
CALL FUNCTION 'RZL_READ_DIR_LOCAL'
  EXPORTING
name           = p_path
  TABLES
file_tbl       = ifile
  EXCEPTIONS
argument_error = 1
not_found      = 2
OTHERS         = 3.

"Delete any old consolidation file in the application folder
DELETE ifile WHERE name CS 'Consolidated'.

" If file type Sales then delete Finance and HR files
IF p_filetp = 'Sls'.
  DELETE ifile WHERE name CS 'Finance'.
DELETE ifile WHERE name NS 'Sales'.
DELETE ifile WHERE name CS 'HR'.
ENDIF.

IF p_filetp = 'Fin'.
 DELETE ifile WHERE name CS 'Sales'.
DELETE ifile WHERE name NS 'Finance'.
DELETE ifile WHERE name CS 'HR'.
ENDIF.

IF p_filetp = 'HR'.
 DELETE ifile WHERE name CS 'Finance'.
DELETE ifile WHERE name NS 'HR'.
DELETE ifile WHERE name CS 'Sales'.
ENDIF.

"Ifile contains all file names in application folder
LOOP AT ifile.
IF ifile-name CS '.csv'.
SKIP.
ELSEIF ifile-name CS 'sales_actual'.   

"File name should start with sales_ actual else the file will be deleted
SKIP.
ELSEIF ifile-name CS '.xml'.
SKIP.
ELSE.
DELETE ifile.
ENDIF.
ENDLOOP.

DELETE ifile WHERE name NS 'sales_actual'.

LOOP AT ifile.
CONCATENATE p_path '/' ifile-name INTO p_file.

CLEAR wa.
OPEN DATASET p_file FOR INPUT IN LEGACY TEXT MODE." ENCODING DEFAULT.
  IF sy-subrc = 0.
DO.
READ DATASET p_file INTO wa.
 IF sy-subrc <> 0.
 EXIT.
ENDIF.
IF SY-INDEX = 1.
CONTINUE.
  ENDIF.
wa-rec = wa.
APPEND wa TO itab.
  ENDDO.
    ELSE.
MESSAGE 'Error in opening the file' TYPE 'E'.
EXIT.
  ENDIF.
CLOSE DATASET p_file.
IF sy-subrc NE 0.
MESSAGE  'Error closing the File' TYPE 'E'.
EXIT.
ELSEIF sy-subrc IS INITIAL.
  WRITE : 'File saved in the location' , p_file.
  ENDIF.

  ENDLOOP.
**  ------------------------------------------------<

CHECK itab[] is not INITIAL.

if p_filetp eq 'Sls'.
CONCATENATE p_path '/' 'Sales_Consolidated_' sy-datum '.csv' INTO f_fname.
ENDIF.

if p_filetp eq 'Fin'.
CONCATENATE p_path '/' 'Finance_Consolidated_' sy-datum '.csv' INTO f_fname.
ENDIF.

if p_filetp eq 'HR'.
CONCATENATE p_path '/' 'HR_Consolidated_' sy-datum '.csv' INTO f_fname.
ENDIF.

OPEN DATASET f_fname FOR OUTPUT IN TEXT MODE ENCODING DEFAULT.
 IF  sy-subrc  = 0.
  LOOP AT itab INTO wa .
 IF sy-subrc = 0.
TRANSFER wa-rec TO f_fname.
 ENDIF.
ENDLOOP.
ELSE.
MESSAGE 'Error in opening the file' TYPE 'E'.
EXIT.
      ENDIF.
CLOSE DATASET f_fname.
  IF sy-subrc NE 0.
 MESSAGE  'Error closing the File' TYPE 'E'.
EXIT.
ELSEIF sy-subrc IS INITIAL.
WRITE : 'File saved in the location' , p_file.
ENDIF.

 

Save, Check and activate the program. Create program variants with values "Sls", "Fin" and "HR".

In Sales process chain, add the program in the beginning with variant value as "Sls".

Add the infopackage step after that and in the extraction tab, select "Load text type file from application server" and write a program to recognize the consolidation file and load it.