Data can be stored either Table Format or File format.

There will be three kinds of files:
▹ Structured files –> E.g. : Flat files / Data files
▹ Semi Structured files –> E.g. : XML / Json / Html file
▹ Un Structured files –> E.g. : Audio , video , Image , Word , PDF and PPT files.

In Realtime we will use mostly Flat files or XML / Json files.

Flat Files:

▹ If the data is stored in the format of File, then it is called Data file or Flat file
▹ Flat Files can be stored under a Folder/directory.
▹ Folders/Directories can be stored under file server.
▹ Every flat file is having an extension. E.g. : .csv , .txt , .out , .dat
▹ Every flat file is having one delimiter (Column Separator).

There are two Types of flat files:
1. Fixed Width Flat Files – Each Column is Separated by Single or multiple Spaces
2. Delimited Flat Files. – Each Column is Separated by a character like comma or “|”

In Realtime we will use Delimited Flat files.

Below are the examples of most used Delimiters in Realtime:
▹ “,” (Comma Delimiter)
▹ “|” (Pipe Delimiter)
▹ Tab (Tab Delimiter)
▹ “;” (Semi colon Delimiter)
▹ “~” (Tild Delimiter)

In Realtime, we will Store the flat files under Secure agent server path as given below.

C:\Program Files\Informatica Cloud Secure Agent\apps\Data_Integration_Server

Working with flat Files in IDMC/ IICS:

Please Perform the below steps, if you want to work with flat files in IDMC / IICS:

1. Create Source Flat File Connection.
2. Create Target Flat File Connection.
3. Create a Mapping.
4. Configure Source.
5. Change the Data types of the Source File as per the source file data in mapping.
6. Configure Target.
7. Use different Transformations in the mapping.
8. Develop the mapping and use any transformations in between source and target as per the requirement.

1. Create source file Connection:

Login to IDMC/IICS, Go to Administrator page, Go to Connections, then create a new source connection as shown in below.

2. Create Target file Connection:

Login to IDMC/IICS, Go to Administrator page, Go to Connections, then create a new Target connection as shown in below.

3. Create a Mapping:

Go to Data Integration page, create a new mapping, then select the source , and provide the source file properties as shown in below.

4. Configure source:

Once you Select your Source File, you need to click on formatting options to change the delimiter of the file and other options as shown in the below.

5. Change source File datatypes:

When you configure the source file in a mapping, by default all the datatypes of the columns will be shown as string 255.

But you have to change these datatypes of the source file according to the source file data as shown in below.

6. Configure Target file in a mapping:

We can Select the Existing Target object (File) or create a new file at runtime as per the requirement as shown in the below.

7. Advanced Properties under Target file:

There are few advanced options under target file as mentioned below:

1. Append if Exists – Add the rows at end of the target flat file each time session runs.
2. Header options – Add the filed names at the first line of the target flat file.
3. Output file directory – Overrides the default directory mentioned in the target connection.
4. Output file name – Overrides the target file name mentioned in the static file name.

8. Develop the mapping and use any transformations in between source and target as per the requirement.