UNIX Commands

UNIX

What is the full form of UNIX?
UNiplexed Information Computing System (UNICS), also known as UNIX.

A UNIX operating system is a multitasking operating system that allows you to initiate more than one task from the same terminal so that one task.

LINUX is an advanced version of UNIX , It has several features similar to Unix, still have some key differences. And Linux is open-source software and can be used freely without any licensing fees.

Why UNIX is popular than Windows?

UNIX-based systems are inherently more secure than the Windows operating system.

Shell in UNIX: A shell is an environment in which we can run our Unix commands, programs, and shell scripts

Example:
▹ Bourne shell ( sh)
▹ Korn shell ( ksh)
▹ Bourne Again shell ( bash)
▹ C shell ( csh)

Tools Used for UNIX in Realtime: To work with UNIX, we have two free tools available. PUTTY & WINSCP

PUTTY: It is a UNIX CLI (Command Line Interface) tool which is used to run only Unix Commands.
WinSCP: It is a UNIX GUI (Graphical User Interface) tool which is used to access Files in Unix Operating system.

Why should I learn UNIX, when I am working as a Informatica Developer?

I should learn Unix due to the below reasons.

1. We have Command Task in Informatica, we will use UNIX/Windows commands in Command Task.
2. In Realtime, Informatica Server will be Installed under UNIX. So INFA_SHARED folder will be under unix server. To access INFA_SHARED Folder and also Source files, Target files and other files we should learn UNIX commands.
3. We can Schedule Informatica Jobs in UNIX by using Crontab.

How to Practice UNIX Commands without installing UNIX in your system?

Please watch the above👆🏻 Video to Practice UNIX Commands without installing UNIX in your system

Most used Unix Commands in Informatica: I have explained the Most used UNIX commands in Informatica in these two videos given blow👇🏻 this section.

1. $pwd – Gives the Present working directory in UNIX
2. mkdir – Create a directory in UNIX

Example:

mkdir dir1

Lets say you want to create more than one directory instead of invoking mkdir multiple(three) times-like.

Example:

mkdir dir2
mkdir dir2/dir3
mkdir dir2/dir3/dir4

3. rmdir: Remove a directory in UNIX
4. cd: Change Directory in UNIX

Example:
cd dir2

cd .. comeback from the directory

5. cal : to check calendar
6. date: Displays the system date and time.

Syntax: date [+format]

Example: Display the date in dd/mm/yy format date +%d/%m/%y 

7. Creating a file in Unix :

a) Using Cat Command
cat > abc.txt — Creates a file and press control d to exit.
Cat filename – to display the contents of file
Cat>>filename – to append file.

b) touch file2.txt — Creates an empty file name file2.txt
touch File1_name File2_name File3_name — Touch command can be used to create the multiple numbers of emplty files at the same time
cat filename : Display Content of a File
cat -b filename : Display the line numbers by using the -b

8. ls : Listing Directories and Files

All data in Unix is organized into files. All files are organized into directories. These directories are organized into a tree-like structure called the filesystem.

“ls” command to list out all the files or directories available in a directory.

‣ ls -l List with long format – show permissions
‣ ls -a List all files including hidden file starting with ‘.’ (Hidden Files)
‣ ls -R Recursive directory tree list
‣ ls -t sort by time & date
‣ ls -r list in reverse order
‣ ls -ls list with long format with file size
‣ ls -lrt

We use “*” to match 0 or more characters, a question mark “?” matches with a single character.

ls file* — Displays all the files, the names of which start with file
ls *.txt — display all the files ending with just “.txt”

9. whoami : Displays the user id of the currently logged-in user

10. who : Displays the list of users currently logged in
11. wc : Counting Words in a File

Example: wc file_name

12. wc filename1 filename2 filename3 — can give multiple files and get information about those files at a time.

‣ wc -l state.txt prints the number of lines present in a file
‣ wc -l state.txt capital.txt With more than one file name
‣ wc -w state.txt prints the number of words present in a file
‣ wc -c state.txt displays count of bytes present in a file
‣ wc -m state.txt displays count of characters from a file.
‣ wc -L demo_file to find the length of longest line in the file

13. cp — Copying Files
syntax: cp source_file destination_file
Example: cp filename1 filename2 : Copy the contents of “filename1” to “filename2”

14. mv –Renaming Files
syntax: mv old_file new_file
Example: mv filename newfile — will rename the existing file “filename” to “newfile”

15. rm — Deleting Files
Example: rm filename — completely remove the existing file “filename”
rm filename1 filename2 filename3 — can remove multiple files at a time4

16. chmod — (change mode) To change the file or the directory permissions

‣ “4” stands for “read”,
‣ “2” stands for “write”,
‣ “1” stands for “execute”, and
‣ “0” stands for “no permission.”
‣ To set the read and write permission for other users, execute the below command: chmod o+w *.txt
‣ To Remove thre write permission for other users, execute the below command : chmod o-w *.txt
‣ To set execute permission for all users :

chmod a+x File1

chmod 777 testfile

17. Sending Email:

mail [-s subject] [-c cc-addr] [-b bcc-addr] to-addr

18. Send a test message to admin@yahoo.com:

mail -s "Test Message" admin@yahoo.com

You can connect two commands together so that the output from one program becomes the input of the next program. Two or more commands connected in this way form a pipe (|)

19. ps — display currently running processes.

20. kill — kill the current process.

21. man: (Manual) – Interface for working with the online reference manuals. (HELP)
Syntax: man [-s section] item
Example:
Show manual page for the ‘cat’ command $ man cat

22. find: Used to search for files and directories as mentioned in the ‘expression’
Syntax: find [starting-point] [expression]
Example:
$ find — List all files found in the current directory and its hierarchy
$find . -name cust.dat — Find all the files whose name is cust.dat from all directories(.)
$ find /root/infa_shared/SrcFiles -name File1.dat — Find all the files under /root/infa_shared/SrcFiles directory with the name File1.dat
$find /root/infa_shared/SrcFiles -iname File1.dat — Find all the files whose name is file1.dat and contains both capital and small letters in /root/infa_shared/SrcFiles directory.
find / -type d -name dir10 — Find all directories whose name is dir10 in / (root) directory.
find . -type f -empty — Find all the Empty Files in all directories

23. du: (disk usage) Estimate disk usage is blocks
syntax: du [options] [file]
Example:
$ du – Show number of blocks occupied by files in the current directory
du -sh * – summary of directories (-s) in human-readable format (-h : Byte, Kilobyte, Megabyte, Gigabyte, Terabyte and Petabyte):
du -sk * — summary of directories (-s) in kilobytes (-k)

24. df: (disk free) Show number of free blocks for mounted file system
Syntax: df [options] [file]
Example:
$ df -l — Show number of free blocks in local file systems

Unix Filter Commands :
‣ grep: Find lines in stdin that match a pattern and print them to stdout (Standard output or Screen ) .
‣ sort: Sort the lines in stdin, and print the result to stdout.
‣ uniq: Read from stdin and print unique (that are different from the adjacent line) to stdout.
‣ cat: Read lines from stdin (and more files), and concatenate them to stdout.
‣ more: Read lines from stdin, and provide a paginated view to stdout.
‣ cut: Cut specified byte, character or field from each line of stdin and print to stdout.
‣ paste: Read lines from stdin (and more files), and paste them together line-by-line to stdout.
‣ head: Read the first few lines from stdin (and more files) and print them to stdout.
‣ tail: Read the last few lines from stdin (and more files) and print them to stdout.
‣ wc: Read from stdin, and print the number of newlines, words, and bytes to stdout.
‣ tr: Translate or delete characters read from stdin and print to stdout.

25. grep Command — search for string in the file. – it will show all the lines which contains the string. searches a file or files for lines that have a certain pattern .
g/re/p which means — Gobally search for a regular expression and print all lines containing it.
Example:
‣ grep “this” demo_file — Search for the given string in a single file
‣ grep “this” demo_* — Checking for the given string in multiple files.
‣ grep -i “this” demo_file — Case insensitive search using grep -i ’ (Both upper and lower case)
‣ grep -r “this” * — Searching in all files recursively using grep -r (look for the string “this” in all the files in the current directory and all it’s subdirectory.)
‣ grep -l this demo_* — Display only the file names which matches the given pattern using grep -l
‣ grep -n “this” demo_file — Show line number while displaying the output using grep -n
‣ grep “[a-e]” file1 — Match all lines that contain any of the letters ‘a’, ‘b’, ‘c’, ‘d’ or ‘e’.
‣ grep “[^aeiou]” file1 — Match all lines that do not contain a vowel
‣ grep “^hello” file1 — Match all lines that start with ‘hello’. E.g: “hello there”
‣ grep “done$” file1 — Match all lines that end with ‘done’. E.g: “well done”
‣ $grep -e “Agarwal” –e “Aggarwal” –e “Agrawal” geekfile.txt
‣ ^ — exclude “[^aeiou]” or (start)
‣ $ — ending

26. sort — command arranges lines of text alphabetically or numerically

Assume the below initial contents of file1.txt for the following examples:
01 Priya
04 Shreya
03 Tuhina
02 Tushar

$ sort file1.txt — Default ordering
01 Priya
02 Tushar
03Tuhina
04 Shreya

$ sort -r file1.txt — Sort in reverse ordering:
04 Shreya
03Tuhina
02 Tushar
01 Priya

$ sort -k 2 file1.txt — Sort by the second field:
01 Priya
04 Shreya
03 Tuhina
02 Tushar

26. uniq : command line utility that reports or filters out the repeated lines in a file. (Displays the unique lines in the file)
Example:
uniq kt.txt
uniq -c kt.txt — It tells the number of times a line was repeated. (count)
uniq -d kt.txt — It only prints the repeated lines. (-d dupliacate)
uniq -u kt.txt — only show lines that are not repeated (unique)

Delete duplicate lines: Important interview question

sort file.txt | uniq -u — remove duplicate line in a file in unix
sort file.txt | uniq -u | cat file.txt — remove and display the contents of the file

27. head Command: Head by default, prints the first 10 lines of each FILE to standard output
head file1.txt — displays first 10 lines in the file1.txt

28. tail Command : tail by default last 10 lines of each FILE to standard output
tail file1.txt — displays last 10 lines in the file1.txt
Examples:
head -10 filename | tail -1 filename — to display 10th line of the file
head -n filename | tail -1 — to display nth line of the file

29. diff command: This command is used to display the differences in the files by comparing the files line by line. (Compare two or more files)
Example:
diff a.txt b.txt

30. tr command (translate): “tr” command in UNIX is a command line utility for translating or deleting characters.
Syntax: $ tr [OPTION] SET1 [SET2] (Translate)
Example:
$cat greekfile | tr “[a-z]” “[A-Z]” –convert lower case to upper case
$cat greekfile | tr “[A-Z]” “[a-z]” –convert upper case to lower case

31. cut command: which is used to extract sections from each line of input (Cut the fileds from a file)
Syntax: cut OPTION… [FILE]…
Example:
cut -c 2,5,7 state.txt — command prints second, fifth and seventh character from each line of the file.

32. zip command : a command-line utility that helps you create Zip archives.
Example:
zip archivename.zip filename1 filename2 filename3

33. unzip command : unzip command extracts all files from the specified ZIP archive to the current directory.
Example:
unzip archivename.zip

34. echo command: used to display line of text/string that are passed as an argument
Example:
echo Hello world

35. rev command: used to reverse the lines characterwise
Example:
rev file1.txt — it reverse the contenets of the file.

How to reverse a string in unix?
echo “java” | rev — it will display “avaj”

36. sed command (stream Editor): it can perform lot’s of function on file like, searching, find and replace, insertion or deletion.
Syntax: sed OPTIONS… [SCRIPT] [INPUTFILE…]
Example:
sed ‘s/unix/linux/’ geekfile.txt — replaces the word “unix” with “linux” in the file.

37. awk command: Awk is a scripting language used for manipulating data and generating reports.The awk command programming language requires no compiling, and allows the user to use variables, numeric functions, string functions, and logical operators.

1. awk Operations:
   (a) Scans a file line by line
   (b) Splits each input line into fields
   (c) Compares input line/fields to pattern
   (d) Performs action(s) on matched lines
2. Useful For:
   (a) Transform data files
   (b) Produce formatted reports
3. Programming Constructs:
   (a) Format output lines
   (b) Arithmetic and string operations
   (c) Conditionals and loops

Example:
awk ‘{print}’ employee.txt — prints every line of data from the specified file
awk ‘/manager/ {print}’ employee.txt — prints all the line which matches with the ‘manager’

38. history command: to give the entire history of the commands which we have used. hisotry>a.txt