dealing with huge data writing to files using shell scripts/Opening the file descriptors for reading and writing

Its not so comportable while writing manipulated huge data to files using shell script/command.

We normally depend on direct o/p commands such as "> "or ">>" to write to or read from files.

In shell scripts aslo using file descriptors we can achive time write/read in less time.


the following was taken from below link.
http://bash.cyberciti.biz/guide/Opening_the_file_descriptors_for_reading_and_writing

Bash supports the following syntax to open file for both reading and writing on file descriptor:

exec fd<>fileName

• File descriptor 0 is used if fd is not specified.
• If the file does not exist, it is created.
• This syntax is useful to update file.

Example

Create a shell script called fdreadwrite.sh

#!/bin/bash
FILENAME="/tmp/out.txt"
# Opening file descriptors # 3 for reading and writing
# i.e. /tmp/out.txt
exec 3<>$FILENAME
 

# Write to file
echo "Today is $(date)" >&3
echo "Fear is the path to the dark side. Fear leads to anger. " >&3
echo "Anger leads to hate. Hate leads to suffering." >&3
echo "--- Yoda" >&3
 
# close fd # 3
exec 3>&-