ADVERTISEMENT
Responsive Ad Unit • Systems Architecture & Low-Level Engineering

File I/O

File Input/Output (File I/O) forms the foundation of UNIX system programming. UNIX follows the principle that most objects can be accessed through file descriptors, including regular files, directories, pipes, terminals, sockets, and many devices. Applications interact with these objects using standardized POSIX system calls.

Learning Objectives

  • Understand the UNIX File I/O model.
  • Learn file descriptors.
  • Study the basic file system calls.
  • Understand how files are opened and closed.
  • Prepare for advanced file operations.

1. What is File I/O?

File I/O refers to the mechanism through which applications read data from files and write data to files. In UNIX, File I/O is performed using unbuffered system calls provided by the operating system kernel. These system calls operate directly on file descriptors.

2. UNIX File Model

UNIX treats many resources as files. The same programming interfaces can be used to access regular files, terminals, pipes, sockets, and numerous device files. This unified design simplifies application development by providing a common I/O interface.

3. File Descriptors

A file descriptor is a small non-negative integer returned by the kernel when a file is successfully opened. Applications use the file descriptor to identify an open file when performing subsequent I/O operations.

4. Standard File Descriptors

Descriptor Name Purpose
0 Standard Input Read input.
1 Standard Output Write normal output.
2 Standard Error Write error messages.

5. File Descriptor Table

Application ↓ File Descriptor (3) ↓ File Descriptor Table ↓ Open File Description ↓ Inode ↓ Storage Device

Each process maintains a file descriptor table. Entries in this table reference open file descriptions maintained by the kernel.

6. open()

The POSIX open() system call opens an existing file or creates a new file when appropriate flags are specified. On success, it returns a file descriptor.

Prototype
int open(const char *pathname, int flags);

7. creat()

The creat() system call creates a new file for writing or truncates an existing file before opening it. Modern applications typically achieve the same behavior using open() with appropriate flags.

8. close()

The POSIX close() system call releases a file descriptor. After a successful call, the descriptor no longer refers to the previously opened file.

9. Basic File Operation Workflow

open() ↓ Kernel allocates file descriptor ↓ read() / write() ↓ close()

10. Practical POSIX Example

#include #include int main(void) { int fd; fd = open("example.txt", O_RDONLY); if (fd != -1) { close(fd); } return 0; }

The POSIX open() function returns a file descriptor that is later released using close().

11. Interview Questions

  1. What is File I/O?
  2. What is a file descriptor?
  3. Which file descriptors are reserved by default?
  4. What does open() return?
  5. What happens if open() fails?
  6. What is the purpose of creat()?
  7. Why should close() always be called?
  8. Can sockets use file descriptors?
  9. What does the kernel maintain for each process?
  10. Why is the UNIX File I/O model considered uniform?

12. read()

The POSIX read() system call transfers data from an open file descriptor into a memory buffer. The number of bytes actually read may be less than the number requested.

Prototype
ssize_t read(int fd, void *buf, size_t count);

13. write()

The POSIX write() system call transfers data from a memory buffer to an open file descriptor. The number of bytes written may be smaller than the requested count.

Prototype
ssize_t write(int fd, const void *buf, size_t count);

14. Return Values

Function Success Failure
read() Number of bytes read -1
write() Number of bytes written -1

15. End-of-File (EOF)

When read() reaches the end of a file, it returns 0. This indicates that no more data is available to be read.

16. Partial Reads and Writes

A successful read() or write() call does not guarantee that the requested number of bytes was transferred. Applications should always check the returned byte count and, if necessary, continue reading or writing until the desired amount of data has been processed.

17. File Offset

Each open file description maintains a current file offset. Successful read() and write() operations automatically advance the file offset by the number of bytes transferred.

Beginning of File ↓ Current Offset ↓ read() / write() ↓ Offset Advances ↓ End of File

18. lseek()

The POSIX lseek() system call changes the current file offset associated with an open file descriptor. It allows applications to reposition the offset before performing subsequent I/O operations.

Prototype
off_t lseek(int fd, off_t offset, int whence);

19. lseek() Reference Positions

Constant Description
SEEK_SET Beginning of the file.
SEEK_CUR Current file offset.
SEEK_END End of the file.

20. Practical POSIX Example

#include #include int main(void) { int fd; char buffer[64]; fd = open("example.txt", O_RDONLY); if (fd != -1) { read(fd, buffer, sizeof(buffer)); close(fd); } return 0; }

The program opens a file, reads data into a buffer, and then closes the file descriptor.

21. Linux Commands

Command Description
cat file.txt Display file contents.
head file.txt Display the beginning of a file.
tail file.txt Display the end of a file.
wc -c file.txt Count bytes in a file.
od -c file.txt Display file contents in character format.

22. Interview Questions

  1. What does read() return on success?
  2. What indicates End-of-File?
  3. Why can partial reads occur?
  4. Why should the return value of write() always be checked?
  5. What is the purpose of lseek()?
  6. What is the difference between SEEK_SET and SEEK_CUR?
  7. What happens to the file offset after a successful read?
  8. Can lseek() move beyond the end of a file?
  9. What is an open file description?
  10. Which POSIX functions modify the current file offset?

23. Sparse Files (File Holes)

A sparse file contains regions where no actual data has been written. These unwritten regions are known as file holes. The filesystem records these regions efficiently without allocating physical storage until data is written.

Data Block ↓ Hole (No Physical Data) ↓ Data Block

24. File Descriptor Duplication

File descriptor duplication creates an additional file descriptor that refers to the same open file description. Both descriptors share the same file offset and file status flags.

25. dup()

The POSIX dup() function duplicates an existing file descriptor and returns the lowest-numbered available descriptor.

Prototype
int dup(int oldfd);

26. dup2()

The POSIX dup2() function duplicates a file descriptor onto a specified descriptor number. If the target descriptor is already open, it is closed before being reused.

Prototype
int dup2(int oldfd, int newfd);

27. dup3()

The dup3() function performs descriptor duplication similar to dup2() while allowing additional flags to be specified.

Availability Category
Linux Linux Extension (Not POSIX)

28. Descriptor Sharing

Property Shared?
File Offset Yes
File Status Flags Yes
Descriptor Flags No

Duplicated descriptors reference the same open file description, so changes to the file offset are visible through both descriptors.

29. fcntl()

The POSIX fcntl() function performs a variety of operations on open file descriptors, including duplication, retrieving descriptor flags, modifying file status flags, and managing advisory file locks.

Prototype
int fcntl(int fd, int cmd, ...);

30. Common fcntl() Commands

Command Purpose
F_DUPFD Duplicate a file descriptor.
F_GETFD Read descriptor flags.
F_SETFD Modify descriptor flags.
F_GETFL Read file status flags.
F_SETFL Modify file status flags.

31. Common File Status Flags

Flag Description
O_RDONLY Open for reading only.
O_WRONLY Open for writing only.
O_RDWR Open for reading and writing.
O_APPEND Append writes to the end of the file.
O_NONBLOCK Enable non-blocking I/O.

32. Practical POSIX Example

#include int main(void) { int newfd; newfd = dup(STDOUT_FILENO); if (newfd != -1) { close(newfd); } return 0; }

The program duplicates the standard output file descriptor and closes the duplicated descriptor before terminating.

33. Linux Commands

Command Description
ls -ls sparse_file Compare logical and allocated file size.
stat file.txt Display file metadata.
file file.txt Identify the file type.
hexdump -C file.txt Display file contents in hexadecimal.
od -x file.txt Display file contents as hexadecimal words.

34. Interview Questions

  1. What is a sparse file?
  2. Why are sparse files storage efficient?
  3. What does dup() return?
  4. How does dup2() differ from dup()?
  5. Is dup3() part of POSIX?
  6. Which properties are shared by duplicated descriptors?
  7. What is the purpose of fcntl()?
  8. What is the difference between descriptor flags and file status flags?
  9. Which fcntl() command retrieves file status flags?
  10. What does O_APPEND do?

35. pread()

The POSIX pread() function reads data from a specified file offset without changing the current file offset associated with the open file description. This makes it useful when multiple parts of a program need to access different regions of the same file independently.

36. pwrite()

The POSIX pwrite() function writes data at a specified file offset without modifying the current file offset. This enables writing to arbitrary locations in a file while preserving the existing file position.

37. Scatter/Gather I/O

Scatter/Gather I/O allows multiple memory buffers to be processed using a single system call. This reduces the number of system calls required for complex data transfers.

38. readv() and writev()

Function Description
readv() Reads data into multiple buffers.
writev() Writes data from multiple buffers.

39. Data Synchronization

Applications often write data into kernel buffers before it is stored on the underlying storage device. Synchronization functions request that buffered information be written to stable storage.

40. Synchronization Functions

Function Purpose
sync() Requests synchronization of modified filesystem data.
fsync() Synchronizes a specific file and its associated metadata.
fdatasync() Synchronizes file data and required metadata.

41. Atomic I/O Operations

An atomic operation is completed as a single indivisible action from the perspective of other processes. Certain POSIX file operations are defined to behave atomically under specified conditions, helping to avoid intermediate inconsistent states.

42. APUE / POSIX vs Linux

Interface Category
open() POSIX
read() POSIX
write() POSIX
close() POSIX
lseek() POSIX
pread() POSIX
pwrite() POSIX
readv() POSIX
writev() POSIX
dup3() Linux Extension

43. Best Practices

  • Always verify the return value of every I/O function.
  • Handle partial reads and writes correctly.
  • Close file descriptors when they are no longer needed.
  • Prefer POSIX interfaces for portable applications.
  • Use pread() and pwrite() when file offsets must remain unchanged.
  • Use synchronization functions when persistent storage is required.
  • Avoid unnecessary system calls by using scatter/gather I/O where appropriate.

44. Module Summary

This module introduced the POSIX File I/O model and covered file descriptors, opening and closing files, sequential reading and writing, file offsets, sparse files, descriptor duplication, descriptor management, synchronization operations, and scatter/gather I/O. These interfaces form the foundation for UNIX system programming and are widely used throughout higher-level libraries and applications.

45. Key Takeaways

  • UNIX uses file descriptors to access open files and many other I/O objects.
  • open() returns a file descriptor used by subsequent I/O operations.
  • read() and write() may transfer fewer bytes than requested.
  • lseek() changes the current file offset for seekable files.
  • dup() and dup2() duplicate file descriptors that share the same open file description.
  • fcntl() manages descriptor properties and file status flags.
  • pread() and pwrite() perform offset-based I/O without changing the current file position.
  • readv() and writev() support scatter/gather I/O using multiple buffers.
  • fsync() and fdatasync() request synchronization of buffered data to storage.
  • Portable applications should rely primarily on standardized POSIX interfaces.

Next Module

The File I/O module is now complete. Continue to: Network IPC: Sockets

Next → Network IPC: Sockets
Knowledge Check

Concept Mastery Assessment — Module 17

Verify your conceptual understanding with 12 targeted technical assessment questions. Instant feedback and explanations are provided for every choice.

Question 1 of 12
What is the primary architectural purpose of Low-Level File I/O (open, read, write, close, lseek) in operating systems?
Explanation
Low-Level File I/O (open, read, write, close, lseek) is designed to manage kernel file table, inode table, file descriptors, atomicity with o_append, seek offsets, and write barriers. within the operating system architecture.
Question 2 of 12
Which system interface or data structure is centrally associated with Low-Level File I/O (open, read, write, close, lseek)?
Explanation
Under POSIX and Linux, Low-Level File I/O (open, read, write, close, lseek) operates through standardized system calls and kernel control structures.
Question 3 of 12
What safety or concurrency constraint must developers consider when implementing Low-Level File I/O (open, read, write, close, lseek)?
Explanation
Proper synchronization, avoidance of race conditions, and handling reentrant execution are critical when working with Low-Level File I/O (open, read, write, close, lseek).
Question 4 of 12
When a system call related to Low-Level File I/O (open, read, write, close, lseek) fails in C, how does the POSIX runtime conventionally report the failure?
Explanation
Standard POSIX system calls return -1 or NULL on error, assigning an error integer (e.g. EINVAL, EPERM, EAGAIN) to the thread-local errno variable.
Question 5 of 12
What is the primary performance tradeoff associated with Low-Level File I/O (open, read, write, close, lseek)?
Explanation
Operations involving Low-Level File I/O (open, read, write, close, lseek) balance performance against kernel transition, memory bandwidth, and cache locality tradeoffs.
Question 6 of 12
What security principle governs access to resources in Low-Level File I/O (open, read, write, close, lseek)?
Explanation
The kernel validates credentials, effective permissions, and Linux capabilities before granting access to protected operations.
Question 7 of 12
How does execution of Low-Level File I/O (open, read, write, close, lseek) transition between User Space and Kernel Space?
Explanation
User space transitions to kernel mode securely via software traps or SYSCALL instructions that switch processor rings to Ring 0.
Question 8 of 12
What happens to resources allocated by Low-Level File I/O (open, read, write, close, lseek) when the owning process terminates?
Explanation
Process resources (open fds, memory maps) are automatically reclaimed by the kernel on termination, while kernel-persistent objects survive.
Question 9 of 12
In production Linux server environments, how is Low-Level File I/O (open, read, write, close, lseek) optimized for high throughput?
Explanation
Linux optimizes throughput via zero-copy data paths, asynchronous kernel events, and scalable concurrency primitives.
Question 10 of 12
What standard governs the portability of Low-Level File I/O (open, read, write, close, lseek) across Unix, Linux, and macOS platforms?
Explanation
IEEE POSIX standards define consistent APIs across Unix-like platforms ensuring portable source code execution.
Question 11 of 12
Which command-line diagnostic tool is most effective for inspecting and troubleshooting Low-Level File I/O (open, read, write, close, lseek) at runtime?
Explanation
Utilities like strace, lsof, and the /proc filesystem provide direct visibility into kernel interactions and resource status.
Question 12 of 12
What is a critical production best practice when developing systems software involving Low-Level File I/O (open, read, write, close, lseek)?
Explanation
Defensive programming requires explicit error checking on all system operations, proper resource cleanup, and avoiding memory corruption.
0 / 12 (0%)
Great job! Review the concepts above and proceed when ready.
ADVERTISEMENT
Responsive Ad Unit • Technical Courses & Certifications