Project 7: File-Related Syscalls

Project 7 Operating Systems
Page 1
Programming Project 7:
File-Related Syscalls

Overview and Goal
In this project, you will implement the syscalls relating to file I/O: Open, Read, Write, Seek, and
Close. These syscalls will allow user-level processes to read and write to files stored on the BLITZ
DISK file. The goal is for you to understand the syscall mechanism in more detail, to understand what
happens when several processes operate on a shared file system, and to understand how the kernel
buffers and moves data between the file system on the disk and the user-level processes.
Download New Files
The files for this project are available in:
http://www.cs.pdx.edu/~harry/Blitz/OSProject/p7/
The following files are new to this project:
TestProgram4.h
TestProgram4.c
Program1.h
Program1.c
Program2.h
Program2.c
The following files have been modified from the last project:
makefile
DISK
The makefile has been modified to compile TestProgram4, Program1, and Program2. The DISK file
has been enlarged, since the previous version was too small to accommodate TestProgram4,
Program1, and Program2.
Project 7 Operating Systems
Page 2
All remaining files are unchanged from the last project.
The Task:
Implement the following syscalls:
Open
Read
Write
Seek
Close
Update your code as necessary to account for the possibility of open files. You’ll probably need to
modify these routines, too:
Handle_Sys_Fork
ProcessFinish
Note that files may be open in a process that invokes the Exec syscall. These files should remain open
in the process after the Exec completes successfully. Does this require a change to your code?
Take a look at the project 5 document for details on the specifications of each of these syscalls. Please
re-read that document thoroughly. In particular, the following sections of that document contain
important information that you’ll need to complete this project.
The “Stub” File System
The “diskUtil” Tool
The FileManager
FileControlBlock (FCB) and OpenFile
Limitations to the “Stub” Filesystem:
New files cannot be added to the disk. We will not implement the Create syscall in this project. All
files must already be on the DISK before they can be opened and accessed. (Files can be added to the
BLITZ DISK with the diskUtil command.)
Also, we will not be able to change the size of a file. If the user-level program invokes Write with
arguments that would cause the bytes transferred to be beyond the current end of the file, the bytes that
are not before the file end should be transferred, but the bytes beyond the file end will not be transferred.
The Write will return the number of bytes successfully written. For example, if the file size is 10 and
Project 7 Operating Systems
Page 3
Write is invoked with a length of 5 on a file whose current position is 7, then 3 bytes should be
transferred and the 2 bytes beyond the end of the file will not be transferred. The Write should return 3.
Implementing Handle-Sys-Open:
Here are some actions you’ll need to take when to implementing the Open syscall:
Copy the filename string from virtual space to a small buffer in this routine.
Make sure the length of the filename does not exceed MAX_STRING_SIZE
Locate an empty slot in this process’s fileDescriptor array.
If there are no empty slots, return –1.
Allocate an OpenFile object (see FileManager.Open).
If this fails, return –1.
Set the entry in the fileDescriptor array to point to the OpenFile object.
Return the index of the fileDescriptor array element.
Suggestions Concerning the Algorithm For Handle-Sys-Read:
Here is a possible approach to implementing the Read syscall.
You’ll need to begin by checking that the fileDesc argument really is valid. The user must provide a
legal index into the fileDescriptor array and the file referenced must have been previously opened. If
not, the Read syscall should return –1.
You’ll also need to check that the requested number of bytes (the sizeInBytes argument) is not negative.
If it is, then return –1.
We’ll discuss problems related to the buffer argument later.
When a user-level program invokes the Read syscall, it asks for a sequence of bytes to be fetched from a
file on disk. Unfortunately, this sequence may span several sectors in the disk file, so several calls to
DiskDriver.SynchReadSector will be needed to read all the data.
Each call to DiskDriver.SynchReadSector will read an entire 8K sector from the disk into memory.
The caller will provide the address of where in memory to read the sector, which we can refer to as the
“sector buffer” area in memory.
The data requested by the user-level process will not necessarily begin or end on a sector boundary. So
some buffering and movement of the data will be necessary. Take a look at FileManager.SynchRead,
which will perform a lot of the work you’ll need to do when implementing the Read syscall. In
particular, it will call DiskDriver.SynchReadSector to read a sector into the sector buffer and then it
Project 7 Operating Systems
Page 4
will copy the desired bytes from the sector buffer to wherever they should go. (This is called the “target
address.”)
Each FileControlBlock already has a sector buffer associated with it. This is an 8K memory region that
was allocated from the pool of memory frames when the FileManager was initialized. [You do not
need to allocate any memory buffers in this project.]
If the requested sequence of bytes spans several sectors, FileManager.SynchRead will read as many
sectors as necessary (re-using the one sector buffer) and, after each disk I/O completes, it will copy
bytes in several chunks to the target area. (Also, if the sector buffer is dirty before it starts, it will first
write the old sector out to disk.)
When a user requests a sequence of bytes to be read from a file, the user provides a pointer to a “userspace target area,” which tells the kernel where to copy this data to. User-level code often refers to its
target area as a “buffer” but be careful to avoid confusing this area with the “sector buffer,” which is part
of the FileControlBlock in kernel space. The user will supply a virtual address for the user target area
by supplying values called buffer and sizeInBytes to the Read syscall.
When implementing the Read syscall, you’ll need to translate the target address from a virtual address
into a physical address, so you can know where in memory to move the data.
Unfortunately, the user target area may cross page boundaries in the virtual address space. In general,
the pages of the user’s address space will not be in contiguous memory frames.
FileManager.SynchRead cannot deal with this; it expects its target area to be one contiguous region in
memory.
This means that you cannot simply call FileManager.SynchRead once to get the job done. You’ll need
to break the user target area into chunks such that each chunk is entirely within one page. Then, you can
call FileManager.SynchRead to read each of these chunks in a loop.
Below is some pseudo-code showing how this loop might work. It works by breaking the entire
sequence of bytes to be read into several chunks. Each chunk is entirely on one page and does not cross
a page boundary. It computes the length and starting address of each chunk. It translates the starting
address into a physical address. Then it calls FileManager.SynchRead to read the chunk and moves on
the next chunk.
The key variables are:
virtPage Virtual address into which to read the next chunk (virtual address page number)
offset Virtual address into which to read the next chunk (offset into the page)
chunkSize The number of bytes to be read for this chunk
nextPosInFile The position in the file from which to read the next chunk
copiedSoFar The number of bytes read from disk so far
At the beginning of each loop iteration, virtPage and offset tell where the first byte in the next chunk
should go. Each iteration computes the size of the next chunk, does the read, and then adjusts all the
Project 7 Operating Systems
Page 5
variables. nextPosInFile tells where in the file the next chunk to be read begins. Initially, it will be
given by the current position in the file, but will be adjusted after each chunk is read.
virtAddr = buffer asInteger
virtPage = virtAddr / PAGE_SIZE
offset = virtAddr % PAGE_SIZE
copiedSoFar = 0
nextPosInFile = ...
-- Each iteration will compute the size of the next chunk
-- and process it...
while true
-- Compute the size of this chunk...
thisChunkSize = PAGE_SIZE-offset
if nextPosInFile + thisChunkSize sizeOfFile
thisChunkSize = sizeOfFile - nextPosInFile
if copiedSoFar + thisChunkSize sizeInBytes
thisChunkSize = sizeInBytes - copiedSoFar
-- See if we are done...
if thisChunkSize <= 0
exit loop
-- Check for errors...
if (virtPage < 0) or
(page number is too large) or
(page is not valid) or
(page is not writable)
deal with errors
-- Do the read...
Set “DirtyBit” for this page
Set “ReferencedBit” for this page
Compute destAddr = addrSpace.ExtractFrameAddr (virtPage) + offset
Perform read into destAddr, nextPosInFile, chunkSize bytes
-- Increment...
nextPosInFile = nextPosInFile + thisChunkSize
copiedSoFar = copiedSoFar + thisChunkSize
virtPage = virtPage + 1
offset = 0
-- See if we are done...
if copiedSoFar == sizeInBytes
exit loop
endWhile
Project 7 Operating Systems
Page 6
Note that we are marking the page in the user’s virtual address space as “dirty.” Think carefully about
why a “read” would mark the page dirty. When implementing the Write syscall, the operation will not
cause the page to become dirty!
When a user-program reads data from disk to memory, the page in memory is changed. So the page that
receives the data from disk must be marked as dirty. When a user-program writes data from memory to
disk, the page in memory is unchanged. The page would not be marked dirty.
You also need to set the Referenced Bit for all pages accessed, regardless of whether the user is
invoking the Read or Write syscall, since in either case, the page has been used.
Normally, the MMU will set the Dirty Bit and Referenced Bit when appropriate, but this will only
occur when paging is turned on. When the kernel handles the Read and Write syscalls, it will be
accessing the pages directly, using their physical memory addresses. This will bypass the MMU, so the
kernel code will need to change the bits explicitly.
[Setting the Dirty Bit correctly will be necessary if we implement virtual memory in a later project. A
failure to set the dirty bit correctly might occasionally result in data that gets lost when a page that
should be copied to disk is not copied. A failure to set the referenced bit may have repercussions for the
page replacement algorithm, causing it to malfunction. The resulting thrashing or poor performance
might be very, very difficult to debug and fix!]
It is possible that the user will provide bad arguments to the Read syscall. For example, the user target
area may lie outside of the virtual address space or may be in a page that is not writable. In such a case,
the kernel should detect the error and return –1. Furthermore, if such a problem with the parameters
occurs, the kernel should not perform any disk operation. In other words, the user target area should be
entirely unchanged.
This is not how the above code works. If, for example, the user target area runs past the end of the
virtual address space, the above code may read several chunks successfully before encountering the
error and aborting.
To perform correctly, you’ll actually need to do the work using two loops. You’ll execute one loop to
completion, then execute the second loop. Both loops will be just like the one shown above, except that
the first loop will not actually perform the reading. The sole purpose of the first loop is to check the user
target area and return from the syscall if problems. The second loop will repeat the computations
exactly and will also perform the reading operations.
FileManager.SynchRead will never fail; as coded it always returns true.
A similar approach to Handle_Sys_Read can be taken for Handle_Sys_Write.
Project 7 Operating Systems
Page 7
Implementing Handle-Sys-Seek:
Here are some actions you’ll need to take when to implementing the Seek syscall:
Lock the FileManager since we will be updating shared data (the OpenFile.currentPos)
Check the fileDesc argument and get a pointer to the OpenFile object.
Make sure the file is open. (Recall that a null entry in fileDescriptors means “not open.”)
Deal with the possibility that the new current position is equal to –1, which has a special meaning.
Deal with the possibility that the new current position is less that –1. (Zero is okay.)
Deal with the possibility that the new current position is greater than the file size.
Update the current position.
Return the new current position.
Remember to unlock the FileManager, regardless of whether you are making a normal return or an
error return.
The User-Level Programs
The p7 directory contains a new user-level program called:
TestProgram4
Please change InitUserProcess to load TestProgram4 as the initial process. Then run it several times,
once for each of the tests it contains.
After you have finished coding and debugging, please run each test once and hand in the output from
each test. A separate document shows more-or-less what the correct output should look like.
Use the same code to execute all tests. Please hand in only one copy of your Kernel.c code and do not
hand in any output that was produced by a different version of your code!
Do not change TestProgram4, except to uncomment one of the lines in the main function.
During your testing, it may be convenient to modify the tests as you try to see what is going on and get
things to work. Before you make your final test runs, please recopy TestProgram4.c from our
directory, so that you get a fresh, unaltered version.
Please hand in hardcopy of Kernel.c. You only need to hand in...
Project 7 Operating Systems
Page 8
Handle_Sys_Open
Handle_Sys_Close
Handle_Sys_Read
Handle_Sys_Write
Handle_Sys_Seek
and any other code you wrote / modified, like Handle_Sys_Fork and ProcessFinish.
Desired Output
There is some sample output, collected together into a separate file called DesiredOutput.pdf in the p7
directory.