Assignment #5 - ScatterGather v1.3

Submit Assignment

The next assignment will continue our implementation of the user-space device driver for the ScatterGather drive. The service creates a peer-to-peer system in which you place the data for your local filesystem on a bunch of external hosts (called Nodes). This assignment is the third of four SG projects in this course.

Note that you will be required to use your code from the previous assignment as the starting place for the next assignment. Thus, if you have terrible code or incomplete implementation, you will have to fix it in this assignment. Remember that students that do a good job and stay on top of the assignments will not only get good grades, but the projects will be much easier.

NOTE: The due date for this project is hard. There is no 10%/day extension and THERE WILL BE NO EXTENSIONS OF ANY KIND GIVEN (because we need to have the projects to grade to assign final grades in time).

Project Overview

You are to continue writing the device driver that sits between a virtual application and virtualized hardware devices. The application makes use of the abstraction you will provide called the ScatterGather driver. This fourth assignment will add four new features/requirements to the implementation; more complex workloads, increased cache size, sequence numbers, and a hidden workload. As before, the sg_sim application is provided to you and will call your device driver functions with the basic UNIX file operations (open, read, ...). You are to write the device driver code to implement the file I/O operations.

The first feature of this assignment is to extend the sg_open/ sg_read/ sg_write/ sg_close functions (see slides) to handle more complex file input and output for the system. In particular, you will implement the system with quarter block reads and writes (256 bytes). The functions are defined in the sg_driver.c file and their use is described in the associated lecture and slides (and previous assignment). Please refer to that documentation to help you understand how to complete this assignment. You MUST use your submission for assignment #4 as the starting point.

The second feature to be to increase the LRU cache to include many more cache lines. See the SG_MAX_CACHE_ELEMENTS variable in the sg_cache.h file. In all other respects the cache should perform in all other respects as assignment #3 (including outputting the cache hit rates).

The third feature is to manage sequence numbers. You will need to manage the sequence numbers for both the local node and each remote node in the system. Details are also in the slides.

11/27/2020 Assignment #5 - ScatterGather v1.3

https://psu.instructure.com/courses/2086230/assignments/12402041 2/3

The last feature is that you will be tested against a workload you have not seen (and will not be given). The workload will include quarter block reads and writes as in this assignment, but may include any number of files and blocks in files. Thus, your code MUST use dynamic allocation of structures and tables (e.g., for the nodes/blocks) to process this file. Note that 30% of the grade for this assignment is determined by your code's ability to process this workload.

See slides and video introduction for more information.

Honors Option

Modify your code to hash blocks (using something like the gcrypt MD5 function) before sending (posting). You need to hash every block coming back from the Scatter/Gather system and compare the hash values. If they differ, you should return an error from the read/write.

Instructions

Login to your virtual machine. Create the repository through github classroom using this link (https://classroom.github.com/a/325ROIt-) . Then from your virtual machine, clone the starter source code provided for this assignment (use the git helper document for the commands):

% mkdir cmpsc311

% cd cmpsc311

% git clone [email protected]:PSUCMPSC311/scattergather5-.git

(mailto:[email protected]:PSUCMPSC311/scattergather-.git)

[ START EDITING ]

Once cloned, you will have the starter files in the scattergather5-, including the source code, libraries, some workload and manifest files and a Makefile.

You are to complete the driver functions defined (see slides). Note that you may need to create additional supporting functions within the same file. Include functional prototypes for these functions at the top of the file.

Add comments to all of your files stating what the code is doing. Fill out the comment function header for each function you are defining in the code. A sample header you can copy for this purpose is provided for the main function in the code.

To test your program (after running make) with these provided workload files, run the code specifying a workload file as follows: ./sg_sim -v cmpsc311-assign5-workload.txt

Note that you don't necessarily have to use the -v option, but it provides a lot of debugging information that is helpful. If the program completes successfully, the following should be displayed as the last log entry:

ScatterGather.com simulation completed successfully!!!

It is good practice to commit and push your code regularly in case you run into issues with your machine (Note: you must commit all files that are marked as changed when running 'git status'). Go to your github page and verify that you actually pushed your code.

11/27/2020 Assignment #5 - ScatterGather v1.3

https://psu.instructure.com/courses/2086230/assignments/12402041 3/3

You MUST commit all of the files in your repository including all .c, .h, Makefile, .a (but not .o or sg_sim).

To turn in:

1. Submit on canvas the commit ID that should be graded, along with your github username.

2. Before submitting the commit ID, clone to a temporary directory and test your code at that commit using the following commands (copy to and run in a temporary directory -- NOT the directory you used to develop the code):

Copy/clone code to temporary directory.

% cd

% git checkout

% make

% ./sg-sim -v cmpsc311-assign5-workload.txt

% git checkout master (to go back to your latest code)

NOTE: The due date for this project is hard (the last day of class). There is no 10%/day extensions for late submissions and THERE WILL BE NO EXTENSIONS OF ANY KIND GIVEN (because we need to have the projects to grade to assign final grades in time). 

source code, libraries, some workload and manifest files and a Makefile.

You are to complete the driver functions defined (see slides). Note that you may need to create additional supporting functions within the same file. Include functional prototypes for these functions at the top of the file.

Add comments to all of your files stating what the code is doing. Fill out the comment function header for each function you are defining in the code. A sample header you can copy for this purpose is provided for the main function in the code.

To test your program (after running make) with these provided workload files, run the code specifying a workload file as follows: ./sg_sim -v cmpsc311-assign5-workload.txt

Note that you don't necessarily have to use the -v option, but it provides a lot of debugging information that is helpful. If the program completes successfully, the following should be displayed as the last log entry:

ScatterGather.com simulation completed successfully!!!

It is good practice to commit and push your code regularly in case you run into issues with your machine (Note: you must commit all files that are marked as changed when running 'git status'). Go to your github page and verify that you actually pushed your code.

11/27/2020 Assignment #5 - ScatterGather v1.3

https://psu.instructure.com/courses/2086230/assignments/12402041 3/3

You MUST commit all of the files in your repository including all .c, .h, Makefile, .a (but not .o or sg_sim).

To turn in:

1. Submit on canvas the commit ID that should be graded, along with your github username.

2. Before submitting the commit ID, clone to a temporary directory and test your code at that commit using the following commands (copy to and run in a temporary directory -- NOT the directory you used to develop the code):

Copy/clone code to temporary directory.

% cd

% git checkout

% make

% ./sg-sim -v cmpsc311-assign5-workload.txt

% git checkout master (to go back to your latest code)

NOTE: The due date for this project is hard (the last day of class). There is no 10%/day extensions for late submissions and THERE WILL BE NO EXTENSIONS OF ANY KIND GIVEN (because we need to have the projects to grade to assign final grades in time). 

source code, libraries, some workload and manifest files and a Makefile.

You are to complete the driver functions defined (see slides). Note that you may need to create additional supporting functions within the same file. Include functional prototypes for these functions at the top of the file.

Add comments to all of your files stating what the code is doing. Fill out the comment function header for each function you are defining in the code. A sample header you can copy for this purpose is provided for the main function in the code.

To test your program (after running make) with these provided workload files, run the code specifying a workload file as follows: ./sg_sim -v cmpsc311-assign5-workload.txt

Note that you don't necessarily have to use the -v option, but it provides a lot of debugging information that is helpful. If the program completes successfully, the following should be displayed as the last log entry:

ScatterGather.com simulation completed successfully!!!

It is good practice to commit and push your code regularly in case you run into issues with your machine (Note: you must commit all files that are marked as changed when running 'git status'). Go to your github page and verify that you actually pushed your code.

11/27/2020 Assignment #5 - ScatterGather v1.3

https://psu.instructure.com/courses/2086230/assignments/12402041 3/3

You MUST commit all of the files in your repository including all .c, .h, Makefile, .a (but not .o or sg_sim).

To turn in:

1. Submit on canvas the commit ID that should be graded, along with your github username.

2. Before submitting the commit ID, clone to a temporary directory and test your code at that commit using the following commands (copy to and run in a temporary directory -- NOT the directory you used to develop the code):

Copy/clone code to temporary directory.

% cd

% git checkout

% make

% ./sg-sim -v cmpsc311-assign5-workload.txt

% git checkout master (to go back to your latest code)

NOTE: The due date for this project is hard (the last day of class). There is no 10%/day extensions for late submissions and THERE WILL BE NO EXTENSIONS OF ANY KIND GIVEN (because we need to have the projects to grade to assign final grades in time).