Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

DRAFT SYLLABUS FOR

Principle & Practice of Parallel Computing

Course Information:

CSE 375-475

Fall 2021

Course Description

It's the era of data, and having knowledge on how to design and develop correct high-performance algorithms and applications for computing data is a fundamental requirement for prospective successful software engineers and designers. CSE-375/475 focuses on that, covering both theoretical and practical aspects, providing students with the sufficient knowledge to implement and reason about parallel applications. A particular focus is given to concurrency, which often represents a barrier for many developers given its complexity in providing correct computation due to the presence of simultaneous accesses on shared data. In this regard, the course covers the traditional lock-based programming, and also state-of-the-art (software and hardware) solutions to code concurrent applications without expositing locks to programmer. For Fall 2020, the course will be online; lectures will be recorded and available for download; office hours and grading sessions will be done at flexible time during face-to-face one-to-one Zoom meetings. The instructor relies on advanced technological tools to boost the online learning experience.

Course Learning Objectives

By the end of this course, students will be able to:

- Understanding of parallel hardware constructs, to include instruction-level parallelism, supercomputer architecture, multicore processor design;

- Understanding of language design issues related to parallel programming;

- Understanding of correctness criteria for parallel and concurrent computing;

- Ability to identify and classify dependencies;

- Ability to identify and correct data races;

- Ability to write performant parallel programs

List of Objectives:

(1) Dependency Analysis

(2) Parallel Programming Patterns

(3) GPU programming

(4) Multicore Architectures

(5) Cache Coherence and Memory Consistency

(6) Correctness Models for Concurrent and Parallel Programs

(7) Principles of Distributed Computing

(8) Locking and Non-Blocking Synchronization

(9) Hardware and Software Speculation/Transactional Memory

Prerequisites

- Knowledge of operating system resource allocation

- Knowledge of an assembly language

- Knowledge of microprocessor design

- Knowledge of memory hierarchy design

- Fluency in a programming language

Textbooks

- Michael McCool, James Reinders, Arch Robison, "Structured Parallel Programming: Patterns for Efficient Computation", Morgan Kaufmann, 1st Edition, 2012, ISBN: 9780124159938

○ Available from Lehigh network at https://asa.lib.lehigh.edu/Record/10682725

- The Art of Multiprocessor Programming, Revised Reprint, 2021

○ Available from Lehigh network at https://asa.lib.lehigh.edu/Record/00011370825

Expectations

This is a programming-focused course. The purpose of the course is to become confident in thinking and developing non-trivial parallel and concurrent programs. The only way to learn how to develop a parallel/concurrent application is to well understand the problem, design a solution, translate it into application code, then test and debug it to discover misbehaviors. Only solving issue after issue will give you the confidence. Given the nature of the course, I expect you to be engaged and proactive. Don’t give up if you cannot solve an issue in your program; think harder and share the problem with the me if needed.

Assignments

- Non-graded assignments:

○ Reading assignments

○ 1 assignment for refreshing C++ (required, only extra points)

- Graded assignments:

○ 1 in-class student presentation for each student = 10 points (10%)

○ 2 programming assignments - 20 points = 40 points (40%)

○ 1 project - 40 points = 40 points (50%)

○ Final exam consists of a report (around 10 pages) of the project, a presentation of the project design and the achieved results.

Policies

Grading Scale:

94-100 = A

90-93 = A-

87-89 = B+

84-86 = B

80-83 = B-

77-79 = C+

74-76 = C

70-73 = C-

67-69 = D+

64-66 = D

60-63 = D-

<60 = F

Statement on Academic Integrity

Student Senate Statement on Academic Integrity

The work you do in this course must be your own. This means that you must be aware when you are building on someone else's ideas—including the ideas of your classmates, your professor, and the authors you read—and explicitly acknowledge that you are do so. Feel free to build on, react to, criticize, and analyze the ideas of others but, when you do, make it known whose ideas you are working with. If you ever have questions about drawing the line between others' work and your own, ask me and I will give you clear guidance or you may visit Lehigh Library’s ‘Proper Use of Information’ page at http://libraryguides.lehigh.edu/plagiarism

You should keep in mind that computer programs exhibit an individual's "style" just as much as other forms of authorship. Changing variable names, editing comments, or making other trivial updates in an attempt to hide plagiarism is rarely effective. If you have questions about this policy at any point throughout the semester, just ask.

Attendance

If you miss a class with a reasonable motivation, contact the professor to arrange a meeting and discuss about the missed class.

Late Papers or Missed Exams

There will be given a reasonable amount of time to submit assignments. No late submission will be accepted, unless an extension is coordinated with the professor at least 48 hours before the assignment is due.

The Principles of Our Equitable Community:

The Principles of Our Equitable Community: Lehigh University endorses The Principles of Our Equitable Community (www.lehigh.edu/diversity). We expect each member of this class to acknowledge and practice these Principles. Respect for each other and for differing viewpoints is a vital component of the learning environment inside and outside the classroom.

Accommodations for Students with Disabilities:

Lehigh University is committed to maintaining an equitable and inclusive community and welcomes students with disabilities into all of the University’s educational programs. In order to receive consideration for reasonable accommodations, a student with a disability must contact Disability Support Services (DSS), provide documentation, and participate in an interactive review process. If the documentation supports a request for reasonable accommodations, DSS will provide students with a Letter of Accommodations. Students who are approved for accommodations at Lehigh should share this letter and discuss their accommodations and learning needs with instructors as early in the semester as possible. For more information or to request services, please contact Disability Support Services in person in Williams Hall, Suite 301, via phone at 610-758-4152, via email at [email protected], or online at https://studentaffairs.lehigh.edu/disabilities.

The Principles of Our Equitable Community:

Lehigh University endorses The Principles of Our Equitable Community [http://www.lehigh.edu/~inprv/initiatives/PrinciplesEquity_Sheet_v2_032212.pdf]. We expect each member of this class to acknowledge and practice these Principles. Respect for each other and for differing viewpoints is a vital component of the learning environment inside and outside the classroom.

Lehigh University Policy on Harassment and Non-Discrimination

Lehigh University upholds The Principles of Our Equitable Community and is committed to providing an educational, working, co-curricular, social, and living environment for all students, staff, faculty, trustees, contract workers, and visitors that is free from harassment and discrimination on the basis of age, color, disability, gender identity or expression, genetic information, marital or familial status, national or ethnic origin, race, religion, sex, sexual orientation, or veteran status. Such harassment or discrimination is unacceptable behavior and will not be tolerated. The University strongly encourages (and, depending upon the circumstances, may require) students, faculty, staff or visitors who experience or witness harassment or discrimination, or have information about harassment or discrimination in University programs or activities, to immediately report such conduct.

If you have questions about Lehigh’s Policy on Harassment and Non-Discrimination or need to report harassment or discrimination, contact the Equal Opportunity Compliance Coordinator (Alumni Memorial Building / 610.758.3535 / [email protected])

COVID-19 Statements:

Per Lehigh’s current masking policy, which took effect August 2, both vaccinated and unvaccinated people are required to wear masks when indoors in public spaces, including classrooms. This policy will be strictly applied during lectures for in-person attendance. Any violations may result in ending the class and reporting the case to the administration. More details about Lehigh’s masking policy can be found in the following link: https://www2.lehigh.edu/news/72921-new-mask-requirement-cdc-guidance

Students who will join remotely should respect the in-classroom privacy of their instructors and fellow students by not taking screenshots or recording class sessions. The instructor may record some live Zoom sessions; however, any recorded live sessions will be shared only with students in the class and will be deleted at the end of the semester.

The latest updates of Lehigh’s COVID policies are available on: Latest Updates | Coronavirus (COVID-19) Information Center (lehigh.edu). Any changes in Lehigh’s policies will be reflected in this class’s policy. Important updates will be announced on CourseSite and/or Piazza.

Logistics

All assignments and the project will be submitted using Course Site.

To meet the challenge of teaching and learning during the COVID-19 pandemic, Lehigh instructors and students will be adopting new forms of instruction and interaction; following new guidelines around classroom behaviors; enhancing communications; and doing our best to be patient, flexible, and accommodating with each other. In remote synchronous meetings, students are expected to attend just as they would any other Lehigh class. Zoom classes work best when all students come to class ready to participate and follow the instructor’s guidelines regarding use of web-cameras. You may be asked to turn your camera on during active learning sessions in Zoom. If you have a strong preference not to do so, please contact your instructor to let them know.

Tentative Schedule

WEEK	TOPIC	READ
Week 1	Introduction to Parallelism, Concurrency, and Parallelization	SPP & slides
09/03	Non-graded assignment on C++ released (1 week submission deadline)
Week 2 to 5	Parallel Programming Patterns. GPU programming.	SPP book & slides & tutorial
Week 3 (09/17)	First Programming Assignment on code parallelization (2 weeks deadline)
Week 6	Memory Consistency Models	SPP book & slides & papers
Week 7	Consistency and progress conditions for semantic objects	AMP book & slides & papers
Week 7 to 10	Concurrent High-Performance Data Structures	AMP book & slides & papers
Week 8 (10/19)	Second Programming Assignment on concurrent programming (2 weeks deadline)
10/15	Project proposal due
Week 11	Software and Hardware Transactional Memory	AMP book & slides & papers
Week 12	Final project presentations