198:334 Introduction to Imaging and Multimedia Fall 2023

Instructor: Dr. Ahmed Elgammal -- email: elgammal a-t cs.rutgers.edu

Office hours: Refer to class canvas page for uptodate office hours info

Email policy: please include “CS334” in the subject line.  Ignoring this in the subject line might results in email being missed.  Student should expect to receive a response within 24 hours on weekday

Regular class time: (All classes will be in person on campus)

Lectures: Mondays and Thursday 12:10- 1:30 PM – PH 115

Recitations:    Sec 01 Thu 4:05-5:00 PM SEC-203

Sec 02 Thu 5:55-6:50 PM HLL- 116

There will be no recording of lectures or recitations. Physical attendance is the only option.

Class Web page:  Canvas page

TECHNOLOGY REQUIREMENTS:  Access to canvas for course materials. Access to computer labs

Course Goals

The  aim  of  CS334  is  to  introduce  fundamental  techniques  and  concepts  used  in  computational  imaging  and multimedia. Upon completion of this course, a successful student should be able to design and implement programs that deal with image, video, and audio data.

Description:

This is a basic undergraduate-level class that covers the fundamentals of image processing, computer vision and multimedia computing. The students learn about the basics of image, video, and audio formation, representations, and processing, the basics of multimedia compression and representation. The students will be exposed to dealing with image and video data through programming assignments using Java and Python.

Recommended Background:

Linear algebra, basic probability and statistics.  Java and Python (you don’t need to know Python in advance, but your will need to pick it up quickly early in the course. We will provide help with that)

Pre-Requisites:

.     01:198:112 OR 14:332:351   (Data Structures)

.     01:198:206 OR 14:332:226 OR 01:640:477   (Discrete Mathematics and Probability)

.     01:640:250  (Linear Algebra)

Topics:

.   Introduction to Multimedia: Historical overview, multimedia representations.

.   Multimedia Digitization with digital camera as an example. Standard image formats. Colors in images and videos.

.   Image Computing: Point Operations, Filters, Binary image analysis: The basics of processing 2D images, thresholding, convolution, edge and corner detection, mathematical morphology, and shape descriptors.    Application: implementation of a simple Optical Character Recognition (OCR) System.

.   Object detection and recognition in images: intro to deep learning models using convolution neural networks

.   Fourier Transform: Understanding frequency components of signals, focusing on imaging.

.   Multimedia compression basics: Lossless Compression: Variable length coding, Dictionary based coding. Basics for Lossy Compression: Fourier Transform, Discrete Cosine Transform. Application to image

compression (JPEG compression), Video compression (MPEGs), Audio compression (MP3)

.   Multimedia at the age of AI: embedding of text, images, and other media and their applications (text-to-image, text-to-speech, …)

Programming Assignments:

Course assignments will be using Java, and/or Python. We will use ImageJ, which is an image processing library using Java. We will also use imaging libraries in Python.

Textbooks

.     W. Burger & M. Burge “Digital Image Processing: An algorithmic introduction using Java”, Springer    - Second Edition ISBN 978- 1447166832

Available online through Rutgers Libraries

https://link-springer-com.proxy.libraries.rutgers.edu/book/10.1007%2F978-1-4471-6684-9

.      Ze-Nian Li, Mark S. Drew, Jiangchuan Liu “Fundamentals of Multimedia”, Springer 2014, Second Edition ISBN 978-3-319-05289-2

Available online through Rutgers Libraries

https://link-springer-com.proxy.libraries.rutgers.edu/book/10.1007%2F978-3-319-05290-8

.     Optional:  P. Havaldar and G. Medioni “Multimedia Systems – Algorithms, Standards and Industry  Practices”, Cengage Learning –  978- 1-4188-3594- 1   (recommended for some topics – not required)

Course Load

. Homework/programming assignments and small projects: (55%) ~4 assignments. All assignments are

equally weighted

. Quizzes: ~6 quizzes (15%) In class. All quizzes are equally weighted.

. Midterm: in class  (15%), in late October – early November

. Final: Online (15%)

. Optional - Extra credit Presentation: 5% can be achieved by researching and presenting a relevant

technology review topic – individuals or groups of 2.  Announcement will be made on how to apply.

Tentative Class Calendar (subject to change)

FM: Fundamentals of Multimedia textbook

DIP: Digital Image Processing text book