vissza a tantárgylistához   nyomtatható verzió    

    Computer Vision Applications for Digital Cinema

    A tantárgy neve magyarul / Name of the subject in Hungarian: Gépi látás alkalmazása a digitális filmkészítésben

    Last updated: 2014. február 27.

    Budapest University of Technology and Economics
    Faculty of Electrical Engineering and Informatics

    Mérnök informatikus szak

    BSc képzés
    Course ID Semester Assessment Credit Tantárgyfélév
    VISZA029   2/0/1/f 4  
    3. Course coordinator and department Dr. Wiener Gábor,
    4. Instructors

    Name

     

    Position

     

    Department

     

    Gergely VASS

     

    Lecturer

     

    Department of Computer Science and Information Theory

     

    Gábor WIENER

     

    Assoc. professor

     

    Department of Computer Science and Information Theory

     

    5. Required knowledge

    A knowledge of linear algebra foundations and programming experience (in any language such as C, C++, objective C, Java, Pascal) are required.

    In addition students should have a passion for movies and science.

    7. Objectives, learning outcomes and obtained knowledge

    The course focuses on current issues in digital post-production and the corresponding solutions utilizing computer vision algorithms. Most of the challenges in focus are related to merging real and computer generated images.

     

    The primary aim of the course is to educate and inspire at the same time. By understanding how essential mathematical techniques – such as solving a set of linear equations – are utilized in the high-end filmmaking process, one soon develops a new appreciation for software research and development. The “true”, practical value of mathematics and computer science introduced in this and other courses is revealed through numerous exciting examples from real projects and innovative software solutions.

    8. Synopsis

    The first studio visit will be a guided tour in the Colorfront facilities, introducing the digital workflows and custom softwares used in the studio. Students will see the most advanced digital projection rooms with advanced color grading equipment, high resolution display devices, sound mixing boards and digital cinema cameras, to name a few things. Not only the full digital post-production pipeline will be introduced, but also the adventurous story of developing the first software color corrector for which the team won the Oscar award for Scientific and Technical Achievement. 

    The academic material begins with image processing basics. The computational costs, advantages and limitations, and application of different classes of image filters will be discussed. Students will learn how to smooth and sharpen images, and how to compute the image gradients. Based on these rather simple algorithms robust feature detectors can be built, that identify interest points in images. Most high-level computer vision tools are based on these interest points, which encode all necessary information from the original image.

    Students will receive the skeleton code for the programming assignments as well. Complete C++ classes are implemented with many useful helper functions, making the development process very similar to what Colorfront developers do on a daily basis. Students customize the code and implement image processing functions to get familiar with the concepts learnt in class.

    To lay the mathematical foundations needed for the class one session is dedicated to an overview of linear algebra foundations from a software developer’s point of view: a “practical guide” to analyzing and solving large sets of linear equations. The majority of parameter estimation problems in computer vision are based on such linear constructions. Examples include aligning multiple overlapping photographs to form a single panoramic image, or tracking a moving object in a sequence of frames.

                While not essential to post-production, object and class recognition is a very important field of computer vision. Understanding the various approaches also helps students get familiar with general concepts such as distance metrics, principal component analysis and linear subspaces. By focusing on real problems and their solutions – instead of the often overwhelming theory – students will grasp the power of such mathematical tools. 

    During the second part of the course feature tracking and parameter estimation are the main topic. Students will understand and implement feature detection and tracking methods, from which basic motion information is reconstructed. This is the main idea behind the most powerful post-production tool: camera tracking, or matchmoving. This process is matching the motion of a virtual camera to the real camera used to capture the image sequence, which allows artists to seamlessly insert computer graphics into live-action footage with correct position, scale and orientation. While this very complex problem may seem unique to computer vision, the challenges are common in computer science and it is worth looking at the innovative solutions used in the field. 

    To compare the implementation of feature tracking in the homework assignments and a real software product there is a last studio visit focusing on the application of the theory presented throughout the course. The deep walk-though of the stereoscopic feature set of the Emmy Award winning product will put the course’s material in perspective, and students will understand how these techniques are utilized to achieve certain artistic goals in modern S3D movies. 

    9. Method of instruction

    The course is lecture-oriented with assigned readings and team-oriented programming and image manipulation projects as homework. There will be two visits to the Colorfront studio, one at the beginning and one at the end of the course.

    10. Assessment

    Students will be evaluated based on assigned homework and final written exam.

    13. References, textbooks and resources

    Richard Hartley, Andrew Zisserman: Multiple View Geometry in Computer Vision, Cambridge Univ. Press, 2003. 

     Richard Szeliski: Computer Vision: Algorithms and Applications, Springer, 2010.
    http://research.microsoft.com/en-us/um/people/szeliski/Book/

    14. Required learning hours and assignment
     Number of contact hours         36
     Preparation to the classes        -
     Preparation to the tests           -
     Homework                             52
     Consultation for homework       10
     Assigned reading                    12
     Preparation to the exam          10







    Total                                         120
    15. Syllabus prepared by

    Name

     

    Position

     

    Department

     

    Gergely VASS

     

    Lecturer

     

    Department of Computer Science and Information Theory