Information Theory

A tantárgy neve magyarul / Name of the subject in Hungarian: Információelmélet

Last updated: 2018. július 8.

Budapest University of Technology and Economics
Faculty of Electrical Engineering and Informatics 		Engineering Information Theory, MSc, Branching Common Subject
Course ID Semester Assessment Credit Tantárgyfélév
VISZMA03 1,2 3/0/0/f 4  
3. Course coordinator and department Dr. Pintér Márta,
4. Instructors Gabor Simonyi
5. Required knowledge Probability Theory
6. Pre-requisites
Kötelező:
NEM ( TárgyEredmény( "BMEVISZM101" , "jegy" , _ ) >= 2
VAGY
TárgyEredmény("BMEVISZM101", "FELVETEL", AktualisFelev()) > 0
VAGY
TárgyEredmény( "BMEVISZMA13", "jegy" , _ ) >= 2
VAGY
TárgyEredmény("BMEVISZMA13", "FELVETEL", AktualisFelev()) > 0)

A fenti forma a Neptun sajátja, ezen technikai okokból nem változtattunk.

A kötelező előtanulmányi rend az adott szak honlapján és képzési programjában található.

Ajánlott:
Probability Theory
7. Objectives, learning outcomes and obtained knowledge The course deals with the theoretical problems arising during transfer
and storage of information. The theoretical limits of data compression
and reliable information transmission are presented. Basic properties
of Shannon's information measures are covered and several data
compression techniques are taught. Course topics include the main
principles of channel coding along with basic examples of situations
when such coding is required.

Students completing the course are supposed to

(1) know the theoretical limits of efficiency of variable length source coding

(2) know the main codes realizing the above limits

(3) be acquainted with the main principles of lossy source coding

(4) develop a basic understanding of the main concepts of classical
information theory

(5) be able to rightly model situations when the task is information
transmission in a noisy environment.
8. Synopsis 1. Variable length source coding
Unique decodability, prefix coding

2. McMillan's theorem and Kraft's theorem

3. Jensen's inequality
The entropy function and its main properties

4. Shannon-Fano coding
Huffman coding

5. Lempel-Ziv type algorithms

6. The entropy of a source, Markov source
Conditional entropy and its properties

7. Mutual information and its properties

8. Quantization

9. Lloyd-Max algorithm

10. The discrete memoryless channel model

11. Channel capacity
Fano's inequality

12. Converse of the channel coding theorem
Channel coding theorem

13. Basic principles of error correction, Hamming codes

14. Zero-error codes.

9. Method of instruction 3 lectures per week
10. Assessment There are 2 midterm tests during the semester. To complete the
course with a valid grade 40% of the total score should be achieved on
both of the  midterms. If this requirement is met, the course
grade is calculated by averaging the results of the three midterms
with equal weights.

In the exam period: ---
11. Recaps There will be a make up test for each of the three midterms during the
semester. One more make up test can be written on the week right after
the semester in case one (and only one) midterm is still below 40%.
12. Consultations Upon appointment.
13. References, textbooks and resources Cover - Thomas: Elements of Information Theory, Wiley, 2006.
14. Required learning hours and assignment
Kontakt óra42
Félévközi készülés órákra
Felkészülés zárthelyire78
Házi feladat elkészítése
Kijelölt írásos tananyag elsajátítása
Vizsgafelkészülés
Összesen120
15. Syllabus prepared by Gabor Simonyi