Contents
The lecture provides an introduction to the fundamentals of Shannon information theory. It develops both an intuitive understanding of communication systems and the mathematical foundations underlying reliable information transmission.
The goal is to enable students to derive the main results of information theory, including the fundamental limits of lossless source compression (source coding) and the maximum achievable data rates for reliable communication over channels (channel coding). The lecture also introduces the essential mathematical tools and concepts required for these derivations, including information measures such as entropy, mutual information, and channel capacity, as well as their properties and related concepts such as typical sequences.
- Basics from probability theory
- Event, probability, random variable, random vector, stochastic process, convergence of random series, convergence theorems
- Basics from information theory
- Measures for discrete random varaibles: entropy, conditional entropy, relativ entropy, mutual information, conditional mutual information, inequalities
- Measures for continous random variables: differential entropy, conditional differential entropy, relative entropy, mutual information, inequalities
- Measure for random series
- Typical sequences and asymptotic equipartition property
- Source coding
- Definition and properties
- Source coding for discrete memoryless sources (fixed and variable-length)
- Selected source codes: Shannon-type, Huffman
- Data transmission and channel capacity
- Discrete memoryless channel: channel coding theorem
- Discrete memoryless channel with state: channel capacities
- Gaussian channel: model and channel coding theorem
- Bandlimited Gaussian channel, vector valued channels
Lecturer: Prof. Eduard Jorswieck
Assistant: Jyun-Sian Wu, Pin-Hsun Lin
