Faculty: Faculty of Engineering
Department(s): Computer Engineering
Number of Students: 4
Course: Information Theory
Weekly hours: Theory: 2 Exercises: 1
ECTS Credits: 5
Semester: Fall
Lecture Schedules:
Thursday: 13:00 - 15:45 (online)
Lecturer: Dr. Hiqmet Kamberaj
Room Number:
Phone Number of the lecturer:
E-mail address of the lecturer: km.ude.ubi|jarebmakh#km.ude.ubi|jarebmakh
Course Objectives:
The aim of the course is to give an introduction to Information Theory for students of computer science, communication theory, and statistics in senior and first-year graduate-level courses in Information Theory. In this course, students will learn about entropy and mutual information, the nature of the algebraic structure of information-theoretic, and problems in data compression and communications.
Available topics for Master Diploma Thesis
- Symbolic Mutual Information for Estimating Nonlinear High-order Correlated Fluctuations of Different Dynamical Systems.
- Enhancement of Transition Path Sampling using Biased Potential Energy Functions.
- Other topics are available upon request.
| Skill outcomes | Necessary ( + ) Not Necessary ( –) |
|---|---|
| Written communication skills | + |
| Oral communication skills | + |
| Computer skills | + |
| Working in laboratory | + |
| Working team | + |
| Preparing projects | + |
| Knowledge of foreign language | + |
| Scientific and professional literature analysis | + |
| Problem solving skills | + |
| Management skills | + |
| Presentation skills | + |
Course Textbooks:
- H. Kamberaj, Lecture Notes in Information Theory, International Balkan University, 2020.
Additional Literature:
- T. M. Cover and J. A. Thomas, Elements of Information Theory, Second Edition, John Wiley & Sons, 2006.
- D.J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, 2003.
Study Plan —- International Balkan University - Academic Calendar
| Weeks | Topics |
|---|---|
| 1 | Introduction. Entropy. |
| 2 | Relative entropy. Mutual information. Jansen's inequality |
| 3 | Data-processing theorem. Fanos's inequality. Asymptotic equipartition Property. |
| 4 | Entropy rate of a Stochastic Process. |
| 5 | Data compression. |
| 6 | Channel capacity. |
| 7 | Differential entropy. |
| 8 | Relative entropy and mutual information. |
| 9 | Gaussian Channel. |
| 10 | Review. |
| 11 | Preparatory week. |
| 12 | Final exam week. |
| 13 | Preparatory week. |
| 14 | Preparatory week. |
| 15 | Make up 1 Exam. |
| In June | Make up 2 Exam. |
Attendance
Students are obliged to attend at least 60% of lectures.
Exams
- Achieved success in a course shall be evaluated through a final exam and seminar work.
- The topic of the seminar work for each course shall be chosen in the first two weeks of the lecture. The seminar work shall be delivered in the last (10th) week of the lecture. It shall consist of 7,500-10,000 words (tables and graphs are excluded).
- The maximum number of credit points (a) for the final exam is 60% (b) for seminar work is 40% of the total number of points.
- The student who has not passed the exam may enter the exam 2 (two) more times during the make-up exam sessions.
Student workload:
Please calculate the “Total Student Workload” and then distribute that figure to the different engagements. For calculating the Total Student Workload, we multiply the course ECTS credits with standard figure 30. (ECTS Credit: 5) x 30 = 150 hours.
| Activities | Hours |
|---|---|
| Lecture hours for 10 weeks: | 20 |
| Laboratory and class exercises for 10 weeks: | 10 |
| Student Mentoring for 10 weeks: | - |
| Consultation for 10 weeks: | 4 |
| Exam preparations and exam hours (final, Makeup): | 20 |
| Individual reading work for 10 weeks (Reading assignments/expectations for reading and comprehension is 5 pages per hour. Example: If a book 300 pages, total Individual reading work for 10 weeks is 300:5 = 60 hours. | 58 |
| Homework and work practice for 10 weeks: | 38 |
| Preparation of diploma work, for 10 weeks: | - |
