INFORMATION THEORY AND CODING Ch. Ravi Kumar Assistant Professor, Department of E.C.E, Sir C. R. Reddy College of Engineering, Eluru, India If the rate of information from the source does not exceeds the capacity of a communication channel; then there exists a coding techniques ,such that the information can be transmitted over the channel with an arbitrary small frequency of errors , despite the presence of noise. 1. Measure of information : The outputs from an information source such as data, speech can be regarded as a random process, the information theory defines the self information of an event x i with probability P(x i ) self information : I i = - log 2 P i = log 2 bits This measure of information is large/small for an event of lower/higher probability. A message that there was or will