ČSN IEC 60050-171 - Mezinárodní elektrotechnický slovník – Část 171: Digitální technologie – Základní pojmy

171-07-15 entropy

$H={\displaystyle \sum _{i=1}^{n}p(x_i)\cdot I(x_i)}={\displaystyle \sum _{i=1}^{n}p(x_i)\cdot \log \left(\frac{1}{p(x_i)}\right)}$

where $X=\left\{x_1\mathrm{,}\dots ,x_n\right\}$ is the set of events $x_i\left(i=\mathrm{1,}\dots ,n\right)$, $I(x_i)$ are their information contents and $p(x_i)$ the probabilities of their occurrences, subject to ${\displaystyle \sum _{i=1}^{n}p(x_i)=1}$

EXAMPLE Let $\left\{a,b,c\right\}$ be a set of three events and let $p(a)=\mathrm{0,5}$, $p(b)=\mathrm{0,25}$ and $p(c)=\mathrm{0,25}$ be the probabilities of their occurrences. The entropy of this set is $H(X)=p(a)\cdot I(a)+p(b)\cdot I(b)+p(c)\cdot I(c)=\mathrm{1,5}\mathrm{Sh}$.