Shearer's inequality

Shearer's inequality is an inequality in information theory relating the entropy a set of variables to the entropies of a collection of subsets. It is named for mathematician James Shearer.

Concretely, it states that if X₁, ..., X_d are random variables and S₁, ..., S_n are subsets of {1, 2, ..., d} such that every integer between 1 and d lies in at least r of these subsets, then

${\displaystyle H[(X_{1},\dots ,X_{d})]\leq {\frac {1}{r}}\sum _{i=1}^{n}H[(X_{j})_{j\in S_{i}}]}$

where ${\displaystyle H}$ is entropy and ${\displaystyle (X_{j})_{j\in S_{i}}}$ is the Cartesian product of random variables ${\displaystyle X_{j}}$ with indices j in ${\displaystyle S_{i}}$. [1]