Extreme value theorem

Statement   If ${\displaystyle f(x)}$ is continuous on ${\displaystyle [a,b]}$ then it is bounded on ${\displaystyle [a,b]}$

Suppose the function ${\displaystyle f}$ is not bounded above on the interval ${\displaystyle [a,b]}$. Then, for every natural number ${\displaystyle n}$, there exists an ${\displaystyle x_{n}\in [a,b]}$ such that ${\displaystyle f(x_{n})>n}$. This defines a sequence ${\displaystyle (x_{n})_{n\in \mathbb {N} }}$. Because ${\displaystyle [a,b]}$ is bounded, the Bolzano–Weierstrass theorem implies that there exists a convergent subsequence ${\displaystyle (x_{n_{k}})_{k\in \mathbb {N} }}$ of ${\displaystyle ({x_{n}})}$. Denote its limit by ${\displaystyle x}$. As ${\displaystyle [a,b]}$ is closed, it contains ${\displaystyle x}$. Because ${\displaystyle f}$ is continuous at ${\displaystyle x}$, we know that ${\displaystyle f(x_{{n}_{k}})}$ converges to the real number ${\displaystyle f(x)}$ (as ${\displaystyle f}$ is sequentially continuous at ${\displaystyle x}$). But ${\displaystyle f(x_{{n}_{k}})>n_{k}\geq k}$ for every ${\displaystyle k}$, which implies that ${\displaystyle f(x_{{n}_{k}})}$ diverges to ${\displaystyle +\infty }$, a contradiction. Therefore, ${\displaystyle f}$ is bounded above on ${\displaystyle [a,b]}$. ${\displaystyle \Box }$

Statement   If ${\displaystyle f(x)}$ is continuous on ${\displaystyle [a,b]}$ then it is bounded on ${\displaystyle [a,b]}$

Proof    Consider the set ${\displaystyle B}$ of points ${\displaystyle x}$ in ${\displaystyle [a,b]}$ such that ${\displaystyle f(x)}$ is bounded on ${\displaystyle [a,x]}$. We note that ${\displaystyle a}$ is one such point, for ${\displaystyle f(x)}$ is bounded on ${\displaystyle [a,a]}$ by the value ${\displaystyle f(a)}$. If ${\displaystyle e>a}$ is another point, then all points between ${\displaystyle a}$ and ${\displaystyle e}$ also belong to ${\displaystyle B}$. In other words ${\displaystyle B}$ is an interval closed at its left end by ${\displaystyle a}$.

Now ${\displaystyle f}$ is continuous on the right at ${\displaystyle a}$, hence there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(a)|<1}$ for all ${\displaystyle x}$ in ${\displaystyle [a,a+\delta ]}$. Thus ${\displaystyle f}$ is bounded by ${\displaystyle f(a)-1}$ and ${\displaystyle f(a)+1}$ on the interval ${\displaystyle [a,a+\delta ]}$ so that all these points belong to ${\displaystyle B}$.

So far, we know that ${\displaystyle B}$ is an interval of non-zero length, closed at its left end by ${\displaystyle a}$.

Next, ${\displaystyle B}$ is bounded above by ${\displaystyle b}$. Hence the set ${\displaystyle B}$ has a supremum in ${\displaystyle [a,b]}$ ; let us call it ${\displaystyle s}$. From the non-zero length of ${\displaystyle B}$ we can deduce that ${\displaystyle s>a}$.

Suppose ${\displaystyle s<b}$. Now ${\displaystyle f}$ is continuous at ${\displaystyle s}$, hence there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(s)|<1}$ for all ${\displaystyle x}$ in ${\displaystyle [s-\delta ,s+\delta ]}$ so that ${\displaystyle f}$ is bounded on this interval. But it follows from the supremacy of ${\displaystyle s}$ that there exists a point belonging to ${\displaystyle B}$, ${\displaystyle e}$ say, which is greater than ${\displaystyle s-\delta /2}$. Thus ${\displaystyle f}$ is bounded on ${\displaystyle [a,e]}$ which overlaps ${\displaystyle [s-\delta ,s+\delta ]}$ so that ${\displaystyle f}$ is bounded on ${\displaystyle [a,s+\delta ]}$. This however contradicts the supremacy of ${\displaystyle s}$.

We must therefore have ${\displaystyle s=b}$. Now ${\displaystyle f}$ is continuous on the left at ${\displaystyle s}$, hence there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(s)|<1}$ for all ${\displaystyle x}$ in ${\displaystyle [s-\delta ,s]}$ so that ${\displaystyle f}$ is bounded on this interval. But it follows from the supremacy of ${\displaystyle s}$ that there exists a point belonging to ${\displaystyle B}$, ${\displaystyle e}$ say, which is greater than ${\displaystyle s-\delta /2}$. Thus ${\displaystyle f}$ is bounded on ${\displaystyle [a,e]}$ which overlaps ${\displaystyle [s-\delta ,s]}$ so that ${\displaystyle f}$ is bounded on ${\displaystyle [a,s]}$.  ∎

By the boundedness theorem, f is bounded from above, hence, by the Dedekind-completeness of the real numbers, the least upper bound (supremum) M of f exists. It is necessary to find a point d in [a,b] such that M = f(d). Let n be a natural number. As M is the least upper bound, M – 1/n is not an upper bound for f. Therefore, there exists d_n in [a,b] so that M – 1/n < f(d_n). This defines a sequence {d_n}. Since M is an upper bound for f, we have M – 1/n < f(d_n) ≤ M for all n. Therefore, the sequence {f(d_n)} converges to M.

The Bolzano–Weierstrass theorem tells us that there exists a subsequence {${\displaystyle d_{n_{k}}}$}, which converges to some d and, as [a,b] is closed, d is in [a,b]. Since f is continuous at d, the sequence {f(${\displaystyle d_{n_{k}}}$)} converges to f(d). But {f(d_nk)} is a subsequence of {f(d_n)} that converges to M, so M = f(d). Therefore, f attains its supremum M at d. ∎

The set {y ∈ R : y = f(x) for some x ∈ [a,b]} is a bounded set. Hence, its least upper bound exists by least upper bound property of the real numbers. Let M = sup(f(x)) on [a, b]. If there is no point x on [a, b] so that f(x) = M then f(x) < M on [a, b]. Therefore, 1/(M − f(x)) is continuous on [a, b].

However, to every positive number ε, there is always some x in [a, b] such that M − f(x) < ε because M is the least upper bound. Hence, 1/(M − f(x)) > 1/ε, which means that 1/(M − f(x)) is not bounded. Since every continuous function on a [a, b] is bounded, this contradicts the conclusion that 1/(M − f(x)) was continuous on [a, b]. Therefore, there must be a point x in [a, b] such that f(x) = M. ∎

In the setting of non-standard calculus, let N  be an infinite hyperinteger. The interval [0, 1] has a natural hyperreal extension. Consider its partition into N subintervals of equal infinitesimal length 1/N, with partition points x_i = i /N as i "runs" from 0 to N. The function ƒ  is also naturally extended to a function ƒ* defined on the hyperreals between 0 and 1. Note that in the standard setting (when N  is finite), a point with the maximal value of ƒ can always be chosen among the N+1 points x_i, by induction. Hence, by the transfer principle, there is a hyperinteger i₀ such that 0 ≤ i₀ ≤ N and ${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}$  for all i = 0, …, N. Consider the real point

${\displaystyle c=\mathbf {st} (x_{i_{0}})}$

where st is the standard part function. An arbitrary real point x lies in a suitable sub-interval of the partition, namely ${\displaystyle x\in [x_{i},x_{i+1}]}$, so that  st(x_i) = x. Applying st to the inequality ${\displaystyle f^{*}(x_{i_{0}})\geq f^{*}(x_{i})}$, we obtain ${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))\geq \mathbf {st} (f^{*}(x_{i}))}$. By continuity of ƒ  we have

${\displaystyle \mathbf {st} (f^{*}(x_{i_{0}}))=f(\mathbf {st} (x_{i_{0}}))=f(c)}$.

Hence ƒ(c) ≥ ƒ(x), for all real x, proving c to be a maximum of ƒ.[3]

Statement      If ${\displaystyle f(x)}$ is continuous on ${\displaystyle [a,b]}$ then it attains its supremum on ${\displaystyle [a,b]}$

Proof      By the Boundedness Theorem, ${\displaystyle f(x)}$ is bounded above on ${\displaystyle [a,b]}$ and by the completeness property of the real numbers has a supremum in ${\displaystyle [a,b]}$. Let us call it ${\displaystyle M}$, or ${\displaystyle M[a,b]}$. It is clear that the restriction of ${\displaystyle f}$ to the subinterval ${\displaystyle [a,x]}$ where ${\displaystyle x\leq b}$ has a supremum ${\displaystyle M[a,x]}$ which is less than or equal to ${\displaystyle M}$, and that ${\displaystyle M[a,x]}$ increases from ${\displaystyle f(a)}$ to ${\displaystyle M}$ as ${\displaystyle x}$ increases from ${\displaystyle a}$ to ${\displaystyle b}$.

If ${\displaystyle f(a)=M}$ then we are done. Suppose therefore that ${\displaystyle f(a)<M}$ and let ${\displaystyle d=M-f(a)}$. Consider the set ${\displaystyle L}$ of points ${\displaystyle x}$ in ${\displaystyle [a,b]}$ such that ${\displaystyle M[a,x]<M}$.

Clearly ${\displaystyle a\in L}$ ; moreover if ${\displaystyle e>a}$ is another point in ${\displaystyle L}$ then all points between ${\displaystyle a}$ and ${\displaystyle e}$ also belong to ${\displaystyle L}$ because ${\displaystyle M[a,x]}$ is monotonic increasing. Hence ${\displaystyle L}$ is a non-empty interval, closed at its left end by ${\displaystyle a}$.

Now ${\displaystyle f}$ is continuous on the right at ${\displaystyle a}$, hence there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(a)|<d/2}$ for all ${\displaystyle x}$ in ${\displaystyle [a,a+\delta ]}$. Thus ${\displaystyle f}$ is less than ${\displaystyle M-d/2}$ on the interval ${\displaystyle [a,a+\delta ]}$ so that all these points belong to ${\displaystyle L}$.

Next, ${\displaystyle L}$ is bounded above by ${\displaystyle b}$ and has therefore a supremum in ${\displaystyle [a,b]}$ : let us call it ${\displaystyle s}$. We see from the above that ${\displaystyle s>a}$. We will show that ${\displaystyle s}$ is the point we are seeking i.e. the point where ${\displaystyle f}$ attains its supremum, or in other words ${\displaystyle f(s)=M}$.

Suppose the contrary viz. ${\displaystyle f(s)<M}$. Let ${\displaystyle d=M-f(s)}$ and consider the following two cases :

(1)    ${\displaystyle s<b}$.   As ${\displaystyle f}$ is continuous at ${\displaystyle s}$, there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(s)|<d/2}$ for all ${\displaystyle x}$ in ${\displaystyle [s-\delta ,s+\delta ]}$. This means that ${\displaystyle f}$ is less than ${\displaystyle M-d/2}$ on the interval ${\displaystyle [s-\delta ,s+\delta ]}$. But it follows from the supremacy of ${\displaystyle s}$ that there exists a point, ${\displaystyle e}$ say, belonging to ${\displaystyle L}$ which is greater than ${\displaystyle s-\delta }$. By the definition of ${\displaystyle L}$, ${\displaystyle M[a,e]<M}$. Let ${\displaystyle d_{1}=M-M[a,e]}$ then for all ${\displaystyle x}$ in ${\displaystyle [a,e]}$, ${\displaystyle f(x)\leq M-d_{1}}$. Taking ${\displaystyle d_{2}}$ to be the minimum of ${\displaystyle d/2}$ and ${\displaystyle d_{1}}$, we have ${\displaystyle f(x)\leq M-d_{2}}$ for all ${\displaystyle x}$ in ${\displaystyle [a,s+\delta ]}$.

Hence ${\displaystyle M[a,s+\delta ]<M}$ so that ${\displaystyle s+\delta \in L}$. This however contradicts the supremacy of ${\displaystyle s}$ and completes the proof.

(2)    ${\displaystyle s=b}$.   As ${\displaystyle f}$ is continuous on the left at ${\displaystyle s}$, there exists ${\displaystyle \delta >0}$ such that ${\displaystyle |f(x)-f(s)|<d/2}$ for all ${\displaystyle x}$ in ${\displaystyle [s-\delta ,s]}$. This means that ${\displaystyle f}$ is less than ${\displaystyle M-d/2}$ on the interval ${\displaystyle [s-\delta ,s]}$. But it follows from the supremacy of ${\displaystyle s}$ that there exists a point, ${\displaystyle e}$ say, belonging to ${\displaystyle L}$ which is greater than ${\displaystyle s-\delta }$. By the definition of ${\displaystyle L}$, ${\displaystyle M[a,e]<M}$. Let ${\displaystyle d_{1}=M-M[a,e]}$ then for all ${\displaystyle x}$ in ${\displaystyle [a,e]}$, ${\displaystyle f(x)\leq M-d_{1}}$. Taking ${\displaystyle d_{2}}$ to be the minimum of ${\displaystyle d/2}$ and ${\displaystyle d_{1}}$, we have ${\displaystyle f(x)\leq M-d_{2}}$ for all ${\displaystyle x}$ in ${\displaystyle [a,b]}$. This contradicts the supremacy of ${\displaystyle M}$ and completes the proof. ∎