Can we exploit some apparent paradoxes of infinity to construct an injective function \(f(x)\) defined over some closed and bounded interval, that is differentiable at some point \(a\), and such that the inverse function \(f^{-1}\) fails to be continuous in \(f(a)\)?
Suppose for simplicity that \(a = 0\), \(f(0) = 0\), and \(f'(0) = 1\). Our task seems at first sight impossibile: for the function to be differentiable in \(0\) with \(f'(0)= 1\) the images of \(x\), for \(x\) close enough to \(0\), must somehow ‘squeeze’ around the straight line \(y=x\). And this in turn, together with the required injectivity, apparently implies that the inverse function is at least continuous in \(0\), if not differentiable.
Furthermore, one may be misled by the well known relationship
\begin{equation}\left( f^{-1}\right)'(f(x))=\frac1{f'(x)}\end{equation}
which, however, is valid if either we know already that the inverse function is differentiable in \(f(x)\) or if \(f(x)\) is continuously differentiable in a neighborhood of \(x\).
Let us dig deeper into the the hypothesis, though. We want \(f^{-1}\) not to be continuous in \(f(0) = 0\), meaning that there must exist a \(\varepsilon>0\) such that for any \(\delta >0\) there is a \(x\), with \(|x| < \delta\), for which \(|f^{-1}(x)|\geq \varepsilon\).
Translating this into a requirement on \(f\) yields the following. There exists \(\varepsilon > 0\) such that, for each \(\delta > 0\) we can find a \(|x|\geq \varepsilon\) that gives \(|f(x)| < \delta\). In other words, we want \(f(x)\) to attain arbitrarly small values, for ‘large enough’ values of \(x\), which is not in contradiction with the requirement on \(f\) we have in our hypotheses. The hardest part of the requirement is that of obtaining a function which has the above mentioned property and yet is injective.
At this point, the famous Infinite Hotel paradox comes into play. Instead of hosting more (actually infintely more) guests in a fully booked infinite hotel, we want to host a (doubly) infinite number of discontinuities in a function that otherwise would have a continuous inverse.
Recall in fact that if the function \(f\) is continuous in a neighborhood of \(0\) then it is either monotonic (in which case it has a continuous inverse) or non monotonic (in which case it fails to be injective). So our \(f\) needs to be continuous (and differentiable) in \(0\) but not in any neighborhood of \(0\).
We concentrate on the interval \([0,1]\). A function with the desired properties on the interval \([-1,1]\) can be easily obtained, for example, by a simmetry w.r.t. the origin. Let us start with the map \(g(x) = x\), which satisfies all the requirements save for the fact that its inverse is continuous in \(0\). In order to preserve differentiability in \(0\) we force the graph of the function to lie between \(y=x\) and \begin{eqnarray} h(x) = -x^2+x.\end{eqnarray}
We introduce our first ‘hole’ in \(x=\frac13\) by letting \[g\left(\frac13\right)= h\left(\frac13\right)=\frac29.\]
In this way, \(\frac13\) will be the image of some larger value of \(x\). However, in order to preserve injectivity, we need to free the ‘room’ now occupied by \(\frac29\), which forces us to set \[g\left(\frac29\right) = h\left(\frac29\right) = h\circ h\left(\frac13\right) = \frac{14}{81}.\]
We can proceed indefinitely, towards \(0\), every time letting \[g\left(h^{(n)}\left(\frac13\right)\right) = h^{(n+1)}\left(\frac13\right),\]
where \[h^{(n)} (x) = \underbrace{h\circ h \circ \cdots \circ h}_n (x)\] for \(n>0\) and \(h^{(0)}(x) = x\).
What we have done so far guarantees that \(g\) is still injective for \(0\leq x< \frac13\). We can now fill the ‘room’ left by \(\frac13\), e.g., by letting \[g\left(\frac23\right) = \frac13.\] We can picture the situation below, where the yellow dots mark some of the discontinuities we created. The red diamond shows the problem we have to solve now!
The partial simmetry of the scenario depicted above, however, might suggest a workaround. Let us draw the parabola \[x=y^2-y+1,\] whose graph is the one simmetric to the graph of \(h(x)\), with respect to the line \[r: y=-x+1,\]also depicted above. Thus, we can shift the diamond on the parabola, by letting \[g\left(\frac79\right) = \frac23.\]Now of course we have the same problem in \(x = \frac79\)… But we can proceed simmetrically with what we have done near \(0\). The result is shown below.
The function we have at this point is injective, but still has a continuous inverse in \(0\). However, a new iteration can be made starting with \(\frac15\). Using inverses of primes will allow us to avoid ‘interferences’ between the ‘holes’ created in the previous iteration! In fact, if we let \(p_k\) be the sequence of primes, then for \(n=0,1,2,\dots\) and each \(p_k\), with \(k=1,2,\dots\), \[h^{(n)}\left(\frac1{p_k}\right) = \frac{m}{p_k^\ell}\] for some positive integers \(m\) and \(\ell\). Hence for \(n\), \(k\), and \(h\) not all equal, \[h^{(n)}\left(\frac1{p_k}\right) \neq h^{(k)}\left(\frac1{p_h}\right).\]
A formal definition of \(g(x)\) can be obtained by first defining, for \(0\leq x \leq \frac12\), \[g_1(x)=
\begin{cases}
h(x) & \left(\mbox{if}\ \ x=h^{(n)}\left(\frac1{p_k}\right),\ \ \mbox{for some}\ k>1\ \mbox{and some}\ n\geq 0\right)\\
x & (\mbox{otherwise}).
\end{cases}\]
Note that \(g_1(x)\) is injective (what is its range?). Therefore we can use the simmetry with respect to line \(r\) to define, in \(0\leq x \leq 1\),
\[g(x) = \begin{cases}
g_1(x) & \left(0\leq x\leq \frac12\right)\\
\frac1{p_k} & \left(\mbox{if}\ \ x=1-\frac1{p_k},\ \ \mbox{for some} \ k>1\right)\\
1-g_1^{-1}(1-x) & (\mbox{otherwise}).
\end{cases}\]
The sketch of the graph of \(g(x)\), after a few iterations on \(k\), is shown in the figure below, where differently colored dots refer to different primes.
If we now let \[f(x) = \begin{cases} g(x) & (0\leq x \leq 1)\\ -g(-x) & (-1\leq x <0).\end{cases}\]we have a function with the desired properties.
As a final note, observe that \[\lim_{k\to \infty} \frac1{p_k} = 0,\]and that, for each \(k\),\[\lim_{n\to \infty} h^{(n)}\left(\frac1{p_k}\right) = 0.\] As a consequence, any neighborhood of zero contains all but a finite number of elements \(x\) such that \(g(x) = h^{(n)}\left(\frac1{p_k}\right)\) for any \(n\) and \(k\). Therefore, if \(\overline x \neq 0\) is a point in the domain of \(g\) such that \(g(\overline x) = \overline x\), then there exists \(\delta_{\overline x}>0\) such that \(g(x) = x\) for all \( x\in [\overline x-\delta_{\overline x}, \overline x + \delta_{\overline x}]\). We conclude that \(g(x)\) is continuous and differentiable at all points \(\overline x\) such that \(g(\overline x) = \overline x\), that this everywhere except a countable number of points.