Improving AI Applications
With a New Calculus Operator

What if you could improve the performance of your AI algorithms? Describe natural phenomena better? And enhance classical math theorems? Perhaps with a new Calculus operator, on top of the derivative and the integral?

Here we present a new and simple operator in basic Calculus. It renders prominent deep learning frameworks and various AI applications more numerically robust. It is further advantageous in continuous domains, where it classifies trends accurately regardless of functions’ differentiability or continuity.

I used to teach my Advanced Calculus students to infer a function’s momentary trend from its gradient sign. Years later, in our AI research, my team encounters this notion often. Upon analyzing custom loss functions numerically, we treat momentary trends concisely and efficiently:

Derivative Sign

Applications

The instantaneous trend of change, embodied by the derivative sign, has been increasingly used in AI applications. Those range from optimization to computer vision and neural networks. Let’s mention some prominent examples. The “Fast Gradient Sign” method ([10], [14]) leverages the sign of the loss function’s gradient for adversarial examples generation. Further, optimization techniques that leverage the trend of change for robust and rapid convergence, such as Rprop, are recommended for some scenarios of deep learning ([3], [6]) and active learning ([4], [15], [26]). Prominent meta-learning frameworks such as [2], [42] leverage the gradient sign to tackle the exploding gradient problem. Memory efficient algorithms such as Quantized Stochastic Gradient Descent ([1]) and Quantized Neural Networks ([12]) quantize the gradient (where calculating its sign is a special case), and TernGrad ([39]) uses the gradient sign directly for more efficient gradient propagation. Other applications, either to explicit numeric analysis or to theorem proving, include Statistical ML ([21], [28], [32]), Neural Networks ([5], [8], [19], [20], [22], [25], [24], [29], [40]), Reinforcement Learning ([13], [16], [34]), Explainable AI ([9]), and Computer Vision ([4], [7], [11], [17], [18], [23], [27], [30], [33], [35], [41]). You’ll find a more comprehensive survey here. On top of numerical applications, researchers traditionally apply the derivative sign (or that of higher-order derivatives) to monotony classification tasks in continuous domains.

Points for Improvement

Numerical Robustness

Suppose we implement one of the algorithms above and approximate the derivative sign numerically with a difference quotient:

$$\operatorname{sgn}\left[\frac{dL}{d\theta}\left(\theta\right)\right]\approx\operatorname{sgn}\left[\frac{L\left(\theta+h\right)-L\left(\theta-h\right)}{2h}\right]$$

We do this to debug gradient code, or because a custom loss function shaped by domain considerations isn’t differentiable. The division by the small number $2h$ is redundant, since the sign of the quotient is the sign of its numerator:

$$\operatorname{sgn}\left[L\left(\theta+h\right)-L\left(\theta-h\right)\right].$$

Two more problems of sign-based numerics come from asking a rate question when only a direction is needed. The first is the $\epsilon$-clamp. When a computed gradient blows up near a degenerate input, a common reflex is to floor the vanishing denominator at a small $\epsilon$. Suppose the true sensitivity grows like $C\,t^{-\alpha}$, with $\alpha>0$, as the distance $t$ to the degenerate input shrinks. Once $t$ is small enough, the clamped value stops growing and stays at a height of order $1/\epsilon$. The clamp has traded a growing power law for a flat ceiling, and the factor by which the clamp underestimates the truth grows without bound as $t\to0$. The sign of the increment never needed that denominator.

The second problem is ties. At a kink such as $\left|x\right|$ or $\max\left(x,0\right)$ at $x=0$ the derivative doesn’t exist, so every automatic differentiation library returns a conventional value there, and the literature on nonsmooth automatic differentiation documents that equivalent programs can return different values at the same tie. The one-sided signs of the increment need no convention. For $\left|x\right|$ at $0$ they report a decrease toward $0$ from the left and an increase away from it to the right, whichever program computes them.

Analytical Robustness

Next, consider the derivative sign’s analytical role in classifying monotony. At critical points, where the derivative vanishes or doesn’t exist, it often says nothing about the trend, although the function may be clearly trending. As a simple example, consider the family

$$f\left(x;k\right)=\begin{cases} x^{k}, & x\neq0\\ 0, & x=0 \end{cases} \qquad k\in\mathbb{R}\setminus\left\{ 0\right\},$$

together with $f\left(x;0\right)=x^{0}\equiv1$. The following diagrams show some of these right-derivatives at $x=0$ for different values of $k$. For every $k\neq1$ (at $k=1$ the tangent coincides with the graph), the right-derivative at $0$ is either $0$ or infinite. Yet for every $k\neq0$ the values just to the right of $0$ lie strictly above $f\left(0;k\right)=0$: each of these functions rises as it leaves $0$ to the right. The instantaneous rate of change often misses the instantaneous trend at critical points. Engineers work around this by computing higher-order derivatives, or by evaluating the function or its derivatives near the critical point. More involved differentiation methods exist as well.

Remark

In the above example, if we accept derivatives in the extended sense, then $\operatorname{sgn}\left(\pm\infty\right)=\pm1$ recovers the trend. Infinite derivatives are often treated as undefined, though, so the answer depends on a convention. There are also functions whose derivative doesn’t exist even in the extended sense while the trend is plain. For example, take

$$f\left(x\right)=\begin{cases} x+x\left|\sin\left(\frac{1}{x}\right)\right|, & x\neq0\\ 0, & x=0 \end{cases}$$

at $x=0$. Its difference quotient at $0$ is $1+\left|\sin\left(1/h\right)\right|$, which keeps oscillating between $1$ and $2$, while $f\left(h\right)$ always has the sign of $h$. Discontinuities of several kinds can also have a clear trend (see below). We could define the trend of such functions through the signs of their Dini derivatives, at the price of working with upper and lower limits. That route only works in one direction: a positive lower right Dini derivative forces the values on the right above $f\left(x\right)$, but the flat function $e^{-1/x^{2}}$ discussed below has a definite trend while all its Dini derivatives at $0$ vanish. We’d rather define the trend directly.

The trouble runs deeper than critical points. Saying that $f$ is increasing at $a$ can mean something about the rate $f'\left(a\right)$, a comparison of nearby values with $f\left(a\right)$, or monotonicity on an interval around $a$, and these can come apart. A function can satisfy the first two readings and fail the third, as we’ll see below, once the detachment is defined. So it would help to have a simple operator that is numerically more stable than the derivative sign, that answers the second question on its own terms, and that defines the trend whenever there is one, including at critical points such as discontinuities, cusps, extrema, and inflections.

The Idea

Whenever you scrutinize your car’s dashboard, you notice Calculus. The mileage is equivalent to the definite integral of the way you did so far, and the speedometer reflects the derivative of your way with respect to time. Both physical instruments merely approximate abstract notions.

The gear stick shows your travel direction. Usually, its mathematical counterpart is the derivative sign: if the car moves forward, in reverse, or stands still, then the derivative is positive, negative, or zero, respectively. However, calculating the derivative only to read its sign is occasionally superfluous. As Aristotle and Newton famously argued, nature does nothing in vain. Following their approach, we probably needn’t go through a rate calculation to define the instantaneous trend of change. If the trend of change is an important term in the analysis of processes, perhaps we ought to express it directly, instead of as a by-product of the derivative?

This occasional superfluousness of the derivative causes the issues we just saw in numeric and analytic trend classification. To tackle them, let’s attempt to simplify the derivative sign as follows:

$$\begin{align*} \operatorname{sgn}\left[f_{\pm}'\left(x\right)\right] & =\operatorname{sgn}\lim_{h\to0^{\pm}}\left[\frac{f\left(x+h\right)-f\left(x\right)}{h}\right]\\ & \colorbox{yellow}{$\neq$}\lim_{h\to0^{\pm}}\operatorname{sgn}\left[\frac{f\left(x+h\right)-f\left(x\right)}{h}\right]\\ & =\pm\lim_{h\to0^{\pm}}\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right] \end{align*}$$

Note the deliberately wrong transition in the second line. Swapping the limit and the sign is not allowed in general, because the sign function is discontinuous at zero. So the resulting operator, the limit of the sign of $\Delta y$, doesn’t always agree with the derivative sign. In the third line, the multiplicativity of the sign function cancels the division. Both facts may work in our favor, given the issues we saw earlier with the derivative sign. Perhaps the limit of the sign of the change is worth studying in trend classification tasks?

This way of defining a trend parallels the definition of the derivative. There, the slope of a secant turns into a tangent as the two points approach each other. Here, we take the direction of change across a small interval next to the point and let the limit process turn it into the momentary trend. Feel free to gain intuition by hovering over the following diagram:

Numerical Stability

The numerical approximation of the (right-) derivative sign and that of the one-sided limit

$$\lim_{h\to0^{+}}\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]$$

both come down to the sign of a finite difference, $\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]$, for some small value of $h$. The derivative sign gets there through a redundant division by $h$. This roundtrip costs an extra logarithmic- or linearithmic-time division (depending on the precision) and might overflow, since $h$ is small. So I find it useful to think of trend approximations as approximations of the limit above, instead of the derivative sign. This makes the trend calculation itself noticeably faster than the numerical derivative sign; in a full algorithm the saving is diluted by its other operations, so how much remains depends on the algorithm.

Similar considerations lead us to apply quantization directly to $\Delta y$ when we’re after a general quantization of the derivative and not only its sign. That is, instead of quantizing the derivative, we calculate $Q\left[f\left(x+h\right)-f\left(x\right)\right]$, where $Q$ is a custom quantization function. Unlike the trend operator, this quantization doesn’t preserve the derivative’s value once the division is skipped. It is good enough in algorithms such as gradient descent, though. If the algorithm multiplies the gradient by a constant anyway (the learning rate, for example), we may skip the division by $h$ in each iteration and fold it into that pre-calculated constant.

We can roughly estimate the share of computation time spared by looking at the computations other than the trend operator itself. For example, suppose we spare a single division in each iteration of gradient descent, and the function is defined numerically on a finite discrete set of points. Then avoiding the division matters, because it is the most time-consuming operation in each iteration. If the optimized function is computationally involved, for example one that includes logical operators, then sparing a division still helps, but less.

Another numerical advantage over the derivative sign concerns the case where the first derivative vanishes. As the Taylor expansion of the trend operator shows below (Claim 14), it equals, up to a sign coefficient, the sign of the first derivative that doesn’t vanish. So for the trend, only the sign of the leading nonzero derivative matters, not its size.

The exploding gradient issue also hinders the derivative sign, and the trend operator mitigates it: the sign of the increment stays bounded however fast the rate grows. Other numerical issues of the derivative carry over to the derivative sign and are often avoided with the trend operator.

Why the Sign of the Increment Is the Right Primitive

Three properties make the sign of the increment a better building block for sign-based numerics than the sign of a rate.

First, it is an order notion. If $\phi$ is strictly increasing, then $\phi\left(u\right)-\phi\left(v\right)$ has the sign of $u-v$, so replacing a loss $L$ by $\log L$, by a rank, or by any other strictly increasing transform leaves every trend unchanged. The derivative has no such invariance. Its magnitude depends on the scale, and even its sign can disappear: with $f\left(x\right)=x$ and $\phi\left(y\right)=y^{3}$, the composite has derivative $0$ at $x=0$ while $f'\left(0\right)=1$, and both functions rise through $0$.

Second, it reads only comparisons of values. It is available for losses that are piecewise constant, defined at discrete points only, or not differentiable at all.

Third, when a direction alone isn’t enough, the sign pairs naturally with a rate of vanishing. If $f\left(x+h\right)-f\left(x\right)=c\,h^{\nu}+o\left(h^{\nu}\right)$ as $h\to0^{+}$, with $c\neq0$, the pair $\left(\operatorname{sgn}c,\nu\right)$ says which way the function leaves $x$ and how fast. In tropical geometry the same pair is called the signed valuation (Allamigeon, Gaubert and Skomra, 2020); the trend operator defined below is its first coordinate, taken on one side. The exponent $\nu$ is what the slope of $\log\left|\Delta y\right|$ against $\log h$ estimates over a range of small steps, and an $\epsilon$-clamp flattens exactly that slope. For the functions met in practice (more precisely, functions definable in an o-minimal structure, a class that contains every semialgebraic function and every function built from polynomials, $\exp$ and $\log$), the sign of the increment always settles as $h\to0^{+}$, so the operator is always defined.

Given this operator’s practical merit in discrete domains, let’s proceed with theoretical aspects in continuous ones.

How Does it Work

Let’s check how coherently this operator defines a local trend, compared with the derivative. Recall the family of monomials from the introductory section, where we tried to define the local trend by the sign of the derivative. We add another degree of freedom and let $f\left(x;a,k\right)=ax^{k}$. To gain intuition, let’s compare the right limit of $\operatorname{sgn}\left(\Delta y\right)$ with the right-derivative for hand-picked cases. Let $k\in\left\{ -1,0,0.5,1,2,3\right\}$, capturing a discontinuity, constancy, a cusp, linearity, an extremum, and an inflection, respectively. We allow opposite values of $a$ to cover all types of trends. Feel free to tweak the following radio buttons and observe the limit process in action for both operators:

In all those cases, the right limit of $\operatorname{sgn}\left(\Delta y\right)$ matches the way we think about the trend: it always equals $\operatorname{sgn}\left(a\right)$, except in the constant case, where it is zero, as expected. It’s possible to show this directly with limit calculus; see some examples below. We also concluded in the introductory section that the derivative sign captures these momentary trends only for $k\in\left\{ 0,1\right\}$. This operator does better at critical points.

Why Does it Work

A more rigorous justification comes from noticing that the definition of a strict local extremum and the definition of this operator say the same thing. Unlike its counterpart in basic calculus, the following claim gives a condition that is both necessary and sufficient:

Theorem 1. Analog to Fermat’s Stationary Point Theorem. Let $f:\left(a,b\right)\rightarrow\mathbb{R}$ be a function and let $x\in\left(a,b\right).$ Then $x$ is a strict local extremum of $f$ if and only if the two-sided limit

$$\lim_{h\to0}\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]$$

exists and differs from zero. The limit is $-1$ at a strict local maximum and $+1$ at a strict local minimum. No continuity or differentiability is assumed.

The avid calculus reader will notice that this theorem follows immediately from the Cauchy definition of a limit. The proof rests on an observation used throughout this page: since the sign function takes only three values, a limit of signs exists exactly when the sign is eventually constant.

Proof

Write $s\left(h\right)=\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]$, a function with values in $\left\{ -1,0,+1\right\}$. Suppose $s\left(h\right)\to L$ as $h\to0$. By the Cauchy definition with $\epsilon_{0}=\frac{1}{2}$, there is a $\delta>0$ such that

$$0<\left|h\right|<\delta\Longrightarrow\left|s\left(h\right)-L\right|<\frac{1}{2}.$$

The values $-1,0,+1$ are at distance at least $1$ from each other, so the only value within $\frac{1}{2}$ of $L$ is $L$ itself, and $s\left(h\right)=L$ for $0<\left|h\right|<\delta$. Conversely, if $s\left(h\right)=L$ whenever $0<\left|h\right|<\delta$, then $\left|s\left(h\right)-L\right|=0<\epsilon$ there for every $\epsilon>0$, and the limit is $L$.

Now take $L=-1$. The condition reads

$$\exists\delta>0:\;0<\left|h\right|<\delta\Longrightarrow f\left(x+h\right)<f\left(x\right),$$

which says exactly that $x$ is a strict local maximum. In the same way, $L=+1$ says exactly that $x$ is a strict local minimum. A limit equal to $0$ would make $f$ constant near $x$, and then $x$ is not a strict extremum. $\,\,\,\,\blacksquare$

The classical first derivative test controls the sign of $f'$ on whole intervals on both sides of $x$, and so it also certifies that $f$ falls monotonically into $x$ and rises monotonically out of it. Theorem 1 compares nearby values only with $f\left(x\right)$. It detects the strict extremum with no regularity at all, and says nothing about how two nearby points on the same side compare with each other. A good test case is $b\left(x\right)=e^{-1/x^{2}}$ with $b\left(0\right)=0$, the building block of smooth bump functions. Every derivative of $b$ vanishes at $0$, so no derivative test of any order decides anything there. Yet $b\left(h\right)>0=b\left(0\right)$ for every $h\neq0$, and Theorem 1 declares a strict minimum at once.

Feel free to scrutinize the relation between the Semi-discrete and the continuous versions of Fermat’s theorem in the following animation:

Where Does it Work

Next, let’s check in which scenarios this operator is well defined. We’ll pick functions with different characteristics around $x=0$. For each such function, we ask which of the following properties hold at $x=0$: continuity, differentiability, and the existence of the operator at stake, that is, the existence of a local trend from both sides. Scrutinize the following animation to gain intuition with some basic examples:

We would also like to find out which of those properties hold across an entire interval (for example, $[-1,1]$). To that end, we add two interval properties: Lebesgue and Riemann integrability. The following widget explores those properties with slightly more involved examples than the animation above. Switch between the granularity levels, hover over the regions, and click the functions in the table to confirm which conditions hold in each case:

Remark

We may extend the discussion to properties that hold almost everywhere in an interval, instead of throughout it. This is beyond this article’s scope, but as a taste consider

$$f\left(x\right)=\sum_{n=1}^{\infty}2^{-n}\phi\left(\frac{x-a_{n}}{b_{n}-a_{n}}\right),$$

where $\phi$ is the Cantor-Vitali function (extended by $0$ to the left of $0$ and by $1$ to the right of $1$), and $\left(a_{n},b_{n}\right)$ runs over all the intervals in $\left[0,1\right]$ with rational endpoints. One can show that this function is strictly increasing, so it has a trend everywhere, while its derivative vanishes almost everywhere. Here the instantaneous trend agrees with the function’s monotone behavior on the interval, and the vanishing derivative doesn’t. Further, by the Baire category theorem, all continuous functions except a meager set are nowhere differentiable, so a notion of trend that doesn’t rely on differentiability enlarges the set of functions whose monotony can be analyzed concisely. Finally, consider $g\left(x\right)=x^{1+\mathbf{1}_{\mathbb{Q}}\left(x\right)}$ on $\left(0,1\right)$. In that domain $g$ is discontinuous everywhere, but it is detachable from the left almost everywhere.

Defining the Instantaneous Trend of Change

Let’s summarize our discussion thus far. The momentary trend, a basic analytical concept, has been embodied by the derivative sign for engineering purposes. It drives core numeric algorithms across AI, optimization and other computerized applications. More often than not, it doesn’t capture the momentary trend of change at critical points. The one-sided limit of $\operatorname{sgn}\left(\Delta y\right)$, in contrast, is more numerically robust in terms of finite differences, and it defines trends wherever they exist, including at critical points. Given these merits, it deserves a definition of its own. Since it “detaches” functions, turning them into step functions with discontinuities at extrema, let’s define the one-sided detachments of a function $f$ as follows:

$$f_{\pm}^{;}\left(x\right)\equiv\pm\lim_{h\to0^{\pm}}\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]\in\left\{ -1,0,+1\right\}.$$

We say that $f$ is right- or left-detachable at $x$ if the corresponding one-sided limit exists, and detachable if both exist. For convenience, from now on $f^{;}$ denotes either one of the one-sided detachments ($f^{;}_{+}$ or $f^{;}_{-}$), without assuming that they agree.

The $\pm$ coefficient keeps the detachment consistent with the derivative sign, so both one-sided values are read from left to right. Substituting $h\mapsto-h$ in the left limit gives

$$f_{-}^{;}\left(x\right)=\lim_{h\to0^{+}}\operatorname{sgn}\left[f\left(x\right)-f\left(x-h\right)\right],$$

so $f^{;}_{+}\left(x\right)=+1$ says the values just right of $x$ exceed $f\left(x\right)$, and $f^{;}_{-}\left(x\right)=+1$ says the values just left of $x$ are below it. The pair $\left(f^{;}_{-},f^{;}_{+}\right)$ is $\left(+1,+1\right)$ when $f$ rises through $x$, $\left(-1,+1\right)$ at a strict local minimum, and $\left(+1,-1\right)$ at a strict local maximum. For $\left|x\right|$ at $0$ it is $\left(-1,+1\right)$, whatever value a library assigns to the derivative there.

The proof of Theorem 1 showed that a limit of signs exists exactly when the sign is eventually constant. So $f^{;}_{+}\left(x\right)=+1$ means that $f\left(x+h\right)>f\left(x\right)$ for all sufficiently small $h>0$; $f^{;}_{+}\left(x\right)=-1$ means that $f\left(x+h\right)<f\left(x\right)$ for all such $h$; and $f^{;}_{+}\left(x\right)=0$ means that $f$ is constant on some interval $\left(x,x+\delta\right)$. That last case is an actual plateau, a much stronger statement than a small change or a vanishing derivative. Geometrically, for the right detachment to equal $+1$, the value $f\left(x\right)$ must bound the values of $f$ on a right neighborhood of $x$ strictly from below. The derivative sign assumes an ascending tangent instead. The detachment forms no quotient and measures no magnitude.

The detachment is a partial operator, and it should stay partial. Take $g\left(x\right)=x^{3}\sin\left(1/x\right)$ with $g\left(0\right)=0$. Then $g'\left(0\right)=0$, but the sign of $g\left(h\right)-g\left(0\right)$ keeps changing as $h\to0$ from either side, so $g$ is differentiable at $0$ and detachable from neither side. The sign doesn’t even need to change: $q\left(x\right)=x^{2}\sin^{2}\left(1/x\right)$ with $q\left(0\right)=0$ has a non-strict minimum at $0$, and $q\left(h\right)-q\left(0\right)$ alternates between $0$ and positive values, so again no detachment exists. Assigning the value $0$ in such cases would mistake oscillation for a plateau. On tame ground the problem disappears. When $f$ is definable in an o-minimal structure, the three sets of $h>0$ with $f\left(x+h\right)>f\left(x\right)$, with equality, and with $f\left(x+h\right)<f\left(x\right)$ each break into finitely many points and intervals. They cover every $h>0$, so one of them includes a whole interval $\left(0,\delta\right)$, and the function is detachable everywhere. Feel free to scrutinize the logical steps that led from the derivative sign to the definition of the detachment in the following animation (created with Manim):

The Derivative, the Detachment and Monotonicity

With the detachment at hand, the questions hidden in "increasing at a point" can be kept apart:

1. Rate: what is the sign, and the size, of $f'\left(a\right)$?
2. Trend: what are $f^{;}_{-}\left(a\right)$ and $f^{;}_{+}\left(a\right)$, that is, do the values just left and right of $a$ lie below, at, or above $f\left(a\right)$?
3. Monotonicity near $a$: is $f$ monotone on some interval that contains $a$?

The first question divides by the step, the second only compares values with the value $f\left(a\right)$ at the point, and the third compares all pairs of nearby points.

A Positive Derivative With No Monotone Neighborhood

Let $f\left(0\right)=0$ and $f\left(x\right)=x+2x^{2}\sin\left(1/x\right)$ for $x\neq0$. The three questions get three different answers at $0$.

The rate is positive. The difference quotient at the origin is

$$\frac{f\left(h\right)-f\left(0\right)}{h}=1+2h\sin\left(\frac{1}{h}\right)\longrightarrow1,$$

so $f'\left(0\right)=1$.

The trend is upward on both sides. For $0<h<\frac{1}{2}$ we have $1+2h\sin\left(1/h\right)\geq1-2h>0$, so $f\left(h\right)>0$, and since $f$ is odd, $f\left(-h\right)<0$. Hence $f^{;}_{-}\left(0\right)=f^{;}_{+}\left(0\right)=+1$.

Yet $f$ is monotone on no interval around $0$. Away from the origin,

$$f'\left(x\right)=1+4x\sin\left(\frac{1}{x}\right)-2\cos\left(\frac{1}{x}\right),$$

and along the sequences $x_{n}=\frac{1}{2\pi n}$ and $y_{n}=\frac{1}{\left(2n+1\right)\pi}$, both tending to $0$, we get $f'\left(x_{n}\right)=-1$ and $f'\left(y_{n}\right)=3$. Monotonicity on an interval would force $f'$ to keep one weak sign wherever it exists there: $f'\geq0$ for an increasing function, $f'\leq0$ for a decreasing one. But in every neighborhood of $0$, $f'$ takes the value $-1$ at some $x_{n}$ and $3$ at some $y_{n}$, so no interval around $0$ qualifies. On a plot, $f$ is squeezed between the parabolas $y=x\pm2x^{2}$, which hug the line $y=x$ and looks like that line on any small window, which makes the example easy to underestimate. The full report at the origin reads:

$$\begin{gathered} f'\left(0\right)=1,\\ \left(f^{;}_{-}\left(0\right),f^{;}_{+}\left(0\right)\right)=\left(+1,+1\right),\\ \text{no monotone neighborhood of }0. \end{gathered}$$

All three statements are true, and none follows from the others. The example is classical (it appears in Gelbaum and Olmsted’s Counterexamples in Analysis), and so is the comparison with $f\left(a\right)$ itself: Burkill’s A First Course in Mathematical Analysis calls a function strictly increasing at $c$ exactly when it rises through $c$ in this sense.

Nine Local Pictures

At a detachable point, the pair $\left(f^{;}_{-},f^{;}_{+}\right)$ takes one of nine values:

"Rises through $x$" is a comparison with $f\left(x\right)$ only, with no claim about an interval: there is a $\delta>0$ such that

$$x-\delta<u<x<v<x+\delta\Longrightarrow f\left(u\right)<f\left(x\right)<f\left(v\right).$$

The central example above rises through $0$ in exactly this sense and is monotone on no neighborhood of $0$.

The derivative’s value can’t tell these nine pictures apart. Each of them already occurs at a differentiable point with zero derivative: given $\ell,r\in\left\{ -1,0,+1\right\}$, set $f\left(x\right)=0$, and for $h>0$ set $f\left(x+h\right)=r\,h^{2}$ and $f\left(x-h\right)=-\ell\,h^{2}$. The difference quotient is $r\,h$ on the right and $\ell\,h$ on the left, and both vanish as $h\to0$, so $f'\left(x\right)=0$, while $f^{;}_{-}\left(x\right)=\ell$ and $f^{;}_{+}\left(x\right)=r$. Standard examples at $0$ are $x^{3}$ with $\left(+1,+1\right)$, $-x^{3}$ with $\left(-1,-1\right)$, $x^{2}$ with $\left(-1,+1\right)$, and the zero function with $\left(0,0\right)$, while $x^{3}\sin\left(1/x\right)$ has no detachment at all.

The value at the point matters, and jumps show how. For $0\leq c\leq1$ let $H_{c}$ be the step function that equals $0$ for $x < 0$, $c$ at $x=0$, and $1$ for $x > 0$. Its pair of detachments at $0$ is $\left(0,+1\right)$ for $c=0$, $\left(+1,+1\right)$ for $0 < c < 1$, and $\left(+1,0\right)$ for $c=1$. All three functions are nondecreasing, so monotonicity can’t tell them apart, while the detachments record where $H_{c}\left(0\right)$ sits between the two one-sided levels.

How the Three Answers Relate

Away from a zero derivative, the trend is the sign of the rate: if a one-sided derivative exists, is finite and differs from zero, the corresponding detachment exists and equals its sign (part 1 of Theorem 10 below). The detachment earns its keep where the derivative is zero, infinite, missing, or answering a different question. The flat function $b\left(x\right)=e^{-1/x^{2}}$ is the extreme case: its difference quotient at $0$ is $e^{-1/h^{2}}/h\to0$, so every one-sided rate there, Dini derivatives included, is zero, and still $\left(b^{;}_{-}\left(0\right),b^{;}_{+}\left(0\right)\right)=\left(-1,+1\right)$.

Monotonicity on an interval implies the trend, and not conversely. If $f$ is strictly increasing near $a$, both detachments equal $+1$ (Lemma 9 below). If $f$ is merely nondecreasing, each detachment exists and lies in $\left\{ 0,+1\right\}$. Look at the right side. If $f\left(a+h\right)=f\left(a\right)$ throughout some interval $\left(0,\delta\right)$, the detachment is $0$. Otherwise there are points $a+h_{0}$ with $h_{0}$ as small as we like and $f\left(a+h_{0}\right)>f\left(a\right)$. Given any small $h>0$, choose such an $h_{0}\leq h$; monotonicity gives $f\left(a+h\right)\geq f\left(a+h_{0}\right)>f\left(a\right)$, so the detachment is $+1$. The left side is symmetric. The central example shows that the converse fails: the detachments compare each side with the value at the point only and never compare two points on the same side.

That also locates the missing hypothesis in a familiar shortcut, the idea that $f'\left(a\right)>0$ makes $f$ increase near $a$. The usual way out is a continuous derivative: if $f'$ is continuous at $a$ and $f'\left(a\right)>0$, then $f'>0$ on some interval around $a$, and the mean value theorem gives strict increase on that interval. Without such control of the sign of $f'$ across an interval, the single value $f'\left(a\right)$ can’t control the nearby derivatives, and the central example shows what happens then.

Finally, the trend and monotonicity are both order notions, and the rate is not. For any strictly increasing $\phi$,

$$\left(\phi\circ f\right)^{;}_{\pm}\left(x\right)=f^{;}_{\pm}\left(x\right),$$

because $\phi\left(u\right)-\phi\left(v\right)$ has the sign of $u-v$; a strictly decreasing $\phi$ flips both detachments. Monotonicity on an interval is preserved in the same way. The derivative is not, as the example $\phi\left(y\right)=y^{3}$ in the Numerical Stability section showed. This is why the detachment still answers at vertical tangents, at jumps and at extrema where every derivative vanishes. The rate breaks down at those points, while the comparison of values does not.

Single Variable Semi-discrete Calculus

Equipped with a definition of the instantaneous trend of change, we may formulate analogs of calculus theorems with the following trade-off. These results report the function’s trend instead of its rate. In return, they hold for a broad set of detachable functions, differentiable or not. They are also outlined in [31].

Two conventions keep the statements short. On the right we write $\Delta f=f\left(x+h\right)-f\left(x\right)$, and on the left $\Delta f=f\left(x\right)-f\left(x-h\right)$, always with a small $h > 0$. In both cases the definition says that $\operatorname{sgn}\left(\Delta f\right)$ equals the corresponding one-sided detachment for every small enough $h$. Since a limit of signs exists exactly when the sign is eventually constant, every proof below has the same shape: find a one-sided neighborhood on which the relevant signs are frozen, and read off the sign of the increment there. One elementary fact does most of the work. If two numbers $A$ and $B$ don’t have opposite signs, then

$$\operatorname{sgn}\left(A+B\right)=\operatorname{sgn}\left[\operatorname{sgn}\left(A\right)+\operatorname{sgn}\left(B\right)\right].$$

Simple Algebraic Properties

Claim 2. Constant Multiple Rule. If $f$ is detachable at $x$, and $c\in \mathbb{R}$ is a constant, then $cf$ is also detachable there and

$$\left(cf\right)^{;}=\operatorname{sgn}\left(cf^{;}\right).$$

Proof

The sign function is multiplicative, so for every $h$:

$$\left(cf\right)_{\pm}^{;} = \pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left(c\right)\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right]=\operatorname{sgn}\left(c\right)f_{\pm}^{;}=\operatorname{sgn}\left(cf_{\pm}^{;}\right).\,\,\,\,\blacksquare$$

Claim 3. Sum and Difference Rules. Let $f$ and $g$ be detachable at $x$. If $f^{;}g^{;}\neq-1$ there, then

$$\left(f+g\right)^{;}=\operatorname{sgn}\left(f^{;}+g^{;}\right),$$

and if $f^{;}g^{;}\neq+1$ there, then

$$\left(f-g\right)^{;}=\operatorname{sgn}\left(f^{;}-g^{;}\right).$$

In words, the sum rule holds unless the two trends point in opposite directions, and the difference rule holds unless they point in the same direction.

Proof

For every small enough $h$, $\operatorname{sgn}\left(\Delta f\right)=f^{;}$ and $\operatorname{sgn}\left(\Delta g\right)=g^{;}$. If $f^{;}g^{;}\neq-1$, these two signs aren’t opposite, so

$$\operatorname{sgn}\left[\Delta\left(f+g\right)\right]=\operatorname{sgn}\left(\Delta f+\Delta g\right)=\operatorname{sgn}\left(f^{;}+g^{;}\right)$$

for all those $h$, and the limit of a constant is that constant. The difference rule is the sum rule applied to $f$ and $-g$, whose detachment is $-g^{;}$ by Claim 2. $\,\,\,\,\blacksquare$

Claim 4. If $f$, $g$ and $f+g$ are detachable at $x$, and the one-sided detachments of $f$ and $g$ aren’t both zero there, then the following holds at $x$:

$$f^{;}g^{;}=\left(f+g\right)^{;}\left(f^{;}+g^{;}\right)-1.$$

Proof

Checking the nine sign combinations shows that for real numbers $A,B$, not both zero,

$$\operatorname{sgn}\left(A\right)\operatorname{sgn}\left(B\right)=\operatorname{sgn}\left(A+B\right)\left[\operatorname{sgn}\left(A\right)+\operatorname{sgn}\left(B\right)\right]-1.$$

Take $A=\Delta f$ and $B=\Delta g$. Since the detachments of $f$ and $g$ aren’t both zero, $A$ and $B$ aren’t both zero for small $h$, and the identity holds for each such $h$. Every factor is eventually constant, so it survives the one-sided limit. $\,\,\,\,\blacksquare$

Remark

Don’t worry about the sum rule failing for functions whose detachments are additive inverses. We’ll handle those cases for smooth functions later on, with Taylor expansions.

Product and Quotient Rules

Let’s begin with a simple example. If $f^{;}=g^{;}=+1$ and $f=g=0$ at a point $x$, then the product’s detachment is $\left(fg\right)^{;}=+1$ as well. One might be tempted to guess that the product rule is simply $\left(fg\right)^{;}=f^{;}g^{;}$. However, shift $g$ downwards so that $g\left(x\right) < 0$, and keep $g$ continuous. Near $x$ the product is now a small positive increment of $f$ times a negative number, so $\left(fg\right)^{;}=-1$, although the detachments of $f$ and $g$ haven’t changed. The product’s detachment depends on the signs of $f$ and $g$ at the point, too.

In Semi-discrete Calculus we aren’t restricted to continuous functions, though. What if $g$ is discontinuous? As long as $g$ keeps its sign near the point, the product’s detachment stays $-1$. But if $g$ vanishes near $x$, or turns positive there, then $\left(fg\right)^{;}$ can be $0$ or $+1$. So we need some control over the signs of $f$ and $g$ near the point. I suggest an intuitive one: sign-continuity.

We say that $f$ is sign-continuous (s.c.) at $x$ if $\operatorname{sgn}\left(f\right)$ is continuous there, and sign-discontinuous (s.d.) at $x$ if every partial limit of $\operatorname{sgn}\left(f\right)$ at $x$ differs from $\operatorname{sgn}f\left(x\right)$. In the second case, for $\bar{x}$ close enough to $x$, the sign of $f\left(\bar{x}\right)$ never equals that of $f\left(x\right)$. A function that is either sign-continuous or sign-discontinuous is sign-consistent. We restrict ourselves in the following discussion to sign-consistent functions.

Sometimes the value and the detachment settle the question by themselves. Take right detachments. If $f^{;}_{+}=0$, then $f$ is constant on a right neighborhood. If $f\,f^{;}_{+} > 0$, then $f\left(\bar{x}\right)$ moves away from zero, and keeps its sign. In both cases $f$ is sign-continuous from the right, and we call it inherently sign-continuous (i.s.c.). If $f=0$ and $f^{;}\neq0$, then $f\left(\bar{x}\right)\neq0$ on both sides, and $f$ is inherently sign-discontinuous (i.s.d.). The remaining case, $f\,f^{;}_{+} < 0$, says only that $f\left(\bar{x}\right)$ moves toward zero, so whether it crosses zero must be assumed separately. On the left the roles flip: $f\left(\bar{x}\right)$ moves away from zero when $f\,f^{;}_{-} < 0$.

You’ll find here a simulation that enumerates the combinations of $f^{;}$, $g^{;}$, the signs of $f$ and $g$, and their sign-continuity, and computes the product’s and quotient’s detachments numerically. Its results are available in two Google sheets, here.

Recall that the product rule for derivatives reads $\left(fg\right)'=f'g+fg'$. The rules below follow a similar formula in part of the cases and a different intuitive formula in others. They hold for detachable functions that need not be continuous. In both rules, $ff^{;}gg^{;}$ is short for the sign of the product $f\left(x\right)f^{;}\left(x\right)g\left(x\right)g^{;}\left(x\right)$, and the detachments are right detachments (see the remark after the proofs for the left side).

Claim 5. Product Rule. Let $f$ and $g$ be detachable and sign-consistent at $x$. If one of the following holds there:

1. $ff^{;}gg^{;}\geq0$, where $f$ or $g$ is s.c., or $f=g=0$
2. $ff^{;}gg^{;} < 0$, where $f$ or $g$ is s.d.

Then $fg$ is detachable there, and:

$$\left(fg\right)^{;}=\begin{cases} \operatorname{sgn}\left[f^{;}\operatorname{sgn}\left(g\right)+g^{;}\operatorname{sgn}\left(f\right)\right], & ff^{;}gg^{;}\geq0\text{, and }f\text{ or }g\text{ is s.c.}\\ f^{;}g^{;}, & \text{else}\text{.} \end{cases}$$

Proof

Let $\bar{x}=x+h$ with a small $h > 0$, and write $s_{f}=\operatorname{sgn}f\left(x\right)$ and $s_{g}=\operatorname{sgn}g\left(x\right)$. The product’s increment splits in two ways:

$$\Delta\left(fg\right)=\Delta f\cdot g\left(\bar{x}\right)+f\left(x\right)\cdot\Delta g=\Delta f\cdot g\left(x\right)+f\left(\bar{x}\right)\cdot\Delta g.$$

For small $h$, $\operatorname{sgn}\left(\Delta f\right)=f^{;}$ and $\operatorname{sgn}\left(\Delta g\right)=g^{;}$. Three cases cover the claim.

1. $ff^{;}gg^{;}\geq0$ and one function is s.c. Say $g$ is s.c., so $\operatorname{sgn}g\left(\bar{x}\right)=s_{g}$ for small $h$. In the first splitting, the two terms have the signs $f^{;}s_{g}$ and $s_{f}g^{;}$. Their product is the sign of $ff^{;}gg^{;}$, which isn’t negative, so the terms aren’t of opposite signs, and the sign of their sum is $\operatorname{sgn}\left(f^{;}s_{g}+g^{;}s_{f}\right)$. If $f$ is the s.c. function, the second splitting gives the same result.
2. $f=g=0$ and neither function is s.c. Then $\Delta\left(fg\right)=f\left(\bar{x}\right)g\left(\bar{x}\right)$, where $\operatorname{sgn}f\left(\bar{x}\right)=\operatorname{sgn}\left(\Delta f\right)=f^{;}$, and likewise for $g$. So $\left(fg\right)^{;}=f^{;}g^{;}$.
3. $ff^{;}gg^{;} < 0$ and one function is s.d. One of the products $ff^{;}$ and $gg^{;}$ is positive and the other negative. Say $ff^{;} > 0$. Then $f$ is i.s.c., so the s.d. function is $g$, and $g\left(x\right)\neq0$. Near $x$, $f\left(\bar{x}\right)$ has the sign $s_{f}$ while $g\left(\bar{x}\right)$ is zero or has the sign $-s_{g}$. So $f\left(\bar{x}\right)g\left(\bar{x}\right)$ is zero or has the sign opposite to $f\left(x\right)g\left(x\right)\neq0$, and $\operatorname{sgn}\left[\Delta\left(fg\right)\right]=-s_{f}s_{g}$. Since $f^{;}=s_{f}$ and $g^{;}=-s_{g}$, this is $f^{;}g^{;}$. $\,\,\,\,\blacksquare$

Claim 6. Quotient Rule. Let $f$ and $g$ be detachable and sign-consistent at $x$, where $g\neq0$ near $x$. If one of the following holds there:

1. $ff^{;}gg^{;}\leq0$, where $f$ or $g$ is s.c., or $f=0$
2. $ff^{;}gg^{;} > 0$, where only one of $f,g$ is s.c.

Then $\frac{f}{g}$ is detachable there, and:

$$\left(\frac{f}{g}\right)^{;}=\begin{cases} \operatorname{sgn}\left[f^{;}\operatorname{sgn}\left(g\right)-g^{;}\operatorname{sgn}\left(f\right)\right], & g\text{ is i.s.c., or }f,g\text{ are s.c., or }f=0\text{ and }f\text{ or }g\text{ is s.c.}\\ \operatorname{sgn}\left[f^{;}\operatorname{sgn}\left(g\right)+g^{;}\operatorname{sgn}\left(f\right)\right], & \text{else, if }ff^{;}gg^{;}\geq0\text{ and }f\text{ or }g\text{ is s.c.}\\ f^{;}g^{;}, & \text{else.} \end{cases}$$

Proof

With the notation of the previous proof, the quotient’s increment can be written in two ways:

$$\frac{f}{g}\left(\bar{x}\right)-\frac{f}{g}\left(x\right)=\frac{\Delta f\cdot g\left(x\right)-f\left(x\right)\cdot\Delta g}{g\left(x\right)g\left(\bar{x}\right)}=\frac{\Delta f\cdot g\left(\bar{x}\right)-f\left(\bar{x}\right)\cdot\Delta g}{g\left(x\right)g\left(\bar{x}\right)}.$$

Since $g$ doesn’t vanish, sign-consistency leaves two options for it. Either $g$ is s.c., and the denominator is positive near $x$; or $g$ is s.d., so $g\left(\bar{x}\right)$ has the sign $-s_{g}$, the denominator is negative, and $g^{;}=-s_{g}$.

1. $g$ is s.c. and $ff^{;}gg^{;}\leq0$. In the first numerator the terms have the signs $f^{;}s_{g}$ and $-s_{f}g^{;}$, whose product is the sign of $-ff^{;}gg^{;}\geq0$. They aren’t opposite, so the quotient’s detachment is $\operatorname{sgn}\left(f^{;}s_{g}-g^{;}s_{f}\right)$, the first formula.
2. $ff^{;}gg^{;} > 0$ and exactly one function is s.c. Use the second numerator. If $g$ is s.c. and $f$ is s.d., then $f\left(x\right)\neq0$ and $f\left(\bar{x}\right)$ is zero or of sign $-s_{f}$. The terms have the signs $f^{;}s_{g}$ and $s_{f}g^{;}$ (or zero), whose product is positive, and the denominator is positive. If $f$ is s.c. and $g$ is s.d., the terms have the signs $-f^{;}s_{g}$ and $-s_{f}g^{;}$, and the denominator is negative. Either way the quotient’s detachment is $\operatorname{sgn}\left(f^{;}s_{g}+g^{;}s_{f}\right)$, the second formula.
3. $g$ is s.d. and $ff^{;}gg^{;}\leq0$. By the hypothesis, $f$ is s.c. or $f=0$. If $f$ is s.c. and $ff^{;}gg^{;}=0$, the second numerator’s terms have the signs $-f^{;}s_{g}$ and $-s_{f}g^{;}$, which aren’t opposite, and the negative denominator gives the second formula again. If $f$ is s.c. and $ff^{;}gg^{;} < 0$, then $gg^{;} < 0$ forces $ff^{;} > 0$: $f\left(\bar{x}\right)$ keeps its sign while $g\left(\bar{x}\right)$ flips, so the quotient jumps to the opposite sign, and its increment has the sign $-s_{f}s_{g}=f^{;}g^{;}$. Finally, if $f=0$ is not s.c., the first numerator is $\Delta f\cdot g\left(x\right)$, of sign $f^{;}s_{g}$, and the negative denominator turns it into $-f^{;}s_{g}=f^{;}g^{;}$, the third formula. (If $f=0$ is s.c., $f$ vanishes near $x$, and all three formulas give $0$.)

A direct check shows that these three cases are exactly the three branches of the statement. $\,\,\,\,\blacksquare$

Remark

1. In the quotient rule, if $f,g$ are both continuous then, because $g\neq0$, $g$ is s.c. If $f$ is also s.c., the first formula holds by its second sub-condition. Otherwise, by continuity, $f=0$, and the first formula holds by its third sub-condition. So the first formula holds for every pair of continuous functions that satisfies the statement. In the product rule, the first formula doesn’t cover all continuous functions. For example, take $f=g=x$ at $x=0$. Both are s.d., so the product’s right detachment follows the second formula, $\left(+1\right)\cdot\left(+1\right)=+1$.
2. Left detachments. Claims 5 and 6 are stated for right detachments. For the left side, reflect: with $\check{f}\left(t\right)=f\left(-t\right)$ we have $f^{;}_{-}\left(x\right)=-\check{f}^{;}_{+}\left(-x\right)$, and applying the claims to the reflected functions gives the left versions. The formulas built from $\operatorname{sgn}\left[\cdots\right]$ don’t change, since both detachments flip sign inside them. The bare product flips once, so the last branch becomes $-f^{;}_{-}g^{;}_{-}$, and "inherently sign-continuous" means $f^{;}_{-}=0$ or $ff^{;}_{-} < 0$. For $f=g=x$ at $0$ this gives the left detachment $-1$ for $x^{2}$, as it should: $x^{2}$ falls into its minimum.
3. There are many ways to bound the signs of $f,g$ other than sign-continuity. For example, we could bound the signs of $f,g$ explicitly on a given neighborhood of $x$. A product rule built on such bounds holds in more cases than Claims 5 and 6. For consistency with Differential Calculus, I preferred to introduce the intuitive trait of sign-continuity here. It corresponds to the traditional requirements of continuity and differentiability, and the first-time reader may feel more comfortable with it.
4. We define sign-discontinuity as above so that it bounds the function’s sign near the point. The mere lack of sign-continuity doesn’t impose such a bound.

Mean Value Theorems

We already introduced an analog to Fermat’s stationary point theorem in a previous section. Let’s formulate analogs of other basic results.

Claim 7. Analog to Rolle’s theorem. Let $f:\left[a,b\right]\rightarrow\mathbb{R}$ be continuous in $\left[a,b\right]$ and detachable in $\left(a,b\right)$ such that $f\left(a\right)=f\left(b\right).$ Then there exists a point $c\in\left(a,b\right)$ where:

$$f_{+}^{;}\left(c\right)+f_{-}^{;}\left(c\right)=0.$$

Proof

By Weierstrass’s theorem, $f$ attains a maximum $M$ and a minimum $m$ on $\left[a,b\right]$. If $m=M$, $f$ is constant, and every point of $\left(a,b\right)$ has both detachments zero. Otherwise, since $f\left(a\right)=f\left(b\right)$, one of the values $m,M$ is attained at some $c\in\left(a,b\right)$. Say it is the maximum; the minimum is symmetric. Then $f\left(c+h\right)\leq f\left(c\right)$ for all small $h$ of both signs, so $f_{+}^{;}\left(c\right)\in\left\{ -1,0\right\}$ and $f_{-}^{;}\left(c\right)\in\left\{ 0,+1\right\}$. If neither is zero, they are $-1$ and $+1$, and their sum is zero. If $f_{+}^{;}\left(c\right)=0$, then $f$ is constant on some interval $\left(c,c+\delta\right)$, and any point $\bar{c}$ inside it has $f_{+}^{;}\left(\bar{c}\right)=f_{-}^{;}\left(\bar{c}\right)=0$. The case $f_{-}^{;}\left(c\right)=0$ is the same on the left. $\,\,\,\,\blacksquare$

Theorem 8. Analog to Lagrange’s Mean Value Theorem. Let $f$ be continuous in $\left[a,b\right]$ and detachable in $\left(a,b\right)$, with $f\left(a\right)\neq f\left(b\right).$ Then for each $v$ strictly between $f\left(a\right)$ and $f\left(b\right)$ there are points $c_{v},c'_{v}\in f^{-1}\left(v\right)\cap\left(a,b\right)$ with

$$f_{+}^{;}\left(c_{v}\right)=f_{-}^{;}\left(c'_{v}\right)=\operatorname{sgn}\left[f\left(b\right)-f\left(a\right)\right].$$

Proof

Without loss of generality $f\left(a\right) < f\left(b\right)$. By the intermediate value theorem, the level set $L=f^{-1}\left(v\right)\cap\left[a,b\right]$ isn’t empty, and it is closed because $f$ is continuous. It contains neither $a$ nor $b$. Let $c_{v}=\max L$. On $\left(c_{v},b\right]$, $f-v$ is continuous, never zero, and positive at $b$, so it is positive throughout. Hence $f\left(\bar{x}\right) > v=f\left(c_{v}\right)$ for every $\bar{x}\in\left(c_{v},b\right]$, and $f_{+}^{;}\left(c_{v}\right)=+1$. In the same way, $c'_{v}=\min L$ has $f < v$ on $\left[a,c'_{v}\right)$, and $f_{-}^{;}\left(c'_{v}\right)=+1$. $\,\,\,\,\blacksquare$

As a corollary, "what goes up must come down", in an exact form. Suppose, in the setting of Theorem 8, that the level set $f^{-1}\left(v\right)$ is finite. Between two consecutive points of it, $f-v$ keeps a constant sign. At each point $c$ of the level set, $f_{+}^{;}\left(c\right)$ is the sign of $f-v$ on the gap to the right of $c$, and $f_{-}^{;}\left(c\right)$ is minus the sign on the gap to its left. Summing over the level set, the terms telescope:

$$\sum_{c\in f^{-1}\left(v\right)}\left[f_{-}^{;}\left(c\right)+f_{+}^{;}\left(c\right)\right]=2\operatorname{sgn}\left[f\left(b\right)-f\left(a\right)\right].$$

Both sides of each point are needed here. A point where $f$ touches the level $v$ from below and turns back contributes $+1$ from the left and $-1$ from the right, so it cancels out, as it should.

Feel free to interact with this illustration of Theorem 8 and its relation to other mean value theorems. For each value of $v$ in $\left( f(a),f(b) \right)$, we highlight at least one point whose detachment equals the function’s general trend in the interval.

Fundamental Theorem

We say that $f$ is continuously detachable on an interval if both its one-sided detachments exist at every point and are continuous there. Since they take values in $\left\{ -1,0,+1\right\}$, that means they are constant on the interval.

Lemma 9. A function $f$ is strictly monotonic in an open interval if and only if $f$ is continuously detachable there with $f^{;}_{-}=f^{;}_{+}\neq0$. If $f$ is strictly monotonic in a closed interval with end points $a < b$, then also

$$f^{;}=f^{;}_{+}\left(a\right)=f^{;}_{-}\left(b\right).$$

Proof

First direction. If $f$ is strictly increasing, then $f\left(x+h\right) > f\left(x\right) > f\left(x-h\right)$ for every $h > 0$ that keeps us in the interval, so both one-sided detachments equal $+1$ everywhere, including the right detachment at $a$ and the left one at $b$ for a closed interval. The decreasing case is symmetric.

Second direction. Assume $f^{;}_{-}=f^{;}_{+}=+1$ throughout the open interval, and let $x_{1} < x_{2}$ be two of its points. Let $s$ be the supremum of the points $t\in\left(x_{1},x_{2}\right]$ such that $f\left(u\right) > f\left(x_{1}\right)$ for every $u\in\left(x_{1},t\right]$. The right detachment at $x_{1}$ makes this set non-empty, so $s > x_{1}$. The left detachment at $s$ gives $f\left(u\right) < f\left(s\right)$ for $u$ slightly to the left of $s$, and those $u$ satisfy $f\left(u\right) > f\left(x_{1}\right)$, so $f\left(s\right) > f\left(x_{1}\right)$ as well. If $s < x_{2}$, the right detachment at $s$ would extend the property beyond $s$, contradicting the choice of $s$. Hence $s=x_{2}$ and $f\left(x_{2}\right) > f\left(x_{1}\right)$. $\,\,\,\,\blacksquare$

Remark

Both one-sided detachments are needed in Lemma 9. The function that equals $x$ for $x < 0$ and $x-1$ for $x\geq0$ has right detachment $+1$ at every point, yet it drops at $0$, where its left detachment is $-1$. For a continuous function, the right detachments alone are enough. Suppose $f$ is continuous on $\left[x_{1},x_{2}\right]$ and $f^{;}_{+}\equiv+1$ on $\left[x_{1},x_{2}\right)$. The maximum of $f$ on $\left[x_{1},x_{2}\right]$ is attained, but not at any point $m < x_{2}$, since $f^{;}_{+}\left(m\right)=+1$ puts larger values just right of $m$. So $x_{2}$ is the only maximizer and $f\left(x_{1}\right) < f\left(x_{2}\right)$. The same argument applies to every subinterval $\left[a,b\right]\subseteq\left[x_{1},x_{2}\right]$, so $f\left(a\right) < f\left(b\right)$ whenever $a < b$, and $f$ is strictly increasing. The case $-1$ is symmetric, with the minimum in place of the maximum.

On an interval where $f$ is continuously detachable the two one-sided detachments also agree at every point. Both are constant there. If they differed with neither equal to $0$, every point would be a strict local extremum, and a function has only countably many of those. If exactly one of them were $0$, $f$ would be constant on one side of every point, and that contradicts the nonzero value on the other side at nearby points. So $f^{;}$ is well defined on such an interval, which is how Claim 12 and Theorem 15 use it.

While the detachment is clearly not invertible, it is directly related to the derivative and the integral. The following theorem states those connections, and can be thought of as the fundamental theorem of Semi-discrete Calculus.

Theorem 10. A Semi-discrete Extension to the Fundamental Theorem of Calculus. The following relations between the detachment and the fundamental concepts of Calculus hold.

1. Let the one-sided derivative $f_{\pm}'\left(x\right)$ exist, finite or infinite, and differ from zero. Then $f$ is detachable from that side, and

   $$f_{\pm}^{;}\left(x\right)=\operatorname{sgn}\left[f_{\pm}'\left(x\right)\right].$$

   In particular, $f'\left(x\right)\neq0$ gives $f_{-}^{;}\left(x\right)=f_{+}^{;}\left(x\right)=\operatorname{sgn}\left[f'\left(x\right)\right]$.

2. Let $f$ be integrable in a closed interval $\left[a,b\right]$, let $F\left(x\right)\equiv \int_a^x f\left(t\right)dt$, and let $x\in\left(a,b\right)$. Assume that $f$ is s.c. at $x.$ Then $F$ is detachable at $x$, and

   $$F^{;}\left(x\right)= \operatorname{sgn}\left[f\left(x\right)\right].$$

3. Let $f:\left[a,b\right]\rightarrow\mathbb{R}$ be a continuous, piecewise monotone function with $f(a)\cdot f(b)\neq 0$, and let $F$ be an antiderivative of $f$ in $\left[a,b\right]$. Then:

   $$\int_{a}^{b}F^{;}(x)\,dx=b\operatorname{sgn}f\left(b\right)-a\operatorname{sgn}f\left(a\right)-\sum_{x_{i}\in f^{-1}\left(0\right)}\left[f_{-}^{;}\left(x_{i}\right)+f_{+}^{;}\left(x_{i}\right)\right]x_{i}.$$

Proof

1. Write the sign of the one-sided derivative as follows:

   $$\begin{align*} \operatorname{sgn}\left[ f_{\pm}'\left(x\right) \right] & = \operatorname{sgn}\left[\lim_{h\to0^{\pm}}\frac{f\left(x+h\right)-f\left(x\right)}{h}\right] =\lim_{h\to0^{\pm}}\operatorname{sgn}\left[\frac{f\left(x+h\right)-f\left(x\right)}{h}\right]\\ & =\pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left[f\left(x+h\right)-f\left(x\right)\right] =f_{\pm}^{;}\left(x\right). \end{align*}$$

   The second transition is where the hypothesis enters: a difference quotient that tends to a nonzero number, or to $\pm\infty$, eventually has that number’s sign. The sign function is continuous away from $0$, so the limit and the sign may be swapped. The third transition uses $\operatorname{sgn}\left(h\right)=\pm1$.

2. Since $\operatorname{sgn}f$ is continuous at $x$, $f$ has the constant sign $\operatorname{sgn}f\left(x\right)$ on some interval $\left(x-\delta,x+\delta\right)$. If that sign is $0$, $f$ vanishes there, $F$ is constant, and $F^{;}=0$. Otherwise the integral of $f$ over any interval of positive length inside $\left(x-\delta,x+\delta\right)$ has the sign of $f\left(x\right)$, and so do both increments

   $$F\left(x+h\right)-F\left(x\right)=\int_{x}^{x+h}f\left(t\right)dt,\qquad F\left(x\right)-F\left(x-h\right)=\int_{x-h}^{x}f\left(t\right)dt,$$

   for $0 < h < \delta$. Hence $F_{\pm}^{;}\left(x\right)=\operatorname{sgn}f\left(x\right)$. Continuity of the sign from one side is enough for the corresponding one-sided statement.

3. Since $f$ is continuous and piecewise monotone, its zero set is a finite union of points and closed intervals, and $f$ is detachable at each zero. Where $f\neq0$, part 1 gives $F^{;}=\operatorname{sgn}f$, and on an interval where $f$ vanishes, $F$ is constant and $F^{;}=0$. So $F^{;}$ is a step function. Let

   $$a=x_{0} < x_{1} < \dots < x_{n} < x_{n+1}=b,$$

   where $x_{1},\dots,x_{n}$ are the points where it changes value, and let $g_{i}$ be its value on $\left(x_{i},x_{i+1}\right)$. Summation by parts gives

   $$\int_{a}^{b}F^{;}\left(x\right)dx=\sum_{i=0}^{n}\left(x_{i+1}-x_{i}\right)g_{i}=b\,g_{n}-a\,g_{0}+\sum_{i=1}^{n}\left(g_{i-1}-g_{i}\right)x_{i}.$$

   By continuity $g_{n}=\operatorname{sgn}f\left(b\right)$ and $g_{0}=\operatorname{sgn}f\left(a\right)$. Each $x_{i}$ is a zero of $f$, so $f_{+}^{;}\left(x_{i}\right)$ is the sign of $f$ just to the right of $x_{i}$, namely $g_{i}$, and $f_{-}^{;}\left(x_{i}\right)$ is minus the sign of $f$ just to its left, namely $-g_{i-1}$. Hence

   $$g_{i-1}-g_{i}=-\left[f_{-}^{;}\left(x_{i}\right)+f_{+}^{;}\left(x_{i}\right)\right].$$

   At every other zero of $f$ the bracket vanishes, so the sum may run over the whole zero set, as in the statement. $\,\,\,\,\blacksquare$

Composite and Inverse Functions Rules

Claim 11. Chain Rule. If $g$ is continuous and detachable at $x$, and $f$ is detachable at $g\left(x\right)$, then $f\circ g$ is detachable at $x$, and the following holds there:

$$\left( f \circ g \right) ^{;}=\left(f^{;}\circ g\right)g^{;}.$$

Here $f^{;}$ is taken on the side from which $g\left(\bar{x}\right)$ approaches $g\left(x\right)$: for the right detachment of $f\circ g$, use the right detachment of $f$ if $g^{;}_{+}\left(x\right)=+1$ and its left detachment if $g^{;}_{+}\left(x\right)=-1$; for the left detachment of $f\circ g$ it is the other way around. When the two one-sided detachments of $f$ at $g\left(x\right)$ agree, as they do wherever $f$ passes through $g\left(x\right)$ in one direction, the side doesn’t matter.

Proof

Take the right side; the left side is analogous. If $g^{;}_{+}\left(x\right)=0$, then $g$ is constant on a right neighborhood of $x$, so is $f\circ g$, and both sides of the formula vanish. If $g^{;}_{+}\left(x\right)=+1$, then $u=g\left(x+h\right)$ satisfies $u > g\left(x\right)$ for small $h > 0$, and $u\to g\left(x\right)$ by continuity. So $f\left(u\right)-f\left(g\left(x\right)\right)$ eventually has the sign $f^{;}_{+}\left(g\left(x\right)\right)$. If $g^{;}_{+}\left(x\right)=-1$, then $u < g\left(x\right)$ and $u\to g\left(x\right)$, so $f\left(u\right)-f\left(g\left(x\right)\right)$ eventually has the sign $-f^{;}_{-}\left(g\left(x\right)\right)$, which is $f^{;}_{-}\left(g\left(x\right)\right)\cdot g^{;}_{+}\left(x\right)$. $\,\,\,\,\blacksquare$

For example, $\left|x\right|$ has the detachments $\left(-1,+1\right)$ at $0$, and $g\left(x\right)=-x$ has $g^{;}_{+}\left(0\right)=-1$. The rule takes the left detachment of $\left|\cdot\right|$ and gives $\left(-1\right)\cdot\left(-1\right)=+1$ for the right detachment of $\left|-x\right|$, which is correct.

Claim 12. Inverse Function Rule. Let $f$ be continuous on a neighborhood of $a$. Then $f$ is continuously detachable on a neighborhood of $a$ with $f^{;}\left(a\right)\neq0$, if and only if $f$ is invertible on a neighborhood of $a$. In that case $f^{-1}$ is continuously detachable on a neighborhood of $f\left(a\right)$ and:

$$\left(f^{-1}\right)^{;}\left(f\left(a\right)\right)=f^{;}\left(a\right).$$

On a neighborhood where $f$ is continuously detachable, the two one-sided detachments agree at every point (remark after Lemma 9), so $f^{;}$ is well defined there.

Proof

First direction. Since the detachments are constant and nonzero near $a$, Lemma 9 makes $f$ strictly monotonic on a neighborhood of $a$, hence invertible there. Second direction. A continuous function that is one-to-one on an interval is strictly monotonic there, so Lemma 9 applies again. In both cases, assume without loss of generality that $f$ is strictly increasing on an interval $J$ around $a$. By continuity, $f\left(J\right)$ is an interval with $f\left(a\right)$ in its interior, and $f^{-1}$ is strictly increasing on it, since $x < y\Longleftrightarrow f\left(x\right) < f\left(y\right)$. By Lemma 9, $\left(f^{-1}\right)^{;}\equiv+1=f^{;}\left(a\right)$ there. $\,\,\,\,\blacksquare$

Claim 13. Functional Power Rule. Given a base function $f$, an exponent function $g$, and a point $x$ in their definition domain, assume that the following conditions hold there:

1. The power function $f^{g}$ is well-defined; in particular $f > 0$ near $x$.

2. $f$ and $g$ are both detachable, and $f-1$ and $g$ are sign-consistent.

3. Either:

   • $f^{;}g^{;}\left(f-1\right)g\geq0$, where $f-1$ or $g$ is s.c., or $f=1$ and $g=0$, or:
   • $f^{;}g^{;}\left(f-1\right)g < 0$, where $f-1$ or $g$ is s.d.

Then the function $f^{g}$ is detachable there and:

$$\left(f^{g}\right)^{;}=\begin{cases} \left[\left(f-1\right)g\right]^{;}, & gg^{;}\left(f-1\right)f^{;}\geq0,\text{ and }f-1\text{ or }g\text{ is s.c.}\\ f^{;}g^{;}, & \text{else.} \end{cases}$$

As in Claim 5, the detachments are right detachments; on the left the second branch becomes $-f^{;}_{-}g^{;}_{-}$.

Proof

The claim follows from these transitions:

$$\begin{align*}\left(f^{g}\right)^{;} & =\left(e^{g\ln f}\right)^{;}=\left(g\ln f\right)^{;}\\ & =\begin{cases} \operatorname{sgn}\left[g^{;}\operatorname{sgn}\left(\ln f\right)+\left(\ln f\right)^{;}\operatorname{sgn}\left(g\right)\right], & gg^{;}\ln f\left(\ln f\right)^{;}\geq0\text{, and }g\text{ or }\ln f\text{ is s.c.}\\ \left(\ln f\right)^{;}g^{;}, & \text{else} \end{cases}\\ & =\begin{cases} \operatorname{sgn}\left[g^{;}\operatorname{sgn}\left(f-1\right)+f^{;}\operatorname{sgn}\left(g\right)\right], & gg^{;}\left(f-1\right)f^{;}\geq0\text{, and }g\text{ or }f-1\text{ is s.c.}\\ f^{;}g^{;}, & \text{else} \end{cases}\\ & =\begin{cases} \left[\left(f-1\right)g\right]^{;}, & gg^{;}\left(f-1\right)f^{;}\geq0,\text{ and }f-1\text{ or }g\text{ is s.c.}\\ f^{;}g^{;}, & \text{else.} \end{cases} \end{align*}$$

The second transition holds because the exponential function is strictly increasing, and a strictly increasing outer function leaves detachments unchanged. The third is the product rule (Claim 5) applied to $g$ and $\ln f$. The fourth uses $\operatorname{sgn}\left(\ln f\right)=\operatorname{sgn}\left(f-1\right)$, which also makes $\ln f$ s.c. or s.d. exactly when $f-1$ is, and $\left(\ln f\right)^{;}=f^{;}$, since the logarithm is strictly increasing. The last transition is Claim 5 again, applied to $f-1$ and $g$, together with $\left(f-1\right)^{;}=f^{;}$: subtracting a constant doesn’t change increments. $\,\,\,\,\blacksquare$

A simulation of the functional power rule is available here. This rule is another example of the detachment’s computational economy relative to the derivative sign. The power rule for derivatives produces a formula with logarithms and a division. The rule above looks more involved on paper because of its conditions, but it only compares signs, and depending on the setting it may be cheaper to compute.

Limits and Trends Evaluation Tools

The following statement expresses the trend through rates. It can be thought of as an interim step towards the familiar Calculus claims about classifying stationary points.

Claim 14. Corollary from Taylor Series. Let $f$ be $k$ times differentiable at $x$, where $k$ is the order of the first derivative that doesn’t vanish there:

$$f'\left(x\right)=\dots=f^{\left(k-1\right)}\left(x\right)=0\neq f^{\left(k\right)}\left(x\right).$$

Then $f$ is detachable at $x$, and

$$f_{\pm}^{;}\left(x\right)=\left(\pm1\right)^{k+1}\operatorname{sgn}\left[f^{\left(k\right)}\left(x\right)\right].$$

Proof

Taylor’s theorem with the Peano remainder gives

$$f\left(x+h\right)-f\left(x\right)=\frac{h^{k}}{k!}f^{\left(k\right)}\left(x\right)+o\left(h^{k}\right)\qquad\left(h\to0\right).$$

The remainder is eventually smaller in absolute value than the leading term, so for small $h$ the increment has the sign $\operatorname{sgn}\left(h^{k}\right)\operatorname{sgn}\left[f^{\left(k\right)}\left(x\right)\right]$. For $h\to0^{\pm}$, $\operatorname{sgn}\left(h^{k}\right)=\left(\pm1\right)^{k}$, and the coefficient $\pm1$ in the definition of the detachment adds one more factor of $\pm1$. $\,\,\,\,\blacksquare$

The hypothesis that some derivative doesn’t vanish can’t be dropped. The flat function $e^{-1/x^{2}}$ (with value $0$ at $0$) is infinitely differentiable with all derivatives zero at the origin, so Claim 14 says nothing there, while its detachments are $\left(-1,+1\right)$.

Remark

Claims 3, 5 and 6 (the sum and difference, product and quotient rules) impose varied conditions. When $f$ and $g$ are smooth enough, Claim 14 applied to the sum or product itself gives the answer without those conditions:

1. For the sum,

   $$\left(f+g\right)_{\pm}^{;}=\left(\pm1\right)^{k+1}\operatorname{sgn}\left[f^{\left(k\right)}+g^{\left(k\right)}\right],$$

   where $k$ is the smallest $i\geq1$ with $f^{\left(i\right)}\neq-g^{\left(i\right)}$ at $x$. An analogous statement holds for the difference $f-g$.

2. For the product, $\left(fg\right)_{\pm}^{;}$ is $\left(\pm1\right)^{k+1}$ times the sign of $\left(fg\right)^{\left(k\right)}$, where, by Leibniz’s rule,

   $$\left(fg\right)^{\left(k\right)}=\sum_{j=0}^{k}\binom{k}{j}f^{\left(j\right)}g^{\left(k-j\right)},$$

   and $k$ is the smallest $i\geq1$ for which this sum doesn’t vanish at $x$. All the middle terms of the sum matter: for $f=g=x$ at $0$ only the middle term of $\left(fg\right)''=2f'g'+f''g+fg''$ survives, and it gives $\left(\pm1\right)^{3}\operatorname{sgn}\left(2\right)=\pm1$, the minimum of $x^{2}$. An analogous statement holds for the quotient $\frac{f}{g}$.

For example, consider the functions $f\left(x\right)=x^{2}$ and $g\left(x\right)=-x^{4}$ at $x=0$. Claim 3 says nothing about $\left(f+g\right)^{;}$, since $f^{;}g^{;}=-1$. The first statement above, with $k=2$, gives

$$\left(f+g\right)_{\pm}^{;}\left(0\right)=\left(\pm1\right)^{2+1}\operatorname{sgn}\left(2+0\right)=\pm1.$$

Theorem 15. Analog to L’Hôpital Rule. Let $f,g:\mathbb{R}\rightarrow\mathbb{R}$ be a pair of functions and $x$ a point in their definition domain. Assume that on one side of $x$ both functions are continuously detachable on some one-sided neighborhood, so that $\lim_{t\to x^{\pm}}f^{;}\left(t\right)$ and $\lim_{t\to x^{\pm}}g^{;}\left(t\right)$ exist. If $\left|f\left(t\right)\right|$ and $\left|g\left(t\right)\right|$ tend, as $t\to x^{\pm}$, to the same limit $L\in\left\{ 0,\infty\right\}$, then:

$$\lim_{t\to x^{\pm}}\operatorname{sgn}\left[f\left(t\right)g\left(t\right)\right]=\lim_{t\to x^{\pm}}f^{;}\left(t\right)g^{;}\left(t\right).$$

The one-sided detachments of each function agree at every point of such a neighborhood (remark after Lemma 9), so $f^{;}$ and $g^{;}$ are well defined there.

Proof

We prove a slightly more general claim. Let $f_{1},\dots,f_{n}$ satisfy the same assumptions, with $\left|f_{i}\left(t\right)\right|\to L\in\left\{ 0,\infty\right\}$ for all $i$. Then

$$\lim_{t\to x^{\pm}}\operatorname{sgn}\prod_{i}f_{i}\left(t\right)=\left(\pm C\right)^{n}\lim_{t\to x^{\pm}}\prod_{i}f_{i}^{;}\left(t\right),$$

where $C=+1$ if $L=0$ and $C=-1$ if $L=\infty$. The theorem is the case $n=2$, where $\left(\pm C\right)^{2}=1$.

It suffices to prove the case $n=1$ and multiply. Write $f=f_{1}$ and focus on the right side. Suppose $f^{;}\equiv+1$ on some $\left(x,x+\delta\right)$. By Lemma 9, $f$ is strictly increasing there, so as $t\downarrow x$, $f\left(t\right)$ decreases to its infimum over $\left(x,x+\delta\right)$. If $L=0$, that infimum is $0$, so $f > 0$ there and $\lim\operatorname{sgn}f\left(t\right)=+1=f^{;}$. If $L=\infty$, the infimum is $-\infty$, and $\lim\operatorname{sgn}f\left(t\right)=-1=-f^{;}$. If $f^{;}\equiv0$, $f$ is constant there, which forces $f\equiv0$ when $L=0$ (and is impossible when $L=\infty$), so both sides vanish. The case $f^{;}\equiv-1$ is symmetric. On the left, $f\left(t\right)$ tends to its supremum instead, which flips both signs; hence the factor $\pm C$. $\,\,\,\,\blacksquare$

Remark

1. Without the conditions of Claims 3 and 5, the sum and product of detachable functions aren’t even guaranteed to be detachable. For example, consider the pair of functions, right-detachable at $x=0$:

   $$g_{1}\left(x\right)=\begin{cases} 1, & x > 0\\ 0, & x=0 \end{cases},\qquad g_{2}\left(x\right)=\begin{cases} 1, & x\in\mathbb{Q^{+}}\\ -1, & x\in\mathbb{R^{+}}\setminus\mathbb{Q}\\ 2, & x=0. \end{cases}$$

   Here $g_{1}$ rises from $0$ and $g_{2}$ drops from $2$, but their sum and product oscillate and aren’t detachable at zero. Counterexamples exist even if we assume differentiability on top of detachability.

2. Dropping the continuity assumption on $g$ in Claim 11 may result in a non-detachable $f \circ g$. For example, take $f\left(x\right)=x^{2}$ and, at $x=0$,

   $$g\left(x\right)=\begin{cases}\left|\sin\left(\frac{1}{x}\right)\right|, & x\neq0,\\-\frac{1}{2}, & x=0.\end{cases}$$

   Then $g$ is detachable at $0$ and $f$ is detachable at $-\frac{1}{2}$, but $f\circ g$ oscillates around $\frac{1}{4}$ near $0$.

Calculating Instantaneous Trends in Practice

Let’s calculate detachments directly from the definition, or with Claim 14. In the following examples, the functions are continuous but not differentiable, discontinuous, or differentiable. As a side note, we’ll also examine a case where the trend doesn’t exist.

Let $g\left(x\right)=\sqrt{\left|x\right|}$, which isn’t differentiable at zero. Since $\sqrt{\left|y\right|}$ is a strictly increasing function of $y^{2}$, we can drop the root and the absolute value inside the sign:

$$\begin{align*} g_{\pm}^{;}\left(x\right) & =\pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left(\sqrt{\left|x+h\right|}-\sqrt{\left|x\right|}\right)=\pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left[\left(x+h\right)^{2}-x^{2}\right]\\ & =\pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left[h\left(2x+h\right)\right]=\begin{cases} \operatorname{sgn}\left(x\right), & x\neq0\\ \pm 1, & x=0 \end{cases} \end{align*}$$

At zero the pair of one-sided detachments is $\left(-1,+1\right)$, a strict minimum. At other points the detachment agrees with the derivative’s sign, as Theorem 10 predicts. The Weierstrass function, which is nowhere differentiable, can be shown by similar means to be detachable at infinitely many points.

Next, let $\ell\left(x\right)=\operatorname{sgn}\left(x\right)$ (not to be confused with the sign inside the definition of the detachment), which is discontinuous at $x=0$. Its trends follow from the definition at once:

$$\ell_{\pm}^{;}\left(x\right)=\pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left[\operatorname{sgn}\left(x+h\right)-\operatorname{sgn}\left(x\right)\right]=\begin{cases} 0, & x\neq0\\ \pm \lim_{h\to0^{\pm}}\operatorname{sgn}\left(h\right)=+1, & x=0 \end{cases}$$

The jump at $0$ is an upward passage, with the pair $\left(+1,+1\right)$. The detachment records the direction of the jump and says nothing about its size, or about continuity.

Finally, let’s calculate trends with Claim 14, for functions differentiable enough times. These are the explicit calculations that Taylor-based theorems on classifying critical points usually leave implicit. For instance, consider $f\left(x\right) = -3x^{5}+5x^{3}$. Its derivative $f'\left(x\right)=15x^{2}\left(1-x^{2}\right)$ vanishes at $0$ and $\pm 1$. There $f''\left(x\right)=30x-60x^{3}$ and $f'''\left(x\right)=30-180x^{2}$ give:

$$\begin{align*}f_{\pm}^{;}\left(0\right) & =\left(\pm1\right)^{3+1}\operatorname{sgn}\left[f'''\left(0\right)\right]=\operatorname{sgn}\left(30\right)=+1\\ f_{\pm}^{;}\left(-1\right) & =\left(\pm1\right)^{2+1}\operatorname{sgn}\left[f''\left(-1\right)\right]=\pm\operatorname{sgn}\left(30\right)=\pm1\\ f_{\pm}^{;}\left(1\right) & =\left(\pm1\right)^{2+1}\operatorname{sgn}\left[f''\left(1\right)\right]=\pm\operatorname{sgn}\left(-30\right)=\mp1. \end{align*}$$

So $f$ rises through $0$, an inflection point, has a minimum at $-1$, where the pair is $\left(-1,+1\right)$, and a maximum at $+1$, where it is $\left(+1,-1\right)$.

A few quick ones to try. Find the pair $\left(f^{;}_{-}\left(0\right),f^{;}_{+}\left(0\right)\right)$ for $-\left|x\right|$, for $x\left|x\right|$, and for the floor function $\lfloor x\rfloor$, and name the local picture in each case.

Answers

For $-\left|x\right|$ the pair is $\left(+1,-1\right)$, a strict maximum without a derivative. For $x\left|x\right|$ it is $\left(+1,+1\right)$: the derivative at $0$ is zero, yet the function rises through $0$. For $\lfloor x\rfloor$, the value $\lfloor 0\rfloor=0$ lies above the values $-1$ just to the left and equals the values just to the right, so the pair is $\left(+1,0\right)$: the function rises into $0$, then stays flat.

Remark

For completeness, let’s show that the trend doesn’t exist at $x=0$ for the function

$$s\left(x\right)=\begin{cases} \sin\left(\frac{1}{x}\right), & x\neq0\\ 0, & x=0 \end{cases}$$

Take the two sequences $a_{n}=\frac{2}{\pi\left(1+4n\right)}$ and $b_{n}=\frac{1}{\pi\left(1+2n\right)}$, both tending to zero. Along them the sine is constant, $1$ and $0$ respectively, so the sign of the change is $+1$ along the first and $0$ along the second. By Heine’s definition of the limit, $s$ isn’t detachable at $0$. Indeed, this function has no instantaneous trend at $x=0$.

Multivariable Semi-discrete Calculus

The detachment helps generalize a familiar integration algorithm, which originated in computer vision, to general continuous domains.

The Integral Image algorithm calculates sums over rectangles in images efficiently. It was introduced in 2001, in a prominent AI article ([36]). To integrate a function over a rectangle in the plane, it pre-calculates the antiderivative, and then, in real time, adds up its values at the rectangle’s corners with alternating signs. It can be thought of as an extension of the Fundamental Theorem of Calculus to the plane. The following theorem generalized it further in 2007 ([38]). In our preliminary discussion we introduced works that apply the instantaneous trend of change explicitly, either numerically or analytically. In the following theorem the detachment is used only implicitly.

Theorem 16. (Wang et al.) Let $D\subset\mathbb{R}^{2}$ be a generalized rectangular domain (polyomino), and let $f:\mathbb{R}^{2}\rightarrow\mathbb{R}$ admit an antiderivative $F.$ Then:

$$\iint_{D}f\,\overrightarrow{dx}=\sum_{x\in\nabla D}\alpha_{D}\left(x\right)F\left(x\right),$$

where $\nabla D$ is the set of corners of $D$, and $\alpha_{D}$ takes values in $\left\{ 0,\pm1\right\}$ and is determined by the type of corner that $x$ belongs to.

The derivative sign doesn’t reflect trends at cusp points such as corners, so it doesn’t classify corner types concisely. The detachment operator, in contrast, classifies corners, and thus defines the parameter $\alpha_{D}$ coherently, independently of the curve’s parameterization. That’s a multidimensional version of our discussion above of how the two operators relate. Feel free to interact with the following widget to gain intuition regarding Theorem 16. The textboxes toggle the antiderivative at the annotated point, which is reflected in the highlighted domain. Watch the inclusion-exclusion principle in action as the domains finally aggregate to the domain bounded by the vertices. While this theorem is defined over continuous domains, it is limited to generalized rectangular ones. Let’s attempt to alleviate this limitation.

Interactive · A discrete Green's theorem

Autoplay. Touch any control to take over.

Corners in the combination

J+1K−2L+1M−1N+1O−1P+1Q+1R−1

AllClear

• counted +1, +2
• counted −1, −2
• Darker means counted more times

∬_D f dx dy = F(J) − 2F(K) + F(L) − F(M) + F(N) − F(O) + F(P) + F(Q) − F(R)

Σ α F over the chosen corners

0.0000

∬ of f, weighted by the shading

0.0000

∬_D f

11.2627

Choose corners. Each one adds or removes the quadrant below and to its left.

Computed live for f(x, y) = 1.2 + 0.8 sin(0.9x + 0.4) cos(0.7y − 0.3), with F(x, y) = ∫₀^x∫₀^y f. The shading integral uses quadrature, not F.

We use the detachment to define an integration method that extends Theorem 16 to non-rectangular domains by combining it with the mean value theorem, as follows. Given a simple closed detachable curve $C$, let $D$ be the region bounded by the curve. Let $R\subseteq D$ be a rectangular domain circumscribed by $C$. Let $C_i$ be the sub-curves of $C$ between consecutive points where it touches $R$. Let $D_{i}\subset D\setminus R$ be the sub-domain bounded between $C_i$ and $R$, let $\partial D_i$ be its boundary, and let $\nabla D_{i}$ be the intersection of $D_i$ with the vertices of $R$. By the mean value theorem in two dimensions, for each $i$ there is a point $x_{i}\in D_{i}$ such that

$$\iint_{D_{i}}f\,\overrightarrow{dx}=\beta_{i}f\left(x_{i}\right),$$

where $\beta_{i}$ is the area of $D_{i}$.

Our semi-discrete integration method accumulates a linear combination of the function and its antiderivative along the sub-domains $D_{i}$, as follows:

Theorem 17. Let $D\subset\mathbb{R}^{2}$ be a closed, bounded, and connected domain whose edge is detachable. Let $f:\mathbb{R}^{2}\rightarrow\mathbb{R}$ be a continuous function that admits an antiderivative $F$ there. Then:

$$\iint_{D}f\,\overrightarrow{dx}=\sum_{i}\left[\overrightarrow{\alpha_{i}}\cdot F\left(\overrightarrow{x_{i}}\right)+\beta_{i}f\left(x_{i}\right)\right],$$

where $F\left(\overrightarrow{x_{i}}\right)$ is the vector of the antiderivative’s values at the vertices $\nabla D_{i}$ of the sub-domain $D_{i}$, and $\overrightarrow{\alpha_{i}}$ is the vector obtained by applying a function $\alpha$ to the detachments of the curve $\partial D_{i}$ at those vertices. As above, $\beta_i$ is the area of $D_i$ and $x_i$ is its matching point from the mean value theorem for integrals. The function $\alpha$ is constructed within the integration method (see [31]). The formula holds for any detachable division of the curve. We don’t assume that the curve’s parameterization is differentiable, only that the function is continuous in the union of the $D_{i}$, so that the mean value theorem holds. Feel free to gain intuition from the following widget. Initially, the domain is rectangular: $D=R$. Moving the vertices exposes the yellow sub-domains $D_i$, whose areas are $\beta_i$. The integral over $R$ is calculated by aggregating the antiderivative’s values at the vertices of each $D_i$ (the highlighted points in this diagram). The antiderivative values enter the sum with coefficients that are determined by the curve’s detachments at each vertex (the function $\alpha$).

Interactive · Extended discrete Green's theorem

Autoplay. Touch any control to take over.

Vertex location

L′

−2.00

M′

−2.00

N′

1.00

O′

3.00

Edge curvature

LL′

0.50

L′M

0.50

MM′

2.00

M′N

2.00

NN′

0.50

N′O

0.50

OO′

2.00

O′L

2.00

Orientation

+ −

Subcurves

LL′L′MMM′M′NNN′N′OOO′O′L

• positive tendency
• negative
• zero
• not calculated
• one side only (coefficient ½)
• sub-domain D_i

(S)∮ F = −F(L′) + F(M′) − F(N) + F(N′) − F(O) + F(O′) + Σ ∬_Di f

Corner terms Σ α F

20.8756

Sub-domains Σ ∬_Di f

0.0000

Sum

20.8756

∬ D f (Green, independent)

20.8756

All eight subcurves: corner terms + sub-domains = ∬ over D, to machine precision.

Computed live for f(x, y) = 1.2 + 0.8 sin(0.9x + 0.4) cos(0.7y − 0.3). Drag L′, M′, N′, O′ along their dotted lines, or use the sliders.

Application in Stochastic Processes

Modern Physics is abundant with phenomena described by functions that are continuous everywhere and differentiable nowhere. A prominent example is fluctuations, random invisible movements of objects in their seemingly steady state. These are studied in Thermodynamics and Quantum Mechanics, to mention a few fields. Another prominent example is Brownian motion, whose statistical model, the Wiener process, resembles fluctuations.

Several characterizations of this process have been suggested, and we’ll specify one of them. Its value $W\left(t\right)$ adheres to the following:

1. $W\left(0\right)=0$.
2. $W$ has independent increments: for every $t > 0$, the future increments $W\left(t+u\right)-W\left(t\right)$, $u\geq0$, are independent of the past values $W\left(s\right)$, $s\leq t$.
3. $W$ has Gaussian increments: $W\left(t+u\right)-W\left(t\right)$ is normally distributed with mean $0$ and variance $u$.
4. $W$ has continuous paths: $W\left(t\right)$ is continuous in $t$.

This characterization of the Wiener process results in the following qualitative properties, which hold almost surely, as stated here:

1. For every $\epsilon > 0$, the function $W$ takes both (strictly) positive and (strictly) negative values on $\left(0,\epsilon\right)$.
2. The function $W$ is continuous everywhere but differentiable nowhere.
3. Points of local maximum of the function $W$ are a dense countable set.
4. The function $W$ has no points of local increase, that is, no $t > 0$ satisfies the following for some $\epsilon$ in $\left(0,t\right)$: first, $W\left(s\right)\leq W\left(t\right)$ for all $s$ in $\left(t-\epsilon,t\right)$, and, second, $W\left(s\right)\geq W\left(t\right)$ for all $s$ in $\left(t,t+\epsilon\right)$. (Local increase is a weaker condition than $W$ being increasing on $\left(t-\epsilon,t+\epsilon\right)$.) The same holds for local decrease.

Note how we can restate these properties in terms of the detachment operator:

“The function $W$ is continuous everywhere but detachable only at its dense countable local optima.”

Here "detachable" means from both sides, and the restatement follows from the nine local pictures. A path of Brownian motion is constant on no interval, which rules out every picture with a $0$ in it. It has no points of local increase or decrease, which rules out $\left(+1,+1\right)$ and $\left(-1,-1\right)$. Only the strict minimum $\left(-1,+1\right)$ and the strict maximum $\left(+1,-1\right)$ remain, and the local extrema of a Brownian path are strict, so it is detachable at each of them. The minima, like the maxima, form a dense countable set.

This description is more concise than the former. It also asserts that a property (detachability) holds on a dense subset (the local optima), instead of asserting that differentiability fails everywhere.

On top of its theoretical rigor, the numerical detachment also ran noticeably faster than the numerical derivative in our experiments on detecting fluctuations.

Summary

This was a short Semi-discrete Calculus tour where we presented the main definitions and results. The detachment answers a question the derivative answers only indirectly: on each side of a point, are the nearby values above, at, or below the value at the point? Where the derivative is nonzero, the two agree. Where it vanishes, blows up, or doesn’t exist, the detachment still gives an answer, and the rules above let us compute it. Hope you’ve learned and enjoyed it. Good luck with your Calculus adventures and stay safe!

References

[1] Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. Qsgd: Communication-ecient sgd via gradient quantization and encoding. In Advances in Neural Information Processing Systems, pages 1709-1720, 2017.

[2] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in neural information processing systems, pages 3981-3989, 2016.

[3] Sven Behnke. Hierarchical neural networks for image interpretation, volume 2766. Springer, 2003.

[4] Jeffrey Byrne. Nested motion descriptors. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 502-510, 2015.

[5] Zhixiang Chen, Xin Yuan, Jiwen Lu, Qi Tian, and Jie Zhou. Deep hashing via discrepancy minimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6838-6847, 2018.

[6] Kai Fan. Unifying the stochastic spectral descent for restricted boltzmann machines with bernoulli or gaussian inputs. arXiv preprint arXiv:1703.09766, 2017.

[7] Mathias Gallardo, Daniel Pizarro, Adrien Bartoli, and Toby Collins. Shape-from-template in flatland. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2847-2854, 2015.

[8] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180-1189, 2015.

[9] Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3681-3688, 2019.

[10] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.

[11] Zhe Hu, Sunghyun Cho, Jue Wang, and Ming-Hsuan Yang. Deblurring low-light images with light streaks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3382-3389, 2014.

[12] Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. Quantized neural networks: Training neural networks with low precision weights and activations. The Journal of Machine Learning Research, 18(1):6869-6898, 2017.

[13] Xiangyu Kong, Bo Xin, Yizhou Wang, and Gang Hua. Collaborative deep reinforcement learning for joint object search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1695-1704, 2017.

[14] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533, 2016.

[15] Jean Lafond, Hoi-To Wai, and Eric Moulines. On the online frankwolfe algorithms for convex and non-convex optimizations. arXiv preprint arXiv:1510.01171, 2015.

[16] Debang Li, Huikai Wu, Junge Zhang, and Kaiqi Huang. A2-rl: Aesthetics aware reinforcement learning for image cropping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8193-8201, 2018.

[17] Wei Li and Xiaogang Wang. Locally aligned feature transforms across views. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3594-3601, 2013.

[18] Li Liu, Ling Shao, Fumin Shen, and Mengyang Yu. Discretely coding semantic rank orders for supervised image hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1425-1434, 2017.

[19] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2574-2582, 2016.

[20] Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Jonathan Uesato, and Pascal Frossard. Robustness via curvature regularization, and vice versa. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9078-9086, 2019.

[21] Ali Mousavi, Arian Maleki, Richard G Baraniuk, et al. Consistent parameter estimation for lasso and approximate message passing. The Annals of Statistics, 46(1):119-148, 2018.

[22] Vidhya Navalpakkam and Laurent Itti. Optimal cue selection strategy. In Advances in neural information processing systems, pages 987-994, 2006.

[23] Jinshan Pan, Zhouchen Lin, Zhixun Su, and Ming-Hsuan Yang. Robust kernel estimation with outliers handling for image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2800-2808, 2016.

[24] Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. Crafting adversarial input sequences for recurrent neural networks. In MILCOM 2016-2016 IEEE Military Communications Conference, pages 49-54. IEEE, 2016.

[25] Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos. In Advances in neural information processing systems, pages 3360-3368, 2016.

[26] Aaditya Ramdas and Aarti Singh. Algorithmic connections between active learning and stochastic convex optimization. In International Conference on Algorithmic Learning Theory, pages 339-353. Springer, 2013.

[27] Miriam Redi, Neil O’Hare, Rossano Schifanella, Michele Trevisiol, and Alejandro Jaimes. 6 seconds of sound and vision: Creativity in micro-videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4272-4279, 2014.

[28] Jaakko Riihimäki and Aki Vehtari. Gaussian processes with monotonicity information. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 645-652, 2010.

[29] Swami Sankaranarayanan, Arpit Jain, Rama Chellappa, and Ser Nam Lim. Regularizing deep networks using efficient layerwise adversarial training. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.

[30] Hee Seok Lee and Kuoung Mu Lee. Simultaneous super-resolution of depth and images using a single camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 281-288, 2013.

[31] Amir Shachar. Applying semi-discrete operators to calculus. arXiv preprint arXiv:1012.5751, 2010.

[32] Eero Siivola, Aki Vehtari, Jarno Vanhatalo, Javier González, and Michael Riis Andersen. Correcting boundary over-exploration deficiencies in bayesian optimization with virtual derivative sign observations. In 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1-6. IEEE, 2018.

[33] Michel Silva, Washington Ramos, Joao Ferreira, Felipe Chamone, Mario Campos, and Erickson R Nascimento. A weighted sparse sampling and smoothing frame transition approach for semantic fast-forward first-person videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2383-2392, 2018.

[34] Yansong Tang, Yi Tian, Jiwen Lu, Peiyang Li, and Jie Zhou. Deep progressive reinforcement learning for skeleton-based action recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5323-5332, 2018.

[35] Roberto Tron and Kostas Daniilidis. On the quotient representation for the essential manifold. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1574-1581, 2014.

[36] Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, volume 1, pages II. IEEE, 2001.

[37] Haohan Wang and Bhiksha Raj. On the origin of deep learning. arXiv preprint arXiv:1702.07800, 2017.

[38] Xiaogang Wang, Gianfranco Doretto, Thomas Sebastian, Jens Rittscher, and Peter Tu. Shape and appearance context modeling. In 2007 ieee 11th international conference on computer vision, pages 18. Ieee, 2007.

[39] Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. Terngrad: Ternary gradients to reduce communication in distributed deep learning. In Advances in neural information processing systems, pages 1509-1519, 2017.

[40] Shaodan Zhai, Tian Xia, Ming Tan, and Shaojun Wang. Direct 0-1 loss minimization and margin maximization with boosting. In Advances in Neural Information Processing Systems, pages 872-880, 2013.

[41] Yu Zhu, Yanning Zhang, Boyan Bonev, and Alan L Yuille. Modeling deformable gradient compositions for single-image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5417-5425, 2015.

[42] Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.

[43] Bernhard Riemann. Uber die Darstellbarkeit einer Function durch eine trigonometrische Reihe, Gott. Abh. (1854/1868); also in: Gesammelte Mathematische Werke, Springer-Verlag, Berlin 1990, 259–296.