We typically use the slope-intercept form of the line: \(y = a + bx\), where \(a = y(0)\) is the intercept and \(b = \Delta y / \Delta x\) is the slope (rise over run).
For example: \(y = 1.5 + 0.5x\).
If \(b = 0\), the line is horizontal.
A graph of a line with the equation \(y = 1 + 2x\).
A graph of a line with the equation \(y = 7.5 - 2x\).
# y = 1 + 2xp <-ggplot() +xlim(0, 5) +ylim(0, 10)p +geom_abline(slope =2, intercept =1, linewidth =0.2)# y = 7.5 - 2xp <-ggplot() +xlim(0, 5) +ylim(0, 10)p +geom_abline(slope =-2, intercept =7.5, linewidth =0.2)# to see some other ways of plotting lines?geom_abline
Your Turn
Turn the following code into a function called plot_line(slope, intercept)
Test it for different values of slope and intercept
Choose test values whose lines are visible in the fixed plotting window.
Exponential Functions
The idea of an exponential function is that the rate of change of the function at time \(t\) is proportional to the value of the function at time \(t\). In other words:
\[
\frac{\text{d}[y(t)]}{\text{d}t} = k \cdot y(t)
\]
The general solution to this differential equation is \(y(t) = C e^{kt}\), where \(C = y(0)\).
Think of population growth or growth of an interest-bearing asset.
Example: Compound Interest
An asset that has a value \(A\) is invested with an annual interest rate \(r\). What is the balance in the account, \(P_1\), at the end of the first year?
rate <-0.05# interest rate rP <-100# principal Pn_comp <-1:100# number of compoundings nPn <-function(n, A, r, t) A * (1+ r/n)^(t*n)Pe <-function(A, r, t) A *exp(r * t)pn <-Pn(n = n_comp, A = P, r = rate, t =50)pe <-Pe(A = P, r = rate, t =50)d <-data.frame(n_comp, pn)p <-ggplot(d, aes(n_comp, pn))p +geom_line(linewidth =0.2) +geom_hline(yintercept = pe, color ='red', linewidth =0.2) +xlab("Number of times the interest is compounded") +ylab("Value of an asset") +ggtitle("Value of $100 at r = 0.05 after 50 years")
Your Turn
Suppose a particular population of bacteria is known to double in size every 4 hours. If a culture starts with 1000 bacteria, what is the population after 10 hours and 24 hours?
The Number \(e\)
Recall the equation for asset value after \(t\) years compounded \(n\) times:
\[
P_n(t) = A \left (1 + \frac{r}{n} \right)^{nt}
\]
Let \(m = n/r\) and take the limit as \(m \to \infty\).
Suppose $750 is invested in an account at an annual interest rate of 5.5%, compounded continuously. Let \(t\) denote the number of years after the initial investment and \(A(t)\) denote the amount of money in the account at time \(t\). Find a formula for \(A(t)\). Find the amount of money in the account after 5 years, 10 years, and 50 years.
For \(a > 0\) and \(a \ne 1\), an exponential function of the form \(f(x) = a^x\) is one-to-one, so it has an inverse called a logarithmic function. Logarithm arguments must be positive.
In R and this course, \(\log\) means the natural log, base \(e\). For a log-scale coefficient \(\beta\), the exact percentage change is \(100(e^\beta - 1)\%\); \(100\beta\%\) is a useful approximation only when \(\beta\) is small. For example, \(\log(1.04) \approx 0.0392\).
The following relationship always holds for \(a > 0\): \(a^t = e^{\log(a)t}\) since \(e^{\log(a)} = a\)
\(\log_a(uv) = \log_a(u) + \log_a(v)\)
\(\log_a(u/v) = \log_a(u) - \log_a(v)\)
\(\log_a u^n = n \log_a u\)
In statistics and machine learning, we often want to normalize a vector so its components add to one. The softmax function is
The second term is the log-sum-exp, or LSE. To evaluate it stably, subtract the largest input before exponentiating.
\[
\begin{aligned}
m &= \max_j x_j, \\
\operatorname{LSE}(x)
&= \log \sum_{j=1}^N \exp(x_j) \\
&= m + \log \sum_{j=1}^N \exp(x_j - m).
\end{aligned}
\]
Because \(m = \max_j x_j\), the largest shifted exponent is zero and every other shifted exponent is negative.
Your Turn: write LSE(x) and log_softmax(x), test them on small and large inputs, exponentiate the log probabilities, and check that they add to 1.
LSE <-function(x) { m <-max(x) m +log(sum(exp(x - m)))}log_softmax <-function(x) x -LSE(x)softmax <-function(x) exp(log_softmax(x))softmax(y_small)
[1] 0.09003057 0.24472847 0.66524096
softmax(y_large)
[1] 0.09003057 0.24472847 0.66524096
sum(softmax(y_large))
[1] 1
Limits
A limit is the value a function approaches as its input gets arbitrarily close to a specified point. The limit can exist even when the function is undefined there or has a different value there.
This demo shows what happens to the secant line as it approaches the tangent.
A Cruel Joke
A thousand years from now, your friends drug you, load you onto a ship, and fire it at the Moon. You wake in a spacesuit on the surface: a ship, a 100 m tower beside you, a metal ball stamped 1 kg, and a stopwatch.
The ship’s autopilot has the equations of motion programmed in — but it is missing one number: the Moon’s gravitational acceleration \(g\)
Your plan: climb the tower, drop the ball, and record the time it passes each red mark (spaced 5 m apart, 20 marks down)
Each mark has a sensor: as the ball passes, it flashes a light, so from the top you read the time off your stopwatch at each flash
From those \((t_k, x_k)\) pairs you will infer\(g\) — and fly home
Setting Up the Equation of Motion
Let \(x\) measure downward distance from the release point, so downward is positive. There is no atmosphere on the Moon, so the only force on the ball is gravity. Newton’s second law gives:
\[
m\,\ddot{x} = m\,g \quad\Rightarrow\quad \ddot{x} = g
\]
The mass cancels — that is why the 1 kg label is a red herring. This is Galileo’s insight, confirmed on the Moon by Apollo 15’s hammer-and-feather drop.
Integrating twice (we cover integration in Session 3), and dropping from rest, gives the solution we will use:
So the acceleration is the second derivative of position — a constant \(g\) — which is exactly the unknown the mission needs.
The Drop, in Data
Low gravity is a feature: on the Moon a 100 m drop takes about 11 s, slow enough to read marks with a stopwatch. On Earth it would take 4.5 s.
The inference problem (Session 6) is to recover \(g\) from noisy time readings at the known sensor positions, assuming \(x(t) = \tfrac{1}{2} g t^2\). The curve below uses the accepted reference value \(g_{\mathrm{ref}} = 1.625\ \text{m/s}^2\); it is not an estimate from these observations.
Oxygen is limited, so each drop costs you. One drop gives up to 20 timed marks (this run caught 19) — how many marks, and how many drops, do we need to pin down \(g\) to one decimal place? To two? We return to this in Session 7.
Average Velocity and the Secant
Before we can find \(g\), we need to understand velocity. What was the ball’s average velocity between \(t_1\) and \(t_2\) seconds? This is the slope of the secant line:
We take the limit again, this time of the velocity function \(v(t) = g t\):
\[
a = \frac{dv}{dt} = \lim_{h \to 0} \frac{g(t + h) - g t}{h} =
\lim_{h \to 0} \frac{g h}{h} = \lim_{h \to 0} g = g
\]
The acceleration \(a(t) = g\) does not depend on \(t\) — it is constant. This constant is exactly what the mission needs. Recovering it from the drop data is the inference problem we solve in Sessions 6 and 7.
It would be cumbersome to compute derivatives this way. Fortunately, we have shortcuts.
Rules of Differentiation
We will state a few rules of differentiation without deriving them
# approximate the derivative on each interval and align it to the midpointappr <-tibble(t = (t[-1] + t[-n]) /2,appr_dxdt =diff(x) /diff(t))p +geom_line(aes(y = dxdt), color ="red") +geom_line(data = appr, aes(t, appr_dxdt),color ="blue", linewidth =5, alpha =1/5 )
Examples of the Chain Rule
What is the derivative of \(\sin(x^2)\)? It is \(2x \cos(x^2)\)
