We typically use the slope-intercept form of the line: \(y = a + bx\), where \(a = y(0)\) is the intercept and \(b = \Delta y / \Delta x\) is the slope (rise over run).
For example: \(y = 1.5 + 0.5x\).
If \(b = 0\), the line is horizontal.
A graph of a line with the equation \(y = 1 + 2x\).
A graph of a line with the equation \(y = 7.5 - 2x\).
# y = 1 + 2xp <-ggplot() +xlim(0, 5) +ylim(0, 10)p +geom_abline(slope =2, intercept =1, linewidth =0.2)# y = 7.5 - 2xp <-ggplot() +xlim(0, 5) +ylim(0, 10)p +geom_abline(slope =-2, intercept =7.5, linewidth =0.2)# to see some other ways of plotting lines?geom_abline
Your Turn
Turn the following code into a function called plot_line(slope, intercept)
Test it for different values of slope and intercept
Choose test values whose lines are visible in the fixed plotting window.
Exponential Functions
The idea of an exponential function is that the rate of change of the function at time \(t\) is proportional to the value of the function at time \(t\). In other words:
\[
\frac{\text{d}[y(t)]}{\text{d}t} = k \cdot y(t)
\]
The general solution to this differential equation is \(y(t) = C e^{kt}\), where \(C = y(0)\).
Think of population growth or growth of an interest-bearing asset.
Example: Compound Interest
An asset that has a value \(A\) is invested with an annual interest rate \(r\). What is the balance in the account, \(P_1\), at the end of the first year?
rate <-0.05# interest rate rP <-100# principal Pn_comp <-1:100# number of compoundings nPn <-function(n, A, r, t) A * (1+ r/n)^(t*n)Pe <-function(A, r, t) A *exp(r * t)pn <-Pn(n = n_comp, A = P, r = rate, t =50)pe <-Pe(A = P, r = rate, t =50)d <-data.frame(n_comp, pn)p <-ggplot(d, aes(n_comp, pn))p +geom_line(linewidth =0.2) +geom_hline(yintercept = pe, color ='red', linewidth =0.2) +xlab("Number of times the interest is compounded") +ylab("Value of an asset") +ggtitle("Value of $100 at r = 0.05 after 50 years")
Your Turn
Suppose a particular population of bacteria is known to double in size every 4 hours. If a culture starts with 1000 bacteria, what is the population after 10 hours and 24 hours?
The Number \(e\)
Recall the equation for asset value after \(t\) years compounded \(n\) times:
\[
P_n(t) = A \left (1 + \frac{r}{n} \right)^{nt}
\]
Let \(m = n/r\) and take the limit as \(m \to \infty\).
Suppose $750 is invested in an account at an annual interest rate of 5.5%, compounded continuously. Let \(t\) denote the number of years after the initial investment and \(A(t)\) denote the amount of money in the account at time \(t\). Find a formula for \(A(t)\). Find the amount of money in the account after 5 years, 10 years, and 50 years.
For \(a > 0\) and \(a \ne 1\), an exponential function of the form \(f(x) = a^x\) is one-to-one, so it has an inverse called a logarithmic function. Logarithm arguments must be positive.
The following relationship always holds for \(a > 0\): \(a^t = e^{\log(a)t}\) since \(e^{\log(a)} = a\)
\(\log_a(uv) = \log_a(u) + \log_a(v)\)
\(\log_a(u/v) = \log_a(u) - \log_a(v)\)
\(\log_a u^n = n \log_a u\)
In R and throughout this course, \(\log\) means the natural log, base \(e\). The natural log and the exponential function undo one another:
So a coefficient of \(0.05\) approximately multiplies \(y\) by \(1.05\), corresponding to a 5% increase. The exact increase is \(100(e^{0.05}-1)\% \approx 5.13\%\).
In statistics and machine learning, we often want to normalize a vector so its components add to one. The softmax function is
The second term is the log-sum-exp, or LSE. To evaluate it stably, subtract the largest input before exponentiating.
\[
\begin{aligned}
m &= \max_j x_j, \\
\operatorname{LSE}(x)
&= \log \sum_{j=1}^N \exp(x_j) \\
&= m + \log \sum_{j=1}^N \exp(x_j - m).
\end{aligned}
\]
Because \(m = \max_j x_j\), the largest shifted exponent is zero and every other shifted exponent is negative.
Your Turn: write LSE(x) and log_softmax(x), test them on small and large inputs, exponentiate the log probabilities, and check that they add to 1.
LSE <-function(x) { m <-max(x) m +log(sum(exp(x - m)))}log_softmax <-function(x) x -LSE(x)softmax <-function(x) exp(log_softmax(x))softmax(y_small)
[1] 0.09003057 0.24472847 0.66524096
softmax(y_large)
[1] 0.09003057 0.24472847 0.66524096
sum(softmax(y_large))
[1] 1
Limits
A limit is the value a function approaches as its input gets arbitrarily close to a specified point. The limit can exist even when the function is undefined there or has a different value there.
This demo shows what happens to the secant line as it approaches the tangent.
A Cruel Joke
A thousand years from now, your friends drug you, load you onto a ship, and fire it toward an unknown moon. You wake in a spacesuit beside the ship, looking at a 100 m tower. A metal ball is mounted at the top, you can release it remotely, and you have a stopwatch.
The ship’s computer refuses to launch until you identify the moon and enter its gravitational acceleration \(g\)
Enter the correct answer and the autopilot takes you home. Enter the wrong answer and the ship follows the wrong trajectory — and crashes
You remain on the ground, where you can see the full tower, and remotely release the ball from the top
A sensor flashes a light each time the falling ball crosses a 5 m mark, giving you 20 known distances down the tower
You use the stopwatch to record the time of every flash, producing pairs \((t_k, x_k)\)
From those observations, you must estimate \(g\), match it to the correct moon, and enter your answer
Setting Up the Equation of Motion
Let \(x(t)\) measure downward distance from the release point, with \(x(0)=0\). The ball is released from rest at \(t=0\)
Ignoring the atmosphere and treating \(g\) as constant near the moon’s surface, free fall is modeled by
\[
x(t) = \tfrac{1}{2}\,g\,t^2
\]
For now, take this equation as given. We will derive it after we learn integration
In the game, \(g\) is unknown. Each flash gives a known distance \(x_k\) and a measured time \(t_k\); later we will use those observations to estimate \(g\)
The Drop
For this worked example, we are not estimating \(g\) yet. We use the accepted gravity on Earth’s Moon, \(g_{\text{Moon}} = 1.625\ \text{m/s}^2\). At that value, a 100 m drop takes about 11 s, slow enough to time the flashes with a stopwatch; on Earth it would take only 4.5 s.
The red curve uses the accepted value of \(g\); it is not estimated from these observations. Later, we will reverse the problem and recover an unknown \(g\) from data like these.
Oxygen is limited, so each drop costs you. One drop gives up to 20 timed marks (this run caught 19) — how many marks, and how many drops, do we need to pin down \(g\) to one decimal place? To two? We return to this in Session 7.
Average Velocity and the Secant
Before we can find \(g\), we need to understand velocity. What was the ball’s average velocity between \(t_1\) and \(t_2\) seconds? This is the slope of the secant line:
We take the limit again, this time of the velocity function \(v(t) = g t\):
\[
a = \frac{dv}{dt} = \lim_{h \to 0} \frac{g(t + h) - g t}{h} =
\lim_{h \to 0} \frac{g h}{h} = \lim_{h \to 0} g = g
\]
The acceleration \(a(t) = g\) does not depend on \(t\) — it is constant. This constant is exactly what the mission needs. Recovering it from the drop data is the inference problem we solve in Sessions 6 and 7.
It would be cumbersome to compute derivatives this way. Fortunately, we have shortcuts.
# approximate the derivative on each interval and align it to the midpointappr <-tibble(t = (t[-1] + t[-n]) /2,appr_dxdt =diff(x) /diff(t))p +geom_line(aes(y = dxdt), color ="red") +geom_line(data = appr, aes(t, appr_dxdt),color ="blue", linewidth =5, alpha =1/5 )
Examples of the Chain Rule
What is the derivative of \(\sin(x^2)\)? It is \(2x \cos(x^2)\)
