The math in "There's a delta for every epsilon"
Intuitively, a function f(x) is continuous at a number a if, as x gets closer and closer to a, then f(x) gets closer and closer to f(a). This is what it means to "not lift up your pencil": as you get closer to a in the horizontal direction, you get closer to f(a) in the vertical direction. Now what does it mean for two numbers to get "closer and closer"? Well, it just means that the distance between them gets smaller and smaller. On the real line, we measure the distance between a and x using the absolute value of their difference: |x-a|. So if x is getting closer and closer to a, the distance |x-a| is getting smaller and smaller. Similarly, if f(x) is getting closer and closer to f(a), then the distance |f(x) - f(a)| is getting smaller and smaller. Moreover, if we want the distance between f(x) and f(a) to be less than some number epsilon, we say that |f(x)-f(a)| < epsilon.
Now let's get back to distances getting smaller and smaller. How small do we want the distance between f(x) and f(a) to be? Arbitrarily small! That is, we want to say that no matter how small you want |f(x) - f(a)| to be --- say, less than some distance epsilon --- then f attains these small distances when x is sufficiently close to a -- that is, when the distance |x-a| is less than some number delta. We need to be able to find a delta for *any* given distance epsilon, so we say:
For all epsilon > 0 (since it only makes sense to talk about positive distances!), there is a delta > 0 such that if |x-a| < delta, then |f(x) - f(a)| < epsilon.
This is the epsilon-delta definition of continuity! So now you can see how much fun Tom Lehrer is having in the song: since the epsilon represents a distance, it only makes sense if it's positive!
One last note: the lines " And now and again, / There's also an N" refer to the definition of the limit of an infinite sequence of numbers. But that's another story.
Background image: covers from some of Tom Lehrer's albums