Lines and circles are perhaps the most basic geometric objects. Most of classical Euclidean geometry concerns things built from lines and circles. Triangles, for example, are nothing but the union of three intersecting lines. [Though we usually erase the pieces of the lines extending to infinity, it does no harm to keep them. Any statement about these extended triangles is equivalent to one about the normal, bounded ones, since the set of vertices is the same.] In fact, the Greeks loved lines and circles so much they often considered questions of constructibility, that is, what can be done geometrically using only an unmarked straightedge and a compass. In other words, what geometry is generated by only considering objects which are somehow built from just lines and circles. These questions weren't answered until the nineteenth century, when the mathematics of fields was able to exactly characterize what is possible using these tools.
It is sometimes said that a straight line is a circle with infinite radius. This statement can be formally understood in several ways, but the notion of curvature is helpful in this department. Curvature of arbitrary curves is defined and calculated using the tools of differential calculus. For lines and circles, though, we can appeal to intuition. Imagine driving a car around a circular track, going at constant speed. If the circle is very large, you barely need to turn the steering wheel. That's because the rate at which the angle your car is pointed doesn't have to be large to keep you on the road. Now if the circle is very small, the curve feels tighter, and you must turn the wheel more to stay on the road. This suggests that the curvature should be inversely related to the radius of the circle. In fact, that is exactly the definition. For an arbitrary curve, we would use calculus methods to approximate the curve with a circle, and the curvature of the point would be the curvature of the circle. Now if you drive on a straight road, you need not turn the steering wheel. Our intuition says the curvature of a line should be zero. The "approximating circle" is obtained by taking circles that get bigger and bigger so that their curvature becomes smaller and smaller. So in that sense, a line is a "circle of infinite radius."
So far, though, I've only talked about lines and circles that live in a flat plane. In fact, lines and circles look very different when we change the space they live in. First let's use the following definition of a line: given two points, it is the curve of shortest length connecting the points. We generalize the notion of a line by calling any curve that satisfies that definition a geodesic. If you were an ant living on the surface, and your Aunt Antonia asked you what the quickest way to the neighboring ant hill, you'd tell her to walk along a geodesic. In the upper half-plane model of the hyperbolic plane, geodesics are actually circles which intersect the boundary of the plane in right angles. Taking each term in the proper context, we can really say that in a hyperbolic world, "the lines are circles."
[Note: I recently revived this blog and changed its title, threw up a few of my favorite columns from the last year, and added a review I wrote of a Rumpus Book Club selection. This post is just to explain the new title.]