This means they come with different tool sets, as well. Your typical 3D modeling and animation program comes with a wide range of tools designed to build a world from the ground up, and then animate that world as smoothly as possible. As a result, it has entire toolsets devoted to the more artistic side of modeling and animation, from shape to texture - plus the toolsets devoted to creating seamless timeline-based animations involving multiple objects interacting with their environments. CAD programs instead focus on creating scale-accurate technical designs that would function in the real world the same way they function in their virtual environment. The tools focus more on scale, measurements, and precision, because these models have to be accurate enough to be used in production, construction, or even in physical simulations. Some programs, such as Google Sketchup, do try to combine the two, but with varying degrees of success.
The quality of the output differs, as well. 3D animation and modeling programs focus on high-poly renders with detailed textures and bump maps, with such finely-tuned things as strands of hair and fur, flowing fabric, individual tree leaves, animated particle systems, moving bodies of water, falling rain, etc. The entire goal is to create the most visually appealing output possible. In CAD programs, how it looks isn't as important as how it works. You don't have the same tools on hand to create detailed, high-poly renders with maps and other enhancements. The output from CAD programs is generally much simpler and bare-bones, just as an engineering or drafting diagram should be.
That's not to say you can't produce detailed models in CAD software, though it's much more time-consuming and difficult, and CAD programs really aren't cut out for something like character animation. Most lack bone systems, particle systems, hair systems, and the other key aides that are practically standard in modern 3D modeling and animation programs. Environmental modeling and animation would be extremely difficult as well, without the ability to use certain types of maps and tools.
Conversely, you could also create accurate, functional architectural, mechanical, and engineering models, artwork, and blueprints in a standard 3D modeling and animation program - but again, you'd run into difficulty. While it's easier to make a complex program do something simple than it is to make a simple program to something complex, most standard 3D animation and modeling programs don't bend well towards the workflows used in producing models in CAD programs, especially with any level of accuracy.
So in the end, when you take the long view, there really isn't much difference between CAD programs and other 3D modeling and animation programs. When you get up close and personal, though, the devil's in the details, and it's all about functionality and design. A Ferrari and a Honda are both cars, but one's designed for speed, the other for reliable transport. It's the same sort of difference between CAD programs and 3D animation software.