What's the deal with integration?

To some extent you are correct, but I would not go as far as comparing exact integration to calligraphy. There are enough practical situations where an exact solutions using integration by one of the numerous integration techniques actually can be computed. Moreover, numerical techniques have their problems. They could be costly, taking a long time to compute, and there is the issue of how accurate they are.

So, for integration it is best to know at least some integration techniques as well as some numerical methods. In this way you learn to appreciate the pros and cons of each method and you should be proficient enough to be able to use whichever is most suitable, or a combination of the two, when solving practical problems.

I do think though that there is far too much emphasis in the standard curricula on integration techniques. This is largely due to historical reasons. It used to be a common task of mathematicians to solve the integrals of, say, physicists. Today the physicists just use some software (e.g., Wolfram alpha) to solve their integrals and, if the integral is standard enough, the computer will usually give a much better answer than a mathematician would do.

It's a bit like matrices. To understand matrix multiplication you need to multiply at least a few $4\times 4$ matrices, but there is no point in becoming a grand master of multiplying matrices (though it used to be a valuable skill before computers showed up). So is it with integration. To understand what it is you need to compute at least a few integrals using substitution and integration by parts, and see some general tricks, but today there is little justification to the over-drilling of integration techniques in undergraduate courses.

"Most integrals in application can not even be evaluated in elementary terms anyway."

That is an overstatement. For example, polynomials occur in applications, and their integrals can be evaluated in elementary terms. Just to take a very simple example: The position of a falling object (constant acceleration) is given by a quadratic function of time, which is obtained by integration.

To me, numerical integration is the last solution I should consider because of its cost and because the problem of accuracy. Special functions coming from integration of complex terms have received a lot of attention and specific, taylor made, subroutines have been developed for thei accurate evaluation. Why do you think that so many libraries of subroutines have been developed ?