In classification problems we use the training set to learn a classifier, and then we can evaluate how good this classsifier does "in the real world" by measuring its error on a test set.

So if for example the classifier has x% accuracy on the test set, I would expect it to have similar accuracy (more or less) on new data and it gives some intuitive notion of how well this classifier works. So if I got 95% accuracy on the test I could decide it's good enough, but 80% accuracy is not-so-good.

What is the equivalent of that in regression problems, and in general - in any model that maps samples to real values? You can calculate some measure of error on the test, such as the mean square error, but then how do you evaluate that?

I know it seems like a very basic question, but we didn't cover that in the lecture…

Evaluation of regression models