If we are talking about measures of center, then the average (mean) will change when the outlier is excluded. That's because, median, unlike the mean, is not changed by an outlier. Let's see this through an example.
We have the data set: 0, 225, 226, 227, 230. 0 is obviously an outlier in this situation. Now, let's calculate the measures of center. The median is 226 (because median is the middle value). The mean, on the other hand, is 181.6. There is a huge disparity between these two numbers. But, the median is much closer to the overall group.
Now let's take away the outlier. The median becomes 226.5, not much further from what it was the last time. But, the average drastically changes to 227. This shows how an average (mean) drastically changes with an outlier.