Have you shown your wife the figures that show why it makes little sense to buy a house right now?
His move should not put any chills on your back, people often move from CA once they decide to retire. Once you don't have to worry about work, there are so many nice places in the country...