No, they can't change their mind. It's a legal verbal confirmation you are a refugee. Verbal or written is still legally binding and they can't change it unless they discovered that you mispresented or lied in your application. If not, then positive is still positive.