Who did you talk to to sort things out? Did you go to the police and ask them if you can kick this girl out of your house?
1. If they moved out, even if you don't divorce, he can sponsor her as his common law. It may well be that he is checking to see if she will stay with him and then comes back to you. You know, he will be liable to pay child support for their child as well as if she goes on welfare, if he has sponsored her, he will have to pay it back to the government. He is responsible for her for 3 years if he sponsors her. Divorced or not, you do not have to take him back if he wants to come back.
2. If you want a divorce, you should see a lawyer and talk about what you can do. The lawyer can tell you for sure if you can kick them out of your house or if it would be better that you look for another place to live for you and your kids. The lawyer can help you apply for child support, alimony and all that. I do not know if you can claim separation while you were living in the same house. It is possible but that is why you need a lawyer who can tell you that for sure. If you do claim that you are separated but living in the same house and this is accepted, he will not need to leave your house to sponsor the girl because if it is accepted that you can live in the same house and be separated, he can also say that he is living with her there as his common law.
It seems that you have let yourself be walked all over by your husband. Sure he thinks he can continue to do so. It is up to you to show him that he can't.