I guess it depends on the purpose of your visit. If you think this visit will add value to your application, then I don't think the visit will matter now to CHC since your husband is waiting for his passport anyways. If the purpose is stay with him till the end of this process and if you can afford to live with him for the next two to three months then I guess you should go for it
I haven't visited my wife since I filed the application. At the time of filing the application (Feb), I had a friend who told me that process will take max six months so i didn't bother to plan for a visit. It wasn't until Nov when it dawned on me (through this forum) that I am in it for a long haul
And by then of course I had used up all my vacation for that year. I am still friends with this dude... i have forgiven him
This year, I decided to save my four weeks of vacation just for this trip that i will be making hopefully this month. Oh what a blissful feeling this is...
Patience is a virtue, something that Allah appreciates and it is good for you too!