Were you married or common-law at the time you invited him as a "family friend"? If so, and they bother to look, it may look strange that you called him only a friend and then say in this PR application that you were actually married or in a serious relationship at the time (ie. you lied to try to get him a visitor visa). I'm not saying you did - I'm just saying they might see that as a small lie on your part in the past.
That's the only problem I can see. If you say "Oh we've been married for 5 years or been dating for 5 years" or something and then they see that last year you invited him and did not declare you were dating or married and only called him a friend, then you may be found out and have some issues there. Because on the PR applicaton, you do have to state if he's ever applied for a TRV and, if so, the details and why he was rejected.
If you were really just friends at the time or if you weren't married or common-law at the time, it might be completely fine.
I would declare on the PR app that he was rejected for TRV when he tried to visit you and I would not mention the "friend" detail.
Good luck