Well they refused it based on the interview.
They cannot refused based on interview. During the interview you must of said or did something. Not answer a question to their liking, wasn't truthful, didn't know the answer to a question they asked. Information you provided was different than what was in the application.
I read your messages and it was based on genuineness. They have to provide reasons why they feel the relationship is not genuine. I read age difference, not sufficient time spent together. I don't know if you are black and your spouse is white, plus age difference, that's a big read flag with IRCC.
You can overcome the visits by explaining the cost of airfares from Canada to Ghana is not cheap so your spouse cannot travel every 6 months. Plus she has a job and only has certain amount of vacation time each year.
My personal opinion is that you appeal it. Instead of getting an attorney you can get immigration consultant.
side note - My husband is Ghanaian.