"The most recent false and baseless accusations by Daphne Joy are clearly in response to my decision to seek sole custody of my son," a representative for the rapper shared with PEOPLE Charna Flam ...