How to unlink Google account contacts from phone?

Question

I recently got a new phone and a new number. I finished setting up my WhatsApp when I realised that my contacts were not the correct ones. Turns out Google Accounts linked the contacts from the wrong account to my phone's contacts. I want to unlink the wrong Google Account from my contacts, so that the contacts will no longer show up on my phone, without deleting the contacts in the Google account itself, then link the correct one to my phone. I'm not sure how to do it, so can someone help me please.
I just realised how confusing it may be, so I'm going to try to rephrase it:
Currently, the contacts on my phone are taken from the contacts saved on Google account X. I want it such that my phone no longer uses the contacts from account X, and instead use contacts from Google account Y, without deleting the saved contacts on account X. 
Thank you so much for reading my lengthy essay, please help me with my situation I'm getting desperate