After roughly 15 hours of trying to do the XY alignment I finally tried something different that I haven’t seen here on the forums yet and so far it’s completely fixed my issue with shifting layers.
Now normally it would be ideal to have the rods parallel to each other but in my case I couldn’t get smooth movement across the entire rod no matter how many times I measured them. I think the problem is that they’re spaced wider on one end than the other inside of the case.
Anyways to solve this I took the plastic bearings out of the caridges that guide the printer head. This gave them a bit more wiggle room as they move along the rods. To compensate for the added room I also tightened the belts a bit more than before.
Hope this can help someone else!
Oh also, without those plastic bearings in place the nuts that hold the belt clamp in place can fall out so be extra careful putting them back together!