Volume up button not working

Sorry Deondre, but assuming this is a hardware problem with the flex cable that holds the buttons, then yes, you're going to have to take the battery back out to get to it. Even if you just want to disassemble the volume buttons and their flex cable, you still have to take out the battery just to be able to get to the button flex cable.

Other than having to open the phone back up and take out the battery, replacing the flex cable assembly isn't too bad of a job. That cable has the switches for all of the buttons, along with the flash and a microphone. Unfortunately iFixit doesn't have a guide to replacing that part, but there's a Spanish site called Nadie Me Llama Gallina that has just what you need.

Manuales / Apple iPhone XR / Flex push buttons and flash

The part itself is available in a wide variety of places like Amazon, eBay and AliExpress, among others. Here's an example; it's just one of the first ones that came up on my admittedly cursory search.

Amazon.com: E-REPAIR Power Volume Button Silent Switch Flash Light Flex Cable Pre-Installed Metal Bracket Holder Replacement for iPhone XR : Cell Phones & Accessories

Assuming you moved that cable from the original phone to the replacement housing, it's likely it got damaged in the process; those cables are very delicate and easy to break when removing them; I speak from experience here.

So you should probably take the phone apart and check the mounting of the volume buttons to make sure it's all in correctly as the first step, but if you don't find anything wrong then you most likely need to replace the assembly in order to get the buttons to work again.

Good luck; let us know how it goes.