Yes, you will have to baby it and may never get fruit. Most citrus trees want a very consistent amount of sunlight, which is why they grow in Florida and not here. If you have a greenhouse or somewhere else indoors with great sun exposure then you can make it work, but you have to pay attention to other details like water level and soil chemistry.
I have a small lemon tree in a giant pot that I can move inside and out. In the summer I can keep it outside where it gets enough sun and it becomes fairly low-maintenance, but in the winter I have to move it inside due to the cold, and even in a well-sunlit room it still is clearly in need of attention for the lack of direct sun. I’ve managed to keep it alive yet it still only produces 2-3 usable lemons per year.