Private Browsing tabs will close when you close Firefox completely. A lot of people think that just pressing the home button on their device will close Firefox, but it's actually still running in the background.

Depending on the device, you would either need to hold down the home button or press the menu button on the device to make the list of running apps appear. You can then swipe Firefox off of your screen.