Private Browsing allows you to browse the Internet without saving any information about which sites and pages you’ve visited.

Private Browsing mode only means that Firefox doesn't store data to the hard drive.
You will always be able to remove data otherwise, either via Clear Recent History or by removing history items and cache content manually (forget about this site would make this easy).
If you want a record of visited websites then you need to do that at the OS level or in a password protected router that can't be bypassed.
Trying to do this in a browser (Firefox) is futile as you would only need a new profile or delete data manually or use a portable Firefox version.

• https://support.mozilla.org/kb/Private+Browsing

I am well aware of what private browsing is and what it does. These are family computers and tablets I want to be able to monitor what my family is doing while online.

I am not very technologically inclined and do not know how to mess with the router or the OS on any of these items.

I simply want the option permanently removed from these devices without having to use Internet explorer (or any child-safe browser) as my family's search engine.