Private browsing mode is of course a great privacy feature; in this mode no records of your web surfing session will be stored on the computer.
As in Google Chrome, you can also open both Private browsing mode and normal mode simultaneously in Firefox by installing an extension.
[After installing the extension you will see above like icon on the progress bar, click it to open new private browsing window]
This extension for Firefox replaces firefox's default private browsing mode. Instead, it allows users to have private browsing and normal browsing windows simultaneously on the same computer.
[You can stop private browsing window by clicking the 'Stop Private browsing' (see above screen-shot) ]