How do you log/save the actual pages which are part of a session?
based on the information given:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging
you may thoroughly log the inner workings of HTTP sessions as handled by the browser.
I also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)
How do you achieve that?
lbrtchx
Chosen solution
OK, I was finally able to do what I wanted.
Here is an outline for anyone to benefit from, improve it:
// __ latest firefox development version (aurora)
https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/firefox-52.0a2.en-US.linux-i686.tar.bz2
$ ls -l firefox-52.0a2.en-US.linux-i686.tar.bz2 -rw-r--r-- 1 knoppix knoppix 65124511 Dec 27 05:31 firefox-52.0a2.en-US.linux-i686.tar.bz2
$ md5sum -b firefox-52.0a2.en-US.linux-i686.tar.bz2 0877480001d91ebdb180011db8f84df5 *firefox-52.0a2.en-US.linux-i686.tar.bz2
// __ untar/install it
$ _LOG="tar_-xjvf_firefox-52.0a2.log"
$ date; time tar -xjvf firefox-52.0a2.en-US.linux-i686.tar.bz2 > "${_LOG}" Wed Dec 28 04:09:02 EST 2016
$ wc -l "${_LOG}" 78 tar_-xjvf_firefox-52.0a2.log
$ ls -l "${_LOG}"
// __ run/test it
$ cd firefox
$ pwd .../firefox
$ ./firefox --version Mozilla Firefox 52.0a2
// __ While developing, I find useful running tcpdump, wireshark and firefox in logging mode
// __ run firefox and set up logging via about:config
_DT=$(date +%Y%m%d_%H%M%S)
_LOGFL="${_ODIR}/${_DT}_firefox-52.0a2_MOZ.log" echo "// __ \$_LOGFL: |$_LOGFL|"
export MOZ_LOG=timestamp,rotate:200,nsHttp:5,nsSocketTransport:5,nsStreamPump:5,nsHostResolver:5
export MOZ_LOG_FILE="${_LOGFL}"
./firefox &
Read this answer in context 👍 0All Replies (12)
You can try Wireshark.
- Wireshark: http://www.wireshark.org/download.html
- Wireshark - Wikipedia: https://en.wikipedia.org/wiki/Wireshark
Try wireshark for that? Firefox is already nicely logging the HTTP sessions
What I need is for firefox not to delete the data feeds of html pages, js, ... as it finishes to process, show them.
There should be some sort of "about:cache_duration" or "about:cache_length" option,
What is it?
Modified
Probably, another MOZ_LOG parameter would do, but I don't know what would it be.
MOZ_LOG parameters don't look like about:config parameters
to me.
See:
MOZ_LOG_FILE and MOZ_LOG_MODULES are now the environment variables for logging:
Modified
See:
yes, but in that document it says:
Using about:networking This is available starting with Firefox 52. Navigate to about:networking and go to the Logging section. ...
and the most up to date version of firefox is 50.1 right now
and the beta release is 51.0beta:
https://www.mozilla.org/en-US/firefox/50.1.0/releasenotes/ https://www.mozilla.org/en-US/firefox/51.0beta/releasenotes/
so I am not getting something right or you guys are
not being clear enough to me.
There must be a way to do such a thing, which you
are using while debugging ;-)
Modified
You would have to install the Firefox Developer version (52.0a2) to set all parameters from within Firefox via about:networking. In the current release you only can do that via environment variables.
#!/bin/sh # https://developer.mozilla.org/en/HTTP_Logging # *https://developer.mozilla.org/en/Mozilla/Developer_guide/Gecko_Logging # try one of these (remove the #) # export MOZ_LOG=nsHttp:3 # export MOZ_LOG=nsHttp:5 export NSPR_LOG_FILE=~/loghttp.txt # set the correct path to Firefox cd /usr/local/firefox-RE # start Firefox ./firefox -no-remote -P
Albretch said
I also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)
I doubt that logging will give you a convenient format. I once turned on logging for something. After 25 seconds, I had a text file of nearly 392,000 lines consuming over 42 megabytes. The responses were encoded (sent with gzip encoding) so they are indecipherable -- if they are included in the file, which I can't confirm. Every so often I find decoded response headers but I don't see the actual response content (text or image, as the case may be).
Hopefully there is another way, such as an add-on, that is simpler than fishing around in the cache.
Chosen Solution
OK, I was finally able to do what I wanted.
Here is an outline for anyone to benefit from, improve it:
// __ latest firefox development version (aurora)
https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/firefox-52.0a2.en-US.linux-i686.tar.bz2
$ ls -l firefox-52.0a2.en-US.linux-i686.tar.bz2 -rw-r--r-- 1 knoppix knoppix 65124511 Dec 27 05:31 firefox-52.0a2.en-US.linux-i686.tar.bz2
$ md5sum -b firefox-52.0a2.en-US.linux-i686.tar.bz2 0877480001d91ebdb180011db8f84df5 *firefox-52.0a2.en-US.linux-i686.tar.bz2
// __ untar/install it
$ _LOG="tar_-xjvf_firefox-52.0a2.log"
$ date; time tar -xjvf firefox-52.0a2.en-US.linux-i686.tar.bz2 > "${_LOG}" Wed Dec 28 04:09:02 EST 2016
$ wc -l "${_LOG}" 78 tar_-xjvf_firefox-52.0a2.log
$ ls -l "${_LOG}"
// __ run/test it
$ cd firefox
$ pwd .../firefox
$ ./firefox --version Mozilla Firefox 52.0a2
// __ While developing, I find useful running tcpdump, wireshark and firefox in logging mode
// __ run firefox and set up logging via about:config
_DT=$(date +%Y%m%d_%H%M%S)
_LOGFL="${_ODIR}/${_DT}_firefox-52.0a2_MOZ.log" echo "// __ \$_LOGFL: |$_LOGFL|"
export MOZ_LOG=timestamp,rotate:200,nsHttp:5,nsSocketTransport:5,nsStreamPump:5,nsHostResolver:5
export MOZ_LOG_FILE="${_LOGFL}"
./firefox &
Modified
then the wireshark and tcpdump part
// __ You don't want to run wireshark as root!
$ sudo dpkg-reconfigure wireshark-common
$ sudo adduser -a $USER wireshark
// __ Set up logging file names for easy management
_NM="..."
_DATA_DIR=".../data"
_ODIR="${_DATA_DIR}/${_NM}"
mkdir "${_ODIR}"
_DT=$(date +%Y%m%d_%H%M%S)
_WSLOG="${_ODIR}/wireshark-gtk_${_DT}.log" echo "// __ \$_WSLOG: |$_WSLOG|"
// __ running wireshark
sudo wireshark-gtk -i eth0 -k -w "${_WSLOG}" &
// __ running tcpdump
sudo tcpdump -pni eth0 -s65535 -G 3600 -w "${_ODIR}/tcpdump_%Y%m%d_%H%M%S.pcap" &
jscher2000 said
Albretch saidI also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)I doubt that logging will give you a convenient format ...
That is why I am a bit more comfortable using more than
one tool and keeping an eye on all of them. Also, gzip you can easily unzip
Modified
Using only "nsHttp:3" (LogLevel::Info) in the DE version with about:networking seems to give a log with the HTTP request and response headers and not that much extra data.