Provide benchmarks and performance analysis

[insanitybit]

It may be interesting to provide metrics that let users know the impact of the extension, with and without the blacklist, etc. It may make development a bit easier if you can identify the bottlenecks, though you may already be aware of them.

This is not so much an extension feature as it would be a documentation feature.

[gorhill]

@requiredregistration I tried but I realized it is to instrument web apps or really just web pages, not extensions.

Hum yeah, instrumenting web pages to see the impact of the extension is actually what was asked originally I realize. As usual I misread, I had in mind the performance and memory usage of the extension itself.

[gorhill]

I will just throw notes here over time.

Memory footprint

Environment:

• Version 31.0.1650.63 Built on Ubuntu 13.04, running on LinuxMint 15 (31.0.1650.63-0ubuntu0.13.04.1~20131204.1)
• 64-bit
• Results:
  • No preset lists of blocked hosts (0), right after launch and GC: 16 MB
  • Out-of-the-box preset lists of blocked hosts (42,148), right after launch and GC: 25 MB

I will provide same stats for same setup but for 32-bit, which incidentally I noticed has a significantly smaller footprint.

[gorhill]

Page load for front page of Wired.

Cache cleared before each page load. Using Network tab in dev console.

• Out-of-the-box preset lists of blocked hosts and block-all/allow-exceptionally:
  • 57 requests
  • 731 KB transferred
• Out-of-the-box preset lists of blocked hosts and allow-all/block-exceptionally:
  • 199 requests
  • 5.3 MB transferred
• HTTPSB switched off:
  • 256 requests
  • 5.8 MB transferred
• HTTPSB disabled in extensions page:
  • 268 requests
  • 5.8 MB transferred

I seem to have trouble to replicate the time data from one run to the other so for now I am leaving it out. Number of requests and bandwidth appear reliable though.

Alright, the numbers above are flaky. I tried to find an extension that can perform a simple and straightforward page benchmarking, and none of those I found (or the one suggested above) does the job. Looks like I might have to come up with an extension that might be quite bare, but that is able to provide reliable results. The timer aspect I don't see a problem, but for the bandwidth aspect I am not sure how feasible it is.

[my-password-is-password]

https://chrome.google.com/webstore/detail/page-benchmarker/channimfdomahekjcahlbpccbgaopjll any good?

Only ran 2 tests.

"A" was loading wired.com 10 times with HTTPSB disabled.
"B" was loading wired.com 10 times with HTTPSB enabled ( out-of-the-box settings )

You can change the graph by click the radio-button under the columns that have them.

[gorhill]

I did try this one, no luck. Seems like it doesn't work on chromium/linux. Launching the browser from the command line with the --enable-benchmarking results in a bunch of errors related to the extension:

[25470:25470:0117/002103:ERROR:content_settings_pref_provider.cc(518)] Invalid pattern strings: ,*
[25470:25470:0117/002103:ERROR:content_settings_pref_provider.cc(402)] Invalid pattern strings: ,*
[6:6:0117/002103:ERROR:base_feature_provider.cc(122)] manifestTypes: Allowing web_page contexts requires supplying a value for matches.
[25470:25470:0117/002103:ERROR:base_feature_provider.cc(122)] manifestTypes: Allowing web_page contexts requires supplying a value for matches.
[16:16:0117/002104:ERROR:base_feature_provider.cc(122)] manifestTypes: Allowing web_page contexts requires supplying a value for matches.
...

and nothing happens when I click "Run".

Thanks for taking the time for the data. I see that without HTTPSB you got 3.7 MB, which is what I got sometimes, while other times I got 5.8 MB. I don't know what to make of this, and this is why I added "numbers above are flaky" in the end.

But even if Page-Benchmarker worked on my computer, the fact that it repeatedly reloads the same page is not very realistic. I'm thinking about an extension which would benchmark a session, i.e. a list of URLs which would be more resembling a user browsing around, with some delay between the pages, etc. I'm thinking of trying to hack something for that. Just give a list of URLs, one pass with the extension, another without, another with another blocker, etc. then we have results that are closer to what a user would experience in reality.

[my-password-is-password]

To get it to work i had to use : --enable-benchmarking --enable-stats-table --enable-net-benchmarking

Some dude in the review section for that extension suggested the other options and it worked for me. Try it to see if works on linux

[requiredregistration]

https://groups.google.com/d/msg/google-chrome-developer-tools/wLxCDYxkN0M/8aPUM2qyaDUJ

'You can invoke DevTools on extension. For that enter "developer mode"
on chrome://extensions and choose your background / UI page to
inspect. Then you can do CPU and Heap profiling.'

click profiles.

[gorhill]

@requiredregistration:

You can invoke DevTools on extension

Well if I didn't know that I don't think I would have been able to get HTTPSB where it is now :-)

[requiredregistration]

you're right :) :P

i saw that it can't be used to see where it spends most of its calculation time.

edit: as you said, "I might have to come up with an extension that might be quite bare, but that is able to provide reliable results.", there is no other solution that is faster and better.

[gorhill]

i saw that it can't be used to see where it spends most of its calculation time.

I suppose you are referring to:

I can't make it monitor my extension only including the content scripts

It's not very relevant to HTTPSB, as the content script aspect is really minimal and trivial (even if it was not the case I am not sure I would agree with the poster that this is useless).

The really important part in HTTPSB is the code in the background process, which takes care of filtering all requests and headers, and I did indeed use it earlier in development to identify where performance work was needed, and it has been most useful.

[my-password-is-password]

The extra 2.0 MB was a webm video for section "WIRED VIDEO" on the right hand side of the page. Sometimes it downloads on its own and sometimes it doesn't and you actually have to click to play for it to download.

[gorhill]

To get it to work i had to use : --enable-benchmarking --enable-stats-table --enable-net-benchmarking

I finally did get around to try this, and this doesn't work on my side. I suspect it might work in chrome on windows though. With --enable-stats-table all the pages crash into "Aw Snap!", and if removing --enable-stats-table, I see the chrome.benchmarking API, but chrome.benchmarking.counter returns undefined values.

[gorhill]

In any case, I am hacking together a bare extension which purpose will be solely to benchmark a playlist of URLs in order to benchmark something that is closer to reality. Output will be very simple, in general users will not care about statistical bloat, we just want to see the impact: comparing time/bandwidth, for different extensions, no extension, different settings in one extension.

It's about the claims vs reality, and the claims are usually simple: "Save 30% in bandwidth and 50% in page load time!". No need for any other fancy numbers.

[gorhill]

Added this https://github.com/gorhill/httpswitchboard/wiki/How-to-use-HTTP-Switchboard:-Impact-on-www.wired.com

Using http://raymondhill.net/httpsb/har-parser.html

[gorhill]

https://github.com/gorhill/sessbench