Bug report - user agent reports are entirely empty

I can see a bunch of Mozilla/5.0 user agents, which all have lots of escaped code in them - lots of %2520 in there.

Clicking them for a distinct report gives "0" for every single day, which can't be right, since I can see they've had thousands of hits.

(I'm particularly wanting to use this to monitor the Googlebot, Yahoo Slurp and bingbot. Currently I don't get much visibility.)

jamescridland
Wednesday, June 10, 2015




I notice that if you edit the URL to simply end /detail/Browsers/Googlebot then that appears to work - at least, it shows some data. Might be a plan to have some standard links always available for the major bots.
jamescridland
Wednesday, June 10, 2015




Thanks for beating me to the fix. You can shorten the useragent name on the URL and it will combine agents that start with that prefix.

As to the "all zeros" bit, I bet that has to do with your reports only having started running a few days ago. Give it a month to fill up, and I bet you'll see a month's worth of numbers.

Jason Kester
Friday, June 12, 2015




Not convinced.

https://www.s3stat.com/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/Googlebot

...gives me a total of about five hits from something with Googlebot in the user agent title. And my reports have run for about two weeks now.

Yet Google Webmaster Tools - sorry, sorry - Google "Search Console" - says that the Googlebot visits on average 7,661 pages every day.

I believe the "Search Console", since I also get Google Alerts sent to my email box when it spots my name on a new page; and many of the pages Google has discovered over the past week have had just that.

https://support.google.com/webmasters/answer/1061943?hl=en gives the user agent as Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

I think there's a bug here. Being able to spot the Googlebot is one of the most important uses of logfiles like this, and I can't get there.

jamescridland
Friday, June 19, 2015




Further to this...

https://www.s3stat.com/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/com.apple.WebKit.WebContent
...shows sporadic data

https://www.s3stat.com/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/com
...shows the same thing

https://www.s3stat.com/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/WebContent
...shows nothing.

jamescridland
Friday, June 19, 2015




Hmm.

day20150618.json contains:

"UserAgents":{"-":[13248,9170315],
"Mozilla/5.0%2520(compatible;%2520bingbot/2.0;%2520+http://www.bing.com/bingbot.htm)":[7358,27000222],
"Mozilla/5.0%2520(compatible;%2520Yahoo!%2520Slurp;%2520http://help.yahoo.com/help/us/ysearch/slurp)":[8034,33359353],
"Twitterbot/1.0":[2154,6775262],
"Mozilla/5.0%2520(Windows%2520NT%25206.1;%2520WOW64)%2520AppleWebKit/537.36%2520(KHTML,%2520like%2520Gecko)%2520Chrome/43.0.2357.124%2520Safari/537.36":[1989,10824981],
"Mozilla/5.0%2520(compatible;%2520Googlebot/2.1;%2520+http://www.google.com/bot.html)":[3492,17466693],

...which leads me to suspect Googlebot had 3,492 hits yesterday, and sucked down 17Mb of data. Which is lovely.

But /Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/Googlebot says I only have had five hits in the past two weeks.

Indeed, a click through from the "User Agents" section for UserAgent gives a totally empty report, with ONE exception - that of
/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/-
...i.e. the only User Agent that doesn't have a '/' in it.

Could you possibly take a look? I suspect it's a bug.

jamescridland
Friday, June 19, 2015




I think I see the disconnect. That parameter is not a filter.

So unless there is a browser identifying itself as "googlebot" alone, it won't narrow results on it.

Jason Kester
Tuesday, June 23, 2015




Understood: but...

The useragent for Googlebot is:
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

https://www.s3stat.com/Report/ViewReport.aspx?DistributionID=5747#/detail/Browsers/Mozilla shows me all Mozilla-claiming useragents. Including Googlebot. So, this URL matches the beginning of the useragent string. (Indeed, #/detail/Browsers/M returns similar).

So, theoretically,
#/detail/Browsers/Mozilla/5.0 (compatible; Googlebot
or
#/detail/Browsers/Mozilla%2F5.0%20(compatible%3B%20Googlebot
or
#/detail/Browsers#Mozilla%252F5.0%2520(compatible%253B%2520Googlebot
... ought to work, and show me all Googlebot traffic.
The first two simply redirect me to the front of the distribution; the last one gives me a report with zeros in it.

So, I think the data's in here, really, and I think I'm annoyingly close! Hey ho...

jamescridland
Tuesday, June 23, 2015

[ reply to this topic ]   [ return to topic list ]

© 2017 Expat Software Back to Top