How S3STAT works
is a service that translates the detailed Server Access Logs provided by Cloudfront and Amazon S3 into human readable charts and graphs. It also outputs those logs into Common Logfile Format so that they can be used as input to a weblog analysis package.
At first glance, that seems like it should be easy. Amazon provides APIs to turn this detailed logging on, and an interface to identify and download the logfiles themselves. The problem is, the files are not in a format that can be used by any existing weblog analysis tools. And, rather than generating a single logfile per day, S3 will churn out an average of 200 little logfile fragments every 24 hours. These fragments overlap one another, so they cannot simply be appended to one another, but rather must be sorted together.
Cloudfront's logging capability is similar, but slightly different in every regard. While it is actually easier to turn on logging for a Cloudfront Distribution, the generated logfiles will be delivered in a different format to those generated by S3. If you want to consolidate your stats, you'll need custom code for each service.
All that stuff takes time to code. We know this because we took that time. Trust me, it's more involved than you think! Here are the basic steps that S3STAT
will perform for each of your S3 buckets and Cloudfront distributions every day:
- Server Access Logging configuration, to ensure that the bucket in question has the proper permissions to recieve logfiles, and that S3 has been instructed to deliver them.
- S3 and Cloudfront Logfile Download to our servers for processing.
- Logfile sorting and normalization to translate the logfiles into Common/Combined Logfile Format so that they can be analyzed.
- Analysis and report generation using Webalizer, the industry standard weblog analysis package, running in a special mode to optimize reports for the Image Hosting and Content Delivery Network functionality for which most developers are using Amazon S3.
- Report and Logfile upload to the Amazon S3 bucket that you specified. If you have requested it, we'll also send your logfiles and reports anywhere else you'd like via FTP.
will do that every day, and we'll only charge you $10 a month
to do it. That's just enough to cover our server costs, so everybody gets to be happy. Oh, and your first month is free
, so you don't have to worry that your precious greenbacks will be squandered if you don't like the service.
Try it out and see for yourself.
Sign up now
for a 30 Day Free Trial!