Frequency of Updates

At the moment, we don't have a good way to pull that off as part of our service.

Our nightly job involves spinning up a bunch of EC2 machines and running them for just a few hours while they spin through a queue of buckets and distributions. While this has a lot of advantages for throughput (and lets us keep our prices down to a reasonable level), it does mean that for most of the day we simply don't have any worker machines running.

As you can imagine, that approach doesn't sync up well with running hourly incremental reports. For that, we'd need to keep some worker machines running 24/7, and do some interesting trickery to keep the right amount of machines up at all times to maximize their utilization. It'd be a fun problem to tackle, but ultimately it would be a lot more expensive to run (and therefore probably something that would price it out most people's budget).

So for now, everybody gets their reports once a day, and a few people are left feeling impatient.

If hourly stats are important enough to you that you'd be happy to keep your own EC2 machine alive to do it, it's worth noting that we're also a consulting shop. Contact me through email and I'd be happy to talk about building something custom for you.