Usability Notes - by Chris Baker

web usage statistics, and Dr Seuss

When they were younger, both my kids were fans of Dr Seuss's Sleep Book (as I was in my turn). Among the delights for a small bed-time reader, Dr Seuss provides real time statistics about the number of people currently asleep, and (like a good statistics provider) publishes his methodology:

"We find out how many, we learn the amount,

By an Audio-Telly-o-Tally-o Count.

On a mountain, halfway between Reno and Rome,

We have a machine in a plexiglass dome

Which listens and looks into everyone's home.

And whenever is sees a new sleeper go flop,

It jiggles and lets a new Biggel-Ball drop.

Our chap counts these ballls as they plup in a cup,

And that's how we know who is down and who's up."

There's also a wonderfully goofy illustration of the machine, and I think it was this, rather than any predestination to work on usage statistics that made this one of my favourite parts of the book when I was a child.

In the real world, web usage statistics sometimes seem to offer the power (and intrusion) of the Audio-Telly-o-Tally-o Count, only to snatch it away again, and offer subtly different statistics, with various caveats.

As an example, suppose you own a website, and understandably want to know "how many visits did people make to my site in the last week?"

If you had an Audio-Telly-o-Tally-o Count, your "chap" would magically listen and look into everyone's home or ofice, find people actually visiting the site ...and then all that remains is to count the Biggel-Balls.

Of course, that's not what a usage stats tool does.

When someone requests a page from your website, their browser sends one or more requests to your website's "webserver" to send the text, images and other content that the customer needs to view the page. Along with that request comes some information about the customer's computer - its IP address, operating system, screen size, and some information about the kind of browser the customer is using. It also often tells us the URL of the page the customer came from. The webserver can record all this (in a file called a "server log") along with the time of the request, for analysis later.  This combination of facts is fairly unique to the computer - think of it perhaps as being like a footprint. When the customer requests the next page, all this happens again, creating a further "footprint". As the customer visits further pages, his or her computer creates a series of further "footprints" - the process of following their visit for analysis purposes is a bit like following a trail of footprints down a beach.

For competeness here I should say that not all analysis runs from server logs. In a popular alternative the webserver includes a small program with each webpage that the customer's browser runs when it assembles the page. The "small program" (used for example by Google Analytics) causes a message to be sent out to a log file for analysis later. And there are other methods. For the purposes of the discussion here, it comes to much the same thing.

As another aside, it is of course sometimes possible to requre every user to log in, and then to follow their individually-identified activity with a cookie. That provides more detailed information, but is not always desirable (in some circumstances requiring people to log in makes them go away instead; not everyone will accept cookies, and so on).

Pushing on with the "footprints on the beach" analogy, it's worth noting that we have a science ficton or fantasy beach here - trails can suddenly start as if someone was teleported in by futuristic technology or magic (e.g. the customer came in from a bookmark, or typed the URL of our page rather than following a link that we can detect). Similarly, trails of footprints almost always suddenly stop (e.g. the customer stopped using their browser, or went to another site). The way the Internet works means that customers don't have to do anything formally to leave your site; they just stop requesting pages.

Imagine now  a detective following these following these trails of footprints around on the beach. How do the clues compare with the Audio-Telly-o-Tally-o Count?

The detective has the following problems:

1. "Footprints" are fairly unique to a given computer, but not completely so. If Big Corporation Inc. has bought a batch of identical computers and successfully forbids its staff from customizing them in any way, then all the computers will have identical footrprints. The detective may struggle to sort out all those size 42 Converse Sneakers. The usual counting rule is to count all this as one user (technically one "unique browser") whereas the the Audio-Telly-o-Tally-o Count can magicallly see several people, and so drops several Biggel-Balls.
2. The Audio-Telly-o-Tally-o Count magically sees exactly where people stop using your website and do something else - and where they are still on the website, but not requesting pages. The detective only has the observation that the footprints stopped (the standard is to declare that a user session has ended if there are no more page requests for 30 minutes). Clearly this is arbitrary - the Audio-Telly-o-Tally-o Count might know that the user is still avidly reading a long web page, has broken off to answer the phone etc. So we might get one Biggel-Ball, as opposed to counting a new session each time there is a 30-minute gap.
3. The detective is counting "footprints" of a computer, not the people behind it. So imagine a public library which has one computer, on which people come and go all day, many of them looking at your website. The Audio-Telly-o-Tally-o Count magicallly follows this, counting the people coming and going. The detective, following computer-generated "footprints" does not know that a different human is now filling those shoes. If there's a 30-minute break, of course the detective assumes this is a new session, but if the queue at the computer is moving swiftly enough, then this won't happen often, and several different humans will be counted as one visit.
4. The Audio-Telly-o-Tally-o Count magically watches as a user switches from their desktop PC to their laptop or mobile device, or their computer at home, and can tell that this is one human continuing his or her visit. But each of these devices has a different footprint for the detective - the trainers suddenly stop, and a pair of heels carry on down the beach. So (unless the customer identifies themselves, e.g. by logging in) the detective counts a new visit each time the user switches device.

So, since we do not have an Audio-Telly-o-Tally-o Count, we can't count "visits" exactly in the common-sense meaning of the term. We can count "unique browsers" and "sessions" and combine those into a "visit" - a statistic which has some sources of error, but at least the major sources of error are known and the statistic is captured by a known and reproducible method. Note that the methodology is such that errors will usually result in under-counting:  probably better for the business and its advertisers than getting an inflated idea of the traffic. It's currently the best that can be done, not due to the limitations of your usage analytics tool or usage analytics people, but due to problems with what you can actually measure, and the decisions you have to make to interpret this.

That takes us into the realm of the many things where "I don't see why we can't just..." meets "It's a bit more complicated than that" . But that needs a whole new blog post.

So this time I think the last word belongs to Dr Seuss, and his observation ("One Fish, Two Fish, Red Fish, Blue Fish") that:

"Every day from here to there,

Funny things are everywhere."

 

eBooks doubled their share of iPhone app store "shelf space" this summer

Interesting data from Flurry (a smartphone usage statistics service) suggests that eBooks were 10% of what is in the iPhone app store back in July 2009, but had reached 20% by October 2009. Over the same period, the proportion of games declined slightly (17% in July 13% in October) . The data are published in the October 2009 article of Flurry's "Smartphone Industry Pulse" series

These data don't tell us about whether the books are being downloaded, nor whether they are being purchased (as opposed to being free downloads). Also, of course, there are many other places you can get books for your iphone (e.g. by using the free Stanza Reader). Nonetheless, asFlurry say "We take the spike in demand we noted from an earlier analysis, combined with the flood of supply into the market as an early indicator that iPhone as an eReader is real. "

Kids' usage of parents iPhones

I have just come across research from Greystripe about how parents let their kids use the parental iPhone. Parental iPhones are key for developers of  apps for younger children, who are unlikely to have their own machine. (The typical iPod touch user is a teenager or young man, and the typical iPhone owner is a middle-aged man, according to demographics I reported in an earlier post .)

The research "How Moms use their iPhones" Includes a survey of how the kids use Mum's phone (or "Mom's" since this is US data). 59% of Moms let the kids use their phone. Of these, 41% had bought games for the kids, and 20% had bought educational content. As regards age of the children, 29% of the moms had children between 0 and 4, while 43% had children between 15 and 17. The survey also covered the role of iPhones in shopping (use of the phone to make or email shopping lists, or to locate stores or compare prices and download coupons)

 

Thanks to @ruhanirabin whose tweet alerted me to these results!

More smartphone developers choosing Android, but iPhone still ahead (Flurry data)

Smartphone developers choosing to include the Flurry usage statistics tool in their apps tend to contact Flurry early in application development. So Flurry have a leading indicator of what app developments are getting underway. They have used this information to produce a lot of interesting stats about the smartphone market, in what they call the "Smartphone Industry Pulse (July 2009)"  Data from Flurry features in my last post , but I also wanted to discuss their data on projects getting underway for Android and for iPhone.

In the first half of 2009, the number of Android projects incorporating Flurry increased - Android projects made up 10% in January, up to 20% in June (over 200 projects started in June). Flurry have this rather striking graph:

This graph looks pretty alarming for Apple, until one also knows that the total number of smartphone apps was increasing dramatically over this time. The following chart shows the number of iPhone projects - rising from 200 in February to just over 1,000 in July.

(The corresponding Android graph is available on the the "Smartphone Industry Pulse (July 2009)" post of the Flurry blog, along with much other interesting data).

Some thoughts about this:

First the caveat - these data are only about projects that include Flurry (possibly the pattern would be different if we had data about all the apps that don't plan to track usage)

Secondly - loss of market share by Apple is probably inevitable now that other players are in the market big time.But since when was Apple about market share? I see them as a sort of computer  Bang & Olufsen - making highly designed, high quality and expensive stuff for the part of the market that likes it, and leaving other companies to scrap over the mass-market.

Lastly, I wonder how many of the Andriod projects are apps that are ALSO being developed for iPhone. As far as I know it is difficult and risky to develop the one app for multiple platforms (technical issues, and Apple are said to be sniffy about passing any app not built with their SDK ). So developers may be forced into running twin projects, just we we used to do in the late '90s to develop desktop apps for Mac and PC.

I have unhappy memories about developing for the cranky PowerMacs of that era!  You could use file extensions that had profound powers on the Macintosh OS. On a good day that was excellent, but on most days you ended up with a gang of file extensions fighting each other like a sackful of cats until the OS keeled over. To make things more frustrating, patches to the OS would suddenly declare a particular file extension persona non grata and uninstall it. That happen to one of my projects just before it was about to launch. Hence the joke of the time - "Q. What does MACINTOSH stand for? A. Most Applications Crash, If Not, The Operating System Hangs." [For balance I should say that the counter-joke wasn't bad either: "Q. What does MICROSOFT stand for? A. Most Intelligent Customers Realize Our Software Only Fools Toddlers ] All that put me right off Macs until quite recently - and should remind me to be a bit patient with strictnesses Apple might want to impose about use of their SDK.

One more thing - if you include Flurry on an Apple application, do you have a MacFlurry? ;-)

eBooks second most active smartphone apps category: Flurry

Flurry provide a usage statistics service for smartphone applications, collecting data when the application is downloaded or used. As a result they have interesting data on the state of the smartphone market.

Eyecatching for me is the chart in the "Smartphone industry pulse July 2009" showing that customer use of eBooks is rising quickly and is second only to games:

The data probably underestimate the situation, as far from all eBooks will have Flurry embedded in them.

In another analysis "Mobile apps: Models, Money and Loyalty" Flurry have also looked at how frequently apps are used, and how likely users are to return to them after 90 days. That suggests that books are used intensely (maybe 10 times a week), but are not used much after 90 days. The customer has finished the book by then, presumably.

For the technically minded, Flurry describe the technical workings of their service thus:

Flurry Analytics places a lightweight agent into an application, so that performance data are tracked, logged and reported back for analysis. This information is confidential and available only to the developer to analyze in aggregate. Individual user data is not identifiable. Developers are provided a wealth of metrics around usage behavior, any custom event they choose to track and technical information about the device, firmware version, carrier and more.

Demographics (age) data for iPhone and iPod Touch users

I've finally found some demographic data for iphone and ipod touch users. The data come from a survey done by Admob and comScore, as published on the AdMob blog  and reviewed by BusinessWire (which has figures not included in the blog post). The survey was in the first half of 2009 (presumably in the US, though this is not stated). I was after age data, which gives me this chart:

 
iPod touch ownership was highest among the 13-17 years age group (46%) falling sharply after age 25 (23% of ipod touch users were 18-24, 12% were 25-34 and 12% 35-49. After that it's single figures). iPhone owners are older - only 6% were 13-17; 20% were 18-24; 27% were 25-34; 31% were 35-49. This is pretty much what one would expect:  iPod touches are cheaper and can be bought for a one-off payment (no ongoing mobile phone bill). So they are a possible (generous) Christmas or birthday present. In the UK at least, you still have to sign up for some hefty costs to own an iphone. So probably another case of "you can tell the men from the boys by the cost of their toys" as a friend of mine used to say. Speaking of men and boys, >70% of the owners (of both iphone and ipod touch) were male.

For other data (income, how mobile usage compares with usage of other media) check out the original article at AdMob

These data may go some way to explaining an interesting observation from "Just Another iPhone Blog" - "Do Crap Apps have legs?" The author comments on the curious popularity of "Crap Apps" - (applications that are trivial and/or  puerile).