What's new in the the new pMetrics

There's a lot of great new features in the new pMetrics, although some of them may not be as obvious at first glance. First we have a quick rundown of the major new features for the impatient. After that is a very detailed list of every single thing that's changed, which you'll want to read if you want to know everything that's new.

The downtime for this upgrade was longer than expected because we also had to fix the referrers database table, which was corrupted - the reason why everyone was getting weird searches in their visitors list. Another reason the the upgrade took so long was that we fixed nearly everything that we didn't like about the old database design. With as much data as we have stored, transforming it is a slow process. That aside the whole thing took considerably longer than we thought it would and we are really sorry for being unavailable the entire weekend. There is a 48 hour backlog that is currently processing and we should be all caught up by Tuesday morning.

Quick rundown

• Bounce rate - shown in "the basics" dashboard module. Bounce rate is the percentage of visitors who only have 1 action, and is typically a good measure of how interesting your site is.
• New visitor filtering interface, makes it very easy to find and analyze any subset of visitors. Filters work across multiple days now, although we are limiting it to 7 days max for the moment. When you use data export with any filters applied, those filters apply to the export as well. Screenshot
• New centralized area to manage all site preferences. Click the "cog" icon in your site's tab bar.
• Most active visitors - shows you the people who come to your site the most often within the specified date range
• Visitor hostnames and organizations - much more interesting than a list of boring IPs. We plan to add a site preference to always show visitors by IP for those of you who prefer that instead (we've had a few emails already)
• More accurate numbers for searches, links, and content
• Recent and Newest Unique for searches and links. Very useful to find new sources of traffic to your site, that won't get buried in the long tail
• Tracking code updates - pMetrics is now perfect for tracking Flash and Ajax sites - more info here. Also, you can now customize the "pause" length when someone clicks a download or outbound link (defaults to 500 milliseconds).
• API - all visitor and action filters work with the API now, they didn't before. (browser, os, country, page title, etc). We've also added all the new data types to the API (most active visitors, bounce rate, etc).
• Popular data pages (searches, content, etc) now have trend data just like the dashboard, including the ability to click the trend percentage and see daily history for that one item. Also added "next page" button so you can see beyond the top 50. I can't think of any good reason why I never did this before, because it's basically the most obvious feature in the world.
• Easier to name your visitors - when viewing details for a visitor, next to their IP is a link "name this visitor" which takes you right to the IP tagging page with the IP address filled in.

The long and extremely detailed version

New visitor filtering features

- Slick new filtering interface with popup menus, easy to add/change/remove filteres on the fly

- When you filter visitors, it also analyzes all of them and gives you useful summary data, and compares it to the site average for the selected date range. Pro users have the ability to compare it to another sub-set of visitors too instead of just the site average.

- Filters now work across multiple days! With the old system, if you were looking at "last 30 days" and selected google.com as the referring domain, it would only show you one day of results. This was intentional because the system was not optimized to support querying such a large span of data. It is much more optimized now - BUT - we're currently limiting them to 7 days until we can get a feel for how much impact they will have on our service with real world usage.

- Some new filters: custom data (e.g. usernames or emails), cities, screen resolutions, and links. (Links meaning, the full link someone used to get to your site - before you could only do this with the domain name. Now you can filter your visitors down to just the ones who came from the exact URL you input.)

- "Anything" filters, e.g. domain=* will only return visitors who arrived at your site from another site. This even works on custom data tracking, e.g. custom[username]=* returns only visitors who have a name assigned to them .... Partial wildcards aren't supported yet, but we plan to add them in (e.g. domain=*google* won't work - yet)

- The global maps page has the same filtering interface and options as the visitors page (as it did before with the old filtering interface).

- Actions page got the new filtering interface too, but the filters are obviously different. You can filter by page URL, page title, and action type (pageview, download, outbound, click). You used to be able to filter actions by the same criteria as visitors, such as country or web browser, but that really doesn't make any sense and has been removed. Page URLs and titles support ending wildcards, e.g. "/some/path/" would only return actions on that specific page "/some/path/", while if you entered in "/some/path/*" (asterisk on the end), you will get actions returned such as "/some/path/more/stuff/" and "/some/path/another/path/". ENDING WILDCARDS ONLY! (for now)

- When you use the data export feature while you have any filters applied, those filters will apply to the export as well! Works with both actions and visitors.

New centralized area to manage all site preferences

- Managing site preferences used to be a pain, as they were scattered all over the place - some on your user page (which a lot of you never even see), and also some on the stats pages (like the dashboard prefs). It was a confusing mess, as indicated by the number of emails we got from people looking for these things. So we fixed it! There is now a new main tab on your stats page that takes you to the new site prefs section. From here, you can get info about your site (sitekey, etc), manage your dashboard, manage ip tags and filters, edit your site (timezone, domain, etc), and get your site's tracking code. All in one place. Much better.

Visitors

- Most active visitors. This shows you the people who have visited your site the most in whatever date range you have selected. If they have a name, it shows that and links to it, otherwise it shows and links to their IP. It's grouped by IP though, not username or hostname or organization. Grouping, filtering, and sorting by IP address is the most optimized way to do this, and since it's reading through so much data, it's the only realistic choice.

- When viewing visitor details, there is now a link next to their IP address "Name this visitor", which, you guessed it, takes you right to the IP tagging page with the IP address filled in. We've received billions of emails about making this easier. I agree the old way was horrible. Hopefully this will satisfy!

- When viewing your visitors list, the visitor's name is chosen in this order of priority: Username (from IP tagging or custom data tracking), Organization, Hostname, IP address. This gives you a lot more info up front about your visitors, since a list of IPs is pretty gosh dang boring and useless. When you click on a visitor, it used to always just link to their IP address, even if it was showing their username. Now it links to whatever is actually shown. If they have a name, it will filter you visitor list by that username to show all visits from that one person, even if their IP address is different between visits. If it shows an organization or hostname, clicking on that will filter down the list by that organization or hostname. If you want to see the details for just one visitor, click the "# actions" link to view their session. From their you can click the IP address to find all other visits from just that user.

- Visitors list no longer shows browser and OS icons next to each visitor, to give more room to the referrer and search data, and also the hostname/organization if applicable. You can still see anyone's computer info by viewing their details.

- Visitors list now has a visitor's search as clickable (to initiate filtering by that search), also you can hover your mouse over the search to see the full text instead of being limited to the first 20 characters or so

- Visitor hostnames - up to now you could only see the hostname for any visitor when you clicked on their IP or viewed a session. We limited it to this view because looking that info up is slow and saps resources. Now we are using a product from MaxMind to lookup the hostname for EVERY visitor automatically when we process that visitor's traffic. However, just like the geolocation product, it's not guarateed to be 100% accurate, because IPs are always changing hands - but it should be pretty good. Also, it's not a full hostname lookup, but just the top and second level domain. For example, visitors using Comcast cable internet will show up as "comcast.net", not their full hostname which might be something like "24-60-143-200.hsd1.or.comcast.net".

- Visitor organizations - this is another product from MaxMind that tells us the organization that this IP is assigned to. For home users, this value will typically be their ISP name (e.g. "Comcast Cable"), but otherwise should be the name of the company or corporation that the IP is assigned to, e.g. "Microsoft Corp".

- You can of course filter visitors by either of these two new criteria.

- Two new sub-tabs under visitors tab to view the most popular hostnames and organizations, also added as sub-tabs to the "Locale" dashboard module.

Much more accurate numbers for searches, links, and content

- Referrer and search data is now only logged for the first action of any visit, instead of any and every action. This prevents multiple counts for people who show up at your site, click a link, press back, click a link, press back, etc etc... Each time someone presses back, their browser sends us the referrer data again, even though it's no longer "fresh". We used to log this data and increase the count for the applicable link and search, but now we don't. Potential problem: every once in a blue moon you might have a visitor who shows up at your site, then a few minutes later shows up again from a search engine or link because they searched for something on your site after seeing it on your site. This new system will not log these links and searches, but we think the trade off is well worth it, and this circumstance is so rare it should not be an issue.

- Page titles - Unique pages are determined by a combination of URL and title, but non-javascript users don't send us the title. This has been a big problem, because the pages they view have been seen as unique from the ones everyone else views. Now when we get sent a page with no title, we try to find an entry in the database for this URL that has a title and we link to that one instead.

- Page URLs - Link #hashs are stripped off the end of URLs (for both your pages, and referrers) so that they don't get counted as unique. However, they are left on the end of URLs for file downloads, outbound links, and clicks, because these may be more relevant, particularly if you are using javascript on your pages.

Searches and Links: Recent, and Newest Unique.

- Recent is just a log of all searches and links to your site, newest to oldest (similar to what used to be sub-tabs under the Actions tab, except presented in a much better fashion now. The old way was really just an ugly hack). "Newest unique" means searches and links that have led someone to your site for the very first time (since you installed pMetrics). This helps you identify new traffic sources, and also makes sure you don't miss any "long tail" items. If you're looking at "today", it shows you the newest 50, no matter when they occured. However, if you are looking at the past or a specific date range, then it will only show the ones that occured within that time. Note that this won't work for dates prior to today, as previously we were not tracking the date/time that a link or search first occured. This also means that searches and links you've already had will show up in "newest unique", even though they technically aren't. So this feature won't be much use for a few weeks until most of your common and popular searches/links get tagged as new. Patience!

Tracking code updates

- You can now customize the length of the "pause" when someone clicks on a download or outbound link. It currently defaults to 500 (milliseconds, aka, 1/2 second). To change, set a global javascript variable called clicky_pause_timer (timer, not time) to the value you want, e.g., var clicky_pause_timer = 200; This must go BEFORE the tracking code on your site. The shorter you set it, the more likely that the data won't be logged, because there may not be enough time for the visitor's browser to contact us before moving to the next page. It's a trade off, but now you have the power to tweak it to your liking. Previously, the only option was to completely disable it.

- Those of you using the clicky.log() function - there is a new action type, "click", that is also the new default if you don't specify a type. This makes more sense than defaulting to "pageview" because typically if you want to track when someone clicks on an item, I have trouble thinking of a scenario where that should be counted as a page view. Clicks have a new icon in the action log, they show up with the mouse icon (same icon as in the main "actions" tab). There is also a new "clicks" sub-tab under the content area, to show you the most popular clicks. Most of you probably won't be using this but some of you will.

API

- All visitor and action filters work with the API! For visitors: country, city, language, browser, os, resolution, domain, search, link, hostname, organization. For actions: title, href, action_type [pageview,outbound,download,click]). All values must be URL encoded. For "global" types of data, e.g. things that are not specific to your site only (country, city, language, browser, os), you do have to type them in exactly right, which is unfortunate since you don't necessarily know how we have them stored. e.g. you might try "browser=msie" and get no results, because we have it stored as "Internet Explorer" so you would have to do "browser=internet+explorer". We plan to add in some flexibility here, and also have a list for you to download of all the official values.

- All new data types have been added to the API (bounce rate, most active visitors, recent unique links, etc)

Other minor features, tweaks and bug fixes

- Bounce rate - percentage of your visitors who only viewed one page. This is shown in "the basics" on the dashboard, and also when using visitor filters.

- Popular data pages (searches, content, etc) now have trend data just like the dashboard, including the ability to click the trend percentage and see daily history for that one item. Also added "next page" button so you can see beyond the top 50. I can't think of any good reason why I never did this before, because it's basically the most obvious feature in the world.

- Time has always been rounded to the nearest minute, which is fine for large values, but a bit obnoxious for smaller values such as individual visitor times. Now we always display the seconds for all time values that are less than 24 hours. Also, the minimum time for a visitor is now "5s" instead of "less than a minute".

- On the dashboard, in "the basics", previously you couldn't click the trend link for "total time spent" and "average actions/visit" to see their history. Now you can!

- When viewing daily trend data for one item on the dashboard or any of the other pages that support it, each date is now a hyperlink that will take you to the visitors page for that date with the appropriate filter to instantly view and analyze those visitors. Confused? How about an example: You are viewing daily trend data for visitors from google.com. You click the date "Mon Jan 14" - this will take you to the visitors page, with the "domain" filter set to "google.com" for Jan 14.

- Our date math for selecting entire consecutive months has been fixed. Huh? Ok, sorry this gets a bit confusing as well... Say you were viewing the entire month of Dec 2007, that's 31 days. If you viewed the actions subtab on the dashboard, it would show you the total number of actions in that period, and then going back previous periods too. Those previous periods used to be based on the number of days in the selected period, which means it would be the 31 days before Dec, 31 days before that, etc. Obviously not every month has 31 days, so the selected ranges were a bit odd. This has all been fixed so if you view an entire month, all the previous period before that are whole months as well.

- "Referrer" is now spelled correctly in all places, except API output variable names, which will be changed upon the next API release. I place all blame on Apache for corrupting my otherwise stellar spelling. I'm not alone: http://www.google.com/search?q=apache+referer+spelling

- You can enter in "*" for site mirrors, which means traffic will be logged for any domain (good for sites with lots of sub domains). Better sub-domain support will come in the future.

- Feedburner icon only shows up in the tab bar if you have it configured for that site.

- IP tagging page - now you can click on the name or IP in that list to view all visits by that name or IP.

- Dashboard trends - for "today" trends, these have always been estimates, and not hard figures. They would be too intense to calculate otherwise. This has understandably caused confusion for some of you. Now, when you hover the mouse over a trend, it will say e.g. "~500 yesterday". The ~ signifies it's an approximation and not an exact figure.

The following new features are for premium members only!

- Hostnames
- Organizations
- Bounce rate
- Most active visitors
- Visitor analysis when you have filters applied.
- Visitor comparison analysis (comparing two subsets of visitors ("Pro" only))

Well, that's it. Enjoy :)