One of the less appreciated features of the awesome Firebug extension it’s ability to edit HTML and CSS live.

I use this to do rapid prototyping, generate screen shots for bug reports, etc. in addition to the development tweak it til it’s right use case.

I did a quick screencast of doing a two state UI prototype for screencaptures:

The mozdev guys gave me a shoutout as they planned the tagging system for mozdev projects. We agreed we were using tags to provide a browsing UI, not facilitate search, meaning that no aspect that would never be shared across 2 or more projects was a good candidate for a tag.

I proposed two key semantic guidelines for tags:

• Tags should address the application component (bookmarks, attachments, history, etc)
• Tags should address the user goal/task (find,share,refind,search,forwarding)

This is pretty much what’s happening, with the addition of software product name identifiers. I’ve used the nifty kwout service to capture a clickable screenshot. Copying partial source doesn’t get the CSS on the tag cloud.

mozdev.org - free project hosting for the Mozilla community via kwout

I was also interested in a architectural level of tagging along the lines of a classification of wishlist bugs.

• feature addition (features)
• feature change (modification)
• feature tweak (enhancement)
• application (app)

There are lots of interesting enhancements happening over at Mozdev. The tagging system offers a much richer way to browser extensions at mozdev (needs more data tho!), and offers a nice organic alternative to the perhaps overly rigid addons.mozilla.org.

The tagcloud is not the end all of browsing UIs for tagged content. I’ve been impressed with some noodlings with a faceted-search module for Drupal by David Lesieur (.fr).

The latest in deep cogitation on tagClouds is from the IA Summit ‘08 (B&A podcasts). While the research found utilization of online video and customization high, only 35% of users reporting clicking on tagclouds. I think the jury’s out on tagclouds as a whole — I value them a lot for providing an aesthetic overview of a topic space.

The version of wordPress powering this site was massively out of date (v2.0) and I paid the price. Two spam infilitrations inserted links into older posts and created a whole directory of spam content.

The upgrade was smooth and mostly painless… as Dougal says “Upgrade or Else“. Indeed, while most of the spam was non-destructive, a handful of older posts seem to have been chopped.

The Web Search Data Mining conference held it’s inaugural meeting recently. This is a spinoff of SIGIR, WWW, and related conferences specifically dedicated to learning from user activity traces & web topology.

While I keep informed and occasionally learn new things from Ted talks, UIE podcasts, etc., that doesn’t compare to the density & sophistication of attending a research conference. In a great sign of the times, many of the WSDM talks are online at videoLectures.net.

Check out the wealth of vids on semantic web topics. Give up your daily YouTube’ng, and learn something!

As we wrote about in the PIKII paper (full text), the web has evolved to partially support most of the key tenets of traditional hypertext. A notable exception is trails, or maps through information space, that can be shared, followed, and augmented. For a indepth primer on the history here, see the web that wasn’t. Let’s look historically at efforts in this space.

Trailfire, a web 2.0 company, has a Firefox plugin that facilitates authoring and a playback mechanism with the claim of personalized topical recommendations. Trexy was an earlier foray that focuses on supporting search and uses search queries as starting points for trails. This type of parallel, collaborative search of an information space was most recently addressed with Microsoft’s Search Together prototype, scheduled for release soon.

The most interesting recent entry into the trails implementation space is PMOG. This pseudo role-playing game with it’s passive mode of operation and economy around user attention and web browsing may have realized the concept of trails more fully than anyone else at this stage. The notion of “passively multiplayer online games“, and especially pmog.com got some buzz this year at SXSW, fueled by a panel and strong attendance by the gaming community.

While PMOG has a lot going for it beyond just the notion of trails, let’s consider the history of popularity of trail-based systems with a graph from Alexa (yes, dubious source I know). The winner here is Walden Paths, an academic project that produces a useful tool for teaching with websites - while past it’s peek it’s still a defining project.

Alexa shows PMOG rising fast, Trexy with a small hayday years ago, Trailfire waning, and Waldens Paths also waning but with more reach than all the others combined.

Are trails a good idea? With the popularity of social bookmarking sites, my guess is yes. There is another precedent — third voice, which left annotations scattered around the web. The richer and more contiguous information provided by good trails could go beyond the random web graffiti that third voice generated.

Authoring

The annotations offered by the trail services described here are typically attached at the page level, not the page element level as the most robust specification of annotation requires. Even with simple page level annotations, PMOG and Trailfire take very different approaches to authoring.

I recreated the a PMOG Scrutinizer trail at “Understanding Vision & Web Design” on TrailFire for comparision purposes.

PMOG uses a 2 step authoring process, with step one being light posts and step two stitching them together into a mission. Trailfire has a more robust interface while browsing allowing annotations & sequencing to happen during the browsing activity.

I like that PMOG allows me to gel trails without a massive interruption in the browsing flow. No metadata is specified when you add the trail. That said, the UI for adding marked urls to specific trails (or missions in PMOG lingo) is clunky. This is partially due to the number of marked urls being tied to the currency of the land, so the expectation is that the set of marked URLs will remain relatively small. I crafted a trail/mission with PMOG as I did the research for this post, the buzz on PMOG, 3/15/07.

Playback Subtleties

The area of playback is also a key experience delta between Trailfire and PMOG. Partially due to the recasting of trails as missions, the contents of the trail are hidden from the user prior to playback in PMOG.

Some placeholder ideas for future playback systems:

• Branching: The natural solution for this is a visualization, with additional support for backtracking both from the subtrail and to the subtrail.
• Discovery of overlapping trails: either augmentations or unaffiliated. Trailfire does this aggressively, all the time when it’s active based upon current url.

Additionally, if annotations were attached to elements, there’s are some new challenges about helping the user orient to the site before deep diving to content. Due to cross-site security restrictions and the general weakness of frames, playback is likely to always require a browser mod.

Next?

I believe the key missing piece here for mass adoption is the ability to publish trails to a blog. Imagine a microformat for trails, with creation tools and blog posting integration. Ideally, this could support multiple representations (overview map, episodic rendering with visual previews). Existing efforts have added widgets, but the topical focus of trails makes this type of global include detrimental.

In the meantime, PMOG registration is wide open, though performance is suffering accordingly.

I’ve very happy to see innovation in web search in any fashion. The challenges of simply getting at scale search done creates a barrier to entry for innovators that might be one of the reasons search has remained so static over a decade on the web, while other interactions evolved. Progress has been varied in pace, but e-commerce, personal information management (aka bookmarks, etc), and browser UIs progressed more dramatically in interaction & UX than search has. If you consider the underlying technology, web coverage and the ever expanding challenge, relevance, structured information access, etc., search has advanced hugely.

But is the text box really the ultimate search UI… perhaps, but let’s not give up without a fight.

You can break down much of searching into the tasks of query formulation & result evaluation. Both of these activities happen in repeating cycles, changing in critical ways as the search continues. Query re-formulation has seen more work than supporting repeated selections from the same set of results, as in scenarios where the user returns from a clicked result to choose another. I’ll explore how this activity might be better supported below.

SearchMe

SearchMe is a seriously VC-backed implementation of Apple’s CoverFlow for search result pages — horizontal scrolling with a centered focus and perspective skew visual effect for thumbnail page previews. The site is vapor (aka private beta) at the moment, with only a YouTube video commonly available but the underlying technology is reported to be Adobe’s FLEX UI layer in Flash.

The advantage of thumbnails for recollection of already visited sites has long been proven (Greenberg & Cockburn, 1999). But for new sites, the full thumbnail is not necessarily the best indicator. For highly designed sites, or perhaps sites that are favorite destinations, recognizability is strong, but typically the company logo or even textual excerpts would be better proxies.

The visual preview has value as a “storefront” — is the site in a good neighborhood? e.g. are the proprieters competent in design and technical execution? How big is the building? What schema does it fit?

But for this feature, Ask.com’s on demand thumbnail and other smaller thumbnail view UIs do a better job of lowering expectation violations, *after* the semantic match has been validated with human natural language processing of the result summary. A thoughtful post from eliottng explores this issue in the context of the separate tasks that search enables.

Clustering and Query Refinement

Search me also offers a query filtering feature for choosing between clusters of content triggered by a keyword, named “Category Suggest”.

While “recognition not recall” is one of the most profound of the commandments of usability, many implementations of query suggestion, without the time savings of auto-complete, cost more effort than their worth. This isn’t the case given polysemy, words with multiple interpretations. Disambiguation, especially for lower frequency senses, is a great use case for query suggestions or filtering operations. The technologists in search I’ve known have always been kind of embarassed that technical topics overwhelm the common IRL interpretations of words like java, but the problem is far more widespread.

“Category Suggest” is also a nice spin on the historical query (re)forumation UIs. The query formulation part of the search task model has basically seen two major areas of experimentation: auto-complete and refinement suggestions / result clustering.

There has been more work in the academic industry and search construction UIs, for things like nested booleans. I don’t think this level of upfront user effort is the right approach for everyday search tasks. People are use to executing query refinement chains, and done right, suggestions of advanced query modifiers (like filters, as in category suggest, not simple keyword alteration) could really approach the fluency and generativity of a natural language UI without the gratuitous anthromorphism of conversational UIs.

Not every query needs this type of offering however. And it varies by user, and specifically user experience in the topic domain. Micah Alpern recounted some nice work on calibrating hesitation detection in query formulation for recent query assistance features at Yahoo in our SXSW panel for rapid detection of when to spend user attention on evaluating suggestions.

Prediction: Search UI Innovation Will Not Remain Slow

I’ve written a lot about how pagination affects the user behavior in search, and done much more proprietary investigations thereof. The process of result evaluation is super optimized by the user, though less so than commonly understood given the 30% navigational query percentage.

User engagement with SERPs will increase as innovations like the awesome bar in Firefox 3 take advantage of existing local persistence of routine navigation. A search engine’s ability to make more complex search problems fun and productive will become more critical.

Infinite scroll experiments (live search, humanized reader, etc) are one of the more gutsy innovations in search UI, but are really a move forward in pagination UIs that have some non-trivial challenges still to solve — notably how to suppport patterns other than simple feed forward.

Idea: Making SearchMe Better

Beyond the simple glitz of emulating the popular iphone, and other variants, coverflow UI, the core promise of the interaction style is more effective search that’s not simple read once, top down. Traditional search results pages (SERPs) make entertaining candidate choices and evaluating the relative value across candidates, prior to click, a very challenging prospect. Memorability for individual results is low, especially for novel topics, and can be improved with visual markers. This can help in both initial result selection and subsequent revisits to a result set for deeper exploration. More consistent deliberation and fluency in the result summary evaluation processes could really help users be more productive.

I’d add dwell time implicit feedback to the SearchMe UI, as well as keyboard shortcuts to nominate and demote result previews. This could directly impact the visual rendering of the results, reducing the real estate footprint of rejected alternatives and increasing the saliency of nominees. Visit status is also an obvious must have. There is a lot more potential for adding session state to search activities, but interface level state models for the SERP evaluation process is a good path of exploration.

Now, I really wonder what they’ve got under the hood!

Quick blog from an early evening pitstop at SXSW… Flickr / Fire Eagle shindig was a blast, as has been the entirety of SXSW.

Got some nice eyeball strays from my old Mozilla Hack shirt around the show today, pictured. Looking forward to the Mozilla WebDev Get-together tommorow after my panel on Metrics for Design at 11:30.

In addition to showing off Scrutinizer, I’ve demo’ed Seek a few times.

When a faceted browsing extension for Thunderbird called Seek hit planet.mozilla last week, I was jammed up at work and itching to try it for hours. It’s undoubtably useful and a step in the right direction for mail management.

The first precedent I know of in the faceted browsing for mail was Zoe, a Java POP importer that indexed with Lucene, now defunt. Microsoft Research also produce a useful tool in the space called Phlat. I really think this approach is years beyond the current gmail “flatness”.

There are some issues with day to day usability, notably the need to rebuild the index for every use — see feedback thread @mit or at mozilla.dev.apps. Go grab it and get involved — it could be a great kickoff of a new round of innovation in the Thunderbird world.

The industry is abuzz with a new Google Split test that offers a site search box underneath the first result, for a small set of navigational queries. The blogdom analysis has been rather shallow of this feature, so let me break it down for you.

First, 30% of all queries are navigational. However, in many cases, the actual user intent is more specific than simply visit. In reading thousands of end user “help us improve” reports while working at MSN/Live Search, I found that often users had a very specific intent behind their simple navigational query. They would come back to the search page and tell us about — after they failed to find it on the target site.

So this is the latent need aspect — it might actually be more useful for users to specify a refined, more targeted query and use the search UI that they’re comfortable with, rather than taking the first result and struggling with that sites UI & information architecture. The search engine simply says “Yep, I know *site name* — can I help you a bit more?” and lets the user choose.

So, that’s the UX side of things. This is a truly good idea, that has both positive UX and positive revenue impact. Heck, it might have been sustainable way that MSN/Live could have got a query share boost instead of the shortlived games effort.

On the revenue side, you can sell ads for “Microsoft CRM Solution” at a much higher volume and price than “Microsoft”. So, the SERP page views following a site search are bound to have a much greater revenue potential than the simple navigational queries. You do the math — if you identify 50% of the 30% of navigational queries (net 15%), and get users to use it even 10% of the time…

The implementation challenge here is identifying navigational queries for which sufficient index coverage exists to provide a good UX. Not so hard.

Bravo Google! Addendum: The official post on this feature calls it teleporting.

Back in the mid 90s when page layouts were much less complex and typically scaled with window size well, I used a 21″ monitor on a Mac and did a lot of link drag and dropping between windows.

With the arrival of my 17″ macbook, I managed to dust off the old extension skills and get something shipworthy done in a few hours.

The BigScreen extension adds a toolbar button (or rather lets you add it by contextual click on the toolbar, customize, and dragging it to a location). The button drop down will move a window to the left or right half of the screen, or take the current location and spawn a new window, resizing both to a dual pane full screen layout.

Updates aren’t configured yet, so I haven’t bothered with FF3, but grab it here. Track further progress at lab.mozilla.bigscreen.

While last year’s extend firefox contest’s winners were dominated by features involving thumbnail previews, this year’s winners are more in the style of mashups. I was happy to see FireGestures in the runner ups. It’s an interesting derivation (if rather lightly acknowledged) of Optimoz gestures. While author dropped my window stacking gestures, he did retain the hover gestures I invented for opening multiple links in new tabs. “Wap wap watah“.

Over on the AIR side of the house, with the 1.0 release approaching, developers are beginnning to find the platform weaknesses — like creating image files efficiently. There’s a real contrast to working from an open source platform in this example, but darn it, while the docs aren’t perfect for AIR, they’re well organized (and there are a lot more blogging coders). I’m finding myself more productive developing in AIR than I ever was in XUL/XPCOM and the sex appeal advantage of the graphics layer exceeds that benefit. Papervision is super slick.

An important aspect of parity, both within the browser space and in the rich internet app / mashup space, Full Page Zoom (the bug) has come to Firefox 3.

The always crafty Daniel Glazman writes:

The fullZoom feature of Firefox 3 is cool. Very cool. It’s so cool that combined with the marvelous extensibility of Firefox, I think many extensions are going to use it and offer a wide set of new features based on it.

The extension is for Firefox 3 betas and zooms to the region where your mouse is when you right-click and zoom.

Suprisingly, IE has had some really nice zoom capacities for years with *.style.zoom. I crafted a resolution tester back in ‘04. The IE implementation is a bit different than Firefox’s, seeming to render at scale and then reduce near the graphics layer.

In a potentially nifty implementation for custom reading solutions & accessibility, Firefox seems to scale the page atomically. The full scale and then manipulate approach is available and demo’ed by Mark Finkle in SVG using Foreign Object, but the rotated & scaled demo is still pretty slow on my current mbp and the latest nightly.

Hat’s off to Daniel for finding a great use case for using DOM boundaries to provide a valuable user feature. Recognizing meaningful content by layout schema is a pretty tractable problem, and the DOM extents are in general rich with information. Using IDs to recognize unique UI elements and repeated CSS classes to recognize UI components is a nicely generic solution – more in Edmonds, A., White, R., Morris, D., Drucker, S. Instrumenting the Dynamic Web. Journal of Web Engineering (JWE), Vol. 6, No. 3 (2007), 243-260.

With a 17″ MacBook coming my way on Monday, with the 1900×1200 screen upgrade, I’ll likely be a heavy zoom user. I do have a wide screen browsing Mozilla prototype deriviant hacked up in Adobe AIR (demo vid) with side scrolling.

We’ve just released an AIR-based browser that simulates human vision, to provide insights into your site design and task flows. Props to the gang at Nitobi, who helped me put this AIR code together from my static image prototype, and to my employer StomperNet, who decided to make a big deal of giving this away for free on Xmas eve!

I first had this idea back in ‘03, somewhere between Andrew Duchowski’s eye-tracking class and CHI ‘03 where I caught Hornof & Halverson’s “Cognitive Strategies and Eye Movements for Searching Hierarchical Displays” (PDF). This research showed that the eyes can jump to headings very rapidly, no matter where the eyes were on the page. Unlike your hand-guided mouse, your eyes aren’t subject to Fitt’s Law. Distance is irrelevant, within the context of normal computer ergonomics, only whitespace, contrast, and detectability with peripheral vision matters.

This realization has shaped a lot of my UI surface-level insights over the years. We put together a video on how to use the Stomper Scrutinizer browser to help generate these types of insights. Here’s the Top Ten Things you can do with the Scrutinizer:

1. Simulate eye tracking in a usability task
2. Assess the ease of use of multi-step processes
3. Give your designer a fresh pair of eyes
4. Find out what “pops” in your design
5. Conduct findability challenges
6. Ask: does your visual grid work?
7. Evaluate your site’s contrast levels
8. Insure learnability in your template
9. Avoid button gravity errors
10. Tell the story of how your eyes work

Give the Scrutinizer download a try — if you have the lastest flash player, it will upgrade or install the AIR runtime auto-magically.

And happy holidays!

I keep an eye on the mozdev project flow (rss available) and recently discovered the clever tab cloud extension. It aims to summarize the tabs with a “weighted list”, aka tag or keyword cloud and offers an affordance for tab selection. The extension’s still in development, and bootstraps with the Yahoo webservice for lexical analysis.

It’s a very interesting idea and one that made me revisit the dialogue from the Mozilla Labs introduction of chroma tabs, a color coding extension for tabs basing color on hostname. One commenter suggested color coding tabs by category. While this is highly doable by a fairly standard technique of using the DMOZ open directory content & categories to create a lexical profile, keyword summarization techniques might offer a more nuanced way of identifying content similarity. For the most part, I think browsing sequence is a more useful way to optimize tabs for efficient and accurate selection by mapped, small real estate visual cues like tab coloring.

Coding tabs by navigation sequence

With firefox 3 and the introduction of a history mechanism that more directly represents referring page load, and separates referred page loads from explicitly triggered (url bar, bookmarks, etc), we now have a mechanism to easily identify a browsing sequence across tabs. What I’m imagining is that spawned tabs, say from a google search session and derived trails would share a hue, varying in intensity as the distance (in page loads) increases from an explicitly triggered view.

Two commenters on the chromaTab labs post express similar ideas: one argues for commonly coding across referrer while another captures the referral chain idea with a focus on limiting coloration to behaviorally identified hubs.

What task should be supported?

There’s a very smart bit of code behind chromatabs: a hash system that insures that a domain will be assigned the same color from one session to another. This optimizes for learning of frequently visited site color patterns, potentially helping identification of less frequently visited domains as well.

I’m not sure about a number of dimensions that dictate the value of optimizing for long term learnability versus short termed cues. Empirical studies tell us that around 50% of all page views are revisits, which is no help at all! The more useful datapoint would be where to people make the most tab selection errors — for exploratory activities on new content or habitual information gathering activities, like reading CNN or browsing from TechMeme.

Folks at Mozilla are hard at work on an instrumentation framework which could be deployed to answer this question. By correlating tab selection errors (rapid switch-aways) to visit count and perhaps attributes of the referrer chain, patterns of errors might be identified. It may take a little while to get to that level in the metrics project, but Firefox has the luxury of huge numbers of insightful users who participate in the dialog as an alternative data source. Inviting users to chime in when they experience errors or challenges could increase the rate of learning.

Turning to other methods for making tabs more navigable, I’ve been surprised at how much I like the Flock behavior of opening new tabs adjacent to the spawning tab. This was very disorienting when I began using the new 1.0, but now I find myself somewhat annoyed at the mouse distance required for common new tab scenarios.

Supporting user optimization

If you can’t prevent user error (commandment #5), enable easy recovery (commandment #9). This is where tab preview systems like foxPose or TabCloud come in. For the user who’s lost context by stepping away from the computer or reading email, or for the browser out of control with diverse topics, going directly to an overview is a strategy that users can easily manage against the strength of their recollection by favicon & title.

There are challenges here with foxPose style visual thumbnails: multiple pages from similar sites may not be very visually different, designs may not be distinctive, or for a new site, the design may have made minimal memory impact. A really well done tabCloud approach could produce a high reliability recollection trigger. My bet is that single page representation mechanisms can’t rival representations of the temporal history of the tab and their spawn. Browsing icons is the definitive work in this space and customized graph layout for the common structures of hypertextual sequences. This could be deployed in the context reinstatement use case, ala foxPose, though the mapping to tabs would require some serious cogitation.

Age as a memorable dimension of tabs

A final, novel (UPDATE: Not!) contribution to the potential techniques for marking tags, is applying the notion of aging or wear & tear, to distinguish the age of tabs. Imagine fresh tabs as immaculate color fills with age introducing cracks and scratches. Combined with the color theming of referrer sequence, this make for a garish but highly informative tab bar.

Amazon’s design team has a real challenge in front of them with a huge catalog size. They’ve worked with tab based designs for years, but have recently switched from a top to side nav bar. They describe the changes to users here.

They say just hover, pick, and click… Interesting that hover is regarded to be a end user vocabulary term, but the real problem comes in the “pick” stage.

The width of the stroke around the mouse path conveys relative speed, from my ‘04 Masters Thesis. The large cicle is a pause, and small circle a click.

Here’s an example of a user who “forgot” about this menu restriction, but while executing the movement remembered the frustration of lost menu selection, and executed a course correction. As it turns out, this menu did not suffer from the flaw described here, but the user’s experience with dysfunctional menus fouled his ability to perform optimally even in this case.


It’s been almost 10 years since we’ve been doing cascading menus on the web. Every modern OS has figured out this, or a parallel solution. I can understand when a small business website implements a $40 plug and play menu that this type of efficiency might not be present. Alas, the web is forming user models of how computers interact with people, and these shoddy implementations affect the good implementations present in operating systems and robust UI toolkits like XUL.

So how hard is this to do correctly in DHTML? The draw the diagonal approach would require a touch of math and ongoing mouse movement detection. Alternatively, a simple 500 msec delay between initiating the hover and initiating a second hover would solve this for all but the most lengthy flyouts. I’d love to hear your implementation ideas, and perhaps even a candidate open source flyout menu toolkit where we could implement a reference fix.

Amazon, I’m ashamed of you.

It’s amazing how many of the ideas talked about at the recent FLOSS heuristic review of Firefox have been addressed by extension authors or are easy hacks by the users.

Two of the ideas discussed, gestures and marking menus, have a rich implementation history with my (and many others!) efforts in ‘01 on optimoz gestures and the subsequent pie menu implementation easyGestures.

I also described how site authors could use the latest CSS functionality in Firefox back in 2003 to mark links with target document types (e.g. PDF | GIF | ZIP | EXE | etc), CSS Generated Content: Link Markers and there’s an extension TargetAlert.

A list of others

• (user control) Private browsing mode: Stealther add-on
• (error prevention) PDF Open Lag Cost: PDF Download prompts the user prior to open in browser.
• (efficiency) Bookmarking: the “new folder” button is hidden behind the progressive disclosure control, which is not very discoverable. The OpenBook solution provides this feature. This history of the add bookmark dialogue is long and varied, and I do prefer the memory of the latest incarnation to repeated folder expansion of a previous design, but agree with the lack of discoverability noted in the review.

Some of my favorites from the list that haven’t been addressed with extensions.

• “Bookmarking: when creating a bookmark, user’s may want to organize their other bookmarks at the same time.” True, and unaddressed. I have found value in the flat bookmark editing extension for serious bookmark metadata management.
• “History button: have a natural mapping for the back list and the forward list by placing them next to each other, but not integrated into a single list.” While I can picture this as being very cool, it doesn’t offer any advantages for moving beyond tree-trimbing to communicating the full graph of history and web-based navigation.

Another comment:”Bookmarking and history seem barely sufficient, might be interesting to bring people in and see what their actual needs are. If it gets too big it starts to fall apart, both in terms of implementation and usability. “

At a general level, we know a lot about this from 10+ years of HCI work. Specifically, we know that some of the revisitation needs of users are short term. To address this idea, which has been observed in traditional research methods, several efforts have offered transient ‘marks: the TARK extension and Netscape 9’s Link Pad.

There’s another bit here — the hierarchical nature of bookmarks, and single category membership of any one mark, is simply not the way people work. Tags will help, but no one has figured out a solid traditional GUI (e.g. XUL not HTML page refreshes) UI for selection (not browsing) from tagged data. I’ve take a couple stabs at filtering UIs and even mocked up tags in the bookmark hierarchy.

Here’s an debatable idea: “History sidebar: we say 4 days ago, but it would be much more useful to say “Last Tuesday” — this is a good candidate for user research, but the challenge for traditional methods is that it requires studying the user in their native infospace. Diary studies and other ethnographical methods are awesome for this, but much more challenging to conduct for many practictioners than lab tests. An alternative approach is remote instrumentation effort where experimenters gain control over which UI is used (x days, or last day-name) and users are either split between designs and studied over time or specifically asked to revisit a recent experience.

In general, aspects of history and bookmarks which go beyond basic interaction design and labeling are challenging to evaluate well in the lab as they require the data in the browser to be personal and map over a significant range of time. I’ve been working with AlexP a bit on the upcoming instrumentation work — we may be able to bootstrap experimental evaluations of alternate schemes using this infrastructure.

I caught the inevitable history data growth work items from Burning Edge last night:

• Fixed: 332748 - Limit history to 20000 entries (new pref browser.history_expire_visits) in addition to 180 days.
• Fixed: 385397 - History menu is slow to open with a large history.

So 180 days or 20,000 records, whatever comes first, is the current Firefox warranty on support for revisitation. There’s a lot of engineering going on on the data-structure, but there may also be some smarter strategies. Please clue me in if I’m missing existing strategies in my cursory reading of the bugs and docs.

The first, and obvious, is that history has “visit type” logging which distinguishes between ajax, iframe, and other location non-changing page loads. My first thought would be to drop this traffic pretty quickly. While I find this stream of URLs very interesting from a nature of the web POV, and potentially from offline support, ux logging, etc., it’s (AFAIK) filtered out of the the history UI and not as useful as top level urls.

The more interesting challenge is how you might preserve visit data that’s important from normal visits. Some indicators that a page is important:

• It’s a hub. You follow numerous links
• It spans more than one session. A highly visited reference page (i.e. a regex reference page) could have been visited frequently during a single session, but may not be as useful as true hubs
• Significant actions are taken from the page, like printing, copy/paste, rss subscription, microformat address access, etc.
• Opened but unviewed pages where you ran out of time, needed to restart, crashed, etc. — these actual deserve a special prominence

Figuring out pages (and trails) that are likely to be useful later, and not easy to simply google, is a definite challenge. Do you care the most about that page you visited once for an answer, or the very interesting blog post on one of your daily reads you didn’t get to look at in detail?
The new relational structure of history also allows for first visit, last visit, and visit count to be preserved even though the actual session trails have been discarded. MOZ_PLACES stores urls with dates and counts, while MOZ_HISTORYVISITS stores each page impression. The retention for places should likely be larger than the retention for visits.

So much goodness to be had… I’ll have to chime in later on the history sidebar, see a mockup here. I always set my history to order by last visited to get a flat list that I can filter or quickly select from the last 500ish pixels of urls.

One of the features that caught my eye in the lead up to Leopard was “web clipping”. A new toolbar button in Safari allows you to grab a portion of a webpage and create a Dashboard widget. I’ve created a 30 second video demo (1mb .mp4) (stream at Blip.tv), as the implementation details are interesting and it’s vaguely akin to the old hypertext idea of transclusion.

Identifying the clip uses some basic DOM element location extraction, with snap to an image position and extent or even a text block. This effort doesn’t go as far as my noodlings on DOM tree based webpage section extraction (domexplorer.mozdev.org), but it’s useful. Even though it doesn’t allow, for instance, a two element excerpt from a right nav bar (ex. new projects & updated projects at the mozdev.org homepage), the snap to of the single section of the right nav makes it very easy to extend the selection to get the other element which has the same width.

Events on the wdiget pass through and execute the correct actions, in the default browser thankfully. Some feature requests remain:

• Pixel offsets don’t necessarily work across instances of a page segment. DOM selection would be more resilient, but would limit the selection flexibility.
• I’d like to be able to scale down the capture, ideally with interactive zooming on the dash. Looking into whether I might be able to that with an iframe-css hack… looks like IE still lays claim to the single best proprietary feature in IE, css zoom? You could certainly port this feature to Adobe AIR and use a chromeless window. Firefox can do it with SVG IFrame translation.
• So far, so good with it remembering authentication, but it doesn’t capture in-page clicks on AJAX functions, so no view switching in GoogleAnalytics.
• Dashboard performance has been challenging in the past. I re-enabled it to try this feature and perf is snappier in Leopard, combined with new features to release virtual memory, I”m optimistic.
• I’m out of space on my dash! Wish it integrated with spaces (e.g. the virtual desktop implementation shipping in Leopard)

Oh, and check out the gorgeous typography at www.wired.com. Atlanta is fairly weak on the technology environment, but when I start linking to wired again, memories of the original internet bubble return!

In a fairly weird twist of research funding, a market research firm and AutoByTel have released a search study & press release hyping a new automotive vertical “search engine”, with study details presented very loosely in a video on AutoByTel’s PR page (QT Video). The results describe widespread user experience with failed search sessions in which the user tried really hard.

The study made the buzz at techmeme, with Calcanis hyping mahalo, Greg@SearchEngineLand beating the peronalization drum, and more inspired thinking from Marketing Pilgrim.

While it’s certainly true that a large number of searches fail, in many of these cases, the user goes on about their day. This study focuses on experiences where the user is highly motivated and continues to try and achieve information gain unsuccessfully. The specific type of personalization appropriate here is actually “contextualizaton”, or just in time personalization that is as often as not irrelevant following the session.

A great search engine, or even search engine interface, would facilitate cumulative understanding of user intent across the session in the retrieval process and help users issue more effectively consume results. Vertical search engines simplify the problem and correspondingly severely reduce the range of needs that can be addressed.

Oiy, this post is far from blogging on peer reviewed research. If you’d like to learn something more real about search behavior, take a look at a recent MSR study exposing how advanced searchers differ (across Yahoo, Google, and MSN) from non-advanced users as measured by user of advanced search syntax:

White R and Morris D. Investigating the Querying and Browsing Behavior of Advanced Search Engine Users. Proceedings of ACM SIGIR 2007, July 2007.

BarCamp Atlanta this weekend! I’m planning up a demo of creating a firefox extension, getting 30 fps out of DHTML and SVG animation (ooold example), and SQLite places hacking with Firefox3.  I figure I’ll have time to hack out the structure of the talk once the event starts.  With Eclipse and XUL Booster, getting a new item in a Firefox toolbar is pretty easy.

My Firefox3 places hacking is coming along, though I’ve bypassed the places interface to bookmarks for now and am using the ttrw@mozdev starter kit for querying SQLite. 

There’s another Firefox3 project, Journal, that is taking on the xpcom bookmarks & history APIs directly. Their intention to do a custom home page isn’t new, but the ideas on rewriting titles in history to be more useful is awesome.  I’m noodling on annotating history with engagement metrics (very little or very much) to facilitate retrieval.

It’s very cool that labs.mozilla has a Firefox 2 extension contest, but the allure of cleaner (and more stable!) access to the internal data-sources will likely keep me playing with v3.

Thanks to the Firefox team for ever-amping the transparency of the process! Sure, back in the day, I spent hours digging through bugzilla.mozilla to figure things out, but the MediaWiki instance at mozilla.org is almost as active at a much better level of granularity.

The Fx3UIPlan page details some the interface expressions of the components that have been long in the making, notably within the Places system:

• Auto-complete / URL Bar: check out the UI Mockup. Alex hinted this was coming in my latest post on revisitation, but oooh lala! Necessary backend work is approaching completion: title+url matching and unified search.
• Stars for bookmarks: reinforcing the utility of bookmarks by indicating bookmarked status in the URL bar.
• Thumbnails in the “content pane” via the views menu. There are some interesting challenges in making a set of thumbnails aesthically pleasing, though check my post(s) on combinformation for another approach.

One of the challenges in this effort is the storage of overlapping sets, specifically bookmarks tagged with one or more keywords, in SQL. The wiki references a solid SitePoint article, but Joe Celko’s SQL for Smarties is the definitive reference of techniques.

A random thought came to me while taking the weekend away from the computer… Here’s a mockup showing how the value of tags might be realized within a more traditional cascading menu design. Flattening multiple set membership data into a hierarchy is lame.

By placing a set of tags at the top of each flyout menu communicating the distribution of tags within the folder, the user is reminded of the constituents of the folder branch. Engaging one or more tags could filter the menu to only nodes or branches containing appropriately tagged content, perhaps even flattening the menu and indenting child nodes that match to some maximum menu height.