The Bing team tweeted last night that a new MSN Toolbar is now available at toolbar.msn.com. The new toolbar is supposedly now powered by Bing, Microsoft’s new search engine.

I am not exactly sure what all the new features are but as you can see, the MSN butterfly is now being placed inline with the Bing logo:

Since their infancy, social news sites have been trying to stay one step ahead of social media users trying to artificially promote content to their front page. Digg.com has been the center focus for spamming and gaming in social media because a front-page promoted story equates to thousands of visitors and hundreds, or even thousands of links. In order to combat the gamers and filter out spam, Digg has created an algorithm that is very complex in nature. Never do they cease to fine tune it, in an effort to try to stay ahead of people trying to artificially promote content to their front page.

Social voting: then

Social news sites rely on the community to downvote or bury spammy content, and while the community does a good job, it’s not always enough to keep an organized promotion effort from succeeding. Traditionally, organized social community members have voted on each others’ content to get enough votes to promote to the front page. While there are measures in place to prevent organized efforts, like enforced Terms of Service rules, natural behavior recognition, reciprocal voting pattern observation, etc., it hasn’t been enough. Thus, social media sites have looked for another solution to help them with their epidemic.

Users heavily involved in social media are constantly being pinged all day long from other users in solicitation for a vote on the content they are trying to promote. Frequently, social media users “blind vote” without even viewing the content. Social news sites caught on and established a behavioral metric of content click through to make the vote weight heavier towards the promotion of that content. Social users then just started clicking through and leaving quickly before they voted.

Social voting: now

Recently, StumbleUpon, Digg, and Reddit have all released options for their users to shorten their URLs to share with their friends. These shortened URLs frame the content below a bar at the top of the page. The bars allow users to vote on the content and explore related content if desired.

StumbleUpon’s Su.pr URL shortened bar

Reddit’s short URL bar

Reddit’s FireFox Socialite bar

Digg’s URL shortened bar

Call me a conspiracy theorist, but I think there’s a deeper reason StumbleUpon, Reddit, and Digg have all recently released URL shorteners and framed bars for their content. We now are approaching a new era in the world of social voting and organized promotion. An era of measuring user engagement on content to analyze natural behavior in order to determine if the vote for that content, from the social media user, is genuine or artificial.

Framed content allows social news sites to move beyond the click-through to measure behavioral metrics.  Now social news sites can analyze:

• How long a user was on the site
• If the user scrolled below the fold
• If the user interacted with media objects like Flash or video
• What the average time spent on the site was
• If the content intrigued the user to explore further within the site
• What the user clicks on, copies, bookmarks, prints, etc.
• How many visitors came to the URL and didn’t vote

This opens up gaming to a whole new level. Now social users need to have their voter rings either engage more heavily in their content or emulate fake user-behavior. This is a new level of cat and mouse that gamers will be playing with social news sites. Personally, I always engage in the content, and no longer blind vote, to prevent my social profiles to be viewed as natural and not part of a voting “favor” ring.

For the social communities, this is a great step in the right direction. The analytics should help filter out a lot of the garbage that enters the frontpage of many social media sites. Time will tell how if social news sites are actually analyzing and utilizing this data correctly in their benefit.

Your sites are being framed

As of now, the Diggbar shows only to those Digg users that are signed in and haven’t opted out of the Diggbar. I’m sure if Digg had it their way everyone would use the framed content. However, they caught a lot of flack from webmasters for doing so. There are some fundamental issues with these frames that have upset webmasters. Social frames:

• Prevent canonical URL bookmarking
• Prevent canonical URL sharing
• Prevent cross-pollination on social sites (for example, you can’t Stumble the canonical URL upon visiting a framed Digg URL)

There is a solution. Add this code to the top of your pages or install this Wordpress plugin. However, some webmasters are happy because there are benefits from shorteners like Su.pr. Su.pr, by StumbleUpon, allows statistical analytics of your users and encourages them to checkout other popular content on your site. Ultimately, it’s up to you which route you want to take.

It appears that Bing is overriding the default search provider set by users of Internet Explorer 6. Several IE6 users have complained that since Bing launched, the search provided went from Google (which they set manually) to Bing. To makes things even worse, when the user tries to set the search provider back from Bing to Google, it doesn’t allow them.

To be fair to Microsoft, IE6 is very old and Microsoft is already up to version 8. I emailed Microsoft about the issue and they confirmed the bug. A Microsoft representative sent me the following statement:

We’re aware of the issue with IE6 and Bing and are investigating a solution. This issue is not impacting IE7 or IE8 users. We respect user choice on search providers in IE and all browsers, and designed IE to enable that choice. We will provide an update soon on this issue, and we apologize for any inconvenience it has caused. In the meantime, we encourage customers to upgrade to IE8 here. Alternatively, Firefox users can install the add-in for Bing here.

Controlling the browser is a big deal for influencing search market share. So this bug is fairly serious, being that many novice internet users still use IE6.

Postscript: Microsoft emailed me again at 2:45am on June 3, 2009 to inform me the issue is now resolved with IE6. The issue was server side, so the fix was able to be pushed out remotely to all infected browsers.

Digg is promising a significant change to how its DiggBar framebar operates, one that should solve SEO concerns about how link credit is passed on but won’t entirely remove misgivings about the framing of content.

With the new implementation scheduled for next week, only those who are logged into Digg will see the DiggBar. So if you click on a DiggBar URL from Twitter, you won’t see the DiggBar unless you’ve already been to Digg and have a logged in status. Just having a Digg cookie won’t be enough, Digg cofounder Kevin Rose told me when we talked about the new implementation yesterday. A Digg user must be logged in for a DiggBar to show. Logged in users can also opt-out of seeing the DiggBar.

Those who are NOT logged in will simply be redirected to the destination page, via a 301 permanent redirect. This will include search engines that follow those links. Digg’s been in touch with Google’s Matt Cutts on the change, which I’ve confirmed. The consensus is that Google will register DiggBar links as 301 redirects.

Overall, this change is a good thing. It means that link credit will flow to the destination site, rather than being kept by Digg itself.

There remains the separate issue of framing, how that was unpopular in the past and remains so in many quarters given the recent resurgence. Yesterday, it was pointed out by Michael Gray how Kevin Rose himself wasn’t pleased to find his own content being framed. So why do this to others, I asked him.

Kevin’s response was that he felt the Truveo experience was different. He wasn’t a logged in user there, so wasn’t expecting that type of display. He also didn’t like that it was an all-panel framing — that his content was completely surrounded by all four sides, in a frame.

I have mixed feelings. Showing a frame to only logged-in users seems like a fair compromise. There’s no doubt that for some Digg users, the framebar is useful. It’s also easier for them to use than downloading a separate toolbar (though Kevin said Digg still plans to improve and promote its toolbar).

On the other hand, I still hate frames. I also really worry that this will just cause other services to do the same. Imagine if Google decided that it should put up a Google framebar for anyone logged into one of its services? I still think the internet would explode in fury over such a move. But Digg’s compromise would give Google the cover to do so — as it would give cover to Yahoo or Microsoft, and as it now extends cover to Facebook that already does framing.

I wish Digg had made the tough decision to drop the framebar entirely, so that pressure could then be applied to Facebook and StumbleUpon and others to drop theirs. But the compromise does seem a good one, as long as it doesn’t end up opening the gates for more framing. Certainly sites that object can continue to run framebusting scripts, as we do here. That solves the frame issue for them, and the other change deals with the link credit issue that was developing.

For more about the DiggBar, framebusting, 301 redirection and other issues, see these two past articles from me that go into them in much more depth:

• The Growth Of Framebars & Kevin Rose On The DiggBar
• URL Shorteners: Which Shortening Service Should You Use?

For more, see discussion on Techmeme.

The DiggBar has been out for about a week now. Since then, there continues to be concerns over twin issues of whether it robs sites of link love and frames their contents in a way that’s unfair to publishers. I had a good conversation with Digg cofounder Kevin Rose today about these issues and how Digg is actively looking at ways to solve worries over the tool.

For those unfamiliar with the DiggBar, it allows people to create a short URL that’s useful in services like Twitter. Anyone clicking on a shortened URL made through Digg gets to a page with a DiggBar at the top. For example, here’s how it looks for a post I recently did on my personal blog about newspapers and concerns over Google:

The red arrow points at one feature, how the DiggBar allows anyone to vote on the page they’re viewing. There are other handy features, such as the ability to see any comments people have made at Digg about the page:

There’s no doubt that if you use Digg a lot, you’ll probably love the DiggBar. But the bar does two things that aren’t making some people (including me) very happy. It doesn’t pass along link credit, and it frames web content.

Link Credit Issues

Last week, my URL Shorteners: Which Shortening Service Should You Use? article went into depth about how various URL shorteners work. A key issue is whether these shorteners tells search engines to credit the destination URLs they point at. Those issues what’s called a “301 redirect” do this correctly (my What Is Google PageRank? A Guide For Searchers & Webmasters article covers more about link credit issues and why it is important to search rankings).

The DiggBar does not do a 301 redirect (nor can it, as this would prevent the DiggBar from showing at all). If you shorten a page using DiggBar service, then Twitter the short URL you receive, any links that Google or other search engines find via that short URL will send credit to Digg, not to the destination page you shortened.

A sidenote here. Twitter automatically puts a “nofollow attribute” on any links that people tweet. That’s a method to tell search engines that the links shouldn’t be counted as “votes” as part of their ranking processes. However, tweets often appear off the Twitter.com domain. In some of these places, the nofollow attribute (or tag) doesn’t get used. So tweeted links can get counted by search engines, and it remains important that URL shorteners pass along credit to the destination pages.

Digg had a blog post out yesterday explaining that they had done some things they believed would stem concerns about link credit not flowing properly. SEO expert Greg Boser dissected that post, finding it didn’t hold up. I also looked at it today and found problems:

1) Using the noindex tag prevents the pages that Digg makes with shortened URLs from being spidered by Google and other search engines, but that does not solve the issue of them still accumulating all the link credit rather than this going to the destination URL. Also, so far despite using noindex, some of these pages are getting listed in Google. Here’s another example of this. (Looking at the source code, that page lacked a noindex tag an a canonical tag. It seems like originally, the DiggBar didn’t add these tags. Now that they are present, it will take search engines a few days to weeks to catch-up).

2) Using the canonical tag as a form of redirection doesn’t work, because that tag is still treated as a “hint” by search engines rather than an must obey instruction. It also doesn’t work across different domains (IE, Digg.com can’t point at content off Digg.com’s own domain and use the tag to tell the search engines anything).3) The “source URL” solution Digg discusses doesn’t solve anything. What this means is that if you’re on the Digg home page, stories are listed there from across the web. For example, here’s a popular one right now from the Daily Telegraph:

http://digg.com/d1oOii

Digg uses a short URL to point you at that story, which in turn brings the DiggBar up on the top of the page. However, if you can’t run JavaScript (as search engines operate), then you get the long “source URL” like this:

http://www.telegraph.co.uk/scienceandtechnology/science/evolution/5131017/Egg-collected-by-Charles-Darwin-found-at-Cambridge-University-after-200-years.html

Digg’s thought was that by showing the long “source” URL to search engines, then the long URL ultimately will get all the link credit. However, there are plenty of places where the short URL will be found across the web by search engines because it is listed with regular HTML, rather than through JavaScript.

Framing Issues

Back in the late 1990s, framing was a big issue. For those unfamiliar, frames allow a web site to pull in content from other web sites into their own pages. It was much loathed for a variety of reasons. It often led to bad user experience. It caused serious issues for search engines, making it difficult for them to spider content properly. Some felt it was a copyright violation — that the site doing the framing was effectively copying their material without permission.

Framing largely disappeared for all of these issues. But now it’s coming back, and Digg’s use with the DiggBar may have been the tipping point.

Last October, StumbleUpon added framing of sites, so that anyone starting a browsing experience from StumbleUpon’s home page gets a framebar like this, as the red arrow points at:

Back in December, Facebook added its own framing of content through a framebar that appears when you click on posted links from within the service. Again, the red arrow points to an example that you can see for yourself here:

Ask.com started framing search results in February. The red arrow below points to the framebar, which appears when you click from Ask search results to a web page that’s listed, like this:

Ask used to do this when it first started out back in the 90s, then dropped framing apparently because so many sites moved away from that model. Now with harder economic times, it apparently finds value in trying to take over the top part of your browser window.

Of course, Google also frames web sites with its Google Image Search service. I believe it has operated this way years and years ago from when image search first started:

There was actually a lawsuit over this, which Google won. Despite that, this might be a good time for Google to reconsider the framing it does.

Also, if Google were ever to frame web sites when you click to them from search results in the way Ask does, the web would almost certain erupt in anger. I don’t think this will happen, of course — but if it’s not something we’d allow Google to do, it’s not something we should be allowing any sites to do.

Framebars Versus Toolbars

Clearly Digg didn’t start the new wave of framing, but it certainly has taken much more heat than Facebook or StumbleUpon over it. I think it’s the combination of URL shortening with framing that’s tipped people over the edge. That mixed framing with the popularity Twitter, where most people these days seem to be using URL shorteners. I think it creates worry that there will be no stopping framing or framebars now.

I feel for services like Digg and Facebook and StumbleUpon. The framebars they’ve created are useful and certainly easier than having users install toolbars for their browsers. But they remain frames, and they bring with them all the negatives about frames that we had in the past.

I’d hope that perhaps there’s an industry move to develop some standards around framebars. For example, if they’re going to be used, perhaps they are less intrusive to a publisher if shown at the bottom of a browser window, rather than at the top. Perhaps there’s also a way to ensure that the URL showing in the browser window remains that of the “source” site with the framebar also displayed (it’s been a long time since I played with frames, so I’m not sure this can be done).

Other ideas might include developing a standard script that publishers can use if they want to break frame code but also inform visitors from a particular site (such as Digg or Facebook) that they can get similar functionality using software toolbars. Perhaps pop-up toolbars in a separate window could work, though there are issues with pop-up blockers.

I don’t know the right answer. Personally, I think the easiest thing would be for everyone to just say no to frames. If you want your dedicated users to have toolbar-like functionality, then have them install an actual toolbar, not a framebar.

Kevin Rose On DiggBar

How’s Digg viewing the uproar? “It’s been a crazy learning experience for us,” Rose said. “We want to follow best practices.”

Rose explained that initially, Digg wanted simply to do a toolbar to help their most active users more easily Digg content or comment on it.

“We wondered what can we create that allows people to go visit that site with a single click and still get a Digg experience. That was kind of the idea behind creating the bar in the first place,” he said.

More as an afterthought, when seeing how popular it was to shorten URLs on Twitter, Digg added on a shortening aspect to the DiggBar.

“The goal in creating this wasn’t, ‘Let’s be the universal URL shortener.’ It was ‘Let’s make a tool that can enhance the experience for Digg users’.”

Rose said someone at Digg did speak with a software engineer at Google, as mentioned in their blog post, about the best way to pass along credit to Google — but he didn’t know who that was. Fair to say, they’ll get the straight scoop shortly, as Rose is now set to speak to Matt Cutts, who heads Google’s spam fighting efforts and who also closely watches over webmaster issues.

As for the DiggBar’s future itself, Rose said the company is taking in all the feedback to determine what’s the next best step.

“I want to make it known by all means that we’re sitting down and thinking about this stuff and trying to come up with solutions that work for anyone,” he said.

What’s A Webmaster To Do?

While Digg reexamines the DiggBar, there are webmasters who will remain concerned. My original article on URL shorteners has code you can use to block framebars. Wikipedia has a page about this, too, and you can see John Gruber’s code here. By the way, we actually had that code on our site before the DiggBar came out, just as a general best practice of breaking frames.

Of course, if you like the idea that people can more easily Digg (or Stumble or Facebook) your content, then you might not have an issue with using the frames.

I’d still recommend that if you’re wanting to shorten URLs for your own sites, use a service that’s primarily built for that and which does 301 redirect.

The Yahoo Search Blog announced that my favorite search browser plugin, Inquisitor has made its way to the iPhone. You can get it right now on your iPhone in the iTunes Store for free by clicking here.

The features include:

• Yahoo Search Assist search suggestions
• View search results in one click, within the application
• Site summaries load before the web site itself loads
• Yahoo News results (if applicable)
• Personalized search results based on your search history
• Search history shows you your search logs
• Clear results by shaking your phone

So I had time to play with this iPhone app and I had some questions. I cannot change my default search provider from Yahoo. The reason I love Inquisitor for Safari is that it lets me plug in my search providers and very advanced search operators that I use often. When I asked Yahoo about changing the provider, they told me “consumers can only use Yahoo.” A Yahoo spokeswoman added, “We made that decision because we wanted to ensure the best experience for our mobile consumers– speed on mobile is a critical issue and we can go to the yahoo web results faster via BOSS as we don’t need to parse results. Using other providers reduces the speed on the mobile app.”

Outside of that, they me take you through a screen by screen of Inquisitor for the iPhone.

When you load up Inquisitor on your iPhone for the first time, you will see this screen:

As you type in the search box, search suggestions begin to show up. This is powered by Yahoo’s Search Assist:

You can click on a result or hit search at the bottom of the screen. Here are the search results, which I assume are provided by Yahoo Search:

If you scroll down to the bottom of those search results, you will see search suggestions:

Clicking on a search result will first show a snippet of what the page is about at the bottom, while it loads the page at the top:

Once the page loads, the snippet disappears:

There is a back and forward button and a refresh button on the bottom (it seems to use the built in Safari controls within the application). Plus you can email or visit that page directly in Safari mobile:

Closing the application and opening it again or shaking the phone will take you back to a page with your search history and a search box:

Here is a screen capture of the settings area for Inquisitor. To access this page, you go to the iPhone’s home screen, click on the gray “Settings” icon and scroll down and click on Inquisitor.

Here you can control if you want to have the application auto correct your searches, change your country, control the number of results you see, turn on or off search history and safe search and control if you want the shake to clear the results.

Overall, this is a very nifty search tool for the iPhone. However, I do wish there were more settings to control the search preferences, like the desktop alternative.

Note: We covered Google’s iPhone Mobile Search app three times here, including:

• Using Google Mobile App On My iPhone
• Video: Google Mobile iPhone App With Voice Recognition Now Available
• Activate The “Bells and Whistles” On Google’s iPhone Mobile App

The Live Search blog announced that Internet Explorer 8 is now showing instant answers from Live Search in the search box. This feature is on by default and will show instant answers for searches related to financial information, weather conditions, movie show times, calculations, equations, conversions, and definitions.

Here is a screen capture:

Microsoft’s Live Search still isn’t built into Firefox, but you can add it to your Firefox search box manually. Now, a new add-on from Live Search does this complete with the ability for the search box to auto-suggest terms you might be searching for. For more, see the Live Search blog.

My favorite search browser plugin, Inquisitor, is now available for Firefox and Internet Explorer.

Inquisitor was Apple Safari’s only solid third-party search browser plugin back in the day. Yahoo acquired them back in May and then provided an upgrade in mid-August. Now, Yahoo has announced they have made this plugin available for Firefox and IE.

If you are running Firefox 2 or 3, you can download the addon over here. If you are running IE 7 or 8, you can download the plugin over here. I highly recommend you give it a try and check out some of the customizations I deployed on Inquisitor years ago.

“Semantic web” is one of those phrases that’s tossed around loosely these days, used to describe just about anything that goes beyond basic keyword search to surface relevant content on the web. But true semantic web applications, those that enable people to share content beyond the boundaries of applications or web sites, are still relatively few and far between.

That’s what makes Headup, a new Firefox plugin from the aptly named SemantiNet, so intriguing. Headup monitors your web activity, and based on what you’re viewing can seamlessly and contextually “understand” how dispersed data across the Web is related to what you’re viewing. Then it provides snippets or links to that information without the need to search for it.

Even more intriguing, Headup has seemingly leapfrogged most current approaches to building out the semantic web that require web sites and applications to add layers of metadata conforming to a rigorous set of standards. Instead, Headup uses Microsoft’s new Silverlight platform to analyze content in real-time, without requiring any additional information from web sites.

How does this work?

According to Tal Keinan, Founder & CEO SemantiNet, Headup uses a sort of meta database—a graph that describes which sources can provide which types of information about certain things. It also taps into your social networks to leverage the information your friends have left scattered about the web. Then, as you’re surfing the web, Headup constantly compares what you’re looking at with this meta database and your friends’ knowledge to offer you additional information when it’s available. And it’s unobtrusive, displaying a ‘+’ symbol next to the relevant content. Clicking on this symbol brings up a separate window with related links and other information.

For example, say you’re browsing a music CD on Amazon. Headup recognizes that you’re reading about music, so it goes to work and shows you things like:

• How many of your Facebook friends like this band
• A link to hear the band’s latest music streamed via Pandora
• Links to let you see band pictures on Flickr
• How to find tickets for a concert by the band in your city via Zvents
• How many of your friends are using Twitter or Friendfeed to discuss the upcoming concert
• What restaurants near the concert hall are top-rated by Yelp

Currently, Headup is limited to pulling in information from the services mentioned above, and a few others including Digg, Gmail, Google, Yahoo and Wikipedia. Keinan says SemantiNet is working to add many other sources, but wanted to start out with this smallish set to get feedback from users. Eventually, he said he expects Headup to work with a huge number of web sites.

What’s with the mashup of Firefox and Microsoft’s Silverlight? SemantiNet says Silverlight offers user privacy capabilities that far exceed Flash and also allows Headup to work as a cross-platform PC/Mac product.

Headup is similar to Adaptive Blue’s Blueorganizer plugin for Firefox, which adds a button to your toolbar with a menu containing contextual shortcuts for books, music, movies, wines, recipes, stocks and other types of info. But Headup appears to go much deeper, with a broader array of content types.

Headup launched today in a limited beta release, so I’m going to wait to take a deep dive into the technology until it becomes more widely available. But I’m impressed with what I’ve seen so far, and the tool looks like it’s well on the way to becoming an invaluable part of my searching toolkit.

More information:

• Headup’s website
• SemantiNet’s website (try signing up for the limited beta here)
• The Semantic Web, by Tim Berners-Lee, James Hendler and Ora Lassila—one of the best introductions to the topic, by the creator of the original world wide web.