This week we published an update to the Federated Search Implementer’s Guide.  This update includes some new information as well as some corrections to the original document.  Most crucially, the original document erroneously indicated that the URL template contained an attribute called “format” – when it was referring to the “type” attribute.  So if you followed the guide exactly, your OSDX file wouldn’t work properly with Windows 7 (or anybody else following the OpenSearch spec).

This update also includes additions such as information about providing “collection” templates to display results without the user entering a query, details about URL protocol support, and more.

Link: Windows 7 Federated Search Provider Implementer’s Guide

The new page roughly sums up its use and purpose:

A client-side OpenSearch data source that sits in between the Microsoft Windows OpenSearch provider and the external data source.  With a search connector (a .searchconnector-ms file), Windows Explorer calls your implementation with the query parameters. Your implementation returns results formatted in RSS or Atom format. That allows your implementation to provide custom authentication UI and connect to the data source using its proprietary API.

More details at:  http://msdn.microsoft.com/en-us/library/dd378289(VS.85).aspx

Unfortunately we still don’t have any details on how you actually hook up an implementation of this.  The doc / SDK folks are working on this but it’s part of a larger effort that’s taking a bit of time.  In the meantime I’m going to see if I can post an example here or on CodePlex.  Most likely I’ll be posting a detailed .NET example followed by a simpler C++ version.

If you have questions about whether this API may end up being useful to you, let me know, and hopefully I can help clarify what is or isn’t possible.

This document covers subjects such as crafting the best OpenSearch Description File for your site, returning files and custom properties, customizing the results view, optimizing your source for performance, and more. If you’re interested in developing an OpenSearch source for Windows 7, check it out! And as always, please share your feedback!

Link: Windows 7 Federated Search Provider Implementer’s Guide

In case you’re wondering which of the features unveiled at PDC is “my baby,” that would be the OpenSearch-based search federation feature in the Windows Explorer, detailed in the Find & Organize session which you can watch here.

We also have a Channel 9 video covering the new Libraries feature and other user experience improvements in Explorer for Windows 7.  This video is part 1 of a two part series, the second of which covers the Search Federation feature (and yours truly).

For now, the best place to learn more about OpenSearch in Windows 7 is to watch the session video at the link below.

PDC 2008 session – Windows 7: Empower users to find, visualize and organize their data with Libraries and the Explorer

This afternoon we made a follow-up post addressing some of the suggestions, comments, and concerns that showed up in the comments to the original entry.

If you have more feedback, please keep it coming.  I and others will try to respond in the comments over there, or in future follow-ups.

Also, if you like the gadget, please go to the Gadget Gallery page and give it a good rating

Download Indexer Status Gadget on Live Gallery

This afternoon Steven made his second post on the blog, discussing the Windows 7 engineering organization.  He describes the rough dimensions of the various feature teams, including a pretty complete list of the teams and an idea of their approximate size (ballpark estimate being 40 developers, 40 testers, and 20 PMs). 

If you look at his list you can spot my team, which goes by the name Find and Organize.  So who are we?  Well, as you might guess and Steven describes, we own the end-to-end Search features of Windows (the “find”) as well as the browsing and data management experience (the “organize”).  We often abbreviate this as “FnO” and many people pronounce it as “fff-no” (rhymes with “snow”), though I am not one of them.

Some examples of things we own:

• The Windows Search indexer
• The indexing configuration UI and the built-in filters / handlers
• The Windows Explorer UI, including things like:
  • The navigation pane
  • Address / Breadcrumb bar
  • Search box
  • Window frame
  • The Filesystem data source
  • The default Shell View (DefView), default Context Menu (DefCM), etc.
• The Explorer data source APIs (IShellFolder and friends)
• Known folders and APIs (”Documents” and such), recycle bin, shortcuts
• The common file dialogs
• Many/most of the Windows “common controls”
• Search in the Start menu

Whew, I often forget just how much stuff we own.  And my feature isn’t even on the list!   Oh yeah, did I mention we also owned the entire Windows Search 4.0 release?

We also aren’t the only team that “owns” the Explorer.  The Core User Experience team owns other pieces of Explorer like the taskbar and the rest of the Start menu.  You’ll often hear people from both teams refer to themselves as the “shell team,” or others refer to both teams collectively by that name.

So who makes up the FnO team?  Well, as I said, there are quite a few of us.  I’m certainly not the only one of us who blogs or has an online presence:

Jonas Barklund has a blog, he’s a developer whose focus is generally on query parsing.

Thayn Moore has a blog and is also a developer, and works on a bunch of stuff including, among other things, mail indexing.

Ben Karas has a blog and is a lead developer for most of the explorer UI.

Paolo Marcucci is a PM who doesn’t seem to blog these days, but posts on Channel 9 a lot.

I’m sure there are others I don’t know about.  So if you work on my team and have a blog, let me know.

As for me, well, you know where my blog is.  As for what I do every day… we’ll talk more about that soon enough.

Update: I saw a comment on Channel 9 ask about how we function with 40 developers on one team.  Well, the truth is these “feature teams” are actually divided into smaller teams that focus on closely related components.  So no, there isn’t really one manager with 40 developers reporting directly to him or her. 

Windam – I wonder why this would be released for Vista since search is already a well integrated feature to begin with.
Is it just because(optional)?

Maudit - Pardon my ignorance, but what the difference between Windows Search 4.0 and the one in Vista ultimate sp1, does it streamline into windows ?

A similar question was asked on Channel 9.

The answer is quite simple:

• Windows Vista includes “Windows Search 3.0″
• Windows Search 4 is an upgraded version of the indexer built-in to Vista.
• This release does not change the User Interface, it looks exactly the same.  The improvements are around responsiveness, resource usage, reliability, API support, etc.

A good analogy here might be DirectX.  Windows XP shipped with DirectX 8.1.  When DirectX 9 was released for XP, it didn’t change the way anything looked or behaved, but it made your system better.  You may apply a similar understanding to WS4.

• Faster queries, often several times faster
• Faster indexing / reduced resource usage
• Support for EFS (encrypted folders)
• Support for indexing delegate mailboxes
• Fixes for all the top reported Watson crashes / errors
• New group policy options

This release also adds the following Vista / Server 2008 features to Windows XP / Server 2003 systems:

• Remote query support (using the server’s indexed when querying a network share)
• Support for Vista-style preview handlers (Office 2007 documents, etc)

Read the KB article here for more details and complete feature list.

Download Links

Vista / 2008                    32-bit    |    64-bit

XP                                   32-bit    |    64-bit

2003 / WHS                    32-bit    |    64-bit

Vista users – don’t forget to grab the indexer status gadget!

Vista Team Blog – Announcing the Windows Search 4.0 Preview

KB Article describing Windows Search 4.0 (with download links)

This release is mainly an update to the Windows Search indexer, and provides countless performance improvements, bug fixes, and reliability / recoverability features.

The XP/2003 version has been updated with more features previously exclusive to Vista – such as the ability to search remote indexes for network shares, and the ability to host Vista-style preview handlers in the preview pane.

WS4 also provides some cool new query capabilities for developers, which I will describe and give some examples of in future posts.

The most noticeable difference is probably how fast it is.  Those geniuses down the hallway in indexer land really pulled off some impressive feats with this release.

Since there are six different downloads depending on your OS, I’ll just refer you to the KB article for downloading the preview release.

Let us know what you think!

Tom Laird-McConnell, who used to be my boss until a few months ago, wrote one of his extremely rare blog posts last week on this subject.  To celebrate the occassion, I will link you there for the detailed answer to this question.

Definition of the PKEY_Identity / System.Identity property at Tom’s Handy Dandy Space

At its core, the Windows Search indexer doesn’t really know anything about files, e-mails, or anything like that.  In fact, all it really knows is how to do the following things:

1. Index contents and metadata associated with a URL and store it in a row.
2. Retrieve rows that match a specific query.
3. Shape the results in interesting ways (sorted, grouped, etc)
4. Retrieve properties / metadata associated with a row.

The indexer relies on other Windows Search components to handle the specifics, such as converting a URL into data to be indexed.  That’s where Protocol Handlers, IFilters, and Property Handlers come in.

Protocol Handlers

A Protocol Handler allows the indexer to crawl a specific kind of data store.  For example, the File System Protocol Handler allows the indexer to crawl files stored on your hard drive.  Windows Search includes a few Protocol Handlers including those for the File System, MAPI (ie. Outlook), and the Client-Side Cache for Offline Files (Vista only).  Other examples include the Protocol Handlers for Lotus Notes, the IE History / Cache, or Mozilla Thunderbird.

At a basic level, a Protocol Handler is just a piece of code that takes as input a URL (like “file://C:/Foo/” or “mapi://{USER-SID}/Brandon’s Mailbox/Inbox”) and performs two important tasks:

1. Enumeration of child URLs (such as “file://C:/Foo/Bar/” or “file://C:/Foo/Bar/Taxes.docx”)
2. Binding of URLs to either an IFilter, or a Stream (which can be bound to whatever IFilter is registered for its content type)

IFilters

An IFilter is responsible for taking an item such as a file (usually in the form of a Stream) and emitting the contents and properties of that item for indexing.

For example, the MS Word IFilter knows how to take the stream from a .DOC or .DOCX file and return both the contents and useful properties (like the author’s name or the date it was last modified) into the index.

Property Handlers

Property Handlers are similar to IFilters, except that they’re designed to simply return properties for items and not complex textual content.

We’ve also released a new set of code samples to go along with it.

I’ll be digging in and writing more about building on the SDK very soon!

WDS 3.01 also brings back full Group Policy support (previously available in the WDS 2.6 Enterprise releases). 

Special congratulations to the indexer team, many of whom worked some pretty heavy hours through January to make sure this release had that extra layer of polish on it.  Thanks guys!  Now go home already!

Download links:

Windows XP 32-bit

Windows Server 2003 (32-bit)

Windows XP/2003 x64 Editions

Group policy ADM file is included in the package.  Run the installer with the /extract parameter to extract the contents directly.

There’s also an updated Advanced Query Reference for 3.x / Vista.

The WDS website and related docs are in the process of being updated as I write this.

Download Windows Desktop Search 3.0 for Windows XP (32-bit)

Download WDS 3.0 for Windows Server 2003 (32-bit)

Download WDS 3.0 for Windows XP/2003 (AMD64)

This release represents a HUGE amount of work across multiple teams and we’re all proud to be delivering the first WDS release that runs as a Windows service, supports 64-bit systems, and unifies our Desktop Search platform and APIs across Windows XP, 2003, and Vista.

As always, post comments with your feedback!  Or head to the official forum.