Brandon Live!

The message in my previous post apparently showed up because Apple doesn’t support 64-bit OSes (ie. Vista x64) with the iPhone.  Yuck.  Come on Apple… you were so close.

iTunes and the iPod both work fine, and the iPhone is recognized and shows up as a disk / camera.  But iTunes refuses to see it and sync.  Why couldn’t they get this right?  I hope they fix it soon… having to sync through my laptop (32-bit) is kind of annoying.

For making it reasonably easy to leave.  Unlike some companies.

I don’t really have complaints about the service.  But since I no longer have the car that had the XM radio built in, it didn’t make much sense to pay for it any longer.  The new car has Sirius, although I’m not yet sure if I’ll bother signing up considering how little I used the XM radio in the last one.

They made a quick attempt to sell me a new radio at a discount, which is perfectly fine.  But they weren’t insistent, and seemed quite understanding of my situation when I mentioned why I was cancelling.  They take calls on Sunday, and I spoke to a real person with hardly a 30-second wait.

So even though I’m no longer a subscriber, XM and I are on good terms in my book.

I’ve added a new FAQ page to the site as well as a few FAQ entries to get things started.  I’ll keep adding more as time goes on.  If you have a question you’d like to see answered, post it in the comments or e-mail me.  But just because you ask, doesn’t mean an answer is guaranteed or that it will be timely.

The Indexer 

At its core, the Windows Search indexer doesn’t really know anything about files, e-mails, or anything like that.  In fact, all it really knows is how to do the following things:

1. Index contents and metadata associated with a URL and store it in a row.
2. Retrieve rows that match a specific query.
3. Shape the results in interesting ways (sorted, grouped, etc)
4. Retrieve properties / metadata associated with a row.

The indexer relies on other Windows Search components to handle the specifics, such as converting a URL into data to be indexed.  That’s where Protocol Handlers, IFilters, and Property Handlers come in.

Protocol Handlers

A Protocol Handler allows the indexer to crawl a specific kind of data store.  For example, the File System Protocol Handler allows the indexer to crawl files stored on your hard drive.  Windows Search includes a few Protocol Handlers including those for the File System, MAPI (ie. Outlook), and the Client-Side Cache for Offline Files (Vista only).  Other examples include the Protocol Handlers for Lotus Notes, the IE History / Cache, or Mozilla Thunderbird.

At a basic level, a Protocol Handler is just a piece of code that takes as input a URL (like “file://C:/Foo/” or “mapi://{USER-SID}/Brandon’s Mailbox/Inbox”) and performs two important tasks:

1. Enumeration of child URLs (such as “file://C:/Foo/Bar/” or “file://C:/Foo/Bar/Taxes.docx”)
2. Binding of URLs to either an IFilter, or a Stream (which can be bound to whatever IFilter is registered for its content type)

IFilters

An IFilter is responsible for taking an item such as a file (usually in the form of a Stream) and emitting the contents and properties of that item for indexing.

For example, the MS Word IFilter knows how to take the stream from a .DOC or .DOCX file and return both the contents and useful properties (like the author’s name or the date it was last modified) into the index.

Property Handlers

Property Handlers are similar to IFilters, except that they’re designed to simply return properties for items and not complex textual content.

The three processes used by the Windows Search service are SearchIndexer.exe, SearchProtocolHost.exe, and SearchFilterHost.exe.  Sometimes you may even see multiple instances of the latter two running simultaneously (especially if multiple users are logged in).

So why are they divided up in this way?  To find out, let’s look at what each of the processes does.

SearchIndexer.exe

This process runs as a system service under the SYSTEM account.  It is responsible for maintaining the index, servicing queries, as well as deciding what to crawl and when.

SearchProtocolHost.exe

This process sometimes runs under the SYSTEM account, and other times runs in the context of the current user.  It hosts a Protocol Handler responsible for enumerating items in a specific store (such as the File System, Outlook, UNC shares, Lotus, etc).

Why is it seperate?

• Access - Sometimes it needs to run in the context of the SYSTEM account (ie. to index the filesystem, even when a user is not logged in).  Other times it needs to run in the context of the user, so that it can access data that is ACL’d for that user (network shares, Offline files) or accessed via a program the user is running (Outlook, Thunderbird).

• Reliability - If a protocol handler, which may be written by a third-party, crashes - it will not crash the indexer itself.  This reduces the risk of index corruption, and ensures that you can still issue queries even if a protocol handler crashes or hangs.

• Security - Isolating code that interacts with possibly untrusted data stores can mitigate vulnerabilities in said code.

SearchFilterHost.exe

This process hosts the actual IFilters. These filters are responsible for processing individual items, such as files, in a data store.

Why is it seperate?

• Security - This process is tightly locked down. For example, it cannot even read the filesystem. It runs with reduced privileges (kind of like Protected Mode IE). Why is this important? Well think back to the WMF file vulnerability a year or so ago. Google Desktop Search would trigger the vulnerability whenever it indexed one of those such files. If you received it as an e-mail attachment, you would have a 0-click attack because they don’t sandbox the indexing process. This wasn’t a problem for WDS users because we have always isolated filtering to a seperate locked-down process.

• Reliability - Same as with the Protocol Handlers.  IFilters are very often third-party code, and may be subjected to corrupted files.  Keeping them seperated improves robustness to crashes / hangs in third-party code or when dealing with corrupted data.

Interesting… apparently Google doesn’t eat their own dogfood! 

A friend and colleague of mine has a funny story about someone from Google asking for his resume, and not accepting it in Google Docs format, but instead requesting it as an MS Word file =)