Indexing & Duplicate Detection
Hotbot:
- Indexes text minus stop words; no words in images.
- No duplicate detection; spam detection.
Yahoo!
- Indexes URLs, titles, and categories.
- If nothing found, resorts to an Alta Vista search.
- Duplicates can be a problem.