@YetAnotherGeekGuy This may be an unpopular opinion, but for search, I don't think a centralized index is a bad solution. When Google was *search*, it was great. It only turned to shit when they replaced search with AdWords everywhere. That said, two alternatives, in order of more complexity for additional decentralization:
- a number of search-as-service operators, to which server admins can relay messages and offload index maintenance for amortized cost
- federated partial indices, in which co-operating servers each index a part of the whole and searches are distributed
Federating everything everywhere, that is each server indexing its own content and returning results to all searches, would not work. The thundering herds and search lag would be horrible.