With the rapid growth of digital information, it becomes increasingly challenging for people to survive and navigate in its magnitude. It is crucial to study basic principles that support adaptive and scalable retrieval functions in large networked environments such as the Web, where information is distributed among dynamic systems. The talk will discuss key challenges facing classic information retrieval models and present a decentralized, organic view of information systems pertaining to search in large scale networks. The presentation will focus on the impact of network structure on search performance and discuss a phenomenon we refer to as the Clustering Paradox, in which the topology of interconnected systems imposes a scalability limit. Our recent experiments involving large scale TREC collections provide evidence on the Clustering Paradox in the IR context. In an increasingly larger, distributed environment, decentralized searches for relevant information can continue to function well only when systems interconnect in certain ways. Relying on partial indexes of distributed systems, some level of network clustering enables very efficient and effective discovery of relevant information in large scale networks. For a given network clustering level, search time is well explained by a poly-logarithmic relation to network size (i.e., the number of distributed systems), indicating a high scalability potential for searching in a continuously growing information space.

Weimao Ke is a Ph.D. candidate at the School of Information and Library Science, University of North Carolina at Chapel Hill. His research is centered around information retrieval (IR), particularly the investigation of intelligent systems that support better connection and interaction between people and information. His recent focus is on decentralized IR functions that can adapt and scale in continuously growing and increasingly interconnected information spaces. His broad interests also include complex networks/systems, text mining, information visualization, bibliometrics, machine learning, multi-agent systems, and the notion of information. Weimao received his Masters degree in Information Science from Indiana University Bloomington. Before the graduate study, he earned his B.E. in chemical engineering from East China University of Science & Technology and worked as information system developer and project manager for six years in the IT industry.