Important Dates
Personal lifelogging is the process of capturing multiple aspects of one's life in digital form and is becoming an increasingly important research topic. Lifelog organisation and retrieval continues to pose significant challenges for the IR community and has recently been the focus of comparative benchmarking exercises at both NTCIR and ImageCLEF. However there has not been much effort on the evaluation of interactive lifelog retrieval systems.

Consequently, the Lifelog Search Challenge workshop at ICMR2018 is the first annual workshop in the series. LSC at ICMR 2018 will be a highly interactive and entertaining workshop modelled on the successful Video Browser Showdown annual competition at the MMM conference. LSC is proposed as a participation workshop, which means that all participants will write and present a paper, as well as taking part in the live interactive search challenge. Consequently, the workshop will have two parts, (1) oral presentations, and (2) the search challenge.

The format of the workshop will be as follows:
  • A conventional morning session with a (local) keynote speaker and oral presentation of research papers, with a paper from each participant.
  • A lunchtime session which is the interactive search engine challenge which will be open for all ICMR delegates to attend. All participating groups will take part in the real-time competition, which will be configured in the manner described below.

The oral presentations (with associated papers) will describe the approaches taken to develop the interactive system for the search challenge. The workshop will have an associated dataset released (see DATA page) based on existing NTCIR-13 test collections. This dataset will consist of four weeks of rich multimodal lifelog data (wearable camera images, biometrics, user activities and locations). Associated with the dataset will be 6 testing and 16 evaluation multimodal topics representing challenging real-world information needs. Details of the dataset will be published as in an appropriate venue in 2018. All participants in the LSC will be invited to submit to a special issue of the MTAP journal in mid-2018.

For the interactive search challenge, each participant will be given a desk with arranged in a semi-circle around a dedicated network switch to avoid any latency issues. Each team will have developed a prototype interactive lifelog search engine for the dataset. At the front of the room is a large screen upon which the host shows each topic and a real-time scoreboard with countdown clock. Each team has a score which is based on the speed at which they find the known item represented by the search topic. When an item is found they submit it to a central server which checks to see if it is correct and a score is allocated to a team based on how long it took them to find the relevant content and the accuracy of submissions. We expect to process 16 topics within the session with each topic requiring 2-3 minutes. The topics will be based on a standard Known-Item approach. Each topic will be represented by an image from the collection along with a related description based on the collection metadata. Rather than simply presenting the topic, however, each topic will be presented dynamically with additional information about the known-item being provided every 15 seconds until the complete topic is displayed. This format of topic display was first used at the VBS2018.