Call for Participation
The Sixth Russian Information Retrieval Evaluation Seminar
April 2008 - October 2008
Russian Information Retrieval Evaluation Seminar (ROMIP) is an annual
retrieval system evaluation campaign that is in many respects similar
to other world information retrieval forums
such as TREC, CLEF, NTCIR.
The distinctive features of ROMIP are:
- focus on problems of information retrieval in Russian language
- Unique large scale Russian language collections
- Tasks that are not available in other related
The seminar is a perfect opportunity to check your information retrieval and
natural language processing approaches and algorithms. You will be able to
use Russian language collections and industry standard evaluation
methodologies. The leading Russian IR researchers and companies
traditionally participate in the seminar and you will be able objectively
compare your results with theirs.
This year the seminar includes two new unique image retrieval tracks. These
tracks are not language specific and have original collections and tasks.
The final workshop will be held in conjunction
October, 2008, in Dubna, Russia.
Official languages: Russian and English.
- Ad hoc search
- in a collection of Web pages
- in a collection of legal documents
- in a mixed collection
- of legal documents
- of Web sites
- of Web pages
- News clustering
- Query-biased summarization
- Search by sample document or text fragment
- Question answering
- Fact extraction from news
- Image tracks
- Content-based image retrieval
- Near duplicates detection
ROMIP Test Collections
The following test collections are created and used for evaluation:
- Web collection Narod.Ru
- Web collection KM.Ru 2007
- Web collection By.web 2007
- Web collection DMOZ
collection of Legal documents 2007
(31.500 documents from 25 news agencies)
- Image collection
(approx. 20.000+ images, Flickr based)
The text collections are in Russian language.
Image collection is language independent.
The test queries sets for
retrieval tasks are from logs of popular search engines.
Requests for participation: May 12, 2008
Collection and task delivery: May 26, 2008
Submission of results: up to July 15, 2008 (depends on track)
Result of evaluation: August 15, 2008
Paper submission: September 10, 2008
Workshop: October 7-11, 2008
How to participate
Please submit your application to romip[AT]romip.ru and provide the following
- List of tracks you are interested in
- Contacts (organization, email of contact person)
- Name of the system
- Brief description of the system/team to be published on the ROMIP website
- Mikhail Ageev (Moscow State University, Russia)
- Alexander Antonov (Galaktika-Zoom, Moscow, Russia)
- Pavel Braslavski (Yandex, Ekaterinburg, Russia)
- Maxim Gubin (Symantec Corp., USA)
- Boris Dobrov (UIS RUSSIA, Moscow, Russia)
- Mikhail Kostin (Mail.Ru, Moscow, Russia)
- Igor Kuralenok (St. Petersburg State University, Russia)
- Igor Nekrestyanov (St. Petersburg State University, Russia)
- Marina Nekrestyanova (NebuAd, St. Petersburg, Russia)
- Vladimir Pleshko (RCO, Moscow, Russia)
- Ilya Segalovich (Yandex, Moscow, Russia)
- Vlad Shabanov (Vertical Search, Moscow, Russia)
- Natalia Vassilieva (HP Labs, St. Petersburg, Russia)
Web site: http://romip.ru.
Please send your questions to ROMIP
forum or directly to the organizers (romip[AT]romip.ru).