Knowledge Discovery Group

Finished Projects

SeMuDocs - Semantic Annotation, Indexing, and Search of Multimedia Documents

Duration: 2013-2015

Project Members: Jun.-Prof. Dr. Ansgar Scherp, Lydia Weiland

Funding: Ministry for Science, Research, and Arts, Baden-Wuerttemberg, Germany

Description: There has been enormous progress regarding media search of a single media type such as images and videos in the past. In contrast, search for multimedia documents such as PowerPoint presentations, Flash documents, and Adobe's Edge documents (HTML 5) is still very limited or not supported at all by the related work. Multimedia documents are composed of media objects like images, videos, audios, and text objects, which are coherently organized in time, space, and interaction. The information about the structure of multimedia documents, however, is not used for indexing and retrieving the content. This is desirable to improve visibility of the multimedia documents in the repositories and to ease reuse of the multimedia documents and the media objects they contain.
Goal of the proposed research project is to improve the search of multimedia documents. To this end, a formal object model and query language for multimedia documents will be developed. The object model allows for representing the arrangement of the media objects in time, space, and interaction and making it available for retrieval tasks. By this, complex queries against the multimedia document repository can be stated that make use of the relations defined between the media objects. In addition, new methods for the semantic enrichment of multimedia documents will be developed. By semantic enrichment, one can derive, e.g., that a text object is a title of a multimedia document or a caption of an image. In combination with existing text classifiers, it is possible to state queries regarding semantic concepts against the multimedia document repository. For example, one can search for topics in domains like sports and entertainment but also look for entities like places, organizations, and persons.
A prototypical implementation of the multimedia search will be evaluated with participants of different age groups and background (including pupils). It allows for deriving new insights into how multimedia document search is used, which was not possible before. As such, the technical as well as empirical results of the proposed research project will significantly influence the development of future multimedia information systems.

SMILER - Social Mobile Location and Event Finder

Duration: 2013-2014

Project Members: Jun.-Prof. Dr. Ansgar Scherp, Lydia Weiland, Dhaval Ranjane, Julian Abe, Julian Seitner, Linh Hoang, Martina Dukadinova, Ramana Dasari, Sven Rullmann

Industry cooperation: Telegate Media AG, Germany

Description: This project considers the question how a mobile application like mobEx (see project description below) can be designed such that it allows for the collaborative creation, modification, and sharing of places and events on the social web. Particular challenge is here that the mobile application SMILER does not only consider a single data source that is used in the backend system. Rather, a set of different data sources are employed that are curated by different organizations and offering different types of data as well as functionalities. This set of data sources are not assumed to be static. Rather, we also address the challenge what happens when new data sources occur or existing ones are discontinued.

mobEx - Mobile location and event finder

Duration: 2013

Project Members: Jun.-Prof. Dr. Ansgar Scherp, Michael Jess, Christian Bikar, Florian Knip, Bernd Opitz, Bernd Pfister, Timo Sztyler

Industry cooperation: Telegate Media AG, Germany

Description: Existing mobile applications do not allow for efficiently and intuitively exploring for events such as concerts, weekly markets, opening hours, etc. and at the same time explore for places such as sights, restaurants, organizations, and persons. Examples are the Christmas market in Mannheim or the opening hours of the bakery "around the corner", where you want to buy some bread rolls on Sunday. Such information is available on the Web from open APIs like the Telegate AG or directories for events such as Eventful, OpenPOI,, Qype, and Flickr. The challenge addressed in mobEx project is to integrate in real-time the data retrieved from the different sources of social media data and make it available to the mobile user. To this end, we develop an efficient matching approach specialized for dealing with spatio/temporal data in the social media context such as events, persons, organizations, and places and the digital media associated with it.

You can try mobEx on your mobile phone. Simply install mobEx from Google's Playstore.

Linked Networked Graphs - Semantic technologies for the next-generation media management solutions

Duration: 2010-2012

Project Members: Jun.-Prof. Dr. Ansgar Scherp, Raphael Feld, Dr. Thomas Franz, Carsten Saathoff, Simon Schenk

Funding: EXIST research transfer, BMWI

Description: Goal of the project Linked Networked Graphs is the development of an infrastructure for semantic media management. To this end, the different processes for media management such as acquisition, distributed storage and search are supported. In contrast to existing solutions, ontologies and background knowledge in form of Linked Data is used. The EXIST research transfer has the goal to found a start-up company. The start-up company is called Kreuzverweis Solutions GmbH ( The founding members are three former scientific assistants of the university T. Franz, C. Saathoff, and S. Schenk, an economist R. Feld, and Mr. Scherp.



  • Homepage kicked off!