Internet Streaming Media Metadata Interchange
with MPEG-7

Eric Rehm
Thomson multimedia
4 May 2001, Hong Kong


Brief look at Singingfish
Indexing Internet streaming media
Automating metadata delivery and processing
Case Study: Using XSL to transform MSNBC schema to MPEG-7

Wholly-owned subsidiary of Thomson Multimedia
B2B Streaming Media Search Service
Pay per query business model
Over 15 M streams indexed
Live with customers since Jan 2000
InfoSpace: Metacrawler, Dogpile
Inside Internet AG: Swiss-Search, Austria-Search
Involved with MPEG-7 standards development since Sept 1999

Slide 4

Service Model

Indexing Streaming Media

High quality metadata improves relevancy of multimedia search results
Crawl.orwork directly with multimedia Content Producers to acquire quality metadata
Solution: Implement FTP push/pull of metadata
Automated processing upon FTP close
Support bulk or incremental operations: add, update, delete, reset
Future: SOAP or other W3C XML protocol


Content Producer Program Metadata Engine

Development Goals

Single metadata schema interface to a database
Control development costs
Partition engineering and content development
Adapt to any content partner metadata
XML, CSV, Excel, Virage VDF, .
Transform content partner metadata to MPEG-7 via:
Custom applications (CSV, Excel) MPEG-7
Proprietary XML schemas XSL MPEG-7

Case Study

Create XSL transformation
MSNBC "Partner XML Format"
MPEG-7 Description

Experimental Results

XSL Stylesheet: 370 lines of lightly commented code


Basic MPEG-7 Tools
Semantic Encoding of MSNBC Keywords into MPEG-7 Structured Annotation DS (Who, What, Where, When, Why, How)
Encoding Controlled Terms using namespaces
Encoding Streaming Media Validity with the Availability DS
Extending an MPEG-7 DS

MSNBC Video Distribution Entry

MSNBC <article>

<article storyorder="12" pubdate="10/23/2000 8:02:00 AM" source="Today show" topnews="12">
<headline>Peace hopes slip farther</headline>
<description>The slim hopes for peace in the Mideast are rapidly fading, NBC&amp;#146;s Martin Fletcher reports Monday from the outskirts of Jerusalem.</description>
<keywords>Israel, palestinian, Yasser Arafat</keywords>

MPEG-7 link to stream



<headline>Peace hopes slip farther</headline>
<xsl:value-of select="headline"/>

< description>, <keywords>

<xsl:value-of select="description"/>

Enhanced <keywords>

<keywords>Israel, palestinian, Yasser Arafat</keywords>
<Name>Yasser Arafat</Name>

Encoding Controlled Terms Genres are described in one namespace (urn:sf:genre).
MSNBC Genres are described in another namespace (urn:msnbc:category )

Encoding Controlled Terms

<category id="News">

Extending an MPEG-7 DS

Extending an MPEG-7 DS

<complexType name="PublicationType">
<extension base="mpeg7:DSType">
<element name="Publisher"
type="mpeg7:AgentType" minOccurs="0"/>
<element name="PublicationLocation" type="mpeg7:PlaceType" minOccurs="0"/>
<element name="PublicationDate" type="mpeg7:TimeType" minOccurs="0"/>
<element name="Rights" type="mpeg7:RightsType" minOccurs="0"/>

Extending an MPEG-7 DS

<complexType name="UsageInformationType">
<extension base="mpeg7:UsageInformationType">
<element name="Publication type="sf:PublicationType" minOccurs="0"/>

Extending an MPEG-7 DS

<UsageInformation xsi:type="sf:UsageInformationType">
<Publisher xsi:type="mpeg7:OrganizationType">
<NameTerm href=urn:sf:publisher:MSNBC/>


Quality search depends on quality metadata
MPEG-7 standards ease development costs
Controlled vocabularies
MPEG-7 MDS can be used to interoperate
XML Schema allows controlled extensions

Thank you

Optional MPEG-7 Background Slides

MPEG-7 Basics

ISO/IEC 15928 Multimedia Content Description Interface
Comprehensive set of audiovisual description tools.
Enabled by key Internet standards:
W3C: XML, XML Schema
IETF standards: URI, URN, URL for resource naming and location
Harmonized with other emerging metadata standards:
Dublin Core, MPEG-21, NewsML, SMPTE Metadata Dictionary, TV-Anytime, and more.
Text and compressed binary encodings
Both encodings have streaming add, delete, update features for delivery over real-time transports: MPEG-2, MPEG-4, IP, etc.
International Standard in October 2001
Ballot period begins 14 March 2001

Basic elements

Content Management & Description

Content Management & Description (Conceptual aspects)

Navigation and Access

Navigation and Access

Content Organization

User Interaction


XML Schema
Data type extensions
MIME type, ISO country, region, currency codes
ISO Character set codes
Revised time data types to support arbitrary fractional seconds denominator for per-frame positioning
2001-05-01T15:23:46N11F30 (11th frame @ 30 FPS)
Type-centric approach using root abstract types
Control available global elements
Allow extension via name spaces and <extension> mechanism

Basic Derivation of MPEG-7 Types

<complexType name="Mpeg7RootType" abstract="true">
<restriction base="anyType"/>

Creation Description Scheme

<complexType name="CreationType">
<extension base="mpeg7:DSType">
<element name="Title" type="mpeg7:TitleType
<element name="Creator type="mpeg7:CreatorType
minOccurs="0" maxOccurs="unbounded"/>