COMM integration
From Semantic Multimedia Wiki
Version: 2010/07/01
Contents |
Introduction
The Multimedia Metadata Ontology (M3O)[1] has been proposed as a revision to the Core Ontology for Multimedia (COMM)[1]. With its introduction the development on COMM ontology has been terminated. Where both ontologies aim at modeling the domain of multimedia annotation, they take distinctive approaches on doing so. COMM lays a strong focus on the MPEG-7 multimedia standard[1], M3O presents a generic scaffold for integrating various media formats and standards and bases on the DOLCE+DnS Ultralight (DUL)[1] foundational ontology and more specifically on the Descriptions and Situations (DnS) pattern [1].
This documents presents the decisions and adjustments made in the alignment of COMM and the M3O.
Overview COMM
The COMM ontology models the domain of multimedia content. It bases strongly on MPEG-7 multimedia standard and uses a pattern-oriented of ontology design. It introduces three annotation pattern, for different facets of multimedia annotation, the decomposition pattern for multimedia decomposition, and the digital data pattern for representing concrete annotations in a digital domain.
COMM structure is as follows:
- comm:core.owl: Defines the core concepts for COMM. Of primary importance are the declaration of the annotation pattern and decomposition pattern, as well as the introduction of comm:nested-data-descriptors, which are part of the digital data pattern.
- comm:localization.owl: Specializes concepts concerning media segmentation. The ontology introduces basic concepts to describe segments and regions in multimedia objects.
- comm:media.owl: Specializes concepts for media object annotation by offering domain specific comm:descriptors, comm:structured-data-parameters and comm:abstract-regions. Works on i.e., comm:media level.
- comm:visual.owl: Specializes concepts for segmentation and annotation of audio-visual multimedia data. Works on comm:image-data level.
- comm:textual.owl: Specializes concepts in relation to textual media data. Works on comm:text-data level.
- comm:datatype.owl: Defines several basic concepts in relation to datatypes, as they might be found in common progaming languages.
General Comments / Legend
- The namespace has been adjusted to http://comm.semantic-multimedia.org/ontology/2010/02/28/.
- Following the notation used with the M3O, all concepts have been camel-cased during the alignment.
- The owl-files contain further detail about the concepts described in this document.
- Inheritance for concepts described in this document has been truncated to the minimum. The following notation has been used for this document.
Namespaces:
comm: http://comm.semanticweb.org/
m3o: http://m3o.semanticMultimedia.org/ontology/2010/02/28/
cal: http://comm.semantic-multimedia.org/ontology/2010/02/28/
dul: http://www.loa-cnr.it/ontologies/DUl.owl#
Abbreviations:
" " = Indendation indicates inheritence. The first level of the inheritence tree always indicates a DUL concept. All concepts that follow are part of M3O or COMM aligned.
"..." = Indicates that more concepts where defined in the original ontology but are not mentioned for reasons of brevity. Refer to actual ontology files for more detailed information.
"(comm:...)" = Indicates that this concept's class name was changed as a result of the alignment. The name in brackets refers to original term of COMM.
Structural Adjustments of COMM
As the the concept of locating segments in multimedia data is less prominent in M3O, we decided to adjust the ontology structure accordingly. The separation between annotation- and decomposition-concepts has been strengthened further. All general concerning the annotation of dul:InformationEntities have been transferred to the COMM_integration#cal:extended.owl extended ontology. Concepts concerning the decomposition of multimedia items have been moved to the COMM_integration#cal:localization.owl localization ontology.
The structure after the integration presents itself as follows:
- cal:extended.owl: Integrates concepts for annotating dul:InformationEntities and introduces basic conecpts for describing structured data annotations.
- cal:localization.owl: Integrates concepts for decomposing dul:InformationObjects and introduces basic concepts for localizing components.
- cal:audioVisual.owl: Integrates concepts for annotating and decomposing audio-visual information objects.
- cal:textual.owl: Integrates concepts for annotating and decomposing textual information objects.
- cal:mediaRealization.owl: Integrates concepts for annotating and decomposing information realizations.
- cal:dataype.owl: Integrates all datatypes of COMM ontology.
Ontologies
cal:datatype.owl
Overview
Origin: comm:dataype.owl, comm:vidual.owl, comm:textual.owl, comm:media.owl
Description: Holds a semantical representation of datatypes, which roughly align to common datatypes used in software engineering and mathematics. All datatypes are specialization to the dul:Region concept. The basis is a set of very basic dataypes, e.g., Boolean, Enumeration, Scalar, Matrix which originate from comm:datatype.owl. All domain specific datatype concepts, formerly part of the domain ontologies of COMM, e.g., COMM_integration#cal:localization.owl, have been integrated into the dataype ontology.
Concepts
dul:Region
cal:AbstractRegion
cal:BasicDuration
cal:MediaDuration
cal:WholeIntervalMediaDuration
cal:BasicTimePoint
cal:KeyMediaTimePoint
cal:MediaTimePoint
cal:Boolean
cal:ParameterTrajectoryEllipseFlag
cal:Enumeration
cal:UniqueIdEncodingEnumeration
cal:RatingSchemeStyleEnumeration
cal:QualityRatingTypeEnumeration
cal:QualityRatingTypeEnumeration
...
cal:RatingSchemeStyleEnumeration
...
cal:UniqueIdEncodingEnumeration
...
cal:Matrix
cal:DoubleMatrix
...
cal:FloatMatrix
...
cal:IntegerMatrix
...
cal:Polygon
cal:RegionBoundary
cal:Rectangle
cal:BoundingBox
cal:Scalar
cal:DoubleScalar
...
cal:FloatScalar
...
cal:IntegerScalar
...
cal:String
cal:AnyUri
cal:Base64Binary
cal:CharacterSetCode
cal:ControlledTermUseType
cal:CountryCode
cal:CurrencyCode
cal:hexBinary
cal:MimeType
cal:Nmtoken
cal:RegionCode
cal:TermUseType
cal:Union
cal:ScalableCodingUnion
cal:FrameStructureUnion
cal:EmphaisUnion
cal:ColorDomainUnion
cal:Vector
cal:DoubleVector
cal:FloatVector
...
cal:IntegerVector
...
cal:extended.owl
Overview
Origin: comm:core.owl
Description: The extended ontology defines basic concepts for multimedia annotation and acts as bases for all dependent domain ontologies (e.g., the COMM_integration#cal:Localization.owl localization ontology This ontology does not comprise any MPEG-7 specific concepts.
Concepts
The concepts of comm:content-annotation, comm:media-annotation and comm:semantic-annotation have been adjusted to match the description and situation pattern implemented in the M3O. The concepts were renamed to match the notation used for the M3O (e.g., cal:InformationObjectAnnotationSituation instead of comm:content-annotation).
dul:Description
m3o:AnnotationPattern
cal:InformationObjectAnnotationPattern (comm:method)
cal:InformationRealizationAnnotationPattern (comm:method)
cal:SemanticAnnotationPattern (comm:method)
dul:Situation
m3o:AnnotationSituation
cal:InformationObjectAnnotationSituation (comm:content-annotation)
cal:InformationRealizationAnnotationSituation (comm:media-annotation)
cal:SemanticAnnotationSituation (comm:semantic-annotation)
dul:Concept
m3o:AnnotatedConcept
cal:AnnotatedInformationObjectConcept (comm:annotated-data-role)
cal:AnnotatedInformationRealizationConcept (comm:annotated-media-role)
dul:Concept
m3o:AnnotationConcept
cal:SemanticLabelConcept (comm:semanticLabel-role)
With M3O giving less focus on the concept of comm:digital-data, the corresponding class was removed in the alignment. The remaining media-specific concepts (i.e. cal:MultimediaRealization, cal:AudioObject) were renamed to highlight their affiliation to dul:InformationObject and dul:InformationRealization concepts. The cal:MultimediaRealization-concept (former comm:media) has been extended by cal:ImageRealization, cal:AudioRealization, cal:VideoRealization and cal:TextRealization for completeness.
dul:InformationObject
cal:MultimediaObject (comm:multimedia-data)
cal:AudioObject (comm:audio-data)
cal:ImageObject (comm:Image-data)
cal:TextObject (comm:text-data)
cal:VideoObject (comm:video-data)
dul:InformationRealization
cal:AudioRealization
cal:ImageRealization
cal:TextRealization
cal:VideoRealization
Integrates the Digital Data Pattern of COMM into M3O. For reference see COMM documentation or the specification within the ontology file. The structure of cal:StructuredDataParameters and cal:StructuredDataRoles have been adjusted to allow for a more convenient use with the m3o:AnnotationPattern. Both concepts are now considered subclass to the m3o:AnnotationConcept, alllowing to use the cal:StructuredDataParameter without the corresponding cal:StructuredDataDescription.
dul:Description
cal:StructuredDataDescription
cal:Descriptor
dul:Parameter
cal:StructuredDataParameter
dul:Role
cal:StructuredDataRole
dul:Concept
cal:AnnotationConcept
cal:StructuredDataParameter
cal:StructuredDataRole
The comm:processing-role (including subclasses) and comm:segment-decomposition were not integrated, as the aspect of segmentation is already sufficiently covered by the M3O Composition Pattern. The comm:structured-data-instantiation was not integrated and should instead be expressed using the information realization pattern.
comm:processing-role
comm:output-role
comm:output-segment-role
comm:input-role
comm:input-segment-role
comm:segment-decompostion
comm:structured-data-instantiation
The following concepts were moved to the COMM_integration#cal:Localization.owl localization ontology to achieve a stronger separation of concerns.
dul:Method
comm:algorithm
comm:segmentation-algorithm
comm:computational-task
comm:mask-role
comm:root-segment-role
cal:Localization.owl
Overview
Origin: comm:core.owl, comm:localization.owl
Description: The localization ontology provides core concepts for specifying segments and regions in multimedia objects.
Concepts
Concepts that are in relation to the segmentation of media data have been introduced from the core ontology.
dul:Role
cal:RootSegmentRole
cal:MaskRole
cal:SpatialMaskRole
cal:TemporalMaskRole
cal:TextualMaskRole
...
dul:Method
cal:Algorithm
cal:SegmentationAlgorithm
dul:Task
cal:ComputationalTask
Only namechanges for:
dul:Description
cal:Descriptor
cal:LocalizationDescriptor
cal:AsciiTextLocatorDescriptor
cal:FigureTrajectoryDescriptor
cal:InterpolationFunctionsDescriptor
cal:KeyTimePointDescriptor
cal:KeyValueDescriptor
cal:MediaTimeDescriptor
cal:ParameterTrajectoryDescriptor
cal:RegionLocatorDescriptor
cal:SpatioTemporalLocatorDescriptor
cal:TemporalInterpolationDescriptor
cal:WholeIntervalDescriptor
dul:Parameter
cal:StructuredDataParameter
cal:LocalizationDescriptorParameter
cal:AsciiTextLocatorDescriptorParameter
cal:FigureTrajectoryDescriptorParameter
cal:KeyTimePointDescriptorParameter
cal:KeyValueDescriptorParameter
cal:MediaTimeDescriptorParameter
cal:RegionLocatorDescritorParameter
cal:WholeIntervalDescriptorParameter
cal:MediaRealization.owl
Overview
Origin: comm:media.owl
Description: The media ontology contains MPEG-7 descriptors for annotating media realizations. It defines concepts for annotation with technical metadata and can be applied to dul:InformationRealizations.
Concepts
dul:Description
cal:Descriptor
cal:MediaDescriptor
cal:MediaFormatDescriptor
cal:MediaInstanceDescriptor
cal:MediaQualityDescriptor
cal:MediaTranscodingHintsDescriptor
cal:NestedMediaDescriptor
cal:AgentDescriptor
cal:AudioChannelsDescriptor
cal:AudioCodingDescriptor
cal:CodingHintsDescriptor
cal:BitrateDescriptor
cal:ColorSamplingComponentDescriptor
cal:ColorSamplingDescriptor
cal:ColorSamplingFieldDescriptor
cal:ColorSamplingLatticeDescriptor
cal:ColorSamplingOffsetDescriptor
cal:ColorSamplingPeriodDescriptor
cal:FrameDescriptor
cal:InlineMediaDescriptor
cal:MediaFormatDescriptor
cal:MediaInstanceDescriptor
cal:MediaLocatorDescriptor
cal:MediaQualityDescriptor
cal:MediaTranscodingHintsDescriptor
cal:MotionHintsDescriptor
cal:MotionRangeDescriptor
cal:PerceptibleDefectsDescriptor
cal:PixelDescriptor
cal:QualityRatingDescriptor
cal:RatingDescriptor
cal:RatingShemeDescriptor
cal:ShapeHintDescriptor
cal:UniqueIdDescriptor
cal:VisualCodingDescriptor
cal:VisualCodingFormatDescriptor
COMM expressed the value range for a specific parameter, e.g., comm:audio-channels-descriptor-parameter, by sub-classing the regions which are applicable values for the parameter, e.g., comm:audio-channels, comm:front-channels, .... As a result those regions are subclass to both dul:Parameter and dul:Region. This violates a best practice of the DUL descriptions and situations pattern.
Through aligning COMM and the M3O we adapt the best practices set elsewhere and express the relation between parameter and region through the "dul:parametrizes only"-relation as proposed in the DnS pattern.
As an example, we might describe audio channel parameters as follows:
comm:audio-channels-descriptor-parameter
comm:audio-channels
comm:front-channels
comm:lfe-channels
comm:number-of-tracks
comm:rear-channels
comm:side-channels
with each subclass of comm:audio-channels-descriptor-parameter also subclass to some comm:abstract-region. Through aligning COMM and the M3O we get:
cal:AudioChannelsDescriptorParameter dul:parametrizes only (cal:AudioChannels or cal:FrontChannels or cal:LfeChannels or cal:NumberOfTracks or cal:RearChannels or cal:SideChannels)
where all parametrized elements have been made subclass to some cal:AbstractRegion. For a stronger separation of concerns, all regions (e.g., cal:AudioChannels) have been transferred to the datatype ontology, where as cal:AudioChannelsDescriptorParameter remains in the media realization ontology.
dul:Parameter
cal:StructuredDataParameter
cal:NestedMediaDescriptorParameter
cal:AudioChannelsDescriptorParameter
cal:AudioCodingDescriptorParameter
cal:BitrateDescriptorParameter
cal:CodingHintsDescriptorParameter
...
cal:MediaDescriptorParameter
cal:MediaFormatDescriptorParameter
cal:MediaTranscodingHintsDescriptorParameter
The following concepts can be sufficiently expressed using the information realization pattern and decomposition pattern of the M3O and have therefor not been included in the alignment.
comm:media-profile
cal:audioVisual.owl
Overview
Origin: comm:visual.owl
Description: The visual ontology contains concepts for describing audio-visual media items. Through the alignment with the M3O, all descriptors are applicable to both dul:InformationObjects and dul:InformationRealizations. The visual ontology defines concepts to locate, describe and technically annotate regions in audio-visual media items and complies with the MPEG-7 multimedia standard. Further information on which aspect of the MPEG-7 standard a concept refers to can be found in the owl file.
Concepts
Concepts were renamed to match the notation introduced with M3O (For example: comm:visual-descriptor-annotation is now cal:VisualDescriptorAnnotationSituation).
dul:Situation
m3o:AnnotationSituation
cal:InformationRealizationAnnotationSituation
cal:VisualDescriptorAnnotationSituation
cal:CameraMotionAnnotationSituation
cal:ColorLayoutAnnotationSituation
cal:ColorStructureAnnotationSituation
...
The following concepts were mapped to annotation pattern of the M3O. For completeness we have introduced subclasses to the m3o:AnnotationPattern concept for every m3o:AnnotationSituation mapped in the previous step. As an example the cal:InformationRealizationAnnotationPattern was introduced to match the cal:InformationRealizationAnnotationSituation.
dul:Description
m3o:AnnotationPattern
cal:InformationRealizationAnnotationPattern
cal:VisualDescriptorAnnotationPattern
cal:CameraMotionAnnotationPattern
cal:ColorLayoutAnnotationPattern
cal:ColorStructureAnnotationPattern
...
The scope for the cal:StillRegionRole dul:classifies-relation has been extended to cover cal:ImageObjects, as well as cal:ImageRealzations (COMM allowed only comm:image-data). The scope for the cal:VideoSegmentRole dul:classifies-relation has been extended to cover cal:VideoObjects, as well as cal:VideoRealzations (COMM allowed only comm:video-data).
dul:Role
cal:StillRegionRole
cal:RootStillRegionRole
cal:RootImageTextRole
cal:ImageTextRole
cal:RootImageTextRole
cal:VideoSegmentRole
cal:RootVideoSegmentRole
cal:RootSegmentRole
cal:RootVideoSegmentRole
cal:RootStillRegionRole
cal:RootImageTextRole
For the following concepts the superclass comm:digital-media has been removed, Instead we will apply the M3O information realization pattern to express concrete data realizations.
dul:Description
cal:Descriptor
cal:NestedVisualDescriptor
cal:CameraMotionSegmentDescriptor
cal:MixtureCameraMotionSegmentDescriptor
cal:NonMixtureCameraMotionSegmentDescriptor
cal:CbacCracCoefficientDescriptor
cal:CbacCracCoefficient-14Descriptor
...
cal:ColorQuantizationComponentDescriptor
cal:ColorQuantizationDescriptor
cal:ColorSpaceDescriptor
cal:ContourShapePeakDescriptor
cal:DominantColorValueDescriptor
cal:FocusOfExpansionDescriptor
cal:FractionalPresenceDescriptor
cal:MixtureAmountOfMotionDescriptor
cal:NonMixtureAmountOfMotionDescriptor
cal:ParametricCoordinateDefinitionDescriptor
cal:SpatialDistributionParametersDescriptor
cal:SpatialLocalizationParametersDescriptor
cal:Vector-16Descriptor
...
cal:TextureBrowsingComponentDescriptor
cal:TrajectoryCoordinateDefinitionDescriptor
cal:TrajectoryCoordinateRepresentationDescriptor
cal:YacCoefficientDescriptor
cal:YacCoefficient-14Descriptor
...
We have removed the comm:structured-data-parameter superclass relation for domain specific parameter values (e.g. comm:color-quantization). The relation has been replaced by "dul:parametrizes only"-relations ( e.g. cal:ColorLayoutDescriptorParameter dul:parametrizes only (cal:CbdcCoefficient or cal:CrdcCoefficient or cal:YdcCoefficient) ). For further explanation see COMM_integration#cal:MediaRealization.owl.
dul:Parameter
cal:StructuredDataParameter
cal:VisualDescriptorParameter
cal:ColorDescriptorParameter
cal:ColorLayoutDescriptorParameter
cal:ColorStructureDescriptorParameter
cal:DominantColorDescriptorParameter
cal:GofGopColorDescriptorParameter
cal:MotionDescriptorParameter
cal:MotionActivityDescriptorParameter
cal:MotionTrajectoryDescriptorParameter
cal:ParametricMotionDescriptorParameter
cal:ShapeDescriptorParameter
cal:ContourShapeDescriptorParameter
cal:RegionShapeDescriptorParameter
cal:Shape-3dDescriptorParameter
cal:TextureBrowsingDescriptorParameter
cal:EdgeHistogramDescriptorParameter
cal:homogenousTextureDescriptorParameter
cal:TextureBrowsingDescriptorParameter
cal:NestedVisualDescriptorParameter
...
cal:textual.owl
Overview
Origin: comm:text.owl
Description: Specifies concepts related to the segmentation of textual media objects.
Concepts
Only namechanging occured for:
dul:Role
m3o:RootSegmentRole
cal:RootTextSegmentRole
cal:RootAsciiTextSegmentRole
dul:Concept
m3o:ComponentConcept
cal:TextSegmentRole
cal:AsciiTextSegmentRole
cal:RootAsciiTextSegmentRole
cal:RootTextSegmentRole
cal:RootAsciiTextSegmentRole
Concepts were renamed to match the notation introduced with M3O (For example: comm:text-segment-Decomposition is now cal:TextDecompositionSituation). Subclasses to m3o:DecompositionPattern were introduced for reasons of completeness.
dul:Situation
m3o:DecompositionSituation
cal:TextDecompositionSituation (comm:TextualSegmentDecomposition)
cal:AsciiTextDecompositionSituation (comm:AsciiTextSegmentDecomposition)
dul:Description
m3o:DecompositionPattern
cal:TextDecompositionPattern
cal:AsciiTextDecompositionPattern
Defining Structured Metadata
The following application example demonstrates the differences between the application of COMM and the M3O. The running example is a color histogram annotation applied to a JPG representation of the photograph of the Yalta Confrence [1].
Color Histogram in COMM
A core concept of COMM is the distinction between media objects, e.g. an image of the Yalta Conference, and the digital file representing them, e.g. a JPG representation of the image described above.
As a result COMM defines three annotation patterns for different aspects of media annotation. For annotating a concrete JPG file, we have to implement the media annotation pattern.
To annotate an image file with COMM we define some dul:Method concept that is satisfied by some comm:annotation-situation. The dul:Method defines a comm:annotated-media-role and some comm:annotation-role. The comm:annotated-media-role classifies the target for the current annotation, specifically image-1 of type comm:media-profile, where comm:media-profile lends from the MPEG-7 standard and expresses the specific composition for this multimedia file.
The comm:annotation-role classifies comm:media-instance-data, a digital data representation of the color histogram for this image.
As COMM pays considerable attention on how multimedia annotation is expressed in a digital file system, the actual information for this color histogram, i.e. the value for the red, green and blue color components, is not directly part of the media annotation pattern. Instead it is parameter to the digital data pattern, which expresses the realization of the color histogram information in a digital domain.
For the digital data pattern we state that the comm:digital-data expresses some comm:color-component-descriptor, which in turn defines some comm:color-component-descriptor-parameter. The actual comm:color-components, one for each color channel, and the comm:number-of-bins-per-component are parametrized by that parameter and are region for the color quantization quality.
Color Histogram in M3O
The M3O specifies the annotation pattern for annotating multimedia items. Following the distinction between dul:InformationObject and dul:InformationRealization, as implemented by the M3O, we consider the color histogram annotation a quality of a concrete image realization, e.g., a JPG representation of a picture of the Yalta conference.
In order to annotate the image realization with its color histogram, we define a m3o:AnnotationPattern that satisfies some m3o:AnnotationSituation. The m3o:AnnotationPattern defines a cal:AnnotatedInformationRealizationConcept that classifies image-1, the target for this specific annotation. It further defines a dmn:ColorHistogramAnnotationConcept that parametrizes a dmn:RGBHistogramRegion, which holds the values for the annotation and is the region for some color composition quality of the image realization.
The annotation itself hasPart some cal:ColorQuantizationComponentDescriptorParameter that parametrizes three cal:ColorComponents, one for each color channel, and one cal:NumberOfBinsPerComponent. All three cal:ColorComponents and the cal:NumberOfBinsPerComponent are part of the dmn:RGBHistogramRegion.
Using this construct we are able to reuse concepts of COMM and the axiomatization expressed in their definition.
COMM-to-M3O-aligned Ontologies
extended.owl [1]
datatpye.owl [2]
localization.owl [3]
audioVisual.owl [4]
mediaRealization.owl [5]
textual.owl [6]
