TR 040 EVALUATION OF IMF FOR BROADCASTERS › docs › techreports › tr040.pdf · TR 040...
Transcript of TR 040 EVALUATION OF IMF FOR BROADCASTERS › docs › techreports › tr040.pdf · TR 040...
TR 040
EVALUATION OF IMF FOR
BROADCASTERS
EBU TECHNICAL REPORT
Geneva
September 2017
This page and others in the document are intentionally left blank to maintain pagination for two sided printing
TR 040 Evaluation of IMF for Broadcasters
3
Executive Summary
Introduction
The Interoperable Master Format (IMF) is an internationally recognised SMPTE standard for the
interchange of file-based, multi-version, finished audio-visual content. This report outlines the
results of investigations carried out by the EBU IMF for Television group1 ("IMF-TV") into the
suitability and potential use cases for IMF in broadcasting workflows. Participation in the EBU-TV
group is open to EBU members and active non-members including vendors. The group has also
worked with the Digital Production Partnership (DPP) in the UK and the North American
Broadcasters Association (NABA) in the USA, where although each group represents the particular
interests of its membership, all are working towards common and more flexible workflows for
versioned content.
IMF and Production
It should be understood IMF is not a format for day to day programme production and post-
production. The Non-Linear Editing systems used during programme production are storytelling
tools that treat each video and audio element as discrete items whose relationship is set by the
storyteller's intent. IMF was originally designed to allow high quality content to be exchanged
between businesses. It requires tools that manipulate finished components of a programme and
skills that are very different to those used in traditional post production.
Current support for IMF comes from larger media organisations who see both business and process
benefits from the standard.
The reason for this interest is the flexible way in which IMF handles content. At the heart of IMF is
the Composition which defines the editorial version of the content by referring to the related
essence (audio, video, subtitles, …) files. IMF requires essence files in a Composition to be stored as
individual components. It is this concept of effectively separating the Composition from the
Essence that makes IMF a very powerful tool for versioning and collaborative content creation.
Analysis
The analysis carried out by IMF-TV Group confirmed that although IMF offers attractive features for
modernising content workflows, its actual day-to-day use is currently limited to a few, but highly
influential parties (OTT providers, Movie Studios, and a few large broadcasters). Currently IMF is
not on the radar of most EBU members.
There are a number of key reasons for this:
There is no immediate need to change current workflows, except where broadcasters are
required to accept or deliver IMF (e.g. international sales).
Currently the use of extensive versioning and high-quality (UHD and HDR) masters is more
applicable to the Movie Studios and international OTT providers than it is for many
(national) broadcasters.
The current IMF dominant codec is JPEG 2000 (J2K) which is not in common use by
broadcasters (except for contribution circuits, where different profiles are used).
There is little expertise on IMF; it is often seen as a 'cinema' or OTT provider's format.
1 See Annex B for the group's main tasks
Evaluation of IMF for Broadcasters TR 040
4
Broadcasters have limited technical resources, which are typically fully utilised by existing
workflow challenges and other high-priority projects.
However, the IMF-TV Group believes IMF workflows will become more and more attractive to an
increasing number of EBU members in the coming years, because:
More OTT parties are appearing on the market, so a common exchange format can reduce
costs and complexity, especially for broadcasters delivering to multiple parties.
The creation of different versions of the same content for online services is increasing for
national broadcasters (e.g. different online versions for mobile watching, in signage
applications in public transport, etc.). This fits the overall trend towards a more
'object-based' production and delivery chain.
Many companies use the Apple ProRes codec for mastering. ProRes is currently in the
process of being added to the IMF family, which will allow IMF to be used with existing
content workflows and libraries.
There is increased understanding of IMF, thanks to various activities (open source
implementations, educational work by the EBU, HPA, the IRT, SMPTE, etc.).
IMF-TV, the DPP and NABA believe IMF offers opportunities to improve existing workflows,
especially when Members are moving to more automated workflows. They see a growing need to
avoid multiple re-encodings, to maximise storage/transport efficiency, to automate audio track
assignment and labelling, and to support versions management.
It is also expected that EBU members will continue to see an increase in the number and amount of
additional services their content needs to provide, including more OTT services, alternate
platforms and the amount and range of access services (such as subtitling).
Conclusion
There is still work to do to bring all the broadcaster business requirements together in such a way
that Members can use IMF without either losing current interoperable workflows or the quality IMF
offers. Members need to understand if and how IMF workflows fit into their current and future
processes
The EBU IMF group should next focus on making sure all main Broadcaster requirements are
supported in IMF Applications and assist Members who want to use the format overcome practical
hurdles (test material, best practice on legacy format support/transformation, overview of product
support, etc.).
Acknowledgements
This report has been prepared by the members of the EBU IMF-TV group. Special thanks go to the
following who have contributed their expertise and time during its preparation;
Bruce Devlin
Dan Tatut
Melanie Matuschak
Pierre-Anthony Lemieux
TR 040 Evaluation of IMF for Broadcasters
5
Contents
Executive Summary .................................................................................... 3
Introduction ..................................................................................................................... 3
IMF and Production ............................................................................................................ 3
Analysis .......................................................................................................................... 3
Conclusion ....................................................................................................................... 4
Acknowledgements ............................................................................................................ 4
1. Introducing IMF ................................................................................ 7
1.1 Overview ................................................................................................................. 7
1.2 IMF for EBU members? ................................................................................................. 7
1.2.1 Classic workflow differences ...................................................................................... 8
1.2.2 Technology enablers ................................................................................................ 8
1.3 IMF concepts ............................................................................................................. 8
2. How IMF works ................................................................................. 9
2.1 The Composition ........................................................................................................ 9
2.2 The Package ........................................................................................................... 10
2.3 Componentised Media ............................................................................................... 11
2.4 Compositions at work ................................................................................................ 12
2.5 Applications ............................................................................................................ 13
3. IMF mastering workflows ................................................................... 14
3.1 Acquisition ............................................................................................................. 14
3.1.1 Air Masters .......................................................................................................... 14
3.1.2 Sub Masters ......................................................................................................... 15
3.2 Versioning and Selling content ..................................................................................... 15
3.3 Archiving of content ................................................................................................. 16
4. A Broadcast Conundrum .................................................................... 17
4.1 Timecode is an illusion .............................................................................................. 17
4.2 File names are a distraction ........................................................................................ 18
4.3 MAM support ........................................................................................................... 18
4.4 IMF for Archives? ...................................................................................................... 18
4.5 Tools support .......................................................................................................... 19
4.6 Access Services ........................................................................................................ 20
4.6.1 Subtitles/Captioning .............................................................................................. 20
4.6.2 Audio Description/Described Video Service .................................................................. 20
4.6.3 Signing Services .................................................................................................... 20
5. Conclusions .................................................................................... 20
5.1 IMF is not (yet) on the radar ........................................................................................ 20
5.2 IMF has strong potential ............................................................................................. 21
5.3 Next Steps.............................................................................................................. 21
Annex A: Abbreviations and Terms used in this report ........................................ 23
Evaluation of IMF for Broadcasters TR 040
6
Annex B: EBU IMF-TV Terms of Reference ....................................................... 25
Annex C: SMPTE ST 2067 standards family ....................................................... 27
Annex D: IMF resources .............................................................................. 29
European Broadcasting Union (EBU) ...................................................................................... 29
Hollywood Professional Association (HPA) .............................................................................. 29
Society of Motion Picture and Television Engineers (SMPTE) ........................................................ 29
Digital Production Partnership (DPP) ..................................................................................... 29
North American Broadcasters Association (NABA) ..................................................................... 29
Annex E: EBU IMF-TV Group Report supplement ................................................ 31
Draft Overview for Broadcast and Online Mastering .................................................................. 31
EBU IMF-TV Group Business Requirements Overview .................................................................. 32
TR 040 Evaluation of IMF for Broadcasters
7
Evaluation of IMF for Broadcasters
EBU Committee First Issued Revised Re-issued
VS, TC 2017
Keywords: Interoperable Master Format, IMF, versioning, mastering, MXF, workflows
1. Introducing IMF
1.1 Overview
The Interoperable Master Format (IMF) is an international recognised SMPTE standard for the
interchange of file-based, multi-version, finished audio-visual content.
It has been said that IMF is a logical evolution of the AMWA AS-02 format. IMF has been designed to
be a flexible and extendable framework for fully file-based systems using established and mature
techniques from the Digital Cinema Package (DCP), Material eXchange Format (MXF) and Extensible
Mark-up Language (XML).
More accurately IMF is a family of SMPTE documents that describe a Core framework and a group of
incremental constraints or "Applications" that are specific to particular use cases. See Annex C for
the current overview of the SMPTE ST 2067 standards family.
A typical IMF Application will specify video codecs and image characteristics plus any additional
(optional) descriptive and technical metadata required. It includes details of the tracks containing
MXF wrapped video and audio essence and any time or content dependent data (e.g. timed text
subtitles), basic descriptive metadata, complex playlists, and delivery or output options.
IMF provides the foundations for a business oriented workflow enabling multiple content versions to
be created from a common set of audio-visual components (audio, video and timed text). Each of
these content versions can be transformed into multiple deliverables, tailored for different target
platforms and audiences. IMF facilitates the distribution of unique versions of programmes between
content owners, service providers and distributors.
1.2 IMF for EBU members?
The initial push for IMF did not come from the broadcast industry. It came from US Studios and OTT
parties that have strong business requirements to harmonise multi-version content exchange. The
vision was to limit duplication of effort and storage for internationally distributed content where
IMF would serve as the master format for the distribution of content versions between businesses.
Evaluation of IMF for Broadcasters TR 040
8
1.2.1 Classic workflow differences
In very broad terms, it could be said that a traditional Studio workflow is:
SINGLE SOURCE to MULTI-TARGET
A single editorial master set
Multiple edits (Cinema, Domestic, International, Airlines…)
Multiple languages
Multiple technical (SD, HD, etc…)
Similarly a traditional broadcast workflow can be said to be:
MULTI-SOURCE to SINGLE TARGET
Multiple masters, one for each of the genre types (news, sport, drama, etc.)
Single version outputs (primarily TV and local OTT)
Studios prefer to have a unified format, capable of carrying a potentially unlimited number of
elements in order to serve the requirements of the various targets, but also flexible enough to
continue to evolve to accommodate new features. Broadcasters on the other hand, tend to specify
the delivery format according to their infrastructure and therefore do not easily benefit of an
all-capable format.
1.2.2 Technology enablers
The media industry, as a whole, is making the transition towards completely file-based workflows
and formats are becoming more and more flexible, allowing content to be 'rendered' on-demand
into different versions by sending collections of component parts of the content instead of creating
multiple finished and flattened files.
This trend is related to the emergence of powerful production and consumer rendering devices that
are more and more heterogenic. They can adapt content to different screen sizes, resolutions,
technical capabilities, dynamic ranges, audio arrangements, etc. They are also able to adapt
content to enable personal preferences to be applied. Examples include subtitles which can be sent
as text instead of images, allowing font, font-size, colour, etc. to be adapted at the receiving end
and object-based audio.
IMF provides a solution for the component-based content versioning. It can combine audio-visual
and data objects into different content versions using XML instruction sets (CPLs) without the need
for continually duplicating content.
It would therefore seem natural for broadcasters to prepare for a completely componentised
content chain instead of continually duplicating content as flattened files. This is an important step
beyond the current situation, as 'going file-based' for many broadcasters until now has simply
meant replacing a traditional linear tape workflow with a linear file workflow.
1.3 IMF concepts
IMF is not a single file; it is a collection of files orchestrated by a Composition.
The Composition defines a specific version of a title. IMF requires the essence needed for a
Composition to be stored as individual “Track Files” which only have to come together at time of
processing. The components of a Composition can be bundled for transport using Interoperable
Master Packages (IMP).
TR 040 Evaluation of IMF for Broadcasters
9
This arrangement allows for a high level of automation and increases content reuse (and thus
minimizes storage and multiple redundant QC passes). A new version only requires additional
components when new essence is needed (e.g. a scene is replaced with a different version to
accommodate a different target audience); unchanged parts of the master do not have to be
duplicated.
Other features supported by IMF are:
content with multiple audio, video and data components
Access Services provisioning in multiple languages
descriptive and dynamic metadata, used by or synchronized with the essence,
creation of multiple distribution formats from the same editorial version
delivery of content in phases, without having to resend already available material
identifier-rich metadata for “robust” automation and auditing
support for high-quality image and audio essence
The IMF group identified three main use cases where such features may be attractive:
Outgoing – content for sale or distribution to others where there is more than one version
Incoming – content bought from distributors or other broadcasters
Archiving – content with multiple versions that require archiving
Increasingly, EBU members make some of their content available to other distributors and alternate
platforms (OTT, package media and streaming services) where alternate editorial versions are
needed. Similarly, EBU broadcasters receiving content from others may need to re-package it for
their linear and non-linear services.
Although the concepts of IMF are now better understood and can be exploited technically,
operationally, it is far more difficult to appreciate IMF’s potential and to understand what changes
are required to move from a traditional flattened “linear” master file and versioning workflow to a
componentised "unflattened" one.
Understanding that IMF exploits the relationship between different content elements required to
make different versions is as large a leap as the move from simple standalone linear editing to fully
networked, shared resource, non-linear post production!
2. How IMF works
A traditional versioning process will create a full-length file for each version of a programme. Much
of the content will be identical in each of these versions. This is an unnecessary and inefficient use
of local or cloud storage and requires excessive and unnecessary QC, file transfer, compliance
checks and repeat processing.
IMF eliminates the need for multiple copies of content by separating it into specific Components
which can be combined to produce different versions.
2.1 The Composition
Each version in IMF is defined by a Composition Play List (CPL). This is an XML file that describes
the make-up of the version and “calls” all the associated essence components (called Track Files)
that are needed to compile it into a programme.
Evaluation of IMF for Broadcasters TR 040
10
This allows multiple versions of the same content, each embodied by a CPL, to reuse share essence
components. In many workflows, multiple child versions (called VF compositions) are generated
from an original version (called OV composition).
A CPL defines the playback timeline for a Composition. It also includes or calls any
metadata applicable to the Composition.
Each Version of the content has a unique CPL which combines the essence components and
data required to complete the version.
A Composition is the combination of a CPL and all the Track Files it references. A Track File is an
MXF file that contains a single essence track such as:
◦ Video essence: currently there are applications for J2K and MPEG-4 Studio Profile.
Potential applications for ProRes and H.264 are being discussed.
◦ Audio essence: 24-bit baseband PCM
◦ Timed text (subtitles and captions): IMSC1 profile of the Timed Text Markup Language
(TTML)
◦ Alternate essence for Audio Description and Signing
◦ Dynamic metadata tracks (metadata that changes during the version)
2.2 The Package
An IMF package is usually referred to as an IMP:
An IMP contains one Packing List (PKL) which is an XML containing a list of all the files
(assets) that belong to the IMP.
Figure 1: Example of an IMP with its components
TR 040 Evaluation of IMF for Broadcasters
11
An IMP can be Complete or Partial:
A Complete IMP contains the complete set of assets for one or more Compositions.
A Partial IMP does not contain the complete set of assets for one or more Compositions.
In addition to the packing list there should also be some or all of the following:
CPL file(s) (XML)
Track Files
Sidecar files (QC reports, production metadata)
2.3 Componentised Media
Components make Compositions!
IMF uses two file formats for media essence assets and descriptive text:
1. MXF for essence data
2. XML for human-readable for descriptive information (metadata)
Figure 2: With IMF, MXF-wrapping is used for essence data
Essence data (video, audio data, subtitles and dynamic composition-specific metadata) are
wrapped into individual MXF data track-files which can be addressed by one or more compositions.
Evaluation of IMF for Broadcasters TR 040
12
Rather than modifying the essence to create different versions, IMF uses an XML instruction set to
combine and order the essence and data needed for each version.
To better understand how this works, the relationship between the components of an IMF package
needs to be examined; it is this relationship that makes Compositions work.
2.4 Compositions at work
In a traditional workflow, every one of the different versions would have been individually created
and stored.
In an IMF workflow a different version is created by creating a new CPL to combine the required
Component parts into a Composition that can be rendered into the required editorial and technical
version.
Figure 3 shows an English Version CPL. It defines a Main Programme made in English with Stereo
and 5.1 Audio and Subtitles.
Figure 3: English master with stereo, 5.1 and Dolby audio metadata
To make a French version of this programme, a new CPL is used to replace the English audio,
subtitles and various video elements that contained English burnt-in text:
TR 040 Evaluation of IMF for Broadcasters
13
Figure 4: French master with French audio, subtitles and video replacements
(only where relevant)
Another example would be to create a 'clean version' (no text or dialogue) using yet another CPL.
Figure 5 illustrates this example.
Figure 5: English master with stereo, 5.1 and Dolby audio metadata
2.5 Applications
IMF essence is defined in an Application which specifies the allowed codec types, frame rates, and
resolutions. The currently published Applications are:
IMF Application 2 (SMPTE ST 2067-20) supports Standard Definition and High Definition
Evaluation of IMF for Broadcasters TR 040
14
(1920 x 1080) SDR at up to 60 fps using JPEG 2000 image coding.
IMF Application 2E (SMPTE ST 2067-21) extends Application 2 to include support for HDR
images up to 4096 x 2160.
IMF Application 3 (SMPTE ST 2067-30) is for essence encoded in the MPEG-4 Visual Simple
Studio Profile2 up to 4096 x 2160, 12-bit 4:4:4.
IMF Application 4 (SMPTE ST 2067-40) is a Cinema Mezzanine specification for essence
encoded in JPEG 2000 up to 8192 x 6224, 16-bit.
The structure of IMF allows additional Applications to be added. These can be based on the
particular business requirements of a single organisation, groups with a common interest, or the
industry at large. This assumes the proposers have the resources to support the process of creating
and maintaining such a Specification. Currently Applications for both ProRes and H.264 are being
considered.
3. IMF mastering workflows
The following gives example IMF workflows for Incoming (Acquisitions), Outgoing (Sales) and a
potential workflow for an Archiving process.
3.1 Acquisition
Virtually all EBU members buy content from international distributors. There are two potential
routes options for Acquired content – Air Master and Sub Master.
3.1.1 Air Masters
A very common method for buying content is to simply acquire an Air Master which is pre-prepared
and “ready to go”. Providing the required Air Master is available in an internationally recognised
format (e.g. AS-10, AS-11, H.264 or MPEG-2), by using IMF there is the potential for cost savings
and guaranteed high quality and reliability of the delivered file.
Figure 6: IMF Air Master workflow example
In the above scenario, the broadcaster orders two versions of the content – BUT only needs receive
the content once, plus the instructions for creating the two desired copies from it (using IMF CPL
and OPL3 ). The play-out file is formatted for the broadcaster's play-out infrastructure (codec,
2 The Sony HDCamSR profile. 3 Currently automated transcoding using OPLs is not possible unless profiles for specific devices are assumed or there is an
TR 040 Evaluation of IMF for Broadcasters
15
frame size, frame rate, audio options, subtitles, etc.) and the on-demand file is formatted for the
broadcaster OTT services. The parameters of the delivered files are "fixed” at the time of delivery.
3.1.2 Sub Masters
Broadcasters can also acquire content as a Sub Master which is in a format that can be post
processed in order to comply with local territorial language, editorial and technical requirements.
This is not a simple operation for the distributors unless the broadcaster can receive and process
the available options from the movie studio. There is always significant extra cost incurred, unless
an internationally agreed sub master format is used and this is a scenario where IMF becomes
valuable.
Figure 7: IMF Sub Master workflow example
Quality can be managed during post production and processing costs kept to a minimum. Assuming
the sub master content is at a higher quality than the buyer's current requirements, it can be used
to generate higher quality versions when needed, e.g. in future after upgrading from HD to UHD
and HDR or from stereo to surround etc.
3.2 Versioning and Selling content
Broadcasters who sell content internationally need to consider versioning and mastering, as well as
international exchange. Sales versions of programmes can be both editorially and technically
different to the broadcaster’s Air Master. The broadcaster or content owner needs to hold many
variants of the same core content or keep going back to the original material or production for
additional material when orders arrive
In some cases, the cost of providing multiple formats, specifications and standards can be more
than the value of the original content. It can also easily be beyond the capability of the content
owner, especially in the case of smaller independent production companies that would require the
use of expensive and repetitive versioning.
Due to limitations in many broadcasters' infrastructure (file size, data rate, image resolution…), it
may not be possible to create an Air Master of a suitable quality for sales and international
exchange.
A high quality sub master with the capability to automate the versioning process is an attractive
solution. It allows:
agreed AND certificated specification process (e.g. via an AMWA UK DPP AS-11 HD V1.1 certificated device). Repeatable
methodologies and parameters to drive codecs have not been yet been standardized/harmonized.
Evaluation of IMF for Broadcasters TR 040
16
Production of content at a quality high enough to satisfy all outputs
On-demand creation of versions
Additional material to be held within the same wrapper (clean backgrounds, etc.)
Figure 8: IMF Versioning & Sales example workflow
Here, the programme maker produces an IMF with all the options required for the commission and
any known sales. The broadcaster can make the Air Master version from the delivered IMF and the
distributors can make the required sales versions. Should more material be required for future sales
or broadcaster deliverables, the content owner need only supply the deltas.
3.3 Archiving of content
Archiving of master programme essence, additional content and associated metadata is a key
operation for any broadcaster. A format that allows storing multiple versions of the same content
and related metadata in an efficient way can be an attractive proposition. However, this does not
mean IMF must sit in the archive.
Figure 9: IMF Archiving example workflow
The key feature of IMF for Archives is the way it handles componentised content. Television
programmes “grow” as they age! Regulation changes add Access Service options that must be
TR 040 Evaluation of IMF for Broadcasters
17
applied to repeat transmissions. Circumstances, changes in the law and international events
sometimes mean further editing must be carried out before repeat transmission. This leads to
multiple flattened versions being stored where the majority of the content is repeated across all
versions. Eliminating redundancy was one of the primary reasons IMF was developed.
In the example in Figure 9, additional material and updated versions are combined into an IMF
Archive package that is sent to an archive.
4. A Broadcast Conundrum
The following points may especially be relevant for broadcasters to consider.
4.1 Timecode is an illusion
Timecode is a historic artefact4 of existing linear workflows. It can introduce conflicts that produce
errors and complexities that inhibit efficient automation.
IMF does not use timecode in the same way a traditional linear broadcast does. In IMF the duration
of the essence and any positional offsets are measured and expressed as Edit Units. An Edit Unit is
the smallest unit of time that can be used to measure an asset. A more useful unit however is the
Edit Rate, which is the inverse of the Edit Unit.
The Edit Rate for Video would normally be the frame rate.
The Edit Rate for Audio would normally be the sample rate.
Edit Rates and Edit Units can be defined independently for a Composition and each Resource. A
Composition Edit Rate will always be the video frame rate of the primary video content.
An important fact to remember is that ALL tracks in a Composition must be exactly the same
duration!
IMF defines the duration of a Sequence as “the sum of the duration of its Resources and [it] shall be
an integer number of Composition Edit Units”.
Figure 10: Illustration of the fact that in IMF all tracks in a
composition (e.g. audio and video) must be of exactly the same duration.
4 It should be noted SMPTE is reviewing timecode (as of September 2017).
Evaluation of IMF for Broadcasters TR 040
18
4.2 File names are a distraction
Filenames are also an historic artefact of existing workflows. They easily are wrong, forever
changing, mistyped or simply misleading. They frequently introduce conflicts that produce errors
and complexities that inhibit efficient workflow automation.
In IMF there is no need to know what the name of a file in an IMP is. IMF uses a Universal Unique
Identifier (UUID) for each Asset. To map the IDs to files, an abstraction layer called the ASSET MAP
is used. When delivered there must also be an Asset Map which contains all the asset identifiers and
which specifies the locations of the required files. Files can be on a local file system or they may
be accessed from HTTP/HTTPS addresses.
An IMP can contain any combination of assets. However, it does not need to contain all the assets
required for a composition. Also, an asset can be in multiple IMPs.
Figure 11: Example of the use of Unique Identifiers in IMF.
4.3 MAM support
IMF is a living format that can and will evolve over time to meet current and future needs of
broadcasters.
As the volume of IMF packages exchanged grows, the need for systems that can import and manage
IMF, understand its structure, have the ability to retain the relationships between the components
and export the content as either Air Masters or intermediate IMF packages, becomes critical. To get
the real benefits of scale, it is important that Media Asset Management (MAM) systems support IMF.
This currently is virtually not the case.
4.4 IMF for Archives?
Most Archives already have systems to handle media and data. A main requirement for efficient
archives with multiple versions will be object-based storage. This does not mean archives have to
store “IMF” itself. Specifically, an archive can adopt an object model compatible with that of IMF,
but store and represent content internally as it wishes and use IMF strictly for interchange with
other systems.
TR 040 Evaluation of IMF for Broadcasters
19
As has been explained, IMF is a set of standards that define an interoperability format. The IMP
(Interoperable Master Package) is the actual incarnation of the format as a structured package.
Storing an IMP might be a solution for certain Archives5, but it would need a mechanism to keep
track of the compositions and asset relationships it contains at a higher level (a MAM).
One thing is clear: transcoding archive content to benefit from IMF is a nonsense. Most broadcasters
have large archives containing multiple legacy codec assets that were made and stored using
multiple legacy production workflows. The conversion of these assets to a single codec would not
just be time consuming; it would inevitably and unnecessarily degrade the quality of content and
use far more storage than is actually needed.
With a (currently imaginary) codec-agnostic Application, it would be possible to transition to an
IMF-based workflow over time without damaging the content quality. Such an Application would not
break the ethos of the interoperability of the format6. Archive content will always be processed
before export unless the files just happen to be in a broadcasters “current” codec (which will
change several times in the lifetime of many assets)! Even then, hybrid content workflows that
process IMF and non-IMF assets will be needed for a considerable period. Further work would be
needed to minimise this workflow overhead.
4.5 Tools support
Firstly it must be understood that versioning is not editing. Non-Linear Editing systems are
storytelling tools that treat each shot and audio element as discrete items whose relationship is set
by the storyteller's intent. These traditional editing tools were adequate when a versioning
workflow created a separate and complete asset for each version.
Mastering and Versioning, though, are craft skills that are very different to Editing. Mastering and
Versioning in an IMF context require support for:
Manipulation of finished and complete segments of essence;
Creating and editing metadata related to the assets and version(s);
Creating and maintaining the relationship between the assets in an IMP;
Identifying and reporting the presence or absence of assets against metadata;
Creating the required CPL(s) for each version in an IMP from a timeline presentation;
Creating and/or importing output profiles as required for each version;
Adding and combining additional material or security (e.g. watermarking, etc.);
Playback of versions form created/imported CPLs;
Creating and applying traceable asset names.
The tools required will need to be capable of managing media and metadata components and the
skills required will require a deep understanding of componentised workflow management and
multiple output content creation.
5 Especially as complete IMPs (i.e. where all the assets required by compositions are present in the package). 6 IMF only supports intra-frame codecs currently, so this may not work for all archived content.
Evaluation of IMF for Broadcasters TR 040
20
4.6 Access Services
4.6.1 Subtitles/Captioning
For subtitle/captioning, IMF supports the IMSC 1 (Internet Media Subtitles and Captions 1.0), which
is a profile of the W3C TTML timed text format. This provides a good international harmonization
point for subtitling, even though it currently does not support some of the Japanese/Asian language
requirements.
Tests so far have demonstrated IMSC 1 implementations seem to be at a relatively early stage of
maturity; the interoperability of IMSC 1 implementations is expected to improve over time. To help
with this process, a test suite7 and a complete open source implementation8 are available.
The EBU-TT family is based on the same W3C TTML specification. EBU-TT-D can be seen as a subset
of IMSC 1.
For broadcasters wanting to use IMF, converting legacy subtitling formats (e.g. EBU STL) into IMSC 1
and/or carriage of the legacy formats may be required.
4.6.2 Audio Description/Described Video Service
Audio Description (AD) files can simply be used as additional fully rendered audio channels in a
track or set of tracks. However, work is still being carried out to correctly identify AD assets and
further work is needed to enable AD to incorporate clean description audio and a control track9.
4.6.3 Signing Services
Currently there is no special support for signing services in IMF. Of course, it would be possible to
reference a fully rendered signed version of an asset as a virtual track but this seems a
cumbersome method and goes against the ethos of IMF as it would create an additional asset
containing repeated material.
5. Conclusions
5.1 IMF is not (yet) on the radar
Currently IMF is not on the radar of most EBU members. There are a number of key reasons for this:
There is no immediate need to change current workflows, except where broadcasters are
required to accept or deliver IMF (e.g. international sales).
Currently the use of extensive versioning and high-quality (UHD and HDR) masters has not
yet reached many (national) broadcasters.
The current IMF dominant codec is JPEG 2000 (J2K) which is not in common use by
broadcasters (except for contribution circuits, where different profiles are used).
Broadcasters have limited technical resources, which are typically fully utilised by existing
workflow challenges and other high-priority projects.
There is little expertise on IMF; it is often seen as a 'cinema' or OTT provider's format.
IMF may also suffer from the misconception that it's the new “MXF”.
7 https://github.com/w3c/imsc-tests 8 https://github.com/sandflow/imscJS 9 As described in e.g. BBC Research White Paper WHP 198.
TR 040 Evaluation of IMF for Broadcasters
21
5.2 IMF has strong potential
It can be said then IMF starts where MXF stops or becomes inefficient. It is a toolkit or an
ecosystem that can enable a new broadcaster paradigm for interoperability between systems and
business entities. For example: almost every broadcaster uses MXF with different codecs, audio
track layouts and metadata combinations. When content is exchanged, IMF can be used to
eliminate many of the interoperability problems.
Efficient IMF workflows are a combination of changing the way we think about versions of titles as
well as changing what we store and deliver.
There is reason to believe that IMF will become more and more attractive to an increasing number
of EBU members in the coming years, because:
More OTT parties are appearing on the market, so a common exchange format can reduce
costs and complexity, especially for broadcasters delivering to multiple parties.
The creation of different versions of the same content for online services is increasing for
national broadcasters (e.g. different online versions for mobile watching, in signage
applications in public transport, etc.). This fits the overall trend towards a more
'object-based' production and delivery chain.
Many companies use the Apple ProRes codec for mastering. ProRes is currently in the
process of being added to the IMF family, which will allow IMF to be used with existing
content workflows and libraries.
There is increased understanding of IMF, thanks to various activities (open source
implementations, educational work by the EBU, HPA, the IRT, SMPTE, etc.).
It is not yet clear if IMF and IMF workflows will benefit all EBU members, but what is clear is that
IMF is a powerful format that brings object-based flexibility to media workflows. Because the
format is open to adaption and revisions, it has the potential to meet broadcaster needs both now
and in the future. As broadcasters diversify their delivery options by accessing platforms that
require different editorial (and technical) versions, a single version of a programme is no longer
good enough.
What is also clear is that traditional standards by themselves are not precise enough to guarantee
compliance with the programme delivery options of broadcasters (which basically are now
collections of standards). A combination of standards and specifications, which constrain existing
standards for business needs, has the potential to provide a solution that satisfies the business
requirements of broadcasters. IMF has the potential to offer Applications as Specifications that are
based on the Core Standards of SMPTE and include relevant recommendations from other
organisations (EBU, AMWA etc.).
5.3 Next Steps
The EBU IMF group should next focus on making sure all main broadcaster requirements are
supported in IMF Applications and assist Members who want to use the format overcome practical
hurdles.
There is still work to do to bring all the broadcaster business requirements (see Annex E) together
in such a way that Members can use IMF without either losing current interoperable workflows or
the quality that IMF offers.
The following tasks have been identified as priorities:
Develop Specifications that meet the current needs of Members and programme makers
Evaluation of IMF for Broadcasters TR 040
22
(e.g. with respect to codecs and bitrates).
Work with the HPA IMF User Group to incorporate additional broadcaster requirements,
especially around the future of the Composition Sidecar (which is a way to associate
external non-IMF files with compositions).
Contribute requirements for international broadcaster Air Master OPLs
Work with SMPTE, the DPP and the HPA IMF User Group to integrate https://EBU.IO/QC
Items and Templates into the core of IMF
Help EBU members who want to use IMF overcome practical hurdles (test material, best
practice on legacy format support/transformation, overview of product support, etc.)
TR 040 Evaluation of IMF for Broadcasters
23
Annex A: Abbreviations and Terms used in this report
AM Asset Map
App Application (IMF)
CPL Composition Playlist
IMF Interoperable Master Format
IMP Interoperable Master Package
OPL Output Profile List
PKL Packing List
STL EBU Subtitling data exchange format
Track File Single Essence OP1A MXF file
TTML Timed Text Mark-up Language
URI Uniform Resource Identifier
UUID Universally Unique Identifier
Air Master File used for direct play-out in traditional broadcast infrastructure
Library Master High or higher quality master for versioning
Evaluation of IMF for Broadcasters TR 040
24
TR 040 Evaluation of IMF for Broadcasters
25
Annex B: EBU IMF-TV Terms of Reference
The EBU IMF group's Terms of Reference at its launch consisted of the following four main tasks:
To investigate the application of IMF in television, cross media and mastering workflows;
To understand and disseminate to EBU members the application, advantages and challenges
of IMF;
If proven to be of advantage and needed, to produce the business requirements for:
◦ An international broadcast master exchange application;
◦ A broadcast archive master exchange application;
◦ A flexible and directly editable broadcast wrapper application (allowing files to “grow” or
change as needed without the need for a new version);
◦ The CPL/OPL framework needed by broadcasters;
◦ Metadata subsets for archiving, production and international exchange.
EBU IMF to develop a compliance testing mechanism.
Evaluation of IMF for Broadcasters TR 040
26
TR 040 Evaluation of IMF for Broadcasters
27
Annex C: SMPTE ST 2067 standards family
Core and common documents
ST 2067-2:2016 Core Constraints
ST 2067-3:2016 Composition Playlist
ST 2067-5:2013
ST 2067-5:2013 Am1:2016
Essence Component
ST 2067-8:2013 Common Audio Labels
ST 2067-100:2014 Output Profile List
ST 2067-101:2014 Output Profile List – Common Image Definitions and Macros
ST 2067-102:2014 Output Profile List – Common Image Pixel Color Schemes
ST 2067-103:2014 Output Profile List – Common Audio Definition and Macros
Applications
ST 2067-20:2016 Application #2 Studio Master (J2K @ HD + SDR)
ST 2067-21:2016 Application #2E Studio Master – Extended (J2K @ 4K + HDR)
ST 2067-30:2013 Application #3 MPEG-4 Studio Profile
ST 2067-40:2016 Application #4 Cinema Mezzanine (J2K @ 8k + XYZ)
Evaluation of IMF for Broadcasters TR 040
28
TR 040 Evaluation of IMF for Broadcasters
29
Annex D: IMF resources
European Broadcasting Union (EBU)
Interoperable Master Format (IMF) Group
https://tech.ebu.ch/imf
The group was set up by the EBU Technical Committee in Q1 2016 to investigate the application of IMF in
television, cross media and mastering workflows. Participation in the group is open to EBU members and
relevant non-members. The main condition for participating is taking active part in the work.
Hollywood Professional Association (HPA)
IMF User Group
https://imfug.com/
The IMF User Group brings together content owners, service providers, retailers and equipment/software
vendors to enhance and promote the use of IMF globally, across domains of applications. The group discusses
technical operational issues that arise in practical implementation, conducts interoperability testing,
develops best practices, and seeks to broaden the awareness of IMF.
Society of Motion Picture and Television Engineers (SMPTE)
TC-35PM Media Packaging and Interchange
https://kws.smpte.org/higherlogic/ws/groups/35pm
The work on the SMPTE ST 2067 suite of documents continues and SMPTE encourages commenters who are
interested and wish to contribute to the work, to join the relevant SMPTE working group and provide
comments
Digital Production Partnership (DPP)
IMF Group
https://www.digitalproductionpartnership.co.uk/news/dpp-smpte-partnership-will-fast-track-imf-f
or-broadcast/
The DPP and SMPTE has announced a partnership to develop a new SMPTE Specifications process with the
intent of delivering a constrained version of an IMF Application for Broadcast and Online. Interested parties
are encouraged to join the DPP to contribute to the process
North American Broadcasters Association (NABA)
Library Mastering Group
http://www.nabanet.com/nabaweb/committees/dpp.asp
NABA and DPP members are currently working on specifications for a High Definition Library Master Format
and an Ultra High Definition Library Master format. These library master specifications will cover the
workflows for organizations wishing to receive a single master from which all other required versions are
produced, such as Air Master, VOD, Netflix, Apple iTunes, Hulu and other O.T.T. Versions. These will be
based on SMPTE IMF Standards Application 2 for High Definition and Application 2e for UHD.
Evaluation of IMF for Broadcasters TR 040
30
TR 040 Evaluation of IMF for Broadcasters
31
Annex E: EBU IMF-TV Group Report supplement
SMPTE ST 2067 is a living suite of documents. SMPTE’s TC-35PM (Media Packaging and Interchange)
and the HPA’s IMF User Group continue to work on all aspects of the Standards while other groups
such as the EBU’s IMF-TV group, the DPP IMF group and the NABA Library Master Group work on
particular use case options based on Broadcast and Online mastering and versioning.
The EBU IMF Report Supplement allows snapshot updates of work being carried out to encourage
Members to participate and shape the future of this new multi-disciplinary paradigm.
Draft Overview for Broadcast and Online Mastering
Work has been carried out and the EBU and DPP IMF groups have identified many common Business
Requirements that have been combined into a proposal for one or more IMF Specifications (IMF
Application Constraints).
Video Formats
Material acquired, post-produced and delivered as:
3840 x 2160 or 1920 x 1080 pixels in an aspect ratio of 16:9.
25 or 50 or 100 frames per second
Progressive scan only.
Colour sub-sample ratio 4:2:0 or 4:2:2 or 4:4:4.
10- or 12-bit.
Codec Data Rates
Minimum data-rates for a versioning master based on acceptable Video Formats.
ProRes ITU-T H.264 J2K
1080p/25 190 Mbit/s 200 Mbit/s 180 Mbit/s
1080p/50 370 Mbit/s 400 Mbit/s 350 Mbit/s
2160p/25 740 Mbit/s 720 Mbit/s 650 Mbit/s
2160p/50 1400 Mbit/s 1400 Mbit/s 1200 Mbit/s
Image Parameters
Image parameters are based on SMPTE ST 2067-20 and ST 2067-21. Including HDR (PQ and HLG).
Audio Requirements
Audio tracks should be encoded as PCM with a sample rate of 48 kHz or 96 kHz at a bit depth of
24 bit/sample
Audio Channel Labels
Audio channels shall contain audio channel labelling as defined in SMPTE ST 377. Each Track shall
be carried in its own MXF Track File, and shall be correctly referenced in the IMF structural
components.
Work is currently underway in SMPTE 35PM to define values and associated ULs for MCA
Audio Content Kind and MCA Audio Element Kind. This will be incorporated into the
Evaluation of IMF for Broadcasters TR 040
32
Broadcast and Online proposals. Persons who wish to contribute to the SMPTE draft
documents are encouraged to join the SMPTE working group and provide comments.
Production Metadata
Production editorial and technical Metadata (including surround sound SMPTE RDD 6 Metadata) is to
be carried in a separate XML file. This Metadata shall be associated with the CPL using the SMPTE
Draft Sidecar file mechanism. The required Metadata shall appear in the SidecarAssetList. Extra
Metadata associated with the composition may be included in the SidecarAssetList.
Work is currently underway in SMPTE 35PM to define an XML document that can be carried
as an IMP asset and that associates other selected IMP assets (called Sidecar Assets) with one
or more IMF Compositions. Persons who wish to contribute to the SMPTE draft documents
are encouraged to join the SMPTE working group and provide comments
Access Services
Closed Captions/Subtitles shall be delivered as an IMF package, as an IMSC1.0 constrained TTML
document encapsulated in MXF. The IMSC1.0 can additionally be constrained to meet the
requirements of EBU-TT (especially EBU-TT-D).
If EBU STL (EBU-Tech 3264) is to be carried in addition to the IMSC1.0 TTML, it shall be
encapsulated as an MXF Track File according to SMPTE ST 2075.
Audio Description files can be fully rendered additional audio services or as a Description only with
a control track as described in BBC Research & Development White Paper WHP198.
Signing currently can be contained as a fully rendered full length Supplementary Video Package
Validation and QC
IMF validation and QC is work currently being carried out by the Video Services’ QC group.
https://EBU.IO/QC will be the source of IMF QC Items and QC Templates
EBU IMF-TV Group Business Requirements Overview
The following is an overview of the EBU IMF-TV group’s poll of EBU members’ requirements. Not all
are achievable at this stage, but work will continue with SMPTE and the HPA IMF User Group to
represent the views and requirements of the Members in future revisions and additions to the IMF
suite of Standards and Specifications.
Video
Progressive only
I-frame codec only
Video quality high enough for all television deliverables
HDR Support (ITU-R BT.[HDR-TV]
ITU-R BT.2020 and ITU-R BT.709 colour
Audio
Audio service per track
Unlimited audio tracks
RDD6 Dolby audio metadata track per audio service
TR 040 Evaluation of IMF for Broadcasters
33
Workflow
Playback of IMF delivered UHD material for technical approval
Content segmentation (extraction)
Content concatenation (insertion)
Editable – chunk size 60 seconds or shot?
Metadata map to Air Ready Masters (e.g. AS-10, AS-11…)
Graphic overlay
Frame Rate adjustment
Branding
Suitability as an input format for encoding farms
Single territory acquisition sub master for incoming acquisitions
Wrapper
MXF!
XML embedded metadata
Carry FIMS/EBU QC template data
Metadata sets to comply or translate from/to FIMS/EBU Core
Simple craft based technical and editorial metadata packages
Video and audio data tracks
Editable programme segmentation track
Segmentation
Segmentation track programme template
Access Services
Closed captioning formatting
One subtitle track per audio service
Access service IDs
Security and delivery
Gateway mechanisms
Virus scanning
Encryption
Delivery of Library and Air master packages
Other issues
Support of a production format for long form content
XDCAM HD / DNXHD145 / DNXHD220 / DNXHD220x
Audience monitoring signal insertion (Discuss)
News/Current Affairs support
Compliance
Wrapper and content block compliance testing
Metadata checking