.NET and Multimedia - Introduction of managed DirectX

Andreas LennartzUniversity of Applied Sciences, Fulda

alenn@gmx.net

Abstract

DirectX is a multimedia API thatprovides a standard interface tointeract with graphics and soundcards, input devices and more. Di-rectX abstracts code from the spe-cific hardware and translates it toa common set of instructions intohardware specific commands. Theactual implementation of DirectXcontains an integration in managedCode, which combines the advan-tages of both world, and is calledmanaged DirectX.

1 Introduction

Currently there are two vendors offering APIsfor developing 3D-Applications available. Onone hand Microsoft provides its DirectX-APIand on the other hand there is the open sourcealternative OpenGL.

OpenGL is an environment for developingportable, interactive 2D and 3D graphics ap-plications. Introduced in 1992, OpenGL isone of the industry’s first choice for 2D and3D graphics APIs (?). Because of its stan-darisation it is platform independent. WithOpenGL, rapid application development witha broad set of rendering, texture mapping,special effects and other powerful visualiza-tion functions can be archieved.

DirectX first appeared in 1995 and was thencalled the ”GameSDK”. Originally it was tar-geted at developers using C and C++. Withthe release of the first managed version (9.0)of the API in December 2002 it is possible touse C# or VB.NET with DirectX (Actually

any CLR compliant language can optionallybe used) (?).

DirectX allow it to make non-hardware-specific function calls. For this purpose,DirectX is accountible for the execution onthe hardware. If the function is supportedfrom the hardware, the hardware accelerationmakes the API functions work fast. If not, Di-rectX has software emulation that takes over.It will be slower, but in spite of that thesame low-level API can be used. Developercan focus on the application development it-self. The layer of DirectX that takes careof hardware-supported features is called thehardware acceleration layer (HAL), and thesoftware emulation layer that kicks in if thehardware doesn’t support a feature is calledthe hardware emulation layer (HEL).

When writing applications with managedDirectX or unmanaged DirectX, managed Di-rectX has the prejudice that it is slower thanunmanaged. But this is only true in certaincircumstances. So far a lot of games werecreated with managed code or using a mixof managed and unmanaged code. Writingin managed code makes the developers moreproductive and thus write more code in lesstime and produces safer code.

2 Managed Code

Managed code is code that has its executionmanaged by the .NET Framework CommonLanguage Runtime (CLR). It refers to a con-tract of cooperation between natively execut-ing code and the runtime. In this contract itis specified that at any point of execution, theruntime may stop an executing CPU. Then itmay retrieve information specific to the cur-

rent CPU instruction address. Informationthat must be queryable generally belongs toruntime state, such as register or stack mem-ory contents.

The necessary information is encoded in anIntermediate Language (IL) and associatedmetadata. The symbolic information that de-scribes all of the entry points and the con-structs is exposed in the IL (e.g., methods,properties) and their characteristics. Withthe Common Language Infrastructure (CLI)Standard, a description is given how the in-formation has to be encoded. Programminglanguages that target the runtime will buildthe correct encoding. Therefore, the devel-oper only has to know that any of the lan-guages that target the runtime produce man-aged code. This code contains encoded filesthat contain IL and metadata. There are a lotof languages to choose from, because there areover 20 different languages provided by thirdparties in addition to the standard languagesfrom Microsoft. (C#, J#, VB .Net, Jscript.Net, and C++).

Before execution of the code, the Virtualmachine compiles the IL into native exe-cutable code. While the compilation happensby the managed execution environment, themanaged execution environment can makeguarantees about what the code is going todo. The enviroment can insert traps and ap-propriate garbage collection hooks, exceptionhandling, type safety, array bounds and indexchecking, etc. E.g, such a compiler makes sureto lay out stack frames and everything justright so that the garbage collector can runin the background on a separate thread, con-stantly walking the active call stack, findingall the roots, chasing down all the live ob-jects. In addition because the IL has a notionof type safety the execution engine will main-tain the guarantee of type safety eliminatinga whole class of programming mistakes thatoften lead to security holes.

”‘Contrast this to the unmanaged world:Unmanaged executable files are basically abinary image, x86 code, loaded into mem-ory. The program counter gets put there andthat’s the last the OS knows. There are pro-

tections in place around memory managementand port I/O and so forth, but the systemdoesn’t actually know what the application isdoing. Therefore, it can’t make any guaran-tees about what happens when the applica-tion runs.”’ see (?)

3 DirectX components

DirectX 9.0 for Managed Code is made up ofthe following major components (The depre-cated parts of DirectX will not be included inthe further discussion):

• Direct3D Graphics provides a single APIthat can be used for 3D graphics pro-gramming.

• DirectInput provides support for a vari-ety of input devices, including full sup-port for force-feedback technology.

• DirectSound provides support for playingand capturing prerecorded digital sam-ples.

• AudioVideoPlayback allows for playbackand simple control of audio and videomedia.

3.1 Direct 3D Graphics

Direct3D Graphics is for device independentaccess to the 3D-graphic hardware and accel-eration. If no hardware acceleration is sup-ported, the software renderer will emulate thehardware. (?)

Complicated 3D-figures and objects areusually modeled with a 3D-software and thensaved to file. The .x file format is an ex-ample for such a file. (?),(?) Microsoft Di-rect3D can create objects from these files us-ing meshes. A mesh is an abstract data con-tainer with the data for a complex model. Itcontains resources such as textures and mate-rials, and attributes such as position data andadjacency data. Meshes itselves are some-what complicated, but accessing and usingthem with Direct3D makes it easier. In thefollowing an example is shown how to load,render, and unload a mesh. With this exam-ple an overview will be given over the func-tionality of Direct3D.

First of all, the mesh object has to be ini-tialized. Also we need a device for the out-put. There are two device types in DirectX:a hardware device and a reference device.The hardware device is the primary devicewhich supports hardware-accelerated raster-ization as well as hardware vertex processing.For this purpose the display adapter of thecomputer has to support Direct3D. The hard-ware device then implements all or part of thetransformations, lighting, and rasterization inhardware. The application itself does not ac-cess the hardware device directly - it calls Di-rect3D functions and methods which accessthe hardware through a hardware abstractionlayer (HAL). If the hardware supports Di-rect3D, these functions and methods are exe-cuted with best performance of the hardware.Direct3D supports additional a device calledreference device. This reference device emu-lates all Direct3D functions and methods viasoftware without any hardware acceleration.Furthermore, if Direct3D functionalities areused which are not implemented in the hard-ware, the reference device will then emulatethe corresponding part of the hardware de-vice.

To create a new Direct3D device, the fol-lowing command has to be executed, e.g. ina Windows Form Class:device = new Device(Manager.Adapters.Default.Adapter, DeviceType.Hardware,this,CreateFlags.SoftwareVertexProcessing,newPresentParameters())

For creating a new device these parametersare needed:

1. A graphics adapter to connect to the out-put device, e.g. to connect one monitorto the graphics card.

2. A parameter which decides whether ahardware of software rendering is done.

3. The class (e.g. the windows form) whichconnects Direct3D with the application.

4. Some flags for setting the behavior of therendering pipeline.

5. Parameters for the presentation behav-ior.

Furthermore in the application defined On-ResetDevice method it is possible to initializean ExtendedMaterial object as an array thatis used to capture the mesh file data to a Ma-terial structure. In the method the device canbe adjusted, for e.g. turning on the z-bufferand white ambient lighting. This function iscalled when the device is initialized and insome different cases reseted.

Now the mesh object has to be loaded.As shown in the following code fragmentas a part of the OnResetDevice method amesh is loaded that represents a texture-mapped 3D tiger from a tiger.x file. In thisMesh.FromFile method call, the SystemMem-ory constant of the MeshFlags enumerationindicates that the mesh is to be loaded intosystem RAM not typically accessible by thedevice.mesh = Mesh.FromFile("tiger.x", MeshFlags.SystemMemory, device, out extendedMaterials);

After the mesh is loaded, a meshMateri-als Material object extendedMaterials has itsmembers filled with the Material structuresof the materials object that came from themesh file. These materials could now be setinside the OnResetDevice method to a specificambient color or could be set to a MeshTex-ture object loaded from a texture of a file. Atexture object file could be loaded with theTextureLoader.FromFile method. For exam-ple, it is possible to set for all extendedMa-terials of a mesh the ambient material color.This could take place by an iteration over thelength of the materials from the mesh. Af-terwards the texture had to be loaded. Thefollowing code fragment (with i as the itera-tion counter) shows this:meshMaterials[i].Ambient = meshMaterials[i].Diffuse;meshTextures[i] = TextureLoader.FromFile(device, extendedMaterials[i].TextureFilename)

Finally, after the mesh has been loaded andinitialized, the next step is to render it. Forthis purpose we have to call the private Ren-der method. This method is called every timean object needs to be rendered. The functionis accountable for making the Mesh objectlooking as desired. Inside the render method,

the rendering pipeline can be adjusted. Torender the mesh itself, it is divided into sub-sets, one for each material that was loaded. Aloop needs to be run for rendering each ma-terial subset. This loop can for example beused to make the following operations on theobject:

• Setting the material property

• Draw the material subset with themesh.DrawSubset() method

With the commands device.EndScene()and device.Present() the end of the render-ing pipeline is indicated and the object willget visible.

Now, it is realized to load, render andpresent a mesh with a Direct3D device. If thegraphic device supports Direct3D, the ben-efits of hardware acceleration would be used.For a deeper look at this example and its codethere are more information in (?)

3.2 Direct Input

With Microsoft DirectInput it is possible toestablish a direct communication between thehardware drivers of an input device. Nor-mally the communication would be providedover a Windows message relay supported bythe Microsoft Win32 API. With DirectIn-put the application retrieves the data directlyfrom the device, even when the application isinactive. Furthermore, with DirectInput allkind of input devices with DirectInput driversare supported, as well as functionalities forForce Feedback. With Force Feedback it isallowed to send messages and properties to auser’s DirectInput device to relay feedback inthe form of physical force.

DirectInput implements the technology ofaction mapping. Action mapping applica-tions can retrieve input data without knowingwhat kind of device is being used to generateit. DirectInput does not provide any specialsupport for keyboard devices or mouse navi-gation. Action mapping provides a possibil-ity to establish a connection between inputactions and input devices. This connection

does not depend on the existence of particu-lar device objects (e.g. specific buttons). Ac-tion mapping simplifies the input loop andtherefore reduces the need for custom gamedrivers and custom configuration user inter-faces in games. (?)

To capture the device input using MicrosoftDirectInput, it is necessary to know fromwhich device the input has to be captured.(And, of course, if it is currently attached tothe user’s computer). To do this, there is aenumeration of DeviceInstance objects whichcan be looped. This Enumeration serves thefollowing purposes:

• Reports what DirectInput devices areavailable.

• Supplies a globally unique identifier(GUID) for each DirectInput device.

• Possibility to create a DeviceInstance ob-ject for each DirectInput device as it isenumerated. With this the type of de-vice and its capabilities can be checked.

An example code fragment for this loopwould beforeach(DeviceInstance di in Manager.Devices)string name = di.InstanceName;

Capturing the device input itself with Di-rectInput can be done in three parts. Firsta device object that represents the input de-vice needs to be created. After the creation ofthe device object the configuration has to bedone. After the configuration the device statecan be checked within the application. Therewill be callback functions registered which arecalled after the corresponding action from theuser (e.g. pressing a button.)

An example for the configuration of a joy-stick device could be:foreach(DeviceObjectInstance doi injoystick.Objects)joystick.Properties.SetRange(ParameterHow.ById, doi.ObjectId,new InputRange(-5000,5000));

Furthermore, in the callback function Up-dateJoystick() which is registered for joystickevents, the state of the buttons could be re-quested with this command:

byte[] buttons = joystick.CurrentJoystickState.GetButtons;

As seen in the examples, DirectInput pro-vides an easy way to access an input device.The benefit of DirectInput devices is the di-rect communication between hardware andapplication, as long as the devices have corre-sponding hardware drivers.

3.3 Direct Sound

Microsoft DirectSound provides a system tocapture sounds from input devices and playsounds through various playback devices.The input devices are realized in several sec-ondary software or hardware sound buffers.Software buffers keep their data always insystem memory and are mixed by the CPU.Hardware buffers can keep their data eitherin system memory or, if the application re-quests it and resources are available, in on-board memory and are mixed by the soundcard processor. Each of them contains a staticsingle sound or a dynamic stream of audio.When sounds in secondary buffers are played,DirectSound mixes them in the so called pri-mary buffer and sends them to the outputdevice. Only the available processing timelimits the number of secondary buffers thatDirectSound can mix. The primary buffer isalways a hardware buffer on the soundcard,which means there is only one available andits memory size is limited. So the primarybuffer holds the audio that the listener willhear. Unlike the primary buffer there ex-ists several settings for the secondary soundbuffer that can be changed. When creatinga new secondary buffer the most importantcommand line is:sound = new SecondaryBuffer (filePath,bufferDescription, SoundDevice)

As shown each secondary buffer has a bufferdescription which contains one or more con-trol properties (like pan, volume, frequency,effects, 3D) from the BufferCaps structure(?). To allow the control of these propertiesthe corresponding flags just have to be en-abled. According to those settings differentoperations can be executed by simply typingin the name of the secondary buffer followed

by a dot and the operation name. For exam-ple sound.Play() starts playing the sound orsound.Format.* gives back additional soundinformation like the number of channels orsamples per second.

When a secondary buffer is created withenabled flag for the control3D property, Di-rectSound offers the possibility to set an al-gorithm to simulate 3D spatial location ofa sound. By default, no head-related trans-fer function (HRTF) processing is performed,and the location of the sound relative to thelistener is indicated by panning and volumeonly. To optimize the 3D effects for the user’ssystem DirectSound also uses the speaker con-figuration. The SpeakerConfig property fromthe sound device allows users to set up thesound system to the given speaker system byswitching the flag to true, for example mono,stereo, 5.1 or 7.1 surround sound.

One of the most important features of Di-rectSound is the possibility to apply real-timeaudio effects to the sounds in playback. Allthe effects are applied in hardware (that is,implemented by the sound card driver) when-ever possible, otherwise emulated via soft-ware. The latter causes an important dropin the performance. There are 9 effects exist-ing: Chorus, Compressor, Distortion, Echo,Flangler, Gargle, Interactive3DLevelReverb,ParamEqualizer and WavesReverb. Each ofthem has its own different settings ((?)). Toimplement an effect on a buffer first the effectflag in the buffer description has to be set totrue and then the SecondaryBuffer.SetEffectsmethod has to be used. This method takesan array of EffectDescription structures thatdescribe the effects. An effect can only beapplied if the sound is not played. An exam-ple and a great summary of the Direct Soundfunctionalities gives The Ultimate ManagedDirectSound 9 Tutorial1. It creates a smallapplication which demonstrates all Direct-Sound elements that have been described be-fore. A big disadvantage of Microsoft Direct-Sound is that it plays only wave and PulseCode Modulation (PCM) files. Waveform au-

1http://www.codeproject.com/cs/media/DirectSound9p1.asp

dio data consists of digital samples of thesound at a fixed sampling rate, whereas PCMthe representation of an analog signal by a se-quence of numbers means. Managed Direct-Sound does neither support compressed WAVformats nor audio formats with more than twochannels. Therefore AudioVideoPlayback isthe better choice for playback other formatslike mp3 for example.

3.4 Audio Video Playback

The AudioVideoPlayback application pro-gramming interface (API) provides like theolder DirectX version the playback and simplecontrol of audio and video files. But Microsofthas highly simplified the usage. All function-alities for the playback are packed into a sin-gle dll-file. Thus the integration of a videointo an application became much easier, be-cause the user only needs to refer to the Mi-crosoft.DirectX.AudioVideoPlayback names-pace. The command:Video ourVideo = new Video("C:\\Example.avi")

creates an instance of the video classand provides the possibility to controlthe playback. For instance the methodourVideo.Stop() stops the playback andourVideo.Play() plays it. The size of the play-back window can be set by ourVideo.Size =new Size(480, 320). If the video file containsaudio, the Video.Audio property returns anAudio object that can be used to change thevolume or the stereo balance for the audio, forexample Video.Audio.Volume = 100.Audio ourAudio = new Audio("C:\AudioFile.mp3")

The Audio object is similar to the Videoobject, but it supports only properties thatrelate to audio, such as Volume and Balance.That means the Audio class is primarily de-signed for very simple playback scenarios, orfor use with the Video class. Whenever agreater control over the audio playback isneeded, it is better to use Microsoft Direct-Sound to play audio files.

The managed DirectX AudioVideoPlay-back API doesn’t support video converting.The only way to do this is using the oldunmanaged C++ code, where several filter-

graphs have to be combined. For further in-formation on that topic see (?).

4 DirectX versus OpenGL

OpenGL is an open standard, so everyone canuse it for free. The fact that it is a standardfor graphics cards makes it running on everyoperating system (as long as the OS consid-ers a graphic user interface). Compared tothat, DirectX is an API developed by Mi-crosoft and consists of several subsystems likeshown in the previous chapters. So only theDirectX graphic component can be comparedwith OpenGL. Considering the DirectX soundpart, the analogue would be OpenAL, whichis an open audio standard. As already men-tioned, DirectX is a Microsoft product, so itcan only be found on Microsoft operating sys-tems where it is preferred to OpenGL. Never-theless Microsoft cannot ignore OpenGL, be-cause it is an industrial standard. So thegraphics card manufaturers support their cus-tomers with the latest OpenGL driver ver-sions and makes OpenGL usable under Win-dows.

Considering OpenGL and DirectX bothare quite similar and offer equal functionali-ties. Anyway there still exist some differenceswhich are illustrated in table 1 on the follow-ing page.

5 Conclusion

With Managed DirectX there exists a struc-tured and easy to use low-level API. Whilein C++ the developer took great pain to sim-plify and integrate unmangaged C++ classes,with the .NET style and the design of names-paces, classes and properties, programmingManaged DirectX is very comfortable forthe developer. Compared to OpenGL, Di-rectX has some very important advantagesfor game developers. With Managed DirectX,the developers also write more productive andtherefore more code in less time and producesafer code.

After a short introduction in Managed Di-rectSound, DirectInput, Direct3D and Au-dioVideoPlayback APIs the reader should forhimself recognize the obviously advantages of

Features OpenGL DirectXobject oriented no yes

operating system many only all Windows versions

useable drivers available for high end graphics cards nearly all graphics cards

driver quality mostly bad often better than OpenGL driver

usage university, research and development, CAD game industry

documentations, tutorials, samples many less than for OpenGL

version release every 5 years every 15 months

trademark of Silicon Graphics Inc. Microsoft

Table 1: The list compares the key features between OpenGL and DirectX

managed DirectX. With regard to the comingWindows Vista where DirectX is an elemen-tal component, DirectX will play a more im-portant role even for application designers inevery part of future application development.