User evaluation and overview of a visual language for real

User evaluation and overview of a visual language for real time image processing on FPGAs

C T Johnston School of Engineering and

Advanced Technology, Massey University

Palmerston North, New Zealand

[email protected]

P Lyons School of Engineering and



[email protected]

D G Bailey School of Engineering and



[email protected]

ABSTRACT FPGAs are often used for image processing, but existing FPGA design tools lack syntactic constructs for some specialized activities that are important in this field, such as timing, resource handling and scheduling. This forces the developer to work at too low a level and makes it difficult to produce a genuinely hierarchically decomposed design.

This paper outlines these deficiencies, as the background for an overview of and justification for each of three views in VERTIPH, a visual programming language for defining image processing algorithms on FPGAs. This updates the overview presented in [1]. The paper then presents the results of two user evaluations of VERTIPH, a pre-implementation paper-based user evaluation which found no major changes were required and a post-(partial)-implementation user evaluation. The latter evaluated the novel parts of the language using participants experienced in the field. The key parts of VERTIPH were found to be useful visualisations for the developers, and the only major problem was the interaction required for defining type-connections between views.

Categories and Subject Descriptors B.7 Integrated Circuits, B.7.2 Design Aids, C.3 Special-Purpose And Application-Based Systems, D.2.2 Design Tools and Techniques, D.3 Programming Languages, D.3.3 Language Constructs and Features, H.5 Information Interfaces And Presentation (HCI).

General Terms Design, Human Factors, Languages.

Keywords Visual programming language, Image processing, HDL, FPGA.

1. INTRODUCTION We have designed and partially implemented VERTIPH, a visual HDL (Hardware Description Language) for defining FPGA (Field-Programmable Gate Array) implementations of

image processing algorithms. This paper contains descriptions of the field, the areas in which conventional Hardware Description Languages fall short, the design of VERTIPH, and some user evaluations that have been performed on an implementation of the first two of three major aspects of VERTIPH.

FPGAs can achieve high throughput and have therefore increasingly come to be used for real-time image processing. Image processing on FPGAs [2] can be implemented using stream processing [3], random access processing [2] and a hybrid of the two.

In stream processing, data is presented as a one-dimensional pixel stream with a suitable access pattern [3]. For image processing, this is typically a raster order (in which pixels are presented from left to right for each image row). This converts the spatial parallelism of the image into a one-dimensional temporal stream of pixels. For non-trivial image processing tasks, there is insufficient time to complete all the required calculations for each pixel within a clock cycle. However, the calculations on successive pixels are generally identical, so they can be pipelined, taking advantage of the temporal parallelism of the stream [2]. Pipelining reduces the logic depth (the number of sequential computational stages that execute in a single clock cycle), enabling a higher clock rate [4]; an HDL should clarify the timing of pipelined designs so that pipelines are efficient and easy to design.

Random-access processing imposes no explicit ordering on the pixels being processed. The data is buffered, and the desired pixels may be read from the buffer in any order. Algorithm speedup is achieved through hardware acceleration of computationally intensive tasks. Random access designs can also make use of parallel units, using multi-port or partitioned memory to enable parallel access to pixels. This means an HDL should make parallel operations easy to design and conceptually separate from the pipelined segments of the design.

Image processing experts are not always experts in hardware design. Their productivity will be enhanced if the design language is intuitive and easy to use, but existing HDLs do not always have appropriate constructs. For example, current languages treat pipelining as a special case of parallel design and make no distinction between the two. This makes it unclear to the developer which operations form part of a pipeline and which are running in parallel, and can result in suboptimal or incorrect designs. A well-designed HDL should therefore

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHINZ'09, July 6-7, 2009, Auckland, New Zealand. Copyright 2009 ACM 978-1-60558-574-1…$5.00.

CHINZ 2009 Environment, Vision & Sound

85

clearly distinguish between conventional parallel architectures and pipelined architectures.

Parallelism opens other possibilities for conflicts. For example, concurrent processors (blocks of computational logic) often share access to resources such as RAM, and it is easy to make access timing errors. An FPGA design system should be able to detect possible conflicts and inform the designer so that appropriate action can be taken.

Developers need to see information relevant to their present task or goal [5]. The information that the developer needs to access when designing the system overview is very different from what is required for the low-level pipelining of operations. Similarly, high-level scheduling needs different control representations from the control structures used at the lower-levels of a design.

Thus, in designing VERTIPH, we have paid particular attention to establishing a clear distinction between pipelining and parallelism, and we have separated an FPGA design into a number of semi-independent views, each tailored to a particular aspect of the design task. To assess users’ perception of the benefit of these design decisions, we have conducted a paper-based user evaluation of the design and a user evaluation of the implementation of the first two of VERTIPH’s three views.

2. EXISTING LANGUAGES Hardware representations based on schematic capture or state transition diagrams are not well suited for expressing complex algorithms.

Current HDLs were generally developed to capture the high-level temporal behaviour of complex digital designs as well as their circuit structure. Industry standard HDLs such as Verilog, and VHDL [6] can be thought of as the assemblers of hardware programming, providing great flexibility from gate level up to the behavioural level. While HDLs offer considerable flexibility in terms of the control logic, it is up to the designer to explicitly specify that logic. This can allow very efficient control of the execution path, but the developer must design both the data flow and the control flow. HDLs such as Handel-C [7], ImpulseC [8] and SA-C [4] have eased this by adding hardware constructs to a software language that automatically constructs the control flow.

Takes 3 clock cycles

seq {a = a+b;b = cx = y;

}

In normal C, the order of instruction

flow goes down the page

Takes 1clock cycle

par {a = a+b;b = cx = y;

}

all operations run in parallel, so there is no order implied by

the layout

Takes 3 clock cycles

seq {a = a+b;b = cx = y;

}

In normal C, the order of instruction

flow goes down the page

Takes 1clock cycle

par {a = a+b;b = cx = y;

}

all operations run in parallel, so there is no order implied by

the layout

Figure 1. Logical flow of instructions

Handel-C is a subset of ANSI-C with hardware-oriented extensions for specifying variable data widths, parallel processing and channel communication between parallel processing blocks. However, its textual nature obscures data flow in parallel designs. Figure 1 shows two example snippets

of code, one sequential and one parallel, which look virtually identical. This is common in text based HDLs.

3. VERTIPH As image processing algorithms are often conceived of as a sequence of relatively independent operations, VERTIPH’s Architecture representation uses a data flow paradigm.

This view is complemented by a lower level computational graphical representation that defines the calculations that make up each operation. Finally, in real-time image processing algorithms on FPGAs, it is important to prevent conflicts when accessing resources like shared memory. A specialized representation for scheduling is therefore justified.

VERTIPH thus provides three graphical views, each representing a different aspect of an algorithm:

• the top-level Architecture View

• the Computational view,

• the Scheduling view.

3.1 Architecture View The Architecture view is aimed at providing the designer with a perspective on the overall system, similar to Khoros [9] and Opshop [10]. The aim is to allow logical separation of image processing operators, to show the data flow through the operators, and to encapsulate related operation data and processors. The use of component blocks allows resources such as frame buffer RAM to be encapsulated and related computational processes to be logically grouped.

Processors and data can also be encapsulated. For example, the colour segmentation and tracking algorithm detailed in [11] represents each detected object in an image by its bounding box. It stores the bounding boxes for each colour class in a data structure, and it incorporates processors for both updating the data and calculating the objects’ positions. The data structure and the processors are logically related and should therefore be kept together.

The Architectural graphical representation, shown in Figure 2, consists of functional blocks and manages data and control flow links though complex data types. The ‘wires’ between the blocks carry data or control information. Several related data items can be collected into a single wire representing a complex data type, thereby reducing the number of connections between blocks and simplifying the diagram.

The Architectural view is hierarchical. Each Architectural block encapsulates either a collection of other Architectural blocks or one or more related Computational blocks. This hierarchy allows very complex designs to be built without overloading the developer with detail. Figure 3 shows an architecture in which one block is expanded; the inputs and outputs of a block are shown as the edge terminals in the lower view. A second block, containing Computational blocks is also expanded. The expanded Computational block shows the I/O from the architecture block and expressions - here, a graphical if-then-else construct. The interface between the outside (Architecture block View) and inside (Computational block View) is mapped


86

via a junction box which allows data types to be broken out, combined or converted.

VideoInImage_data Corrected_pixel

Display

Pixel LocationImage_in

xypoint

Distortion correctionVideoInImage_data Corrected_pixel

Display

Pixel LocationImage_in

xypoint

Distortion correction

Figure 2. Architecture View with blocks, terminals and wires

Figure 3: Overview of the Architecture View

A B C A

B

CTime

(a) (b)

Figure 4. Sequential (a) and parallel (b) operations

3.2 Computational View FPGA designs usually comprise parallel or pipeline networks. Parallel operations and timing, which must be precise in pipelines, both involve complex relationships. This suits a 2D graphical representation more than a linear textual one. VERTIPH (Figure 4) represents (a) sequential operations horizontally along a time axis and (b) parallel operations vertically, like tasks in Gantt charts. In the Johnston-Bailey-Lyons (JBL) notation [1], (based on N-S (Nassi-Shneiderman) diagrams [12]) these time-related operations can be controlled by conventional while and if-else control constructs.. The while construct and the other loop structures (for and until, illustrated in Figure 5) are essentially the same as those in N-S diagrams. However, the N-S representation for choice constructs uses too much room when sequential and parallel operations are expressed. It also places alternatives horizontally, which in

VERTIPH implies sequence. Instead VERTIPH's if-statement is based on an F-shape. Control expressions are written within the horizontal bars and the statements they control appear below them. The vertical bars can also be used for comments. The algorithm shown in Figure 5 is part of an object tracking system described in [11]. Figure 5 which shows an infinite while-loop that defines three cases in handling a video image. It contains a nested if-statement which uses signals received from the camera to control the execution path. A new line of pixels in the input stream triggers the line counter expression. If new pixel data is available, a 5-stage pipeline with a throughput of 1 pixel per clock cycle is activated. The notation for a pipeline is similar to that for a sequence of operations but with a slider and data bars that indicate how the input data progresses through the pipeline, this is a modification of the pipeline design presented in [13]. This gives feedback to the developer on the latency of the pipeline design and it makes it easy to check the scheduling of parallel stages. Function names, such as “rgb to yuv” represent expressions. The final condition in the if-statement detects when a new frame;arrives the controlled expression contains the two function blocks which increment the frame counter and reset the line counter in parallel. They are stacked vertically, to indicate that they operate in parallel.

3.3 Scheduling View Processors (blocks of computational logic) in complex image-processing systems on FPGAs need to be scheduled, to avoid resource contention, to ensure correct program execution, and to respond as data arrives or is requested. Celoxica’s PixelStreams [14], Khoros [9] and other languages schedule processors by passing execute-enable tokens between processors in order to accomplish this. This approach highlights data flow but obscures control flow [15].

The authors’ experience suggests that control should be treated at two levels. At a low level, there are data-dependent controls that choose program actions depending on data received; for example, pixel values can be used to select an identifying label for a uniformly coloured region [11]. This approach is used in the Computational View. The second, higher-level, type of control operates globally. In many algorithms, some or all of the processors should only run during certain “epochs”, with a predefined duration or with external or internal triggers. This allows efficient scheduling of both source- and sink-driven processes and makes control flow more explicit.

Many visual representations for the Scheduling View were considered and have been documented elsewhere [16]; the final representation is an extension of state charts that allows processors to run sequentially, in parallel or as a pipeline.


87

else If(pixel_data)

if(new_line)

while(true)

Line_count++

else if(new_frame)

Line_count = 0

Frame_count++

r

b

g

rgb to yuv Threshold Filter vertically Filter horizontally Bounding box

x

y

bb_regionthres_value

else If(pixel_data)

if(new_line)

while(true)

Line_count++

else if(new_frame)

Line_count = 0

Frame_count++

r

b

g

rgb to yuv Threshold Filter vertically Filter horizontally Bounding box

x

y

bb_regionthres_value

Figure 5. Single phase pipeline for object tracking

Resource and Scheduling

Frame 0Field

retrace1

Frame 1

VerticalBlanking 1

Read0

Write1

Read1

Write0

Vertical Blanking 1

End of field New field

End of fieldNew Frame

New Frame

New Frame

pipelinedprocessorsBounding

boxPixel

labellingFilterColor

convertion

ResetHistogram

Histogramwrite port

ReadHistogram

Histogramread port

concurrentprocessors

(a)

(b)

(c)

b after a c after ba sequentialprocessors


Frame 0Field

retrace1

Frame 1

VerticalBlanking 1

Read0

Write1

Read1

Write0

Vertical Blanking 1



New Frame

New Frame


boxPixel


convertion


Frame 0Field

retrace1

Frame 1

VerticalBlanking 1

Read0

Write1

Read1

Write0

Vertical Blanking 1



New Frame

New Frame


boxPixel


convertion

ResetHistogram

Histogramwrite port

ReadHistogram

Histogramread port


(a)

ResetHistogram

Histogramwrite port

ReadHistogram

Histogramread port


ResetHistogram

Histogramwrite port

ReadHistogram

Histogramread port


(a)

(b)

(c)



Figure 6. Extended state machine editor, Vertical Blanking

selected. In the lower split screen, the three concurrent processors associated with the state (labelled (a)), are shown. The sequential (b) and pipelined processors (c) are not part of

this design; they have been added for completeness

Figure 6 shows the Scheduling View using a combination of a Gantt chart notation and the standard state-machine chart. In this example two state machines are shown - one for the main

algorithm (representing a histogram example [16]) and one for controlling an encapsulated resource.

When a state is selected in the editor, the internal processors are shown in a split window. In the lower part of this window, processors can be added, and their execution order can be edited. The combined notation makes it possible to implement the complex scheduling required by image processing applications.

The lower window in Figure 6 shows all the possible options for the scheduling of processors like the Computational View. Only the concurrent processors, labelled (a), are relevant to the histogram algorithm; the others, (b) and (c) are there for illustration. Time increases from left to right; parallel processors, labelled (a), are stacked vertically; sequential processors, labelled (b), are drawn from left to right on a single horizontal line. As a pipeline is a combination of sequential and parallel processing, we draw a pipeline, labelled (c), using the same notation as in the Computational View. Unlike the Computational View, the data processing times for each block will vary depending on the block’s latency as determined by its internal operations.

Processors are represented by a dark horizontal bar surmounted by a lighter coloured “flag” containing the name of the process. The resources that processors require are also represented as flags, which can be hidden; they are in a different colour and are positioned under the processor bar. In the concurrent processors in Figure 6 (a), flags showing the histogram resources that the processors require are shown in their expanded state.

4. PAPER BASED USER EVALUATION A small scale paper-based user evaluation of the VERTIPH graphical syntax was undertaken with 8 participants including engineering undergraduates, masterate and PhD students. Each participant was provided with a paper mock-up of the interface and paper templates to simulate interface objects. The author played the part of the IDE. Participants had a tray of different cardboard cut-outs to represent the different objects (if-else, loop,


88

pipeline etc) from which they could choose. These were assembled by the participants and expressions were parameterised by pencilling in the operation. These cut-outs were available in a number of different sizes and could be extended. Terminals were added manually and wires between objects drawn in by either the author or the participant. The author added the clock cycle lines to the diagram as required. Each participant was given three tasks: the first was to represent the architectural View for an algorithm to count grains of rice in a grey scale image; the second was to do the high level scheduling for this; the final task was to implement the low level operations of a given algorithm for correcting lens barrel distortion. Participants were allowed as long as they needed to complete the tasks. They were observed during the evaluation process and could ask any questions relating to the problem or development with VERTIPH. When they had finished all three tasks, they completed a questionnaire. Following this there was an open discussion with the author where they could indicate which aspects of the notation they found intuitive and helpful, and which parts needed modifying.

Most of the participants had a low to medium level of knowledge of both image processing and FPGA design. Two of the participants had a high degree of knowledge of both image processing and FPGA design. It was found that these ‘expert’ participants had the best grasp of the concepts, both with solving the tasks and with the goals behind VERTIPH’s design. The other participants either had difficulty solving the image processing operations or they did not grasp why pipelined designs were required.

All the participants said that when explaining a system they would draw a diagram to aid them in the description. This suggests that diagrams are a useful tool in capturing the structure of a design. They all found that the Architecture View was useful in that it encouraged top level design and was an intuitive notation for them. This is not surprising as block diagrams are a commonly used notation in most forms of engineering and are also used in programming.

The participants were split equally on the question of whether VERTIPH is too graphical or is well balanced. Those who found it too graphical found the block level design used in the Architectural View useful for describing the system but found the Computational View too graphical for expressions and would have preferred to use text only. One solution to this would be to incorporate a split view with the assignment statements editable as text in one of the panels. Indeed, this was considered at the design stage, but there was only enough time available to implement one alternative; it was therefore decided to restrict the implementation to a diagram-only notation, as this alternative allowed us to gain useful insights. We were familiar with, and already had some idea of the strengths and weaknesses of, textual notations (see the earlier discussion relating to Figure 1); therefore it seemed more interesting to investigate the graphical notation.

The pipelining notation was useful and aided in the understanding of design. However many of the participants had difficulties solving the pipelining problem they were given because they appeared to have difficulties understanding the problem. They lacked understanding of how to transform the set of equations into a pipeline. Most of the participants said that with practice it would be easier to use this notation.

The JBL notation, based on N-S control notations, was considered to be useful and helped to associate functions with the corresponding controls. All but one of the participants found it to be intuitive to both read and use.

The Resource and Scheduling View was considered to be disconnected from the other views and several participants stated that they would not use it. Other than this view being disconnected from the other views there are two possible reasons for this. With a paper based evaluation, there is no linkage between the Architectural View and the Resource and Scheduling View, so participants had to represent architectural blocks twice, which they reported as being time consuming and pointless. The second possibility is that the problem they were required to solve was too simple to really test the usefulness of the view. The problem needed to be simple so it could be solved in a reasonable time but this meant that it did not require the more complex high-level control structures offered by the state based view. The participants felt that they could define the control at the lower level. One of the more experienced participants noted that, for more complex problems, the usefulness of the high level view of the Resource and Scheduling View would become more apparent.

The developer most familiar with VHDL had several observations on both VERTIPH and its design methodology. She correctly completed all tasks but would have preferred to be able to build the state machine with the states first and then put the operations controlled by them into it. From an implementation point of view, the components of the state machine could be built in either order. We have encouraged the Architecture View as the first view so that developers think about the system as a whole first. This developer would like to have had the option for bottom-up programming; rather than having to define the top level architecture first. This would let her create low level operations independently and then stitch them together at the higher level. While this is possible in VERTIPH, it is not encouraged because a top-down approach can generally lead to more maintainable designs[17].

The developer with the most experience with implementing image processing on FPGAs also took the longest to complete the task, spending most of the time on the Architectural View. The reason for this is interesting; he said in the discussion afterwards that he wanted to get all the architectural blocks correct before committing to use them. Generalizing this implies that the Architectural View requires forward commitment – a negative in the Cognitive Dimensions Analysis terminology [15] - that a choice made at the start would affect the outcome, and can lead to reimplementation if incorrect choices are made at the start. The benefit is that a top-down approach encourages more thought about the problem at the design phase, which can reduce errors later in the design.

Some of the participants did not have the required detailed knowledge of both image processing and FPGA design (despite having taken university courses on both). Their comments were not as beneficial as the more knowledgeable participants’; however they still provided some useful feedback.

No major changes to VERTIPH were made following this initial user evaluation as the key parts (pipelining, JBL control structures and use of a top level Architectural View) were seen as useful and beneficial by most participants. The interface between the


89

Scheduling View needs to be clear to avoid the feeling of disconnect that participants experienced; this can be done through the linking of controlled objects with the state machine and vice versa. Some commented on the editing of equations and this may need some modification as some participants would like to be able to edit equations outside the expression blocks. We have proposed using a split panel to allow this, however this would require a two-step process of editing equations in text then moving separate objects which represent them. It was decided to implement the expression editing within expression blocks and evaluate whether a split view would be desirable in the evaluation of the prototype.

5. USER EVALUATION Once the Architectural and Computational views had been implemented, a user evaluation was conducted to identify aspects of the system that needed improvement. Participants undertook design tasks that focused on control structures, concurrent designs, and designs involving complex pipeline timing, as these are areas where conventional hardware description languages lack expressive power, and which VERTIPH was designed to clarify. The information captured by the evaluation fell into two broad categories: participants’ subjective opinions about the usability of the system, and information about the correctness and style of the designs they produced. We were interested to see whether or not VERTIPH biased its users towards a particular style of solution.

A pilot version of the evaluation was trialled by a member of the signal processing research group at Massey and some of the questions and tasks clarified based on the feedback produced.

The participants who were invited were known to our research group; this allowed us to target people who we were confident had the required experience in both image processing and FPGA design. Of those contacted over half decided not to take part, either because of time constraints or because they felt they did not meet the FPGA design experience requirement. This led to the identification of three participants with experience in implementing image processing algorithms and one with experience in implementing signal processing algorithms on FPGAs. This has led to an evaluation by “experts” which has benefits (in terms of understanding the problem domain) and negatives (in terms of being used to a particular methodology or tool for solving the problem). The three participants with image processing experience were most familiar with lower level HDLs (mainly VHDL), though they also had some experience with Handel-C. One participant had only used Handel-C. All were used to using a mixture of parallel and pipelined design to speed up algorithms when mapping to FGPAs. Most used a mixture of top-down and bottom-up programming, using a top-down approach to get the key parts of the design, then building these up from the bottom by creating smaller functions. All but one had a good understanding of the use of pipelining.

These participants where then introduced to VERTIPH with two simple tasks. The first introduces the participant to the Architectural View, and the second task to the Computational View. This structure was intended to allow us to resolve any problems with the understanding or use of VERTIPH and to gauge initial reactions.

However, in the event, all but one of the evaluations were performed remotely, so generally this information was only

gathered after the participants had completed the evaluation. Therefore problems in understanding were not caught early and this resulted in several participants being confused between the pipeline and sequential visualisations (which have a similar layout, though different operation). Their initial reaction to VERTIPH was generally that they needed to use it more before they could give an informed opinion. The design flow used by participants was in general similar to ours, but influenced by the tools they were used to using. Generally, the standard approach is to develop and test the algorithm in software (such as MATLAB) and then port the design to hardware. This normally involves developing the top level hardware design before constructing the low-level detailed hardware units.

5.1 Evaluation tasks After the participants had become comfortable using VERTIPH they completed the evaluation tasks. These tasks were developed to allow the developer to test key parts of the system without taking too much time to complete. We wanted to find which aspects of VERTIPH are useful for aiding in the design process and which aspects are detrimental. This allows key features of the language to be evaluated and also allows future improvements to be developed. We also wanted to see how users develop their designs and how they implement a given design.

The parts of VERTIPH which we wanted to test are:

• the multi-view design process - both the enforced top-down design process and the separation of the high-level block design from the low-level computational design.

• the Computational View’s separation of parallel and sequential operations.

• the pipeline control structure.

• the if-else and loop control structure.

• the type editor and “junction box” editor.

These are the novel parts of the language compared to other HDLs, so evaluating them will help to answer the research questions of whether it is easy to understand the design and map it to FPGAs.

We needed to allow the participants to explore the design space but still guide them towards solutions that use the features of the language we want to evaluate. To answer our questions within a reasonable period of time we identified two tasks to be completed. Both have an image processing focus but are general enough so that a non-expert could complete them. They also have the majority of their focus on the Computational View as this is the most complex and is also the task that would take the most time in a normal design.

The first task involves implementing an image processing algorithm to correct for barrel distortion. The problem is given to the participant in the form of a poster with the output and a diagram in the form of pseudo code. The participants were asked to implement this design in VERTIPH, They needed to identify that a pipeline is the most sensible approach, although they are free to modify the algorithm to produce a similar algorithm. Though they are guided though a design in this task, it allows


90

them to concentrate on the VERTIPH design rather than solving the problem itself.

The second task (a min-max filter) is more open-ended, and although it is image processing based, it has several simple solutions which should be familiar to signal processing practitioners. It also has the added dimension of handling edge effects. It was assumed that most participants would not consider them but gave the opportunity for design decision questions in the questionnaire and interview, which might test knowledgeable participants. Participants were not required to complete the entire task, as it could become quite time consuming.

All of the participants took a lot longer to complete the tasks than expected (much longer than the participant in the pilot); this led to most of then not finishing the final task. This may have been because of a combination of factors; the learning time required to become sufficiently proficient using VERTIPH for completing the tasks, and the amount of work required for the final task being greater than anticipated.

5.2 Summary of User Evaluations The questionnaire contained a mixture of open and closed questions; it can be found in [18]. Following the questionnaire, the participants were invited to make any other comments in an open discussion. Below is a summary of their comments for the parts of VERTIPH we wanted to evaluate.

Multi view design process: The multi view design process was found to be “different” but could be useful. The participants did not find the enforced top-down design approach to be a problem. The Architectural View was viewed positively with most participants preferring it to the module wiring tools they were used to in VHDL.

Computational View: The computational expression view generated the most mixed reviews. Generally it was thought that the visualisation was useful. However, the participants would like to have been able to edit their expression in text and then either have the Computational View automatically generated or edit the placement of the expressions in the view. Generally the separation of parallel and sequential operations and the pipeline control structures were considered useful. However, the participants would like to have been able to use this more as a visualisation than for editing directly. The control structures were viewed very positively.

Pipeline control: There was also some confusion over how the pipeline control was used, with several participants assuming that a pipeline could be created implicitly by drawing a simple sequence of operations, when in fact operations that are to be pipelined need to be drawn inside an explicit pipeline structure. This is perhaps due to confusing our dataflow diagram at this level and the Register Transfer Level diagrams which some HDL IDEs can create – or possibly because the high level blocks in the Architectural View act as a pipeline. However, this was found to be a useful visualisation when considered as part of the Computational View as a whole.

Control structures: The participants who commented on the control structures found them useful and intuitive. One participant was very complementary about them and wished that their design tool had such a set of control structures.

Type editor and Junction box: The data type editor was viewed positively. All of the user evaluations commented that the junction box interface needs to be improved by adding both internal and external pins and connecting them by default when a new pin is created, rather than having to create the internal connections manually. This, with hindsight, is what should happen, as most of the time the internal and external pins will be the same with the breaking-out or combining of data types not required with many operations and data types.

The notations for pipelining, parallel processing and controls were all found to be useful for the developer as they make the design clearer and easier to understand.

The developers generally made correct design decisions, using pipelined or parallel operations appropriately. They produced quite different solutions to the tasks. Although the solutions were all pipeline-based (the sensible solution) the exact implementation of the system as a whole, the modularisation, the operations used, and the number of steps, were different.

In addition, all the participants commented that a simulator would be desirable. There was also a desire for a number of different specialised hardware objects.

In summary, the participants found most parts of VERTIPH useful, even if they would prefer to use the Computational View as a visualisation of the expressions rather than an editing tool. This implies that VERTIPH provides a good balance between textual and graphical representation, although further refinement is still needed. The evaluations resulted in many useful suggestions for other improvements and future work.

Unlike the paper-based user evaluation, the implementation evaluation provoked no comments about the perceived problem of forward commitment.

6. DISCUSSION VERTIPH is a multi-view design environment for image processing applications on FPGA devices, which separates the design into a number of views, each with an editor that provides the user with graphical tools tailored to different aspects of the design task.

This paper has described and explained the reasons underlying VERTIPH’s three key Views, and given the results of a paper-based evaluation of the whole language and a user evaluation of an implementation of two Views which are capable of being used as a standalone design environment. An Architectural View provides a data-flow representation of the complete problem and encourages the use of top down design. A Computational View defines a processor’s parallel computations and contains expressions and control structures. Finally, the (as-yet unimplemented) Scheduling View allows the high-level architectural blocks to be scheduled; this allows for high-level control at a global level. Processors can be scheduled to run only during certain defined periods, triggered by events, without relying on the low-level computational control structures.

We identified several requirements in our introduction. VERTIPH allows for a mixture of, and maintains a distinction between, parallel and sequential design, which makes pipelining easier to visualise, as demonstrated in both the Computational View and the Scheduling View.


91

VERTIPH can support streamed, random access and hybrid image-processing modes. Both source- and sink-driven control architectures are supported.

Our paper-based user evaluation gave some positive feedback on the benefits and limitations of different views and their ease of use. We found that there could be some issues with forward commitment and that the expression editing might also need to be modified. It was decided to implement the planned design then do a user evaluation.

Besides the minor criticism of junction boxes, the user evaluation of the implementation of VERTIPH was positive. The user evaluation found that the forward commitment issue identified in the paper based evaluation was not a problem. However, expression editing needs some further refinement.

A Cognitive Dimensions Analysis [15] of the implemented parts of VERTIPH has been conducted and is documented elsewhere [18].

7. References [1] C. T. Johnston, D. G. Bailey, and P. Lyons, 2006 "A Visual

Environment for Real-Time Image Processing in Hardware (VERTIPH)," EURASIP Journal on Embedded Systems, (Article ID 72962), vol. 2006, pp. 1-8.

[2] C. T. Johnston, K. T. Gribbon, and D. G. Bailey, 2004 "Implementing Image Processing Algorithms on FPGAs," Proceedings of the Eleventh Electronics New Zealand Conference, ENZCon’04, (Palmerston North, pp. 118-123, November. 2004.)

[3] V. M. J. Bove, M. Lee, Y.-M. Liu, C. McEniry, T. Nwodah, and J. Watlington, 1999 "Media Processing with Field Programmable Gate Arrays on a Microprocessor’s Local Bus," Proceedings of SPIE Media Processors, (San Jose, CA, pp. 12-20, January 1999).

[4] R. Rinker, J. Hammes, W. A. Najjar, W. Bohm, and B. Draper, 2000 "Compiling image processing applications to reconfigurable hardware," Proceedings. IEEE International Conference on Application-Specific Systems, Architectures, and Processors, (Boston, MA, pp. 56-65, 10-12 July. 2000).

[5] P. Lyons, 1999 "Programming in Several Dimensions," Proc. SoftVis '99, pp. 31-39, 3-4 December 1999.

[6] Accellera, "EDA Industry Working Groups - VHDL,"2008, http://www.vhdl.org/, visited on March 2008.

[7] I. Alston and B. Madahar, 2008 "From C to netlists: hardware engineering for software engineers?," Electronics

& Communication Engineering Journal, vol. 14, no. 4, pp. 165-173, 2002.

[8] ImpulseC, "ImpulseC,"2008, http://www.impulsec.com/, visited on March/2008.

[9] K. Konstantinides and J. R. Rasure, 1994 "The Khoros software development environment for image and signal processing," IEEE Transactions on Image Processing, vol. 3, no. 3, pp. 243-252, 1994.

[10] P. M. Ngan, 1992 "The Development of a Visual Language for Image Processing Applications,"Ph.D. Thesis in Computer Science, Massey University, Palmerston North, New Zealand, 1992.

[11] C. T. Johnston, D. G. Bailey, and K. T. Gribbon, 2005 "Optimisation of a colour segmentation and tracking algorithm for real-time FPGA implementation," Proceedings of Image and Vision Computing New Zealand, (Dunedin, pp. 422-427, 28-29 November. 2005).

[12] Nassi I and Shneiderman B, 1973 "Flowchart techniques for structured programming," ACM SIGPLAN Notices, vol. 8, no. 8, pp. 12-26, 1973.

[13] C. T. Johnston, D. G. Bailey, and P. Lyons, 2006 "Towards a visual notation for pipelining in a visual programming language for programming FPGAs," 7th International Conference of the NZ chapter of the ACM's Special Interest Group on Human-Computer Interaction, pp. 1-9, 2006.

[14] Celoxica, PixelStreams Manual, 1 ed: Celoxica, 2005.

[15] T. R. G. Green and M. Petre, 1996 "Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework," Journal of Visual Languages and Computing, vol. 7, no. 2, pp. 131-174, 1996.

[16] C. T. Johnston, P. Lyons, and D. G. Bailey, 2008 "A Visual Notation for Processor and Resource Scheduling," IEEE International Symposium on Electronic Design, Test and Applications (DELTA 2008), Hong Kong, pp. 296-301, 23-25 January. 2008.

[17] H.-P. Juan, N. D. Holmes, S. Bakshi, and D. D. Gajski, 1993"Top-down modeling of RISC processors in VHDL," European Design Automation Conference, 1993, with EURO-VHDL '93,( Hamburg, Germany, pp. 454-459, 20-24 September. 1993).

[18] C. T. Johnston, 2009 "VERTIPH: A Visual Environment for Real Time Image Processing on Hardware", PhD Thesis in Computer Systems Engineering, School of Engineering and Advanced Technology, Massey University, Palmerston North, 2009.


92

User evaluation and overview of a visual language for real

Documents

Transcript of User evaluation and overview of a visual language for real