p. 1

flippable user interfaces for internationalization iyad khaddam and jean vanderdonckt université catholique de louvain louvain school of management louvain interaction laboratory place des doyens 1 ­ b-1348 louvain-la-neuve belgium {iyad.khaddam jean.vanderdonckt uclouvain.be ­ phone +32 10 478525 abstract the language reading direction is probably one of the most determinant factors influencing the successful internationalization of graphical user interfaces beyond their mere translation western languages are read from left to right and top to bottom while arabic languages and hebrew are read from right to left and top to bottom and oriental languages are read from top to bottom in order to address this challenge we introduce flippable user interfaces that enable the end user to change the reading direction of a graphical user interface by flipping it into the desired reading direction by direct manipulation this operation automatically and dynamically changes the user interface layout based on a generalized concept of reading direction and translates it according to the end user s preferences author keywords one single gui while making a ui local results into many different variations of the same initial gui but adapted to the different cultural backgrounds these different variations are subject to a series of challenges their design should be coordinated [4 culturally-aware [13 maintained simultaneously [18 and applicability of a change request depending on local settings a single change request may indeed affect one or several variations of the same gui many different factors could positively influence a successful localization [4,6,10,11,13,14,15,16,18,21 color format metaphor screen layout language images structure density ordering of information instead of producing several variations of the same initial gui for the different localizations it might be interesting to concentrate the adaptation logic into a single gui that handles these variations depending on setting of the end user a single gui could be produced with adaptation thus addressing the challenge of coordination and simultaneous maintenance but leaving the challenges of culture awareness and dependability open in the adaptation logic one of the main critical factors of success is the adaptation of the gui to the end user s language which includes translation e.g by automated translation of all contents and resources [15 and layout adaptation to the language reading direction [8 this adaptation largely fosters the ui acceptance [8 other aspects such as color font and size are mostly lexical factors and less critical while high-level aspects such as metaphors and organization are hard to predict in a systematic way [18 and their impact depends on many cultural parameters that are hard to reproduce [14 this paper introduces the interaction technique of flippable user interfaces in order to support the adaptation to the end users language which subsumes translation transformation of the layout to support a correct reading direction mixing different directions improving the visual properties of the layout such as balance and symmetry [22 the remainder of this paper is structured as follows the next section reports on some related work then the design principles that underlie flippable user interfaces i.e based on a concept of generalized direction are introduced motivated and exemplified the software architecture supporting the implementation of flippable user interfaces as an adaptation interaction technique for addressing internationalization is discussed finally a conclusion delivers the main points of this research and presents some future avenues especially for the end user s acceptation adaptation cultural background internationalization reading direction user interface layout general terms design experimentation human factors verification acm classification keywords d2.2 [software engineering design tools and techniques ­ user interfaces d2.m [software engineering miscellaneous ­ rapid prototyping reusable software h.5.1 [information interfaces and presentation multimedia information systems ­ animations h5.2 [information interfaces and presentation user interfaces ­ graphical user interfaces gui user-centered design windowing systems introduction localization usually refers to the process of designing and developing a graphical user interface gui that is adapted to a particular culture [6,13 continent [4 country [14 or region [18 or a set of them based on [4 the opposite process called globalization usually refers to the process of designing and developing a gui that accommodates the common ground of the largest possible audience from different cultures continents countries and regions based on [4,18,21 making a ui global [21 therefore results into permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page to copy otherwise or republish to post on servers or to redistribute to lists requires prior specific permission and/or a fee eics 11 june 13­16 2011 pisa italy copyright 2011 acm 978-1-4503-0670-6/11/06 10.00 223

[close]

p. 2

related work this paper is aimed at developing an interaction technique i.e flippable user interface as a support for internationalization i.e adaptation to end user s language of guis with transition i.e animated transition between ui before and after adaptation the following state of the art is structured with respect to these three main fields of research adaptation to cultural background three kinds of gui adaptation are usually performed in order to localize a gui technical adaptations [18 that address the needs for making the gui workable and displayable in the localized context of use e.g by use of appropriate alphabet character set national adaptations that address the needs of particularizing the information and their associated actions to a particular country e.g by adding information relevant to a country only by removing unnecessary menu items for a particular task that does not require it in a specific country and cultural adaptations that address the needs of cultural habitudes conventions and meanings [13,14 while most of the adaptation operations are well documented in the literature they are applied mostly on a case by case way they are rarely applied systematically or encapsulated in an adaptation engine the main goals for adaptation towards localization are [8 14,18 communicate in the country s native language support the natural writing symbols punctuation and so on support native date currency weight scales numbers and addresses support natural work habits and the work environment and communicate in an inoffensive manner again while these principles are largely recognized and widely adopted they are seldom translated into rules that automatically transform guis for a localized purpose in the market we can find well-arabized products and most of them are built on microsoft windows operating system as it was the pioneer in providing support for right-to-left rtl languages or on the web erp products were forced to provide rtl support due to market pressure portenari amara explain the arabic language characteristics and explain the challenges of the language in the context of providing os support for arabic they discuss the issues related to encoding character shaping and the cursive or handwritten style of writing in arabic vowels numbers shapes and the mirroring effect on visual screens rejmer et al [18 discussed the internationalization of a web site that performs automated evaluation of w3c accessibility guidelines based on a set of design guidelines for supporting internationalization/localization these guidelines only address western languages latin alphabet thus ignoring other reading directions but identifying important gui properties that are affected by internationalization xul supports rtl ui [16 by providing the dir property at the ui element which is the base of all elements the dir property can have one of two values normal reverse the normal means position elements in the container according to their order in the xml file the reverse value means position elements in the container according to their reverse order in the xml file xul does not directly address the rtl concept but their concept of reverse fixes the orientation issue on the other side it does not address the control localization nor provide support to it when the dir property is reverse this doesn t imply that the final control to be used is a control that supports rtl xul depends on the rendering framework to determine the final control localized version therefore it provides a localized version of firefox for each language the problem we note with xul approach is that each localized version of the product will have a localized version of the design the xul file is copied for each language this imposes a maintenance/update problem quiroz et al [16 implemented a genetic algorithm for automated generation of gui layouts based on user fatigue based on the xul language by applying this algorithm they demonstrate which layout causes the less fatigue but again the layout if only ltr since xul itself is like that.xaml www.xaml.net is a markup system that underlies user interface components of microsoft s .net framework 3.0 and above xaml supports rtl by adding a flowdirection property to the containers and ui elements that takes one of the values lefttoright righttoleft this causes the expected effect of rtl to be applied on the container and/or the element figure 4 gives an example figure 1 a sample ltr and the localized rtl version in arabic language interaction techniques various interaction techniques have been investigated in human-computer interaction hci that are related to the metaphor of flipping a page of a book this metaphor has been extensively used in hypermedia and hypertext applications since a long time e.g in hypercard and is still used today in multimedia presentations for the following reasons i the flipping gesture is familiar with the activity of browsing a book an album e.g flipalbum [7 or a stack of pages or documents since a simple flip distinguishes forward from backward movement ii the flipping gesture is natural and straightforward to produce iii flip drop and turn are basic operations of geometric symmetry [22 that respectively reflect an image around the y-axis the xaxis perform a 90° rotation to the right and iv flipping a window could reveal additional information related to the window e.g as in sun s looking glass 3d desktop in other words the flipping gesture indicates a direction which is appropriate to denote the direction of reading the 224

[close]

p. 3

flipping metaphor has however different interpretation for western countries where the page is flipped from right to left in order to support the reading direction from left to right and top to bottom `fold and drop [5 is an interaction technique enabling end users to drag an icon from a stack of overlapping windows and drop it onto a possibly hidden window by applying gestures on the windows thus releasing the user to constantly switch from one window to another orimado [9 is a variant of this interaction technique for oriental languages while fold drop and orimado also rely on the metaphor of flipping they are applied to window and maintain the direction from right to left without affecting the ui contents since this is not their goal the idea of rotated and peeling the windows figure 2 and snapped and zipped windows has already being introduced as an interaction technique recommended for manipulating multiple windows more efficiently [1 preliminary investigations [1,23 show that this interaction technique generates a high subjective satisfaction rate not just because it is graphical or easy to use but mainly for its metaphor that is pretty close to the real world animated transitions it is well established [2,7,21,23 that it is generally considered easier for users to maintain a mental model of the data across smooth transitions and less time is spent comprehending the new data presentation in other words an animated transition between two states is appreciated because it supports a progressive transition from the initial state to the final state without any disruption between several visual techniques exist that could be applied to gui design whether it is for one localization [23 or for globalization of these guis [4,22 the two main dimensions of internationalization usability guidelines e.g [4,13,14,18 also exist that reply on animated transition to change the state of a gui depending on its culture [11 design principles of a flippable user interface for internationalization in this section we motivate define and discuss the design principles that led us to rely on flipping an interaction technique augmented by an animated transition as a way to support the change of cultural background here mainly western vs eastern cultures with various ways of reading principle #1 provide handles for direct flipping figure 2 example of the rotated and peeled-back metaphors flip zooming [3 is an interaction technique that consists of splitting a screen into a sequence of objects e.g images fragments of texts or combinations of both types putting the focus on of these objects and letting the end user to flip through the sequence of objects from left to right top to bottom to preserve the context zooming in/out is then used on any object of interest the main drawback of this technique is a constant `touch-and-go between flipping to navigate and zooming in/out to see the details flying [12 is an interaction technique used for quickly browsing a large hypertext in order to gain some insight to features such as organization size depth level of detail and layout based on flipping this technique is not intended to support reading the contents but to provide a first idea of how it is structured even in a non-linear order as we can see from existing work it seems interesting to consider the concept of flippable ui that mimics paperbased operations in order to improve the end user s subjective satisfaction and naturalness of interaction while different operations that mimic page flipping have been introduced none of them really reproduce the flipping of a page in any direction that results into an adaptation to the new state this is why animated transitions are made for convey the change of display in order to adhere to the principles of the direct manipulation it is expected that the flipping interaction technique should be supported by handles figure 3 that indicate the direction where to flip left to right ltr right to left rtl top to bottom ttb or bottom to top btt each flipping direction then indicates the reading order of the gui thus provoking its adaptation to the corresponding cultural background in direct manipulation the objects should have a graphical representation preferably one that is close to the real world if any with an incremental interaction that is fast and reversible for this purpose horizontal handles represented in red in figure 3 control transformations along the x axis while the vertical handles represented in blue in figure 3 control transformations along the y axis an arrow-shaped handle may convey a translation a square-shaped handle may convey a change of scale and a circle-shaped handle represented in green in figure 3 may convey a rotation similarly for a 3d object in space these three handles could support respectively nutation along x axis precession along the y axis and rotation along the z axis for instance the card stack had markers on front side and back side so the user was able to flip it backward and saw the backside of the 3d model the central button represented in grey in figure 3 restores the transformation to its initial stage or identity the ellipse and the lines depict the current status of transformation figure 3 handles for direct flipping 225

[close]

p. 4

horizontal vs vertical direction top-tobottom right-to-left left-to-right rtl and ttb bottomto-top ltr and ttb rtl and btt ltr and btt figure 4 the four possible layouts depending on the generalized direction principle #2 perform adaptation according to flipping from the aforementioned literature we summarize the ui adaptation in basically two ways the orientation the mirroring this consists of changing the reading direction of the contents depending on the end user s native language some languages are read ltr while some others are read rtl in order to generalize this attribute we also introduce the vertical dimension since some languages are read ttb or btt therefore each native language is assigned to a reading direction according to the horizontal dimension ltr vs trl and the vertical dimension ttb vs btt thus covering the four possible cases figure 4 the common direction in western countries e.g ltr and ttb is reproduced in the top right position of figure 4 if the gui is flipped to the left it reverses the order from ltr to rtl if the gui is flipped to the bottom it reverses the order from ttb to btt figure 4 keeps the same contents and language in order to remain understandable automatic translation of the language [15 is a service that could be used for translating the labels instructions messages from one language to another even at run-time after changing the layout several operations are required by the language change the localization the trl localization has special characteristics that do not exist in latin-alphabet languages which makes the localization a challenge we identify from the examples before the following characteristics for rtl localization text localization text localization language encoding and character set alphabet direction switching direction of text writing graphics localization rtl sensitive graphics non-horizontally sym metrical images with text inside other localizable images country flag control localization control rendering for instance a label control should support writing from right to left control behaviour controls should be aware of special behaviour for special keys like pressing enter key in a text area principle #3 ensure smooth transition for adaptation to avoid startling and confusing users we employ smooth slow-in and slow-out transitions [21,23 for every visual change occurred after the adaptation has been performed this animated transition has the advantage to preserve most of the visual aspects of gui widgets horizontally or vertically other animated transitions could be investigated depending on the adaptation operation that has been executed thus opening a door for many different ways to ensure a smooth transition between the gui before and after adaptation to a cultural background here localization animated transitions however suffer from the lag problem [21 that may cause end user frustration if the animated transition is too fast or too slow here a flippable user interface does not suffer too much from the lag problem because the animated transition is performed at run-time while the end user is flipping the gui in the next section we describe how this engineering technique has been implemented flipping at run-time is the target because a user may speak different languages and may want to switch from one language to another because the by default language is not always right even in an option menu because a ui may contain various data simultaneously in different formats and languages and because manually developing different layouts for different languages requires extensive development efforts that are not required by our approach 226

[close]

p. 5

implementation this section describes how the three aforementioned design principles have been implemented together in order to make the concept of flippable user interface operational the software architecture is structured as follows figure 5 1 the widget selector this component is responsible of mapping the nodes of a gui specified in a user interface description language uidl to output components it needs the mapping xml file each specific output type swing html needs a specific map.xml but the widget selector is the same for all maps in our case usixml www.usixml.org was used for testing but other similar uidls could be used in the same way a xml map is structured as follows the root is usimapmodel this contains sub elements cuimap that determine how we will map usixml tags in the cuimodel section of the xml file element selection algorithms are simply saying a set of conditions that determine the element selection the control structure in the usimapmodel is the option tag this tag has 2 sections condition and action the condition is a simple boolean expression that can be written in groovy a java like syntax www.groovy.com which is modeled with a condition tag in usimapmodel file the actions are a set of commands that create the suitable widget and fine tune its properties they are modeled using the maps tag this tag contains an init attribute and that is where we usually create the appropriate widget the child node map is a statement to map a property from a usi node to the mapped widget property 2 the cui tree traversal this is simply a tree traversal algorithm currently we implemented a depthfirst traversal algorithm for parsing the concrete user interface cui if some type of rendering needs to consider width-first traversal then the width-first class needs to be implemented and passed to the engine instead of the default traversal depth-first 3 the merge algorithm this is simply a component that merges the tree of components in a custom way suitable to the final output in the case of swing the components are all of type java.awt.component and the merge algorithm will simply add components to the container a sub class of component but calling the method add 4 the cuirendering engine the engine that orchestrates the messages among the above components the engine calls the traversal to start traversing the uimodel which will parse the uimodel tree looking for usixml nodes for each node it will notify the engine who will call the widget selector to resolve this node if the map.xml file contains a mapping for this node it will create the mapped component and return it to the engine who in turn will store it a special storage called usiruntime the usiruntime contains a tree of rendered objects the usiruntime allows us to retrieve the rendered component by using the id of the relative usixml element this allows 2-ways mapping between original usixml element and the rendered element when finished the engine will call the merge algorithm to merge the resulting tree of components and returns the usiruntime object to the caller the usiruntime now contains the roots components if the usixml file contains 2 window elements usiruntime will contains 2 roots and so on the caller uses these roots for any purpose e.g display on screen save figure 6 shows how the control panel could be displayed for finely governing the layout of the gui by moving the handles according to the principle #1 the end user is able to manipulate the geometry of the gui in order to reverse its reading order but also in order to accommodate constraints imposed by the screen resolution this engineering technique therefore supports the concept of variable geometry layouts that could accommodate different physical constraints imposed by the target computing platform a user interface with variable geometry is hereby defined as a gui exhibiting the capability of altering its layout geometry by direct manipulation depending on the context of use thus making it plastic to some extent map.xml usixml file uimodel merge algorithm widget s elector cuitreetr aversal cuirende ringengine figure 5 the software architecture of the flippable ui figure 6 the control panel of the flippable ui 227

[close]

p. 6

when the end user does not want to finely manipulate the layout geometry she could only flip the gui in the desired direction thus obtaining only the four possibilities depicted in fig 4 along with adaptation techniques executed as in principle #2 principle #3 preserves the continuity between the gui before flipping and after flipping conclusion this paper presented the concept of flippable user interface as an interaction technique for supporting internationalization of graphical user interfaces traditionally internationalization of a gui is achieved by providing different sets of translated resources that are then incorporated in the executable code of the interactive application that is then responsible for switching to one layout to another this method is largely static all resources should be provided in resource files translated and compiled and physically defined all layouts are done manually and embedded in the code instead a flippable user interface introduces an engineering technique where the gui definition is dynamically parsed interpreted and computed depending the default language and country settings of the end user switching from one cultural background to another e.g switching from english to arabic is done performed dynamically by flipping the window by the end user in the desired reading order at runtime such a flippable ui could be flipped in any direction vertical or horizontal all other operations that are subsequent to a change of cultural background are then dynamically performed for all widgets like localization of controls text messages labels etc this approach is much more flexible both for the end user since there is no need to select parameters from a menu and for a designer/developer since all the variations are made flexible in the interpretation mechanism acknowledgments the authors would like to acknowledge the support of the itea2call3-2008026 usixml user interface extensible markup language european project and its support by région wallonne dgo6 as well as the fp7-ict5-258030 serenoa project supported by the european commission references 1 beaudouin-lafon m novel interaction techniques for overlapping windows in proc of uist 2001 acm press new york 2001 pp 153­154 2 bederson b.b and boltman a does animation help users build mental maps of spatial information in proc of ieee symposium on information visualization infovis 99 ieee computer society press los alamitos 1999 pp 28­35 3 björk s holmquist l.e redström j bretan i danielsson r karlgren j and franzén k west a web browser for small terminals in proc of uist 99 asheville nov 710 1999 acm press 1999 pp 187­196 4 delgaldo e.m and nielsen j international user interfaces john wiley sons new york 1996 5 dragicevic p combining crossing-based and paper-based interaction paradigms for dragging and dropping between overlapping windows in proc of uist 2004 santa fe october 24-27 2004 acm press new york pp 193­196 6 evers v and day d the role of culture in interface acceptance in proc of interact 97 sydney july 14-18 1997 chapman hall london 1997 pp 260­267 7 flipalbum http www.flipalbum.com 8 halawani s.m the effect of language reading direction on user interface design phd thesis 1996 9 inaba k orimado 2010 http www.kmonos.net/lib/orimado.en.html visited on december 5th 2010 10 ishida r creating html pages in arabic hebrew and other right-to-left scripts w3 consortium geneva 2002 http www.w3.org/international/tutorials/bidi-xhtml visited on december 5th 2010 11 khaddam i and vanderdonckt j adapting usixml user interfaces to cultural background in proc of 1st int workshop on user interface extensible markup language usixml 2010 berlin 20 june 2010 thales research and technology france paris 2010 pp 163­170 12 lai p and manber u flying through hypertext in proc of 3rd acm conf on hypertext hypertext 91 san antonio dec 15-18 1991 acm press new york 1991 pp 123­132 13 mahemoff m and johnston l culturally-aware requirements for internationalized software in proc of 3rd australian conf on requirements engineering acre`98 geelong october 26-27 1998 d fowler l dawson eds deakin university geelong 1998 pp 83­90 14 marcus a and west gould e cultural dimensions and global web user-interface design interactions 7 4 july 2000 pp 32­46 15 pérez-quiñones m.a padilla-falto o.i and mcdevitt k automatic language translation for user interfaces in proc of the int conf on diversity in computing tapia 2005 albuquerque october 19-22 2005 acm press pp 60­63 16 portaneri f and amara f arabization of graphical user interfaces in [2 pp 127­150 17 quiroz j.c louis s.j and dascalu s.m interactive evolution of xul user interfaces in proc of the 9th annual conf on genetic and evolutionary computation gecco 2007 acm press new york 2007 pp 2151-2158 18 rejmer p cooper m and vanderdonckt j lessons learned from internationalizing a web site accessibility evaluator in proc of 3rd int workshop on internationalisation of products and systems iwips 2001 designing for global markets 3 milton keynes july 11-14 2001 digital printing services milton keynes 2001 pp 61­79 19 sears a layout appropriateness a metric for evaluating user interface widget layout ieee transactions on software engineering 19 7 july 1993 707 719 20 schlienger c conversy s chatty s anquetil m and mertz ch improving users comprehension of changes with animation and sound an empirical assessment in proc of interact 2007 lncs vol 4662 springer pp 207­220 21 stasko j animation in user interfaces principles and techniques in proc of user interface software `93 pp 81­101 22 taylor d global software developing applications for the international market springer-verlag berlin 1992 23 vanderdonckt j and gillo x visual techniques for traditional and multimedia layouts in proc of 2nd acm workshop on advanced visual interfaces avi 94 bari 1-4 june 1994 acm press new york 1994 pp 95­104 228

[close]