Recommendation System for Automatic Design of Magazine Covers

Recommendation System for Automatic Design of

Magazine Covers

Ali Jahanian

, Jerry Liu

, Qian Lin

Daniel R. Tretter

, Eamonn O’Brien-Strain

Seungyon Lee

, Nic Lyons

, Jan P. Allebach

School of Electrical and Computer Engineering

Purdue University

West Lafayette, IN 47907, U.S.A

{ajahania, allebach}@purdue.edu

HP Labs, Palo Alto, CA 94304, U.S.A

{jerry.liu, qian.lin, dan.tretter,

eamonn.obrien-strain, claire.sylee,

nic.lyons}@hp.com

ABSTRACT

In this paper, we present a recommendation system for the

automatic design of magazine covers. Our users are non-

designer designers: individuals or small and medium busi-

nesses who want to design without hiring a professional de-

signer while still wanting to create aesthetically compelling

designs. Because a design should have a purpose, we sug-

gest a number of semantic features to the user, e.g., “clean

and clear,” “dynamic and active,” or “formal,” to describe the

color mood for the purpose of his/her design. Based on these

high level features and a number of low level features, such

as the complexity of the visual balance in a photo, our sys-

tem selects the best photos from the user’s album for his/her

design. Our system then generates several alternative designs

that can be rated by the user. Consequently, our system gen-

erates future designs based on the user’s style. In this fashion,

our system personalizes the designs of a user based on his/her

preferences.

Author Keywords

Visual design principles; aesthetics; layout; visual balance;

color design; autonomous; recommendation; personalization

ACM Classiﬁcation Keywords

H.5.2 Information Interfaces and Presentation: User

Interfaces

H.1.2 Models and Principles User/Machine Systems - human

factors, software psychology

General Terms

Human Factors; Design; Theory.

INTRODUCTION

In the information era, the representation of information via

visual media is a desire for both personal and business means,

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are

not made or distributed for proﬁt or commercial advantage and that copies

bear this notice and the full citation on the ﬁrst page. To copy otherwise, or

republish, to post on servers or to redistribute to lists, requires prior speciﬁc

permission and/or a fee.

IUI’13, March 19–22, 2013, Santa Monica, CA, USA.

though these consumers may prefer not to hire a professional

to create designs for them. Personal communication through

internet web services is becoming an essential part of con-

temporary life. New generations of communication devices

also contribute to a need for the design of personal communi-

cation media. Hence, there is a great need and an increasing

request for software applications that provide users with easy-

to-follow steps for creating visual media, such as magazines,

blogs, and other forms of self-publishing media. This need,

in turn, inspires science and engineering to tackle the idea of

visual media’s automatic design.

An automatic design system, however, needs to take into ac-

count the key concepts of design and address the challenges

that designers encounter when creating a design. The key

concepts include: elements of design, principles in design,

and aesthetics of design. By elements of design, we mean lay-

out and visual balance considerations, typography, design of

color, style, and context. Principles in design, however, refer

to a number of rules that a designer may apply to accomplish

the task of design. Aesthetics of design deals with the beauty

of designs and how good a design is, considering both the

form and function of a design. In fact, an automatic design

system has to quantify these subjective concepts. Moreover,

such a system has to perform the task of design automatically

and autonomously. From a broader perspective, this system

should be able to recommend to the user a number of de-

signs and to customize the future designs for a user based on

his/her personal preferences. Therefore, designing effective

sets of interactions for user participation in the process of au-

tomatic design of visual media based on his/her preferences

is required.

In this paper, we introduce a recommendation system for au-

tomatic design of magazine covers. Professional designers

start a design by developing a concept to convey the message

or purpose of the design. Similarly, our system guides the

user to understand the purpose of his/her design. The pur-

pose of a magazine cover design relates to the context of its

cover image [11]. The context of the cover image has sev-

eral aspects, including the color mood and the objects. At

this stage, we focus on the color mood: we recommend a set

of semantic descriptors to the user to understand his/her pre-

ferred color mood in his/her design. The system then selects

a number of photos from the user’s album based on his/her

preferences and generates alternative designs.

We focus on magazine covers as an application of the idea

of automatic design, especially because of the need for self-

publishing designs in recent years. This need has also been

addressed in prior work, such as [6] and [25]. There are now

several tools and services for design of magazine covers. As

an example, HP MagCloud websites [4] provide individuals

as well as businesses the ability to design and upload maga-

zine covers to share with others. Moreover, there are several

on-line tools and apps for magazine cover design for ama-

teurs, which is in contrast with tools that facilitate profession-

als for the design of magazine covers. Consequently, some

well-known magazines such as Wired and Cosmopolitan have

started offering design of personalized magazine cover as pro-

motions to customers. As an example of other applications of

our work, consider the case for providing non-designer de-

signers with the opportunity to design magazine-like docu-

ments to communicate with speciﬁc target audiences. For in-

stance, Greig et al. [14] suggests a system which creates a

travel brochure, but it has to be enhanced by adding features

that recognize how to design an effective cover. Other poten-

tial users include small non-proﬁt organizations that need to

design magazine covers and other magazine-like documents

to generate excitement and credibility about their work, with-

out hiring professional designers [6]. Furthermore, as we sug-

gest in the discussion section, our work could be considered

as a starting point to contribute to the idea of creativity sup-

port tools for both non-designer and designer users.

Our recommendation system is an extension of the automatic

design system ADoMC, or Automatic Design of Magazine

Covers, by Jahanian et al. [21]. Hence, we call our rec-

ommendation system R-ADoMC. Although we dedicate one

section to ADoMC, we describe each part of R-ADoMC, in-

cluding: Input User Interface, which suggests different pur-

poses of a design to the user; Evaluation of Input Photos,

which deals with the user’s inputs; Designs User Interface,

which provides a rating mechanism of the generated designs

for the user; and Personalization of Designs, which recom-

mends more designs based on the preferences of the user. We

ﬁnally discuss the current limitations, challenges, and the fu-

ture work. In the next section, we illustrate a common sce-

nario that typically occurs with our users.

SCENARIO

Our users are non-designer designers, individuals and busi-

nesses who want to design without hiring a professional de-

signer, while still wanting to create aesthetically compelling

designs. Consider the following scenario: John has a pile of

photos, and he wants to make a magazine cover with the best

photo from the group. John also has a number of stories as

his cover lines that he wants to place on the cover. He has

thought about the priorities of his stories in terms of their im-

portance. He knows what he would like to call his magazine

title, but he may or may not have a predesigned masthead for

his magazine title. John wants to make an attractive design,

but he has no idea about the principles of design. The only

thing that he knows is that he wants a “sporty and dynamic”

design. Designers believe that if someone does not know how

to design, he/she can easily make non-pleasing designs [20].

They also believe that any changes made in a design would

lead to a different design, and the designer would have to start

a new design from scratch. From the above input, R-ADoMC

is implemented to create a design from start to ﬁnish. It does

not allow any modiﬁcations on the user’s part. Instead, it pro-

vides a number of designs to the user and lets him/her choose

one. It also recommends a design based on the preference of

the user. In John’s case, R-ADoMC takes his photos and sto-

ries (i.e. cover lines), and then it suggests a number of adjec-

tives to him that describe some possible kinds of design, e.g.,

“clean and clear”, “dynamic and active”, “elegant”, “formal”,

etc. In the next sections, we discuss these adjectives, their

origins, and the way that we deploy them to create a design.

RELATED WORK

We believe the work of visual media’s automatic design,

magazine covers in particular, is a multidisciplinary task

that spans several different topics, including automated lay-

out, color design, typography, computational aesthetics, and

recommendation with personalization. The following sub-

sections review the related work in each area and how we

apply or extend the prior work.

Automated Layout

Automated layout – how to position the textual and graphical

elements in the format of documents including newspapers

and magazines – has been an active area of research activ-

ity. Lok et al. [28] and Hurst et al. [18] provide surveys of

automated document layout techniques. Most of the related

work is based on the concept of grids in design. We also use

the idea of making and breaking grids, comprehensively dis-

cussed in [33]. The idea of automated layout has recently

been used in design of photo-books [42], web-based maga-

zines [17], and semi-automated magazine layout [25]. The

main difference between the prior work and our work is per-

haps the fact that we apply the layout onto a bitmap, or an

image, as the background, or the canvas, rather than a white

space. This requires dealing with the composition and com-

plexity of the images, as well. In fact, our deﬁnition of white

space is different, since we consider the non-salient part of an

image as the white space. In this paper, we will describe how

we ﬁnd the salient and non-salient parts of an image, and how

we use them to convey visual balance in a layout.

Color Design

Color design is a broad concept in design that ties with sev-

eral well-known color models proposed by color theorists.

In computer science and engineering, researchers have de-

ployed some of these color theories, such as color harmony

and complementary colors. [41] uses Matsuda’s harmonious

hue and tone templates as well as Kobayashi’s color image

scale to support users for the selection of colors for prod-

uct design. [9] also uses Matsuda’s harmonious templates for

color harmonization in images. [31] studies color preferences

provided from two color design websites Adobe Kuler [1]

and COLOURLovers [3]. In our work, however, we use the

well-known color theories of Itten [19], Matsuda [41], and

Figure 1: A schematic view of R-ADoMC. The user uploads his/her photos, cover lines, and masthead. Then, he/she chooses

some recommended descriptors to indicate the desired color mood of his/her design. R-ADoMC evaluates the user’s photos and

chooses the best. It then sends the selected photos to the design engine. The generated designs will be shown to the user, who can

rate the generated designs. The ratings will be considered for the user’s future designs to recommend him/her one more design

based on his/her style.

Kobayashi [32] to evaluate input cover images and to au-

tomatically choose colors for the cover lines of a magazine

cover.

Typography

Typography in automatic layout is often considered as an op-

timization problem [18]. Knuth’s work [23] is perhaps the

most effective solution for typography. Recently, [27] sug-

gests an algorithm based on Knuth’s work for text ﬁtting in

arbitrary geometrical regions for digital publishing. The auto-

matic text insertion in our typography is inspired by both [23]

and [27], while dealing with text ﬁtting in geometrical re-

gions. Typography in design also deals with the aesthetics of

typefaces and which typeface to choose for the purpose of a

design [34]. Although we alternate the typeface of the tex-

tual elements in design of a magazine cover, choosing a good

typeface for the purpose of a design is mentioned in the Dis-

cussion section as our future work.

Computational Aesthetics

Quantifying aesthetics of images is another trend of research.

Birkhoff’s [8] model is perhaps the ﬁrst mathematical expres-

sion for aesthetics. [30] deploys Birkhoff’s model for mea-

suring aesthetics of layout of user interfaces on the screen.

From another perspective, [13] develops a genetic algorithm

for computing aesthetics of photo-albums. Datta et al. [10]

applies a number of machine learning techniques in com-

putational aesthetics of photos. However, machine learning

techniques require deﬁning effective features to quantify aes-

thetics and should be considered based on both function and

form. That is why in our work aesthetics considerations are

part of each module of design, e.g., aesthetics of color de-

sign or visual balance as well as the ﬁnal design. Moreover,

unlike other methods, we consider high level or semantic fea-

tures (e.g. “clean and clear” colors) rather than just low level

features (e.g. lightness of color). From a psychological view-

point, [26] suggests an exploratory factor analysis for this

purpose and lists a number of semantic terms in aesthetics of

design. However, to the best of our knowledge, deployment

of high level features in automatic design has not been con-

sidered in previous work. This is, in fact, part of this paper’s

contribution.

Recommendation Systems

Recommending a generated design from scratch for the user

is another contribution of this paper. [16] discusses how de-

signers can beneﬁt from examples recommended by a rec-

ommending system to support their creativity in new designs.

The main difference, however, is that in our work we create

designs and recommend them to non-designer users. In rec-

ommendation systems, typically, prediction of a user’s prefer-

ences is based on a rating data set, accumulated from the same

user and similar users [5]. In our work, we personalize the

future designs for a user by predicting his/her personal pref-

erences in design and by inferring a style for him/her based

on his/her ratings.

SOFTWARE FRAMEWORK

This section describes how we think about the problem of au-

tomatic design from the viewpoint of software engineering

and computer science. We endeavor to quantify the elements

and concepts in design and bring these subjective topics to the

realm of objection. Our approach includes close collaboration

with professional designers, magazine art directors and edito-

rial boards, and journalists. The current software framework

is the projection of a number of rules, mathematical expres-

sions, and algorithms delineated and devised from the lessons

we have learned so far. The following sub-sections describe

each part of the system in more detail. We ﬁrst start by look-

ing at a schematic view of the framework and the relation of

each part with others.

Schematic View

As Figure 1 illustrates, the system is comprised of three ma-

jor parts: Input User Interface, Automatic Design Engine, and

Figure 2: Descriptors for color mood of design. The user can

tell R-ADoMC which color mood he/she wishes to convey

in his/her design. These descriptors are suggested by [32] to

relate the world of colors to the world of styles and human

preferences.

Designs User Interface. The system ﬁrst takes the user’s in-

puts, which includes a collection of his/her photos, a number

of text strings as his/her magazine cover stories (e.g. headline

and byline), a masthead for the title in the format of a text

string or a predeﬁned logo, and a number of predeﬁned ad-

jectives describing the desired color mood of his/her design.

The system then evaluates the user’s photos based on his/her

chosen adjectives. In this way, the system can order the in-

put photos based on their aesthetics, including both form and

functional aspects. The best photos are then sent to the design

engine. This engine is called ADoMC and is based on work

by Jahanian et al. [21]. The designs generated by ADoMC,

which in turn pass a visual composition criterion, are pre-

sented as the ﬁnal designs to the user by listing them in order

of best design. The user can rate the generated designs using

a ﬁve-star rating scale. The system incorporates the user’s

ratings in the personalization part to customize his/her future

designs. In this fashion, the system will suggest to the user

one more design among the other designs that he/she may like

by indicating that it is based on “your style.”

Input User Interface

A magazine cover has a main photo called a cover image,

a title called a masthead, and a number of stories, including

headline, byline, and other cover lines [11]. The user can

upload his/her cover image, masthead, and cover lines to R-

ADoMC. Users generally have a number of photos and may

wonder which photo could be a good candidate for a maga-

zine cover. On the other hand, since the context of the cover

image inﬂuences the purpose of a magazine cover design, we

recommend a list of adjectives to the user to describe several

color moods in the context of a magazine cover. Although

the context of a photo includes aspects other than color, at

this stage of our work, we address only the color aspect. Af-

ter the user chooses a set of adjectives, or color mood descrip-

tors, R-ADoMC can evaluate the user’s photos, rate them, and

generate alternative designs. Figure 2 illustrates the menu of

the color mood descriptors in R-ADoMC.

Evaluation of Input Photos

Computational aesthetics of photos is one of the recent in-

terests in computer science and engineering. However, most

of the related work is focused on a number of low level fea-

tures in a photo, such as color distribution, texture, and ob-

jects [22]. These features are part of the aesthetics of pho-

tos, but they are context-free. Since a design has a mission

and conveys a purpose to the audience, we also need a set

of high level features to be able to describe a design and

its color mood. Our solution to this challenge is inspired

by the work on the Color Image Scale by Kobayashi [32].

Kobayashi has proposed a semantic view to the aesthetics of

colors to relate the world of colors to the world of objects, life

styles, and human preferences. Therefore, we use his work to

quantify the mood of our designs from a semantic viewpoint.

Kobayashi’s Color Image Scale is an attempt to investigate

the meanings that people may perceive from the colors and

color combinations, from a number of psychological, system-

atic, and pragmatic approaches. This viewpoint comes from

the idea of associating the physical attributes of the colors –

hue, value, and chroma — to a higher level of abstract at-

tributes of colors – warm-cool and soft-hard. Kobayashi and

his institute, Nippon Color and Design Research, have con-

ducted more than a decade of psychophysical experiments on

colors. They asked individuals and groups of people to match

a color and a combination of three colors with an adjective.

Through factor analysis, a semantic differential method, and

cluster analysis, Kobayashi came up with a list of adjectives

as color descriptors or “color-images.” Kobayashi then ap-

plied his scale to different color designs, for example, clothes,

food, and several other industrial design applications [24].

Kobayashi later published his Color Image Scale in a book

with the same name [32]. Today a Japanese website uses

his work in a search engine for images [2]. Recently [37]

has deployed Kobayashi’s scale for image retrieval. We de-

ploy the mathematical framework that [37] develops to im-

plement Kobayashi’s color scale for the purpose of image re-

trieval; hence, we use almost the same notation. We, how-

ever, use this framework to measure and evaluate the input

photos for the purpose of a design. Kobayashi ﬁrst deﬁnes

130 Basic Colors. He then deﬁnes 1170 3-Color Combina-

tions and associates them with 180 adjectives (called color-

images) or Labels. Each 3-Color Combination is also associ-

ated with a ﬁve-star rating that indicates the frequency of its

usage. The Labels are later classiﬁed into 15 Patterns. The

Patterns represent a number of selected terms in fashion and

lifestyle. Kobayashi positions his 3-Color Combinations on a

scale of warm-cool and soft-hard, and based on this arrange-

ment, deﬁnes the boundaries of each Pattern. Figure 3 illus-

trates Kobayashi’s Color Image Scale with the 15 Patterns and

some of his 3-Color Combinations.

Our problem is how to locate a given image on Kobayashi’s

Color Image Scale. First we need to ﬁnd a way to represent

an image in sRGB color space in terms of Kobayashi’s 130

Basic Colors. One way is to represent the image by its his-

togram. In other words, the problem is: how to convert the

sRGB histogram of an image to its corresponding histogram

in Kobayashi’s 130 Basic Colors. To do so, we quantize the

sRGB color space to a smaller space with 8 levels of R, 8 lev-

els of G, and 8 levels of B. This leads to 512 sRGB colors, or

bins. We obtain the Kobayashi’s histogram of an image, de-

noted by h

from the sRGB histogram of the image, denoted

Figure 3: Kobayashi’s Color Image Scale. Image taken

from [32]. The contours show 15 patterns, labeled by all-

capital letters. We use these patterns to describe the desired

color mood of the design.

by h

RGB

, according to the following Equation:

= h

RGB

· T (1)

Here T is the transformation matrix. We construct T by ﬁrst

converting both the quantized sRGB space and the 130 Ba-

sic Colors to CIELab color space. This space is perceptually

uniform, and so we measure the distance between each pair of

colors converted from these two color spaces in terms of their

Euclidean distance in CIELab, denoted by 4E

. We can then

deﬁne the elements of the transformation matrix T :



1 −

+ 0.1 if 4E

≤ 10

0 otherwise

(2)

where r = 1, ..., 512, and l = 1, ..., 130.

Hence, we obtain Kobayashi’s histogram h

of an image.

The next step is computation of the distribution of any 3-

Color Combination in the given image from its corresponding

. To do so, we ﬁrst deﬁne the following notation to refer

to a 3-Color Combination:

(3)



, b



(3)

where 0 ≤ b

≤ 130 denotes the index of a Basic Color.

Now we deﬁne h

as the distribution of the all 3-Color Com-

binations in a given image by:

= [σ

, ..., σ

1170

]

(4)

where

σ = min(h

), h

)) (5)

for n = 1, ..., 1170.

Next, we deﬁne matrix L as an index matrix to associate the

180 Labels to the 1170 3-Color Combinations:

L =

. . .

180

(6)

where L is a 1170 × 180 matrix and



1 if label j is associated to Color Combination i

0 otherwise

(7)

Also, we deﬁne the diagonal matrix R, which contains the

ratings of each 3-Color Combination. Hence, we obtain the

distribution of each Label in an image by h

as:

= h

· R · L (8)

Similar to Equations 6 and 7, we can deﬁne the matrix U as

an index matrix to associate the 15 Patterns to the 180 Labels.

Hence, we obtain the distribution of each Pattern in an image

as follows:

P = h

· U (9)

Finally, given the location of all the 180 Labels on the scale

from the matrix E, we can obtain s, the location of the given

image on the scale from:

s = h

· E

(10)

As an example, Figure 4 illustrates the location of an image

on the Color Image Scale. In fact, we can ﬁnd which of the

Patterns, or in our word, color moods an image holds. The

normalized values of matrix P give us a measure of closeness

or similarity of an image to each Pattern. We use this measure

to rate the user’s photos and compute the best matches to Pat-

terns. As we mentioned, this method deals with the high level

features of an image’s aesthetics. However, we also consider

a low level feature in images, which is the layout composition

of the image. We suggest this feature because the composi-

tion of the cover image heavily inﬂuences the visual balance

and layout of the design of a magazine cover. Therefore, if an

image maintains a busy composition, it will not be included

among the ﬁnal designs. At this stage, we equate the layout

composition of an image with its visual saliency and compute

it with the Graph-Based Visual Saliency algorithm [15].

Figure 4: The location of an image on Kobayashi’s Color Im-

age Scale. This image is considered to be CHIC and MOD-

ERN according to this scale. The scale is taken from [32].

Figure 5: A schematic view of ADoMC. Image reproduced

from [21].

Design Engine

After having a cover image selected by our evaluation mech-

anism, the next action is to design a magazine cover with it.

This section is an overview to the automatic design of mag-

azine covers (ADoMC) system [21]. A design process typi-

cally includes three modules: layout, typography, and design

of color. Similarly, ADoMC quantiﬁes and implements these

modules. Figure 5 illustrates a schematic view of ADoMC.

Layout

As noted in [28], layout can be considered as a process that

tries to solve the problem of positioning the visual elements

while considering or determining their sizes. However, cre-

ating aesthetically pleasing layouts ties with visual balance

considerations in design. Visual balance for designers means

holding the property of “looking right” in a layout and can

be considered as “visual weight” [29]. This term, the work

of [30] in measuring aesthetics of layout, and the grid-based

approach in layout [33], form the quantiﬁcation approach of

the visual balance of magazine cover layout in ADoMC, il-

lustrated in Figure 6. In design, the form of the layout should

follow the functionality of the design. For instance, the layout

(a) (b) (c)

(d) (e)

Figure 6: Layout decisions and visual balance considera-

tions. a) An image. b) GBVS [15] binary mask. c) a gen-

eral template for magazine covers. d) Fusion of general cover

template (Figure 6.c) and GBVS binary mask (Figure 6.d).

ADoMC [21] keeps the green region and discards the brown

region, since the green region looks empty and is a good can-

didate to hold the cover lines. This decision is made by com-

puting the visual balance as shown in part e (Figure 6.e). e)

Computation of visual balance: different weights are assigned

to the cells of the yellow grids on the left and right sides of

the cover image: the darker the cells, the heavier the weights.

Summing the product of each cell’s weight and the number of

salient pixels (white, here) gives us the weights of the left and

right grids. The side with a lighter weight will be considered

as an empty space to hold more elements, such as the texts of

the cover lines. Images reproduced from [21].

of the magazine cover should be in a form that directs the eyes

of the audience through the design [11]. Based on the advice

of designers, ADoMC applies a general template for maga-

zine cover layout (illustrated in Figure 6.c). This template,

however, can be reformed based on the visual balance of the

cover image. For visual balance considerations, ADoMC ﬁrst

ﬁnds the white spaces of the cover image. Here, white space

on the background image means the regions of the image

which are less busy relative to the other regions of the im-

age. That is, ADoMC ﬁnds the non-salient segments of the

image by deployment of the Graph-Based Visual Saliency al-

gorithm (GBVS) [15]. ADoMC then fuses the binary GBVS

mask with the general magazine cover template. The result

yields the candidate regions for holding the textual elements

on the cover. Finally, ADoMC applies grids on the image

and assigns different weights to different cells of the grids to

compute which candidate region should hold the textual ele-

ments. In this fashion, ADoMC conveys the visual balance in

the layout of magazine covers.

(a) (b)

Figure 7: Typography of cover lines for a magazine cover.

a) Given the green region computed from Figure 6 as a can-

didate to hold cover lines, ADoMC [21] ﬁts the text of the

cover lines in it. It applies a form of text indentation to follow

the boundaries of the green region. However, this process is

performed in a way such that the size of the text is as large as

possible for the purpose of legibility. b) Result of the auto-

matic text ﬁtting in the green region in (a). Images reproduced

from [21].

(a) (b) (c)

Figure 8: Color Design for masthead and cover lines. a)

ADoMC [21] incorporates this complementary color wheel

based on a continuous version of Itten’s hue wheel. The

crossed region inside the inner disc is discarded and replaced

with the corresponding sectors in the outer ring. b) ADoMC

uses these two templates of Matsuda’s hue templates. The

colors in the shaded sectors are considered to be in harmony.

c) ADoMC uses these two templates of Matsuda’s tone tem-

plates. The colors in the black arcs are considered to be in

harmony. Images of (b) and (c) are reproduced from [9].

Typography

Where and how to put text, including the masthead, head-

line, byline, and other cover lines, is an important decision

in design of magazine covers [11]. In our problem scope,

by typography we mean the decisions that a designer makes

about using different typefaces for different design purposes,

the text size of the cover lines, priorities of the cover lines,

and the problem of legibility of the text. According to profes-

sionals, the family of Helvetica is a good choice for a typical

magazine cover. Better choices, however, relate to the class

of magazine and will be considered in our future work. Simi-

lar to the preceding discussion for layout, we need to convey

the priority and the hierarchy of the cover lines on the cover.

Therefore, ADoMC assigns different sizes to the cover lines

and lays them out based on their priority (speciﬁed by the

user) starting from the top of the cover. Given a candidate

region from the layout process, ADoMC inserts the text by

applying a special form of indentation. This indentation is

a pattern derived from well-known magazine designs. That

is, designers usually give an indentation to the texts to align

them with the boundaries of the salient segments of the cover

image. This technique can be formulated as how to insert

or ﬁt a text in a geometrical region such that its indentation

is most faithful to the shape of the region. ADoMC solves

this problem as an optimization problem in which the maxi-

mum ﬁdelity to the shape of the geometrical region and the

maximum size of the text is desirable. Figure 7 illustrates the

results of this solution for different numbers of cover lines

with different words.

Color

Design of color for textual and graphical elements on a

magazine cover is another challenge for magazine art direc-

tors [11]. In the automatic process of color design, one needs

to address both aesthetics and legibility of the color text. In

order to tackle the aesthetics of colors, ADoMC draws upon

the well-known theories of color harmony. One of the most

signiﬁcant works in the area of quantifying aesthetics of col-

ors is Itten’s color harmony concept [19]. Itten deﬁnes seven

kinds of color contrast: contrast of hue, light-dark contrast,

cold-warm contrast, complementary contrast, simultaneous

contrast, contrast of saturation, and contrast of extension.

Among these contrasts, ADoMC applies the concept of com-

plementary contrast to choose the color of the masthead. For

the color of cover lines, ADoMC then uses Matsuda’s hue

templates [41] and [9], which are perhaps an extension of It-

ten’s color harmony. The problem of illegible text may oc-

cur when the hue or the tone of the text in the foreground

competes with the hue or the tone of its local background.

Hence, for legibility of the text, Matsuda’s tone templates are

deployed as well as the hue templates. Figure 8.a illustrates

how ADoMC customizes a continuous version of Itten’s hue

wheel. Figures 8.b and 8.c also illustrate the four (among

eighteen) of Matsuda’s hue and tone templates that are used.

The reason for this choice is that these templates are more

likely to occur in the contemporary context of well-known

magazine cover designs, as analyzed and suggested by pro-

fessional designers [20].

Designs User Interface

Getting the user to effectively visualize the results is a chal-

lenge when designing a user interface for recommendation

systems. This, in turn, inﬂuences the personalization of the

user’s results. In our problem, we aim to present more than

one design to the user in order to understand his/her design’s

purpose and to customize his/her future designs. We believe

this is an added value for non-designers because it provides

them designs that are aligned with their personal preferences

and styles. In this fashion, we devise an interactive process

between the automatic system and the user while creating a

design, which also happens in practice between a professional

designer and his/her client. Figure 9 illustrates R-ADoMC’s

user interface. R-ADoMC displays the ﬁrst choice design in

the main GUI and arranges two more design choices, along

with a design based on the user’s accumulated ratings in a

separate panel. It also provides a ﬁve-star rating scale op-

tion by the user for these designs [38] provides a number of

guidelines for choosing a rating scale, the options being unary

(“like it”), binary (thumbs up/thumbs down), ﬁve-star, or a

Figure 9: Designs UI for visualization of the designs gener-

ated in R-ADoMC. A rating option is available for each de-

sign, and the ratings will be considered in the user’s future

designs. The “your style” design is a representation of the

idea of personalized designs for the user. The more that the

user works with the system, the more that the system is able

to personalize the future designs.

100-point slider. The degree of accuracy and user friendli-

ness of the ﬁve-star method is a good choice for our work. In

this way, we accumulate a better understanding of what kind

of design the user likes, and consequently, we can customize

his/her future designs.

Personalization of Designs

Is it legitimate to personalize designs? Although form follows

function – or the context of a design should be aligned with

the mission of the design – it can also be personalized. In fact,

one way of thinking about personalization of a design is to

think about the design’s style, which comes from the prefer-

ences of a designer. There is a consensus between artists that

each professional has his/her own style or signature, making

some of the designers identiﬁable based on their designs even

without their explicit identiﬁcation. On the other hand, there

might be alternative good designs for a speciﬁc purpose, and

choosing one from several, ties with the personal preferences

of the user. Hence, providing alternative designs and allowing

a user to choose one design based on his/her personal prefer-

ences is an added value to the function of automatic design

systems. Accordingly, R-ADoMC implements the idea of de-

sign personalization to support the user. As the user works

more with R-ADoMC and rates the generated designs, the

rating data is accumulated for future designs.

There are three different approaches for recommending an

item in current recommendation systems, content-based, col-

laborative ﬁltering, and a combination of both these known

as hybrid [5]. At this stage, our system does not consider any

collaborative ﬁltering because its personalization procedure

is single-user based rather than multiple-user based. As Fig-

ure 9 illustrates, the system suggests the personalized design

as an extra design, labeled as “your style.” To anticipate this

personalized design from the previous ratings of the user, we

deﬁne the prediction of the ratings of the generated designs

as a linear regression problem. To solve this problem, we use

the LASSO algorithm [40] with L

regularization.

EXPERIMENTAL RESULTS

Figure 10 illustrates several ordered results. Generally speak-

ing, the results generated by ADoMC are acceptable to our

professional designers. However, a set of psychophysical ex-

periments to evaluate the results by users is part of our future

work. The system is implemented in Matlab and takes a few

minutes to present the results (for a photo album of about 100

images), on a 64-bit PC machine with a 2-core 3GHz Intel

processor and 4GB of RAM.

DISCUSSION

R-ADoMC is a recommendation system for the automatic de-

sign of magazine covers, and the purpose of this system is to

aid the non-designer designer in creating aesthetically com-

pelling designs. After a user uploads his/her photos, the sys-

tem ﬁrst suggests a number of semantic features, e.g., “el-

egant,” “dynamic and active,” or “sporty,” to describe the

color mood for the purpose of his/her design. Based on these

high level features and a number of low level features, such

as the complexity of a photo’s visual balance, our system

then selects the most suitable photos from the user’s album

for his/her design. Following this, R-ADoMC generates sev-

eral alternative designs that can be rated by the user. Conse-

quently, the system generates future designs grounded in the

user’s style, thus personalizing the designs based on his/her

preferences.

The feedback that we have received from professional design-

ers suggests that our automatic designs are aligned with their

intuition. However, there is a need to devise a solid frame-

work to evaluate R-ADoMC’s generated results. This prob-

lem is highly correlated with aesthetics measurements. Quan-

tifying aesthetics is still a challenge in computer science and

engineering. In our case, while seeking a good solution by

discussing this problem with our partners in the School of

Visual and Performing Arts at Purdue University, we found

that there might be several alternative good designs. So, we

might be able to restrict the problem to what is a bad design.

That inspired us to follow the principles of design. We be-

lieve that our work, among other previous work, emphasizes

the need of devising and designing a set of measures and user

experiments to evaluate aesthetics of the visual design. It is

also worthwhile to mention that one dimension of aesthetics

is beauty, which might be perceived through personal inter-

est and experience, as suggested by Frohlich in “Beauty as a

Design Prize” [12] and also by Barthes [7]. This viewpoint

supports the notion of personalized design, or “Your Style,”

which, as suggested in this paper, may leverage the transcen-

dental experience of beauty. Moreover, as Frohlich suggests,

the perception of beauty can be stimulated by the inclusion of

a more subjective experience in design. This is another sup-

port for the idea of recommendation for automatic design, as

it considers the user’s preferences. In fact, in interaction de-

sign, engaging the non-designer designer in the process of a

design’s creation by having a personal experience can arouse

the experience of beauty, and hence, aesthetics.

(a) (b) (c)

(d) (e) (f)

(g) (h) (i)

Figure 10: Generated designs by R-ADoMC. The results are ranked from the ﬁrst to third choice of design, from the left column

to the right. First row shows a “Pretty”, “Romantic”, and “Elegant” color mood. Second row shows a “Classic”, “Natural” and

“Cool-Casual” color mood. Third row shows a “Dynamic”, and “Modern” color mood. Photos courtesy of Andrew Hancock (a,

e, g, h), Sean Molin (c, d, f, i), and Martin O’Connell (b).

Although our framework provides personalized designs, we

endeavor to develop a framework which generates designs

that are effective in both the function and form of aesthet-

ics. Keeping this in mind, there are a number of limitations

in R-ADoMC that must be addressed. One limitation is that

we reduce the dimensionality of a design’s purpose to the

color mood dimension, as well as visual complexity of the

cover image. Although these two dimensions are important

and inﬂuence the aesthetics of a design heavily, there exist

other dimensions such as the objects in the cover image. This

correlates with the context evaluations of the photos, and so

these contextual factors need to be further investigated. An-

other limitation includes the hypothetical case that a user has

an album of similar photos, e.g., all the photos are classiﬁed

as modern in his/her album. While our system would provide

several designs, we still require another feature or set of fea-

tures to identify the ranking of these designs based on what

the user needs. An additional limitation is that we do not con-

sider speciﬁc typefaces for our typography, and this problem

ties with the class of magazine, which contributes to deﬁning

a magazine’s style. We aim to address design based on the

class of magazines within our future work.

Our current system, R-ADoMC, is a software framework

meant to emulate the process that a professional designer at-

tends to when creating a design. In fact, one of the key com-

ponents of this framework is developing an interactive pro-

cess between the software and the user to understand what

kind of design the user needs or wants. As also indicated

by Swearingen et al. [39], understanding the user’s needs is

important in designing the interactions of recommendation

systems. On the other hand, designing interactions for rec-

ommendation systems can also be expanded to contribute to

the idea of creativity support tools. Creativity support tools

facilitate users to “explore, discover, imagine, innovate, com-

pose, and collaborate” [35]. However, as indicated by Shnei-

derman [36], creative work often begins by viewing previous

projects and similar samples. Since R-ADoMC recommends

alternative designs to the user based on his/her preferences, it

can be an underlying framework for a creativity support sys-

tem in visual design. If the user is unaware of key concepts in

design or principles of design, it is an added value to recom-

mend these semantic descriptors that make sense to him/her

and help to describe a design. It is, however, more ambi-

tious to escalate the creativity of our non-designer user in the

design of visual media. This desire requires a series of effec-

tive interactions between the user and the system to include

the user in the creative process of his/her design. Hence, we

endeavor to evolve R-ADoMC’s framework into a creativity

support tool for the process of visual design and concept de-

sign for media.

ACKNOWLEDGEMENT

We gratefully thank David Sigman, Head of The Patti and

Rusty Rueff School of Visual and Performing Arts at Purdue

University, Hector Sanchez, Design Director of Indianapo-

lis Monthly Magazine, Ryan Kough Gibboney, Art Director

of Purdue Alumnus Magazine, and Petronio A. Bendito, As-

sistant Professor of Visual Communication Design, The Patti

and Rusty Rueff School of Visual and Performing Arts at Pur-

due University, for their input on our automatic designs. We

also gratefully thank Elana Cutter, from the Department of

English at Purdue University, for her editorial reviews and

creative words, which we used in the automatic design sam-

ples. This work is supported by HP Labs, Palo Alto.

REFERENCES

1. Adobe Kuler. http://kuler.adobe.com/.

2. amanaimages EVE. http://amanaimages.com/eve/.

3. COLOURlovers. http://www.colourlovers.com/.

4. HP MagCloud. http://www.magcloud.com/.

5. Adomavicius, G., and Tuzhilin, A. Toward the next

generation of recommender systems: A survey of the

state-of-the-art and possible extensions. Knowledge and

Data Engineering, IEEE Transactions on 17, 6 (2005),

734–749.

6. Allebach, J., Segovia, M., Atkins, C., O’Brien-Strain,

E., Damera-Venkata, N., Bhatti, N., Liu, J., and Lin, Q.

An investigation of document aesthetics for web-to-print

repurposing of small-medium business marketing

collateral. In Society of Photo-Optical Instrumentation

Engineers (SPIE) Conference Series, vol. 7540 (2010),

13.

7. Barthes, R. Camera lucida: Reﬂections on photography.

Hill and Wang, 1982.

8. Birkhoff, G. D. Aesthetics Measure. Harvard University

Press, 1933.

9. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and

Xu, Y. Color harmonization. In ACM Transactions on

Graphics (TOG), vol. 25, ACM (2006), 624–630.

10. Datta, R., Joshi, D., Li, J., and Wang, J. Studying

aesthetics in photographic images using a computational

approach. Computer Vision–ECCV 2006 (2006),

288–301.

11. Foges, C. Magazine Design. RotoVision Press,

Switzerland, 1999.

12. Frohlich, D. Beauty as a design prize.

Human–Computer Interaction 19, 4 (2004), 359–366.

13. Geigel, J., and Loui, A. Automatic page layout using

genetic algorithms for electronic albuming. In Photonics

West 2001-Electronic Imaging, International Society for

Optics and Photonics (2000), 79–90.

14. Greig, D., Hunter, A., and Slatter, D. Layout hierarchies

for interactive design reuse. In Society of Photo-Optical

Instrumentation Engineers (SPIE) Conference Series,

vol. 8302 (2012), 12.

15. Harel, J., Koch, C., and Perona, P. Graph-based visual

saliency. In Advances in Neural Information Processing

Systems 19, MIT Press (2007), 545–552.

16. Herring, S., Chang, C., Krantzler, J., and Bailey, B.

Getting inspired!: understanding how and why examples

are used in creative design practice. In Proceedings of

the 27th International Conference on Human factors in

Computing Systems, ACM (2009), 87–96.

17. Hunter, A., Slatter, D., and Greig, D. Web-based

magazine design for self publishers. In IS&T/SPIE

Electronic Imaging, International Society for Optics and

Photonics (2011), 787902–787902–6.

18. Hurst, N., Li, W., and Marriott, K. Review of automatic

document formatting. In Proceedings of the 9th ACM

Symposium on Document Engineering (DocEng’09)

(2009), 99–108.

19. Itten, J. The Art of Color. Wiley, 1974.

20. Jahanian, A. Automatic magazine cover design.

Master’s thesis, Purdue University, 2011.

21. Jahanian, A., Liu, J., Tretter, D., Lin, Q.,

Damera-Venkata, N., O’Brien-Strain, E., Lee, S., Fan, J.,

and Allebach, J. Automatic design of magazine covers.

In IS&T/SPIE Electronic Imaging, International Society

for Optics and Photonics (2012), 83020N–83020N–8.

22. Joshi, D., Datta, R., Fedorovskaya, E., Luong, Q., Wang,

J., Li, J., and Luo, J. Aesthetics and emotions in images.

Signal Processing Magazine, IEEE 28, 5 (2011),

94–115.

23. Knuth, D., and Plass, M. Breaking paragraphs into lines.

Software: Practice and Experience 11, 11 (1981),

1119–1184.

24. Kobayashi, S. The aim and method of the color image

scale. Color Research & Application 6, 2 (1981),

93–107.

25. Kuhna, M., Kivel

a, I.-M., and Oittinen, P.

Semi-automated magazine layout using content-based

image features. In Proceedings of the 20th ACM

international conference on Multimedia, MM ’12, ACM

(2012), 379–388.

26. Lavie, T., and Tractinsky, N. Assessing dimensions of

perceived visual aesthetics of web sites. International

Journal of Human-Computer Studies 60, 3 (2004),

269–298.

27. Lin, X. Intelligent content ﬁtting for digital publishing.

In Electronic Imaging 2006, International Society for

Optics and Photonics (2006), 60760J–60760J.

28. Lok, S., and Feiner, S. A survey of automated layout

techniques for information presentations. In Proceedings

of Smart Graphics, vol. 2001 (2001), 61–68.

29. Lok, S., Feiner, S., and Ngai, G. Evaluation of visual

balance for automated layout. In Proceedings of the 9th

International Conference on Intelligent User Interfaces,

ACM (2004), 101–108.

30. Ngo, D., Samsudin, A., and Abdullah, R. Aesthetic

measures for assessing graphic screens. Journal of

Information Science and Engineering 16, 1 (2000),

97–116.

31. O’Donovan, P., Agarwala, A., and Hertzmann, A. Color

compatibility from large datasets. ACM Transactions on

Graphics (TOG) 30, 4 (2011), 63.

32. S., K. Color Image Scale. Kodansha Intern, 1991.

33. Samara, T. Making and breaking the grid: a graphic

design layout workshop. Rockport Publishers, 2005.

34. Samara, T. Typography workbook: A real-world guide to

using type in graphic design. Rockport Publishers, 2006.

35. Shneiderman, B. Leonardo’s laptop: human needs and

the new computing technologies. MIT Press, 2003.

36. Shneiderman, B. Creativity support tools: A grand

challenge for hci researchers. Engineering the User

Interface (2009), 1–9.

37. Solli, M., and Lenz, R. Color semantics for image

indexing. In CGIV 2010: 5th European Conference on

Colour in Graphics, Imaging, and Vision, IS&T (2010),

353–358.

38. Sparling, E., and Sen, S. Rating: how difﬁcult is it? In

Proceedings of the Fifth ACM Conference on

Recommender Systems, ACM (2011), 149–156.

39. Swearingen, K., and Sinha, R. Interaction design for

recommender systems. In Designing Interactive

Systems, vol. 6, ACM Press (2002), 312–334.

40. Tibshirani, R. Regression shrinkage and selection via the

lasso. Journal of the Royal Statistical Society. Series B

(Methodological) (1996), 267–288.

41. Tokumaru, M., Muranaka, N., and Imanishi, S. Color

design support system considering color harmony. In

Fuzzy Systems, 2002. FUZZ-IEEE’02. Proceedings of

the 2002 IEEE International Conference on, vol. 1,

IEEE (2002), 378–383.

42. Zhang, X., Gao, Y., Atkins, C. B., Cheatle, P., Xiao, J.,

Chao, H., Wu, P., Tretter, D., Slatter, D., Carter, A.,

Penny, R., and Willis, C. Autophotobook: using

technology to streamline photobook creation. In

Proceedings of the SPIE, vol. 7540 of Society of

Photo-Optical Instrumentation Engineers (SPIE)

Conference Series (2010), 75400L–75400L–15.