Generator¶

The role of the Generator module is to automatically produce designs out of the encoded representations, user inputs and constraints. The ways to produce designs, uses, algorithms, and levels of control are numerous. In order to identify the possibilities of the Generator module it is helpful to consider properties of a taxonomy of generators and try to identify which of these properties are important, or possible, for the Generator module. We refer to the taxonomy of generators from the framework of Procedurally Generated Content (PCG) defined in [ST16] and try to adapt it to the AEC domain. The main practical choices detailed in the taxonomy are:

Online vs. offline: a generator can produce content while the designer is interacting with the tool (online) or in batches during downtime (offline). Online generation “places two or three main requirements on the algorithm: that it is very fast, that it has a predictable runtime and (depending on the context) that its results are of a predictable quality” [ST16]. In principle, generation for a design tool should be online as the designer should immediately receive generated content as a response to their current action. In theory, however, an offline generator could run a priori for many hours, producing a large database of diverse possible designs and then, during the design process, the most appropriate design could be chosen from the database and shown to the user.
Necessary vs. optional content: a generator can produce elements that are central to the design’s function (necessary) or elements which are mostly decorative and could be either omitted or replaced by a different element without affecting the design’s function (optional). In terms of design, an example of collection of necessary content is a building’s layout where all rooms must be connected through doorways, the doors should be high enough for residents to pass through, and so on, while the doors’ wood color or a room’s wallpaper motif is optional content.
Random seeds versus parameter vectors: the level of control that the generator has over the content affects both the type of designs that can be produced and the balance between computational and human initiative. Parametric architecture has already shown how a parameter vector can affect a design; however, the level of control of the generator remains pertinent. A well-defined, narrow set of design parameters can allow the generator to search for optimal designs without making large changes to a human-authored design, but this approach sacrifices diversity. On the other hand, allowing for more freedom, either through larger parameter spaces or from more flexible representations, can lead to novel and creative solutions that local search would not be able to discover.
Stochastic vs. deterministic generation: the degree of randomness (stochasticity) affects the potential of the generator to discover new and creative solutions, but also contributes to the risk of dramatic changes that can move the solution far from the designer’s intentions. Deterministic processes allow the designer to anticipate what the generator will do, and are thus easier to explain. They also might be more appropriate for transformations of one type of content into another (e.g. a hand-drawn sketch to a floor layout). On the other hand, stochastic processes allow the infusion of contingency and surprise in the generation process, a crucial element of a truly expressive generator.
Constructive vs. generate-and-test: the types of algorithms in PCG are often distinguished between those that perform “operations, or sequences of operations, that are guaranteed to never produce broken content’” [ST16] (constructive) versus those that evaluate the quality of the produced content and re-generate a new solution if it is of insufficient quality (generate-and-test).

It is interesting to think of generative design in the context of the AEC through the above properties of the taxonomy proposed. Concerning the way designs are generated, almost by default, the offline method is utilized in generative design settings due to the way the popular design software (Rhino/GH, Revit/Dynamo, Forge) work. Given the expensive evaluation costs for most interesting metrics we want to quantify in the AEC, the offline setting does seem to be currently the best candidate for large scale design exploration, which is what the current proposed systems endeavors to develop, although online generation as a second step would be an interesting addition to the system allowing for mixed-initiative design elements, where the human designer investigates and alters the offline generated collection. When it comes to content, things are much clearer. For architectural and engineering design most, if not all, content produced during early concept phases is necessary. There are very few aesthetic elements that we would care to incorporate in conceptual design at that stage. In lieu of content generation, it’s also hard to imagine many cases where random parameter vectors of any kind could produce meaningful and valid designs. Instead, we expect that at least several geometric primitives (points, lines, polygons, and so on) will have to be present in order to be able to generate and evolve complex designs. Things get more interesting when we consider the horizon of the generative and search processes. At this point in time, most design generation happens strictly in the deterministic domain, again mostly due to the restrictions imposed by the design software and partly by the paradigm of parametric design that is currently popular in practice. Some more stochastic approaches do exist, but those are scarce and typically found in the academic domain, with practical applications preferring instead ease of use and transparency of generative methods. Finally, for very similar reasons, the constructive approach is predominant in the AEC. In fact, determinism and constructivism seem to be the two philosophical pillars of the current state-of-the-art, practical generative design, something that severely limits the potential expressivity of generators and their ability to find truly new, interesting, and well-performing designs.

The most influential choice for the Generator of course is the one of the generative algorithms used to generate designs. There are multitude of generative algorithms that have been used for content generation, across many domains, with varying implementation details and implications for search. Jie Lim et al. [SJLW20] use Cellular Automata (CA) for the generation of mass-customized building blocks in Jakarta’s kampungs (informal urban settlements). A combination of bottom-up rule sets for the CA and top-down strategies of development is used in a process that aims to improve the environmental performance of generated morphologies. Christiane and Thomas [HK05] use CA to generate high-density building form. In contrast to typical CA approaches, their research focuses on the re-modelling of an existing project, showing the value of CA on a more practical real-life design task. Koenig and Bauriedel [KB09] present a CA method to steer precisely the generation of settlement structures with regard to their global and local density as well as the size and number of forming clusters, allowing for precise control over the developing structure, an important functionality when using stochastic generation processes like CA.

Much more research has been done on evolutionary processes for design generation, and especially Genetic Algorithms (GA) in the AEC. Schneider et al. [SFKonig11] present a creative evolutionary methodology for layout problems that explores the design space across the dimensions of intuition and rationality in a variety of ways. The paper highlights the importance of identifying important behavioral descriptors of the design space that relate to the human designer. Byrne et al. [BFH+11] evolve designs by their ability to handle physical, structural, constraints in a process where the physical constraints interact with aesthetic considerations of the human designer. They combine the evolutionary approach with a design grammar that is capable of evolving aesthetically pleasing designs. In another Shape Grammar (SG) approach, Lipp et al. [LWW08] introduce a real-time interactive editing paradigm that allows the human designer to create rules in an intuitive manner. The work explores the important MID implications of real-time generation of such systems. Bielik et al. [BSK12] use DecodingSpaces, a graph-based analysis toolset, to generate a variety of different urban designs according to detailed design parameters and properties of the urban area. For the AEC domain, tools that enable the generation of an initial seed population of designs might prove to be critical for large-scale generative design. Recently, Nauata et al. [NCC+20] developed House-GAN, a relational Generative Adversarial Network model for graph-constrained layout generation which improved the state-0f-the-art in layout generation. They also provide an intuitive interface through which the designer can, in real-time, interactively draw adjacency graphs and see generated layouts. DL models will potentially be crucial for the Generator going forward, especially in design domains where evolution of a design is complex and cumbersome, and can potentially provide a streamlined way to generate design alternatives.

A lot of parallel research in all of these domains has also been conducted in the PCG and Gaming AI field (see for example [KaravolosLiapisYannakakis18], [SSN+], [LYT13], [SYT10] among many others) with a lot of interesting results and generative design approaches. But the most important, newly emerging, domain of research, focusing on a new approach of expressive generation of design alternatives with a focus on design exploration, is that of Quality Diversity (QD). The QD field was initiated by the seminal work of Stanley and Lehman [LehmanStanley11] on the idea of Novelty Search (NS), searching through a design space without objective. This has lead to an explosion of research around NS, which coupled to the idea of quality (in place of objective) creates the notion of Quality-Diversity. The goal of QD is to create a process that produces a diversity of organisms (or designs) that are all high performing in their own niche. The most successful algorithm that came out of QD is MAP-Elites by Clune and Mouret [MC15] which develops a collection of well-performing designs (elites) across a discretized behavioral space. QD and MAP-Elites revolutionize the way we approach design exploration and optimization in the AEC, by focusing on exploration while still finding both interesting and well-performing designs. Instead of one, or a front of, good solution(s) as with typical evolutionary approaches, QD finds a whole collection of designs which can have different characteristics and in fact could be useful in different design tasks and constraints. QD will be the focus of the generation-evaluation-selection process in the proposed system and will heavily influence the inner workings of the Generator and Critic modules.

The Generator, like all other modules of the system, comes with its own set of challenges and opportunities. A core challenge is whether it can consistently create content that is coherent, valid, original, creative, and performative. Architectural design has always been considered too difficult to do well in an automated way. This is where the QD approach can prove significant in allowing for proper exploration of diverse design spaces. Additionally, the need for consistency and computational speed is paramount. Furthermore, the level of control of a generator is also important, especially if a proper MID interface is to be developed. However, having more parameters that can affect the generator may also create broken content which increases the computational effort needed from the generator and mental effort by the human designer. Relatedly, efficient data pipelines and data-efficient representations need to be developed to allow quick transfer of data between the generator and other system modules. While individual designs are typically quite light-weight, having design spaces that range in the 100s of thousands of alternatives can become a bottleneck and has to be dealt with during system design.

Just like before, one of the biggest opportunities in the Generator module seems to be related with learning, in this case the being able to learn how to express specific architectural styles and typologies, and even specific styles of the human designer through the evaluation of his decisions. How to transfer that knowledge to other tasks is also an interesting research and might be a way to speed up subsequent design generation tasks. In combination with the Learner module, the Generator could use (DL) methods to find new ways to generate content. One such method is the “forward pass” method which involves the following process:

Generate a collection of designs without performance evaluation.
Train a model that learns an embedding of the latent space of the design collection generated.
Use the pretrained model to guide generation in two ways:
- unsupervised generation, where novelty can be defined as distance between designs in the latent space;
- supervised generation, creating ‘nobs’ that allow the user to search across the dimensions of the latent space instead of the input space.
The design of the proposed system enables the investigation of several of these ideas, some of which may become crucial for the generative design going forward.

References:

BSK12: Martin Bielik, Sven Schneider, and Reinhard König. Parametric urban patterns: exploring and integrating graph-based spatial properties in parametric urban modelling. In volume. 2012.
BFH+11: Jonathan Byrne, Michael Fenton, Erik Hemberg, James Mcdermott, Michael O’Neill, Elizabeth Shotton, and Ciaran Mcnally. Combining structural analysis and multi-objective criteria for evolutionary architectural design. In volume 6625, 204–213. 04 2011. doi:10.1007/978-3-642-20520-0_21.
HK05: Christiane M. Herr and Thomas Kvan. Using cellular automata to generate high-density building form. In Bob Martens and Andre Brown, editors, Computer Aided Architectural Design Futures 2005, 249–258. Dordrecht, 2005. Springer Netherlands.
KB09: Reinhard Koenig and Christian Bauriedel. Generating settlement structures: a method for urban planning and analysis supported by cellular automata. Environment and Planning B: Planning and Design, 36(4):602–624, 2009. doi:10.1068/b34025.
LYT13: Antonios Liapis, Georgios Yannakakis, and Julian Togelius. Enhancements to constrained novelty search: two-population novelty search for generating game content. In GECCO 2013 - Proceedings of the 2013 Genetic and Evolutionary Computation Conference, 343–350. 07 2013. doi:10.1145/2463372.2463416.
LWW08: Markus Lipp, Peter Wonka, and Michael Wimmer. Interactive visual editing of grammars for procedural architecture. Association for Computing Machinery, 2008. doi:10.1145/1360612.1360701.
MC15: Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. 2015. arXiv:1504.04909.
NCC+20: Nelson Nauata, Kai-Hung Chang, Chin-Yi Cheng, Greg Mori, and Yasutaka Furukawa. House-gan: relational generative adversarial networks for graph-constrained house layout generation. 2020. arXiv:2003.06988.
SFKonig11: Sven Schneider, Jan-Ruben Fischer, and Reinhard König. Rethinking automated layout design: developing a creative evolutionary design method for the layout problems in architecture and urban design. In John S. Gero, editor, Design Computing and Cognition ‘10, 367–386. Dordrecht, 2011. Springer Netherlands.
SSN+: Mohammad Shaker, Mhd Hasan Sarhan, Ola Al Naameh, Noor Shaker, and Julian Togelius. Automatic generation and analysis of physics-based puzzle games.
ST16(1,2,3): Noor Shaker and Julian Togelius. The search-based approach. In Noor Shaker, Julian Togelius, and Mark J. Nelson, editors, Procedural Content Generation in Games, pages 17–29. Springer, 2016.
SYT10: Noor Shaker, Georgios Yannakakis, and Julian Togelius. Towards automatic personalized content generation for platform games. In Proceedings of the 6th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2010. 12 2010.
SJLW20: Varvara Vasilatou Song Jie Lim and Shih Hsin Wuu. The use of ca to generate informal architectural systems. In SimAUD 2020, Society for Modeling & Simulation International. 2020.
KaravolosLiapisYannakakis18: D. Karavolos, A. Liapis, and G. N. Yannakakis. Using a surrogate model of gameplay for automated level design. In 2018 IEEE Conference on Computational Intelligence and Games (CIG), volume, 1–8. 2018.
LehmanStanley11: J. Lehman and K. O. Stanley. Abandoning objectives: evolution through the search for novelty alone. Evolutionary Computation, 19(2):189–223, 2011.

Designing with Intelligence - A General Framework for Mixed-Initiative Generative Design

Generator¶