1. Introduction

Currently, models that can be represented in the form of graphs are widely used in knowledge engineering and decision support. Some examples of such models are:

• Semantic networks, thesauri, ontologies [1];

• Bayesian networks, as well as influence diagrams based on them [2, 3];

• Decision trees [4], probabilistic decision trees [3];

• Markov decision process models [2, 5];

• Models of analytic hierarchy process and analytic network process [6];

• Transshipment models [5];

• Cognitive models based on different types of cognitive maps [7].

It is characteristic that it is the graph form of representation of both the listed and other similar models that is usually the most natural and intuitive for user perception. Indeed, each of these models can be easily associated with a graph with vertices corresponding to the main elements of the object, system or situation under consideration (for example, these can be elements of a decision-making problem or a model of knowledge about a certain subject area), and the edges between the vertices correspond to relations between the respective elements. Depending on the type of a model, the vertices of a graph can be either homogeneous (i.e., represent “equal” elements of the same nature) or heterogeneous (for example, there are qualitatively different types of nodes in decision trees). A similar statement is true for the edges of a graph. Semantic interpretations of vertices and edges determine characteristics of a graph. So, it can be directed or undirected, be or not be weighted, allow cycles or be acyclic, etc. Also, a mathematical apparatus, used within the framework of a specific type of graph model, plays an important role: for example, probabilistic models [2, 3], fuzzy models [7, 8], etc. can be distinguished.

The presence in the discussed models of the graph form of representation naturally leads to the problem of their visualization. It is characterized by the availability of many possible ways to solve it, among which, as a rule, there is no predominantly correct way. Taking this into account, an approach can be used to describe the visualization problem in general which is based on the concept of a visualization metaphor [9]. A visualization metaphor is understood as a set of principles for transferring characteristics of the object under study into the space of a visual model. The visualization metaphor includes two components, applied sequentially:

• a spatial metaphor, describing the general principles of building a visual model (in particular, type and dimension of visualization space, relative position of model elements);

• a representation metaphor, which is responsible for clarifying characteristics of a visual image (as a rule, with the aim of visualizing certain properties of the object under study which are of most significance at the current stage of its analysis).

An important aspect of working with any graph model that affects the efficiency of its application is the simplicity of perception of the model by the researcher. To describe this aspect, a concept of cognitive clarity is often used [10], which means the ease of intuitive understanding and interpretation of a certain amount of information presented in a certain form. Lack of cognitive clarity is usually associated with difficulty in understanding information, with missing a significant part of it, inaccurate or erroneous interpretation of some of its elements, etc. As applied to a graph model, ensuring a high level of cognitive clarity of its representation allows the researcher to notice more important properties of the model “at a glance”, to find more errors made in its construction, and also to interpret the results of its analysis faster.

2. Visualization metaphors as a means of increasing cognitive clarity of graph models

In [11-14], the authors investigated aspects of application of visualization metaphors in the problems of visualization of fuzzy cognitive maps and graph models in general. Summarizing the results obtained, the process of visualizing a graph model using a visualization metaphor can be represented as a diagram in Fig. 1. Thus, a spatial metaphor determines the arrangement principle of graph vertices and edges in a visual space, therefore, it can be based on well-known graph tiling algorithms. Following [15], we say that the result of applying a spatial metaphor is a spatial arrangement of a graph model. In turn, a representation metaphor is intended to focus the researcher’s attention on certain aspects or results of modeling depending on his needs at a particular stage of working with the model. For this, visual features of graph vertices and edges can be used displaying the attribute values of the corresponding model elements in a cognitively accessible form. This is how a visual representation of a graph model is formed. Spatial arrangement and visual representation together form a visual image of a graph model.

 

Fig. 1. Graph model visualization using a metaphor

 

In [14] using an example of visualization of fuzzy cognitive maps, the authors showed that use of visualization metaphors allows structuring and partially formalizing the task of increasing cognitive clarity of a visual representation. They also hypothesized that a similar effect can be achieved through the use of visualization metaphors for graph models of other types.

Accordingly, each of constituents of a visualization metaphor must contribute to enhancing cognitive clarity of a graph model. At the same time, the spatial metaphor provides an increase in cognitive clarity mainly due to optimization of a number of formal indicators characterizing graph tiling, which are taken as criteria for cognitive clarity at this stage. The authors proposed [11] and formalized [12] a basic set of criteria that can be taken as a basis for assessing cognitive clarity of any type of graph models. Also, in [12], an approach was proposed that allows automating comparison of a set of generated tilings of a given graph in order to select the one that provides the greatest cognitive clarity of a visual image.

Thus, at the moment, the task of constructing spatial metaphors of graph models has a general solution, which is later to be elaborated and adapted for various types of models considering their specific features. At the same time, we are also interested in formalization and automation of the process of constructing qualitative metaphors for representing graph models which provide a high level of cognitive clarity of these models in their visual analysis. Next, we will consider some ways to solve this problem.

3. Grounds for formalizing the process of constructing representation metaphors for graph models

It is necessary to highlight a number of characteristic features inherent in graph models that are significant in the context of their visualization.

First, each specific type of graph models is characterized by a certain structure which can be formally described. So, in general case, the main elements of the model (i.e., graph vertices) can belong to one of several types, the conceptual meaning and internal structure of which, as a rule, are specified and described in advance. The same applies to relations between elements (graph edges) – the acceptable types of such relations and the corresponding conceptual interpretations are usually known in advance. Further, to simplify the terminology used, we will call structural components of the corresponding graph – both vertices and edges – model elements.

Secondly, as a rule, model elements are assigned some attributes (i.e. properties, characteristics of these elements) that have certain tolerance ranges and interpretations and can also have an internal structure, i.e. contain a number of simpler attributes. Elementary attributes that have no internal structure, from the point of view of tolerance ranges, usually correspond to elementary data types: text strings, integers or real numbers (often from certain ranges), elements of discrete sets, binary yes/no values, etc. In fact, attributes make up the parametric space of a graph model and can reflect both its initial data (that is, specified when building the model) and the results of its analysis.

Thirdly, quite an obvious solution is to visualize elements and attributes of different types by different methods. Meanwhile, for each specific type, it is possible to distinguish (both intuitively and on the basis of experience) visualization methods that are more preferable from the point of view of ensuring a high level of cognitive clarity.

Let us say that a model element is visualized by creating a visual image of this element, and an attribute of the element is rendered by assigning some visual feature corresponding to the visual image. The graph model as a whole is visualized by creating a visual representation, i.e. a set of visual images of model elements, the visual features of which reflect the attributes of these elements.

With this consideration in mind, the visualization method mentioned above can be more formally understood as establishing a correspondence between a specific type of an element (attribute) and a specific type of a visual image (visual feature), and a representation metaphor as a whole – as a necessary set of such visualization methods.

When choosing methods to visualize attributes, it is advisable to proceed from their characteristics. Thus, a type of tolerance range of the attribute must be considered, in particular, whether it is discrete or continuous. It should also be taken into account whether it is important to know the exact value of the attribute from the point of view of the visual analysis of the model, or some “approximate picture” which provides a qualitative insight into the situation is sufficient. Otherwise, the choice of visualization methods is predominantly subjective. In addition, in some cases it can be dictated by already established traditions. Such a situation often arises in cases of widespread use of software tools for supporting a certain type of model (for example, it is characteristic of Bayesian networks). In any case, possible ways to visualize various types of attributes can be represented in a certain formalized form, thus forming a knowledge base suitable for use in developing representation metaphors for any graph models.

Also, when building representation metaphors, it is important to consider that various attributes of model elements become relevant at different stages of modeling. For example, at the stage of building a model, attributes representing the results of its analysis will be irrelevant (since at this stage they do not yet have definite values). In addition, if the process of model analysis includes a number of logically separate stages (as, for example, in case of cognitive maps where it is customary to distinguish structure and target and scenario stages of analysis), then such a model is characterized by the presence of several separate groups of element attributes. In turn, each of the stages can be logically divided into sub-stages, which leads to emergence of subgroups of attributes, etc. All this allows us to introduce the concept of a graph model representation, by which we mean a special relation between elements and their attributes. The representation selects from the set of all element attributes a subset of those that are to be visualized. Thus, when constructing metaphors for visualizing graph models, the representation can be used as a named template that makes it easier for the analyst to select model elements and attributes for solving a specific visual analysis problem.

The material presented in this section creates the basis for formalization and partial automation of the process of developing representation metaphors for graph models.

4. Principles of forming representation metaphors for graph models

Let us formulate a number of principles that determine the rules of forming representation metaphors for graph models. These principles should be considered when developing approaches to constructing such metaphors. The semantic content of these principles is schematically illustrated in Fig. 2.

1. The principle of partial visualization. As a rule, only some subset of elements and their attributes available in the model are visualized “at one point in time” (or, based on the terminology introduced above, one representation is visualized). This is due both to the high structural and parametric complexity of graph models, which, as a rule, exceeds the analyst’s cognitive capabilities, and to the multi-stage process of studying models, when at a certain stage there is a possibility and need to visualize only a part of the information related to the model.

2. The principle of injective visualization. Different attributes of model elements within the same representation metaphor must be visualized in different ways. In other words, mixing of two or more attributes within one visual feature is not allowed, since this will entail mixing of the corresponding properties of the model in the analyst's perception.

3. The principle of surjective visualization. Each separate visual feature must reflect a specific attribute that is significant in the context of the problem being solved. In other words, the researcher's perception of the model must not be cluttered with information that is irrelevant in the context of the problem, since this can lead to a slowdown in visual analysis.

4. The principle of subordination. Each subordinate element of the model must be visualized in such a way that its visual image makes it possible to unambiguously establish which particular element it is subordinate to. A special case of subordination is logical nesting of one element of the model in another, which should be displayed, respectively, as nesting of visual images.

5. The principle of restructuring. In some cases, it is possible to merge two discrete attributes into one attribute based on the Cartesian product of their tolerance ranges. So, in the example considered below (Table 1), it is allowed to merge the “Type” and “Target” attributes (the resulting attribute takes on the values “unmanaged target”, etc.). The reverse variant of application of this principle is also possible: the original attribute is split into two attributes of different types. For example, the attribute “Influence magnitude” can be divided into “Influence sign” and “Influence intensity” (Table 2). In general, application of this principle allows optimizing a representation metaphor due to a more rational use of the space of available visual features.

 

Fig 2. Illustration of the principles of forming representation metaphors for graph models

5. General approach to construction of representation metaphors for graph models

Considering the formulated principles, it is possible to propose a general approach to construction of representation metaphors for graph models. It can be schematically represented as a generalized algorithm (Fig. 3). Let us briefly describe its main stages.

1. The analyst chooses a subset from the set of elements and attributes of the model to be visualized within one representation metaphor. This stage can be performed in an accelerated version, by choosing a specific representation from a set of possible representations (if there is a similar set which can be formed, for example, on the basis of previous experience with graph models of this type).

2. Checking the possibility of simultaneous visualization of all attributes from the selected subset. In fact, at this stage, an attempt is made to establish a correspondence between the specified attributes and the available visual features taking into account the known methods of visualizing attributes of various types. Available visual features are formalized by means of a basic template of a visual image – a pre-compiled structure that stores a hierarchy of visual features with indications of their types. At this stage, compliance with the principles of injective and surjective visualization as well as the principle of subordination is ensured. If necessary, it is possible to use the restructuring principle, while the decision to merge or split attributes is made by the analyst.

3. If it is impossible to establish at least one correspondence option, the analyst can be offered to narrow down the subset of visualized attributes. The excluded attributes can be visualized in the future using a different representation metaphor. Another way to achieve the desired correspondence is to expand the space of available visual features by modifying the visual image template, which is also performed by the analyst. In each of the two methods, it is possible and advisable to formulate recommendations for the analyst on the choice of an optimal course of action to achieve the required result.

 

Fig. 3. Generalized algorithm for constructing representation metaphors for graph models

 

4. Generating possible options for representation metaphor implementation, which is carried out according to the principle of enumerating visualization methods (i.e., combining acceptable correspondences between attributes and visual features), considering the given preference of these methods.

5. Analyst’s visual familiarization with the obtained options of the representation metaphor and selection of the most preferable one (performed considering subjective informal preferences).

6. Examples of constructing representation metaphors for graph models

6.1. Fuzzy cognitive maps

Let us consider an example of applying the proposed approach to constructing a representation metaphor for V.B. Sylov’s fuzzy cognitive map (FCM) [16] within the framework of a cognitive model of analysis and planning of software projects [17]. In the example, the details related to formalization of data representation and processing will be omitted; the emphasis will be on the key features of the proposed approach.

Elements of a cognitive map are concepts of the studied subject area (software project management) and have a single set of attributes, thus being “homogeneous”, i.e. belonging to the same type. All relations between elements (relations between concepts) existing in the model define cause-and-effect influences between them, have common sets of attributes and, thus, also belong to the same type.

Tables 1 and 2 show concept attributes and cognitive model influences (in  case of concepts, a small sample of attributes is given), for which tolerance ranges and examples of visualization methods that provide sufficient cognitive clarity are indicated. When listing the attributes, the authors relied on the implementation of Sylov’s FCM apparatus within the framework of IGLA decision support system [18].

Suppose the researcher performs the stage of structure and target analysis of the cognitive model and is going to visualize the following concept attributes at the same time: name, type, influence on the system, as well as all available attributes of relations between concepts. Meanwhile, suppose that the basic template of the concept visual image contains only two visual features: the text displayed on it and the background color (which corresponds to the quest for creating the simplest metaphors with high cognitive clarity).

Table 1. Some attributes of a concept within Sylov’s FCM and their visualization methods

 

Table 2. Attributes of influence within Sylov’s FCM and their visualization methods

 

At the second stage of the algorithm, it was found that it was impossible to establish a correspondence between attributes and visual features under the indicated conditions. According to known visualization methods (which could be obtained by formalizing knowledge from Table 1), the displayed text is matched to the concept name, but the background color of the concept cannot simultaneously reflect its two other attributes (type and influence on the system). Accordingly, a violation of the principle of injective visualization has occurred.

The representation metaphor for the cognitive model shown in Fig. 4 can be formed after narrowing the set of attributes to be visualized: the analyst agrees to visualize only the name of the concept and its influence on the system. In contrast, the metaphor in Fig. 5 can be obtained as a result of adding to the visual image template a new graphic element which provides the missing visual feature.

 

Fig. 4. An example of an FCM representation metaphor

 

Fig. 5. An example of an FCM representation metaphor: alternative selection of a subset of attributes and methods of their visualization

 

The key difference between these representation metaphors is the way of visualizing the influences of concepts on the system: the colors of the graph vertices (Fig. 4) and the elements of the bar graph distributed over the set of vertices (Fig. 5) are used as visual features. In the second metaphor this allowed to “release” the vertex color for visualization of the concept type (thus, in this cognitive model, the concepts “Customer requirement volume” and “Requirement specification complexity” are managed, i.e. direct control actions can be exerted on the corresponding parameters of the system, the other concepts are unmanaged). Thus, the advantage of the second metaphor is the simultaneous visualization of the concept type and its influence on the system. Due to this, the analyst can obtain a larger amount of information of interest to him in the course of one act of visual perception of the cognitive model. The “trade-off” for this is the speed of the very act of perception, which slows down due to the complication of the visual image, which, however, in the given example is insignificant.

The rest of the differences between the two metaphors (in terms of visualizing relations between concepts) can demonstrate flexibility of the proposed approach and consideration of the subjective component within its framework: the choice of the final version of the representation metaphor from a number of acceptable ones as well as adjustment of preferable color schemes remain with the analyst. Thus, in the above examples, different color schemes were used to visualize the signs of influences: red/blue (Fig. 4) or green/red (Fig. 5) to convey positive and negative influences, respectively. The influence intensity of one concept on another is transmitted through visual features of the thickness of the edge between them (Fig. 4) or the intensity of its color (Fig. 5).

6.2. Bayesian networks

Let us consider some aspects of representation metaphor formation using another type of graph model, i.e. Bayesian networks.

A Bayesian network consists of random events that characterize the problem under consideration, each of which is described by a discrete random variable with a set probability distribution. Links between random events are directed, each link sets a probabilistic dependency relationship on the corresponding pair of events.

Table 3 shows the main attributes of random events (important in the context of the example considered below), their tolerance ranges and possible visualization methods. In this type of models, links have no other attributes, except for links to the associated events.

Implementation of a representation metaphor in GeNIe, a software tool for supporting modeling based on Bayesian networks and influence diagrams, is worthy of attention [19]. Fig. 6 shows a visual image of the well-known “Asia” model [20], intended mainly to demonstrate the capabilities of Bayesian networks in medical diagnostics.

Table 3. Some attributes of a random event in a Bayesian network and methods of their visualization

 

Fig. 6. An example of a representation metaphor for a Bayesian network:
implementation in GeNIe software tool

 

Analyzing this visual image from the point of view of the representation metaphor used, we can note its structural overload with both graphic and text elements. Although in the example under consideration the effect of slowing down perception is not so significant, in the case of large Bayesian networks (used in real practical problems of diagnostics and including dozens of random events) the overload of the visual image can become critical.

In this regard, it is necessary to distinguish a number of features of the Bayesian network under consideration which make it possible to obtain a simpler representation metaphor with a higher level of cognitive clarity.

First, all random variables in this model are characterized by only two acceptable values. In this case, based on the purpose of the model, one of the values can be interpreted as a favorable (“normal”) event, while the other corresponds to an unfavorable event (some “deviation from the norm”).

Second, the presence of only two possible values for the variables leads to the fact that the probabilities of the corresponding random events are related in an obvious way: if A and B are random events, then p(B) = 1 – p(A).

Third, as part of the preliminary analysis of the simulated situation, knowledge of the exact probability values of the events is not mandatory for the analyst – here, it is more important to provide him with the opportunity to quickly assess the current picture at a qualitative level and only then, if necessary, receive detailed information about the model aspects that are of interest to him.

Thus, in this case, the principle of restructuring can be applied, namely, combining two “antagonistic” attributes describing the probabilities of mutually exclusive events into a single attribute that takes values from the range [0; 1] (the convention can be made, for example, that this number corresponds to the probability of an unfavorable event).

Another possible solution is to display this attribute using gradient color coding based on a two-color scheme. For example, shades of green can be selected to represent an acceptable (i.e., sufficiently low) probability of an unfavorable event, and shades of red for the cases where this probability is greater than some set critical value. It is important that the critical value should be set individually for each random event: for example, for an event associated with the presence of a disease, this value can be very low (even a low percentage of risk is often the basis for taking appropriate measures).

Fig. 7 shows an example of a visual image of the model under consideration, which can be obtained on the basis of the proposed representation metaphor. The fact that some evidence has been added to the model is indicated here by the visual feature of a dashed box for the corresponding random event – in this example, evidence has been added that the patient has dyspnea.

It should be noted that this network, like many other Bayesian networks designed to solve diagnostic problems, has a three-level structure: random events are divided into risk factors (visit to Asia, smoking), hypotheses (tuberculosis, lung cancer, bronchitis) and observations (X-ray results, dyspnea), and the intermediate level contains a disjunction of two hypotheses.

 

Fig. 7. An example of an alternative representation metaphor for a Bayesian network

 

Moreover, as can be seen from the spatial arrangement of the graph in Fig. 6 and 7, this feature can be considered at the stage of spatial metaphor. So, here the principle of level-by-level graph tiling is applied, which has made it possible to distribute its vertices in accordance with their belonging to the above levels.

Depending on the type of the graph model, this aspect of relationship between spatial metaphor and representation metaphor may play a more significant or less significant role in providing cognitive clarity of the graph model. In any case, it should be considered when building efficient mechanisms for visualizing graph models.

7. Results and discussion

The considered examples lead to a general problem that can be formulated as a contradiction between the volume of a representation metaphor and its cognitive clarity. At the same time, the volume of the representation metaphor can be defined as the cardinality of a set of attributes to be visualized within the framework of this metaphor. An equivalent definition can be the cardinality of many different visual features in the resulting visual image (the cardinalities of the two sets are equal, since, in accordance with the principles introduced in section 4, there must be a bijective correspondence between them).

Thus, when trying to visualize a larger number of attributes simultaneously, an obstacle arises in the form of exhaustion of the space of visual features: for example, if a particular visual feature displays some attribute, then it cannot be assigned to display another attribute. An obvious way to overcome this obstacle is to “extensively” expand the space of visual features by increasing the complexity of the visual image (by adding new graphic elements to it). However, the inevitable consequence of this is a decrease in cognitive clarity of a visual image, which is expressed in perception slowdown. This leads to the necessity to maintain a balance between volume and cognitive clarity when constructing a specific representation metaphor.

It is notable that there is a relationship with Hick's law [21], which establishes dependence between the number of elements contained in a certain user interface and the average time it takes for a user to visually discover and select the element he needs. A detailed study of this relationship and the refinement of its parameters is an urgent problem, solution of which can result in efficient recommendations for the formation of high-quality representation metaphors for graph models.

For this purpose, in the future, it is planned to conduct a series of experiments in accordance with the ideas outlined in [22]. In the course of experiments, users will be asked to solve some set of problems of visual analysis of graph models of various types, using representation metaphors. Meanwhile, the volume of the representation metaphor will vary for each of the problems, and time for the user to reach the correct answer to the question posed will be recorded as the main indicator of visual analysis success.

8. Conclusion

The proposed approach can become one of the key components of an integrated approach to building a visualization mechanism for arbitrary graph models. It is assumed that this mechanism should be based on a system of visualization metaphors that provide an increase in cognitive clarity of a graph model throughout all stages of its construction and analysis.

At the same time, an interesting promising opportunity is to provide intelligent switching between visualization metaphors reflecting the optimal order in each specific case for changing the stages of modeling.

It appears that a certain paragon and a criterion for the success of application of the visualization mechanism can be the performance of all necessary actions with the graph model predominantly (or even exclusively) in a visual form. In other words, the visualization mechanism can be considered the more successfully constructed, the more work can be done with its use without involving alternative means and forms of information display.

A number of specific directions can also be indicated for the development of the proposed approach in the near future. They are focused primarily on ensuring the possibility of software implementation of the algorithm that underlies it:

• Formalization of the terminological apparatus introduced within the framework of the approach, in particular, the concepts of a graph model element, an attribute, a representation, a visual image and a visual feature.

• Development of a language (languages) for a formal description of the composition of the graph model and the structure of its visual image or adaptation to this task of any of the existing markup languages (for example, XML or JSON).

• Providing automated generation of a set of graph model representations based on a formal description of its composition.

• Development of a language for formal description of representation metaphors for graph models resulting from the application of this approach.

In addition, experimental research of the discovered contradiction between the representation metaphor volume and its cognitive clarity, as well as the assumed interrelationship of this contradiction with Hick's law is of interest.

In the context of the indicated directions, a software tool for supporting visual analysis of various graph models of knowledge representation and decision making can become a promising applied result of the research carried out by the authors. This tool can be implemented in the form of a library for building visual analysis mechanisms, which can be used in the development or modernization of decision support systems and other software systems the work of which is closely related to information representation in the form of graphs.

Acknowledgements

The reported study was funded by RFBR, project number 19-07-00844.

References

1. Staab, S., Studer R. (Eds.): Handbook on Ontologies. Springer-Verlag Berlin Heidelberg, 2009. doi: 10.1007/978-3-540-92673-3

2. Sucar, L.E.: Probabilistic Graphical Models. Principles and Applications. Springer-Verlag London, 2015. doi: 10.1007/978-1-4471-6699-3

3. Jensen, F.V., Nielsen, T.D.: Bayesian Networks and Decision Graphs, 2nd. ed. Springer Science + Business Media LLC, 2007.

4. Bramer, M.: Principles of Data Mining. Springer-Verlag London Ltd., 2016. doi: 10.1007/978-1-4471-7307-6

5. Taha, H.A.: Operations Research: An Introduction, 10th. ed. Pearson, 2017.

6. Saaty, T.L.: Decision Making with Dependence and Feedback: The Analytic Network Process. RWS Publishing, Pittsburgh, PA, 2001.

7. Borisov, V.V., Kruglov, V.V., Fedulov, A.S.: Fuzzy Models and Networks. Goryachaya Liniya – Telekom, Moscow, Russia, 2012. [in Russian]

8. Piegat, A.: Fuzzy Modeling and Control. Physica-Verlag Heidelberg, 2001. doi: 10.1007/978-3-7908-1824-6

9. Zakharova, A.A., Shklyar, A.V.: Visualization Metaphors. Scientific Visualization 5 (2), 16–24 (2013).

10. Huang, W., Hong, S.H., Eades, P.: Predicting Graph Reading Performance: A Cognitive Approach. In: Proc. Asia Pacific Symposium on Information Visualization (APVIS2006), Tokyo, Japan, 2006, pp. 207–216. doi: 10.1145/1151903.1151933

11. Podvesovskii, A.G., Isaev, R.A.: Visualization Metaphors for Fuzzy Cognitive Maps. Scientific Visualization 10 (4), 13–29 (2018). doi: 10.26583/sv.10.4.02

12. Podvesovskii, A.G., Isaev, R.A.: Constructing Optimal Visualization Metaphor of Fuzzy Cognitive Maps on the Basis of Formalized Cognitive Clarity Criteria. Scientific Visualization 11 (4), 115–129 (2019). doi: 10.26583/sv.11.4.10

13. Isaev, R.A., Podvesovskii, A.G.: Verification of Cause-and-Effect Relationships in Cognitive Models Using Visualization Metaphors of Fuzzy Cognitive Maps. Scientific Visualization 12 (4), 1–8 (2020). doi: 10.26583/sv.12.4.01

14. Isaev, R.A., Podvesovskii, A.G.: Improving the Cognitive Clarity of Graph Models of Knowledge Representation and Decision-Making Using Visualization. Ergodesign. 1 (11), 27–35 (2021). [in Russian]. doi: 10.30987/2658-4026-2021-1-27-35

15. Kasyanov, V., Kasyanova, E.: Information Visualization on the Base of Graph Models. Scientific Visualization 6 (1), 31–50 (2014).

16. Sylov, V.B.: Strategical Decision Making in Fuzzy Environment. INPRO-RES, Moscow, Russia, 1995. [in Russian].

17. Podvesovskii, A.G., Titarev, D.V., Isaev R.A.: Fuzzy cognitive models in software projects analysis and planning. Herald of Computer and Information Technologies, 8, 22-31 (2019). [in Russian]. doi: 10.14489/vkit.2019.08.pp.022-031

18. Zakharova, A.A., Podvesovskii, A.G., Isaev, R.A.: Mathematical and Software Support for Cognitive Modeling of Semi-structured Organizational and Technical Systems. In: CPT2019 International conference Proceedings, pp. 131–141. Pub. NNGASU and SRCIPT, Nizhniy Novgorod (2019). [in Russian].

19. GeNIe Modeler – BayesFusion. URL: https://www.bayesfusion.com/genie/

20. Asia Bayesian network | Live demo. URL: https://www.bayesserver.com/examples/networks/asia

21. Proctor, R.W., Schneider, D.W.: Hick’s law for choice reaction time: A review. Quarterly Journal of Experimental Psychology 10 (4), 145–153 (2018). doi: 10.1080/17470218.2017.1322622

22. Zakharova, A.A., Vekhter, E.V., Shklyar, A.V., Krysko, A.V., Saltykova, O.A.: Quantitative Assessment of Cognitive Interpretability of Visualization. Scientific Visualization 71 (6), 1281–1299 (2018). doi: 10.26583/sv.10.4.11