The language of graph theory is one of the most common
means for describing problems of representing and processing information. The
wide variety of both graph models themselves and their areas of application is
due to the fact that, as noted in [1], graphs are “a natural means of
explaining complex situations on an intuitive level”. This circumstance also
determines the fact that many models currently used in knowledge engineering
and decision support also quite naturally allow a graph form of representation.
Among such models, one can distinguish, for example, semantic networks,
thesauri and ontologies [2], Bayesian networks and influence diagrams [3, 4],
decision trees [5], hierarchical and network decision-making models [6],
transport and flow models [7], cognitive models based on various types of
cognitive maps [8].
Advantages of graph models are most often manifested
in their visual processing, which makes the problem of visualizing such models
relevant. The problem is characterized by multivariance of its solutions [9].
To describe a visualization problem in a general way, an approach can be used based
on the concept of a visualization metaphor [10], which is understood as a set
of principles for transferring characteristics of the object under study into
the space of a visual model. The visualization metaphor has two components
applied sequentially:
•
a spatial metaphor that
describes the general principles of constructing a visual model (type and
dimension of visualization space, mutual arrangement of model elements in it);
•
a representation
metaphor responsible for clarifying characteristics of a visual image (as a
rule, in order to visualize certain properties of an object under study, the
most significant at the current stage of its analysis).
When working with any graph model, simplicity and
convenience of visual perception of the model by the researcher is of
particular importance. To describe this aspect, the concept of cognitive
clarity is often used [11, 12], which refers to the ease of intuitive understanding
and interpretation of a certain amount of information presented in a visual
model. Insufficient cognitive clarity of a model is usually associated with
difficulty in understanding information, omission of its significant part,
inaccurate or erroneous interpretation of some of its elements, etc. On the
contrary, providing a high level of cognitive clarity of the visual model
allows the researcher to “cover at a glance” a greater number of important
properties of the modeled object, increase the probability of detecting errors
made when building the model, and also increase the speed of interpreting the
results of its analysis.
The authors’ research in the field of visualization of
graph models, as well as the development of methods for assessing their
cognitive clarity and ways to improve it, were published in [13-17]. In
particular, in [13-15] the problems and tasks of visualization of a particular
type of graph models – fuzzy cognitive maps – were studied in detail.
One of the results of a more general work [17] was the
identification of a problem that was formulated as a contradiction between the volume
of the graph model representation metaphor and its cognitive clarity, where the
volume of the representation metaphor is understood as the number of different
visual features in the resulting visual image of the model. At the same time,
it was assumed that there is a relationship between this contradiction and
Hick’s law [18], which establishes a relationship between the number of
elements contained in a certain user interface and the average time that the
user spends on visual detection and selection of the element he needs. In this
regard, it was noted that an experimental research of the discovered
contradiction as well as its possible relationship with Hick’s law is of
interest.
In this paper, we propose a more general conceptual
approach to conducting experimental research in the field of visual perception
of graph models, potentially suitable for solving not only the above problem
but also many other research problems that may arise in this area.
A significant part of the terminological apparatus
used in this work was introduced and described in detail by the authors in the paper
[17].
Figure 1 shows a diagram illustrating the approach
proposed by the authors to understanding the idea of cognitive clarity of graph
models and related concepts and phenomena.
Figure 1 – Conceptual diagram of the
proposed approach to understanding the idea of cognitive clarity of graph
models
Based on the presented diagram, the concepts and
phenomena associated with the idea of cognitive clarity are proposed to be structured
into three categories.
Cognitive clarity forming factors,
in a sense, can be interpreted as “causes”
of its occurrence. This category includes everything that can contribute to the
emergence of cognitive clarity of visual images of the studied graph models:
the observed general visualization principles, the metaphors used, specific
rules of thumb (up to highly specific ones) and algorithms.
Cognitive clarity forming factors have yet to be
identified. The concept of the studies aimed at their identification is
proposed in this paper. Theoretically, it is possible to say that
identification and formalization of all factors-“causes” is tantamount to the
construction of an “ideal visualization metaphor” (for a certain type of graph
models and problems of their analysis). From a practical point of view,
obviously, such a goal is completely unattainable, but it can serve as a good
guideline.
Under
cognitive clarity “as such”
it is
proposed to understand a certain set of characteristics and properties of a
visual image which makes the corresponding graph model “cognitively
understandable” for a human analyst. Here it is necessary to note the fact that
this category is not constructive: even if you have already constructed visual
images that have cognitive clarity to a “sufficient” degree, you may know
nothing about the principles of obtaining them (i.e., not own the forming factors)
and, accordingly, not be able to create new visual images with similar
qualities. Thus, constructiveness is typical for the category of cognitive
clarity forming factors.
In addition, although cognitive clarity in its
proposed understanding exists objectively (properties of a visual image do not
depend on whether the analyst perceives it at the moment), it is actualized
only “at the junction” between the visual image and the cognitive functions of
the human analyst. Through these functions, a synergetic combination of a set
of individual properties into a certain integral result (the appearance of
which is well described by the term “emergence” from systems theory [19]) takes
place, due to which the observed effect arises.
It is important that at the level of this category it
becomes possible to measure individual formal components of cognitive clarity
(examples are graph tiling characteristics, such as the number of arc intersections,
etc. [14], as well as parameters of the representation metaphor, such as its
volume [17]). However, it remains impossible to measure the magnitude of the
synergetic effect (because it does not occur at the level of this category
itself) and its practical consequences.
Finally, it is the
effects of cognitive clarity
presence
(which, within the framework of the presented approach, can be
interpreted as its “consequences”) that are the “ultimate goal” of interest in
the very concept of “cognitive clarity” and research on this topic.
This category corresponds to the level of phenomena at
which the efficiency indicators of the graph model visual analysis can be
identified, available for evaluation and measurement. Examples include the
speed of solving a certain problem of visual analysis or the number of errors
made in this process.
The category of effects-“consequences” is also not
constructive: a single knowledge that a certain visual image provides an efficient
analysis does not give anything in terms of a general understanding of how to
build such images. In terms of the scheme under discussion, it can be said
that, in general, it is impossible to reconstruct the chain of cause-and-effect
relationships in the opposite direction, from the effects of cognitive clarity
presence to the factors that form it. The method proposed below for solving
this problem is to create a closed loop with feedback by introducing an
experiment.
Considering the fact that the visualization metaphor
includes two components, it seems appropriate to consider a detailed version of
a diagram of the outlined approach (Figure 2). This version involves allocation
of two levels of the visualization metaphor on the diagram – the spatial
metaphor and the representation metaphor, with the subsequent specification of
the semantic content of the intersections of these levels with each of the three
categories introduced above.
Figure 2 – A detailed diagram of the
proposed approach to understanding the idea of cognitive clarity of graph
models, considering two levels of visualization metaphor
Thus, the main cognitive clarity forming factors of
graph models at the level of spatial metaphor are algorithms for graph tiling.
It is assumed that an algorithm chosen optimally from the point of view of a
given type of a graph model and correctly configured taking into account specifics
of the problem being solved will ensure a graph tiling with the highest degree
of cognitive clarity possible. Such graph tiling will have formal
characteristics that are most conducive to simplifying the analyst’s visual
perception of this particular graph model. For example, it can be characterized
by the absence of arc intersections, well-pronounced symmetry, arc directions
that are convenient for quickly viewing the graph, etc. [14]. Depending on the
type and specific features of a graph model, other, more complex and less
obvious characteristics may contribute to the improvement of perception.
In any case, the analyst’s perception of a graph model
visual image that has a set of such characteristics should lead to a more efficient
(compared to other situations) understanding of the model structure. The
increase in this efficiency can be registered by measuring the analyst’s performance
when solving visual analysis problems directly focused on understanding model
structure.
At the representation metaphor level, cognitive
clarity forming factors can,
firstly, be based on the principles of constructing representation metaphors
for graph models outlined in [17]. These principles set very general and
intuitive rules for the formation of correspondences between model attributes
and visual features, and their observance contributes to the creation of
metaphors with a sufficient level of cognitive clarity and the absence of
“gross” violations. Secondly, it is of interest to formalize and consider visual
perception patterns, both related to the perception of graph models in general
and related to their certain types. Such regularities include, for example, the
already mentioned contradiction between the volume of the representation
metaphor and its cognitive clarity, as well as the hypothesis about its connection
with Hick’s law.
The optimal (under the conditions of a specific visual
analysis problem) distribution of correspondences between attributes and visual
features creates the basis for the most efficient understanding by the analyst
of a set of visualized attributes, which can reflect both the initial
parameters of the model and the results of modeling. At the same time, in order
to evaluate the efficiency, it is required to measure the analyst’s performance
in the conditions of solving visual analysis problems focused on understanding
not structural, but parametric components of the model.
Of considerable interest, both in research and
practical terms, is the content of the middle level of the scheme, which
corresponds to the junction between the two levels of a visualization metaphor
in each of the categories. Thus, in the category of cognitive clarity forming
factors, the main semantic content of this junction is the possible mutual
influence and mutual conditioning of factors related to different levels of
metaphor. In particular, the following questions are useful to ensure efficient
visualization:
•
How much influence does
the choice of factors at the level of spatial metaphor have on cognitive
clarity of the representation metaphor level? In particular, whether a bad
choice of such factors (for example, an obviously erroneous choice of graph tiling
method) can negate the positive effects of even a very well-constructed
representation metaphor.
•
How meaningful is the
feedback? In particular, whether the choice of cognitive clarity forming
factors at the level of spatial metaphor will depend on the representation metaphor
and the research problem as a whole. An example of such dependence can be the
expediency of changing graph tiling when moving to another visual analysis problem
of the same model.
•
What are the
possibilities and limits of acceptable compensation for errors made at one
level by successful solutions at another level? For example, to what extent is the
correct selection of graph tiling capable of compensating for an insufficiently
qualitative representation metaphor (in the context of solving a specific
visual analysis problem), and vice versa.
Finally, in the category of cognitive clarity effects
at the “junction” between the levels of the visualization metaphor, there is a
synergetic combination of effects from both levels of the metaphor that occurs
when the analyst perceives a visual image. In other words, the analyst’s clear
understanding of the graph model structure, combined with understanding of its
parameters and modeling results, as a rule, leads to a much more efficient
solution of the visual analysis problem than in the absence of any of these
components. A detailed study of the mechanisms of such synergy and the search
for methods for quantifying its impact on the analysts’ work efficiency can be
very useful for the development of the proposed approach.
The diagram shown in Figure 3, on the one hand, can be
considered as an addition to the conceptual diagram in Figure 1, and on the
other hand, it demonstrates the “role and place” of the authors’ concept of
cognitive clarity within the framework of the proposed approach to organization
of experimental research.
Figure 3 – Diagram of the proposed approach
to organization of experimental research to study the influence of various
factors on cognitive clarity of graph models
The proposed approach is aimed at solving the problem
of experimentally identifying the factors that contribute most to the formation
of cognitive clarity, i.e. provide the greatest effect. At the same time, the
idea of the approach proceeds, as mentioned above, from the practical
impossibility of “directly” detecting cognitive clarity forming factors based
on the observed effects of its presence.
It is assumed that the cycle presented in the diagram
should begin with a hypothesis about the influence (or, conversely, the absence
of influence) of a certain factor (or their combination) on cognitive clarity
of the resulting visual image of the graph model, which should manifest itself
in specific effects, the magnitude of which is estimated by measuring relevant
indicators.
Based on the formulated hypothesis, an appropriate
experiment plan is built, in accordance with which the experiment is conducted.
It consists in analyst’s solving the task of graph model visual analysis.
During the experiment, the necessary indicators are measured, and then they are
processed, which, among other things, may include matching and combining the
results with those obtained earlier. Based on the results of processing, the
identified dependencies are formulated, which serve as the basis for
confirming, rejecting or refining the hypothesis.
Note that execution of each iteration of this cycle
should take place in the context of a specific type of graph model and a
selected problem of visual analysis.
Thus, the approach involves identification of
cognitive clarity forming factors not in the sense of their detection, but in
the sense of an iterative process of putting forward and testing hypotheses
about the influence (or lack of influence) of certain factors. The task of hypothesis
formation is assigned to the researcher. In other words, the approach generally
does not provide support for the formation of hypotheses, acting primarily as a
methodological and technological basis for their verification. In the future,
however, it is possible to develop an approach towards the automatic formation
of new hypotheses based on the results of testing a number of previously
proposed ones.
As for experimental studies of the impact of graph
models on cognitive clarity, it is of considerable interest to ensure
reproducibility of the results of studies of this type as well as sufficient
transparency of the study itself. In this regard, it seems appropriate to
develop some methodological foundations for preparation and conduct of such
studies as well as processing their results. It is possible to form a certain
general set of recommendations, the implementation of which will help expand
the possibilities of conducting experiments, as well as increase the generality
and reliability of the results obtained.
First, it is advisable to provide for the possibility
of conducting an experiment not only on specific, known types of graph models
(such as, for example, fuzzy cognitive maps), but also on “abstract” types
designed specifically for a particular experiment, taking into account its
hypothesis and conduct specifics. This will allow more flexible adjustment of
experiment parameters to efficiently refine the dependence of interest to the
researcher.
Secondly, the possibility of varying the complexity of
the model should be implemented, i.e. the number of elements in its composition
(in this case, the best option is to implement a separate variation in the
number of vertices and the number of edges of the graph). This will contribute
to the formation of a more general picture of the desired dependence, without
linking it to a model of a certain complexity, and, accordingly, will allow
testing more complex and abstract hypotheses.
Thirdly, it must be taken into account that with a
single solution by an analyst of any problem of visual analysis, the measured
indicators will inevitably be influenced by the random factor. A natural way to
reduce the degree of influence of this factor is to re-solve similar problems
of visual analysis with registering the average values of the indicators.
Fourthly, it is necessary to introduce a mechanism to
counteract the analyst’s habituation to the same graph model, especially in
combination with the same spatial arrangement. The essence of this effect is
that with each subsequent presentation of the same model, even when using
different representation metaphors, the analyst will navigate it faster and
better than with previous presentations. The difference in the degree of the
analyst’s habituation to the model at different stages of the experiment can
lead to a distortion of the real dependence of the analyst’s performance on the
considered cognitive clarity forming factors. Accordingly, considering the
described effect is most relevant when these factors relate to the
representation metaphor used. The desired mechanism can be implemented in two
ways. The first method involves a constant change in the graph spatial arrangement
or, in general, replacing the model with another one that has identical (from
the point of view of the current visual analysis task) characteristics. The
second way is to provide the analyst with the possibility of a preliminary
study of the model (primarily its structure) for subjectively sufficient time.
One of the key concepts of the considered approach is
the type of experiment. We will understand it as the result of formalization of
the visual analysis task from the point of view of its formal goal. The type of
experiment acts as a very general class that combines many real tasks of visual
analysis of various graph models characterized by a similar shape.
We can distinguish the following main types of experiments,
which reflect the nature of typical situations that arise in the visual
analysis of real graph models, and at the same time are quite simple in terms
of implementation:
1.
Selection on the entire
set of elements of a subset that satisfies the specified restrictions on
attribute values (attributes refer to any characteristics of graph model
elements related to certain data types [17]). A special case of this type of
experiment is detection of any one element that fits the specified
restrictions.
2.
Ranking elements (or
some subset of them, also visually distinguished based on values of other
attributes) in ascending or descending order of the value of some target
attribute. A special case is finding an element with the optimal value of the
target attribute (with possible consideration of restrictions on the values of
other attributes). For example, when analyzing cognitive models of semi-structured
systems, this type of experiment corresponds to the problem of identifying
concepts that are of the greatest interest from the point of view of
controlling the system being simulated [20].
Other types of experiment are also possible,
reflecting more complex or, conversely, specific tasks of analyzing graph
models and visual models in general [21, 22].
Taking into account the above, it is possible to
propose a generalized algorithm for setting up and conducting an experiment,
which includes the following steps (Figure 4):
Figure 4 – Generalized algorithm diagram
for setting up and conducting an experiment
1. Selecting a graph model type or creating an “abstract”
type on which the experiment will be conducted.
2. Selecting experiment type, i.e. defining what
exactly is meant by solving a visual analysis problem from a formal point of
view.
3. Setting experiment parameters. There are several
groups of parameters depending on various factors.
a) Parameters depending on the model type. Here, for
example, a subset of model attributes participating in the experiment is
indicated, and also a range of model sizes is specified, within which the
number of its elements should vary (a special case involves conducting an
experiment on models of one specific size).
b) Parameters depending on the type of experiment.
This group formulates in detail conditions for solving a visual analysis
problem (restrictions on attribute values, the attribute to be optimized and
the direction of its optimization, etc.).
c) Parameters determined by the hypothesis. Here cognitive
clarity forming factors and variation ranges of their values are set, as well as
the measured indicator of visual analysis efficiency (or a set of them).
d) General parameters. These include, for example, the
number of repetitions of solving the same visual analysis problems in order to
eliminate the influence of the random factor.
4. Conducting an experiment with the parameters set at
the previous stage. Each iteration of the experiment involves presenting the
analyst with a visual image of the graph model (which, as a rule, is
pre-generated randomly and visualized taking into account cognitive clarity forming
factors under study). After that, the analyst is required to quickly solve the
task of visual analysis and register the achieved result (at the same time the specified
indicators are registered). The task re-execution mechanism itself is used both
to eliminate the influence of the randomness factor (in accordance with the
above requirement) and to vary the parameter values, the change of which is
provided for by the experiment plan (model dimensions, values of cognitive
clarity forming factors, etc.).
5. Processing of the results, as well as, if necessary
and possible, their coordination and combination with previously obtained
results of similar experiments in order to identify the desired patterns and
dependencies more reliably and more accurately evaluate the initial hypothesis.
Depending on the type of experiment, the process of
solving a visual analysis problem can be considered completed when one of the
following conditions is met:
1. The analyst received a completely correct answer to
the question of the problem. In the event that the recorded answer does not
meet the specified correctness criteria, the analyst can be notified of this
using a visual signal, with a possible hint pointing to specific elements
associated with the errors made.
2. The analyst received an answer close enough to the
correct one, taking into account a predetermined error margin.
3. The analyst, in principle, registered some answer.
4. The time allotted for solving the problem has
expired.
Indicators of visual analysis efficiency
(characterizing the effects of cognitive clarity presence), depending on the
hypothesis, can include time for the analyst to achieve the correct or close to
the correct answer, the completeness of the visual analysis problem solution (i.e.,
percentage of its completion in the allotted time), the number of errors made,
etc.
As mentioned above, it was noted in [17] that it is of
interest to experimentally verify the discovered contradiction between the volume
of the graph model representation metaphor and its cognitive clarity, as well
as the possible relationship of this contradiction with Hick’s law. With this
in mind, one of the possible ways to use the proposed experimental approach can
be to study the dependence of the efficiency indicators of graph model visual
analysis on the volume of the visual image of this model, i.e. the number of
different visual features it contains. A priori, it is assumed that from the
point of view of the time indicator for solving the problem, this dependence
should obey a pattern that is similar in structure to Hick’s law.
Thus, the following experimental setup can be
proposed.
•
The time taken to
complete the visual analysis task is used as the target for cognitive clarity.
•
The volume of the
representation metaphor is a variable factor in the formation of cognitive
clarity, the influence of which needs to be investigated.
•
The following
relationship between the factor and the target indicator is considered as a
hypothesis: an increase in the volume of the representation metaphor (which
entails complication of the visual image) leads to a decrease in the cognitive
clarity of the model visual image (which is expressed in an increase in the
time for solving the visual analysis problem).
A possible way to conduct an experiment is to conduct
it in the context of visual analysis of Silov’s fuzzy cognitive maps [8, 20].
At the same time, one of the most relevant problems can be chosen for this type
of model as a specific task of visual analysis – the problem of identifying
concepts that are most preferable from the point of view of providing control
actions on the system under study. This task refers to the second type of
experiments described in section 4: ranking concepts by the value of the
selected target attribute. In this case, this attribute is a systemic indicator
of the concept’s influence on the system [8].
In accordance with the recommendations given in section
4, the complexity of the cognitive model itself (the number of concepts in its
composition) should vary, as well as the repetition of solving problems of the
same complexity a given number of times (in order to counteract the influence
of randomness) should be provided.
Examples of visual images of a fuzzy cognitive model
that have different volumes of the representation metaphor that can be
presented to the analyst during the experiment are shown in Figure 5 (the
difference in volumes of the representation metaphor for these visual images
was discussed in [17]). Let us also note that randomly generated abstract
cognitive models with the required characteristics can be used in the
experiment in addition to cognitive models of real tasks.
|
|
|
|
a)
|
b)
|
Figure 5 – Examples of visual images
presented to the analyst during the experiment: a) smaller volume of
representation metaphor; b) bigger volume of representation metaphor (source:
[17])
Based on the results of the experiment, it is
necessary to assess the degree of similarity of the revealed pattern with Hick’s
law. The technique for assessing such similarity belongs to the statistical
analysis field.
Obviously, in addition to an increase in the time
spent by the analyst on solving the visual analysis problem, a decrease in the
cognitive clarity of the model visual image can manifest itself in
deterioration of values and other indicators. So, in the context of the
described problem, other relevant indicators to be measured may be the following:
•
The degree of solution
correctness (without limiting solution time). This indicator, in fact, will
determine the correctness degree of ranking concepts by the analyst according
to the value of their influence on the system. Violation of the ranking
correctness, generally speaking, can also arise due to the analyst’s
insufficient attentiveness, especially in situations where some two concepts
affect the system with approximately the same force. However, it seems very
plausible to assume that high complexity of a visual image will contribute to
the distraction of the analyst’s attention, which will increase the probability
of human error. A specific nature of such a dependence is not clear a priori
and, apparently, can only be established experimentally.
•
The degree of solution
completeness (for limited time). This indicator is generally “symmetrical” to
the indicator of time spent on the complete solution of the problem. However,
an important point, leading to less predictable results, may be additional
discomfort for the analyst associated with the requirement to meet a limited
time period. Let us note here that this aspect of stress effect on the results
should be studied separately. In any case, it is assumed that more complex and
information-saturated visual images will slow down perception and reduce the
value of the indicator under consideration.
Identification of dependency nature for all of the
above indicators will provide an opportunity to form a more comprehensive and
reasonable idea of how the complication of a graph model visual image leads to
deterioration in its perception by a person, and in what particular negative
consequences this manifests itself. All this will become a theoretical basis
for developing practical recommendations for optimizing the complexity of
visual images of models in various applied problems.
The results of the experiment carried out in the
described setting are planned to be presented in one of the future works. Let
us also note that, in the future, it will be necessary to conduct similar
experiments with other types of graph models and visual analysis problems,
followed by generalization and systematization of the results for a more
complete assessment of the patterns under study.
The article proposes the authors’ interpretation of
the idea of cognitive clarity of graph models and related concepts and
phenomena. On its basis, an approach is proposed to organize experimental research
aimed at studying the influence of certain factors on the cognitive clarity of
graph models by measuring various indicators characterizing the degree of
manifestation of the effects of cognitive clarity presence. An example of
setting up an experiment is given, the purpose of which is to study the
dependence of efficiency indicators of visual analysis of a graph model on its
visual image volume.
The concepts considered in the paper and the proposed
experimental approach provide an extensive basis for promising research aimed
at their refinement, generalization, improvement and application. According to
the authors, the following areas of research seem to be the most relevant:
1.
Approbation of the
proposed approach to organizing and conducting experimental research when
testing specific hypotheses about the influence of certain factors on forming
cognitive clarity of various graph model types.
2.
Studying possible
mutual influence and mutual conditioning of the factors that generate cognitive
clarity and relate to different levels of metaphor (primarily in the aspect of
the questions formulated in section 3 of this article). In addition, an
important subject of research may be the synergetic effect that occurs when the
analyst perceives a visual image due to the combination of cognitive clarity
effects from both levels of metaphor.
3.
Development of methods
for adaptive planning and experiment control, including methods for combining
and coordinating the results obtained at different stages of its
implementation, dynamic adjustment of parameters, etc. It seems that such
methods will help optimize the volume and content of visual analysis problems
presented to the analyst during an experiment, taking into account the
requirements of reliability and statistical significance of the results. These
methods can be based on the approach described in [23].
In addition to the above areas, it is of considerable
interest to study the degree and limits of applicability of the concepts
outlined not only for graph visualization problems but for other classes of
visual models and also, in the long term, for any problems of visualization and
visual information perception of various nature.
Finally, a separate large area of research to be
considered is the role and place of the subjective factor of a human analyst in
the described concepts. This refers mainly to the influence of analyst’s individual
perception and other psycho-emotional characteristics on the course of solving
various problems of visual analysis and the indicators recorded in this case.
The degree of generality and universality of theoretical and practical
conclusions that can be obtained on the basis of the above approach in the
future depends on the completeness of the study of this topic.
1.
Kasyanov, V.N, Evstigneev,
V.A.: Graphs in Programming: Processing, Visualization, and Applications. BHV,
Saint-Petersburg, Russia, 2003.
[in Russian].
2.
Staab, S., Studer R.
(Eds.): Handbook on Ontologies. Springer-Verlag Berlin Heidelberg, 2009. doi:
10.1007/978-3-540-92673-3
3.
Sucar, L.E.:
Probabilistic Graphical Models. Principles and Applications. Springer-Verlag
London, 2015. doi: 10.1007/978-1-4471-6699-3
4.
Jensen, F.V., Nielsen,
T.D.: Bayesian Networks and Decision Graphs, 2nd. ed. Springer Science +
Business Media LLC, 2007.
5.
Bramer, M.: Principles
of Data Mining. Springer-Verlag London Ltd., 2016. doi:
10.1007/978-1-4471-7307-6
6.
Saaty, T.L.: Decision
Making with Dependence and Feedback: The Analytic Network Process. RWS
Publishing, Pittsburgh, PA, 2001.
7.
Taha, H.A.: Operations
Research: An Introduction, 10th. ed. Pearson, 2017.
8.
Borisov, V.V., Kruglov,
V.V., Fedulov, A.S.: Fuzzy Models and Networks. Goryachaya Liniya – Telekom,
Moscow, Russia, 2012. [in Russian].
9.
Kasyanov, V.,
Kasyanova, E.: Information Visualization on the Base of Graph Models.
Scientific Visualization 6 (1), 31–50 (2014).
10.
Zakharova, A.A., Shklyar, A.V.:
Visualization Metaphors. Scientific Visualization 5 (2), 16–24 (2013).
11.
Huang, W., Hong, S.H.,
Eades, P.: Predicting Graph Reading Performance: A Cognitive Approach. In:
Proc. Asia Pacific Symposium on Information Visualization (APVIS2006), Tokyo,
Japan, 2006, pp. 207–216. doi: 10.1145/1151903.1151933
12.
Abramova, N.A., Voronina, T.A., Portsev,
R.Y.: Ideas of Cognitive Graphics to Support Verification of Cognitive Maps. Large-Scale
Systems Control 30.1, 411–430 (2010). [in Russian].
13.
Podvesovskii, A.G.,
Isaev, R.A.: Visualization Metaphors for Fuzzy Cognitive Maps. Scientific
Visualization 10 (4), 13–29 (2018). doi: 10.26583/sv.10.4.02
14.
Podvesovskii, A.G., Isaev, R.A.: Constructing
Optimal Visualization Metaphor of Fuzzy Cognitive Maps on the Basis of
Formalized Cognitive Clarity Criteria. Scientific Visualization 11 (4), 115–129
(2019). doi: 10.26583/sv.11.4.10
15.
Isaev, R.A.,
Podvesovskii, A.G.: Verification of Cause-and-Effect Relationships in Cognitive
Models Using Visualization Metaphors of Fuzzy Cognitive Maps. Scientific
Visualization 12 (4), 1–8 (2020). doi: 10.26583/sv.12.4.01
16.
Isaev, R.A.,
Podvesovskii, A.G.: Improving the Cognitive Clarity of Graph Models of
Knowledge Representation and Decision-Making Using Visualization. Ergodesign 1
(11), 27–35 (2021). [in Russian]. doi: 10.30987/2658-4026-2021-1-27-35
17.
Isaev, R.A.,
Podvesovskii, A.G.:
Visualization of Graph Models: An
Approach to Construction of Representation Metaphors.
Scientific
Visualization 13 (4), 9–24 (2021). doi:
10.26583/sv.13.4.02
18.
Proctor, R.W.,
Schneider, D.W.: Hick’s law for choice reaction time: A review. Quarterly
Journal of Experimental Psychology 10 (4), 145–153 (2018). doi:
10.1080/17470218.2017.1322622
19.
Peregudov, F.I., Tarasenko,
F.P.: Basics of Systems Analysis, 3rd ed. NTL Publishing, Tomsk, Russia, 2001. [in
Russian]
20.
Zakharova, A.A., Podvesovskii,
A.G., Isaev, R.A.: Fuzzy Cognitive Models in Management of Semi-structured Socio-economic
Systems. Information and mathematical technologies in science and management.
4 (20), 5–23 (2020). [in Russian]. doi: 10.38028/ESI.2020.20.4.001
21.
Zakharova, A.A.,
Shklyar, A.V., Rizen Y.S.: Measurable Features of Visualization Tasks. Scientific
Visualization 6 (1), 95–107 (2016).
22.
Zakharova, A.A.,
Vekhter, E.V., Shklyar, A.V., Krysko, A.V., Saltykova, O.A.: Quantitative
Assessment of Cognitive Interpretability of Visualization. Scientific
Visualization 10 (4), 145–153 (2018). doi: 10.26583/sv.10.4.11
23.
Zakharova, A., Korostelyov, D.,
Podvesovskii, A.: Evaluating State Effectiveness in Control Model of a
Generalized Computational Experiment. In: Kravets A.G. et. al. (eds.):
Creativity in Intelligent Technologies and Data Science. CIT&DS 2021.
Communications in Computer and Information Science, Vol 1448. Springer, Cham (2021).
doi: 10.1007/978-3-030-87034-8_16
