The variety of solutions to the
visualization problem is a consequence of subjectivity of visualization
techniques and a lack of justification for their efficient application. The subjectivity
reflects a long historical path of visual art development and an unformed
apparatus for the formal description of its capabilities in solving visual
communication problems.
Justification of application efficiency can
be achieved as a result of analysis of numerous options for solving
visualization problems, however, most attempts stop at the stage of
generalizing the existing experience. Difficulties are associated with the need
to systematize visualization problems and visual communication features that
require activation in accordance with the goals of visualization.
A classification of visualization problems
was proposed earlier in [1], [2]. The semiotic model, which complements the
proposed classification, has created conditions for convergence of both
difficulties. This manifests itself in the fact that visual representation of
information and its interpretation, i.e. processes corresponding to the
specified difficulties, have become elements of a linguistic space [3].
In this space, it is possible to correctly
represent most of the concepts and definitions used in visualization. This combination
of heterogeneous entities is necessary to search for and determine the factors
that can be considered the reason for emergence of new hypotheses corresponding
to visualized data, as well as new knowledge [4]. In other words, a detailed
consideration of internal processes of visualization, which are the basis for
its development as a cognitive tool, should be considered a relevant area of
research.
Creation of a theoretical model of such
processes will fully or partially answer a number of important questions: how
to adapt visualization parameters to characteristics of a particular user or
their group; what the prospects for visualization are in terms of creating a
universal means of information communication; whether it is possible to create efficient
subsystems in visualization tools that allow levelling or using productively interpretation
subjectivity of visual images, etc. One of the expected results when answering
these questions will be the use of a visual representation of both the initial
data and the knowledge corresponding to them, in a generalized form. In other
words, visualization, as a way of information communication, will get an
opportunity to become a knowledge formalization tool initiating processes of
transferring knowledge between subject areas, both at the level of hypotheses
and in the form of verified models and systems.
Known patterns of visual perception largely
confirm the assumption of a high degree of persuasiveness of any visual images.
This is both a serious disadvantage and the most important advantage of
visualization. Moreover, the boundary between these qualities is very blurred.
What is the point?
Visual representation of any information is
perceived by the observer as an object with properties of completeness,
constancy, integrity, etc. If time for interpreting an image is short, then,
according to the linguistic model, it remains a symbol that reflects an
indisputable fact. The same result can be obtained using a number of expressive
techniques, so the impact of such visualization on the viewer belongs to the
category of “informing”. Consistency of the received new information with the
user’s own knowledge system in this case becomes a signal to complete the
interaction with visual information.
Knowledge, as information with a higher
degree of formalization, involves supplementing the initial data with information
about their significance in a particular subject area, reliability and
generalization. However, when solving the problem of knowledge representation,
it is difficult to point out differences between data and knowledge because, in
most existing solutions, these differences are presented as additional data
using known syntactic rules. An experimental assessment [5] carried out as part
of the study of the semiotic model suggests that a conceptual interpretation of
visualized data occurs only as a result of creating special conditions for the
observer [6], [7].
At the level of organizing the researcher’s
interaction with information, such conditions can be obtained in various ways,
both known and completely new. However, in the development of visual
communication tools, the same goal is pursued with more or less success in each
case: provoking the user, initializing the activity aspect in the course of
visual communication. If this goal cannot be achieved, then the visualized
knowledge remains data that has no practical value for the researcher at the
current time.
Studying
existing approaches to organization of initiating effect on the user and examining
the problem of increasing its effectiveness cause the emergence of new
visualization tools as well as the development of a specialized visualization
tool classifier, that complements the semiotic model. The need to separate the
means of visual analytics depending on mechanisms of initiation of cognitive
processes is associated with the obvious desire of visualization system developers
to increase efficiency. The correspondence between the toolkit and the problem
solved with its help is one of the most important directions in data
interpretation techniques. The classifier under consideration can be a simple
and convenient method for achieving this correspondence [8], [9], [10].
In a
more complete formulation, it is an external indication or subordination to a
request. It is the most common way of organizing interaction between a
researcher and a visual analytics tool. It can be organized in several ways
including preliminary informing about the search goal, a demonstrative analogue
or a sign indication. In each such case, a mental perception pattern is formed
which reduces the time spent by the researcher on interpretation and analysis
of visualized data. Feedback responsible for comparing the results of the
analysis and its goal plays a significant role in dynamic analysis. Hence, in
this case, substitution of the dynamic analysis for a quick comparison with a
mental template and deciding about the degree of closeness between the template
and the current result of visual perception should be considered a negative
cognitive effect.
In other
words, it is subjective proneness to conflict. It is a variant of interaction
with a visual image that naturally occurs in situations where both the direction
of the analysis and verification of its results are determined by the
researcher himself. In other words, if there is not enough data for the search of
and making the only correct solution to the problem, then visualization tools
should not form a false sense of accuracy of interpretation and associated problem-solving
hypothesis. Application of some features of the researcher’s perception for
correcting his actions can become a more useful variant of communication
between the user and initial data, aimed at collecting and involving new data
in the solution and generating additional hypotheses. For example, in a
situation of inconsistency with the user’s own current expectations, the
incompleteness of data emphasized by means of visualization (image incompleteness,
gap, limited representation, unreasonable movement, etc.) gives rise to a subjective
conflict and forces to take actions necessary to eliminate it. Thus, cognitive
search is initiated allowing to supplement the available information.
It is compliance
with an achievable goal. It is a broader statement of the problem of the researcher’s
interaction with the initial data. It is assumed that a potential opportunity
is created to correct both available knowledge and data study goals simultaneously
with initial data interpretation. In the “data
–
visualization
–
user” system, a dynamic component arises,
that is the user’s knowledge that changes system behavior. In this formulation
of visualization problem, the most ambiguous is the user’s behavior. Therefore,
the key issue that arises at the visual analytics system design stage is a
choice of possible methods and corresponding means of influencing the user
[11]. The purpose of such influence is to switch the researcher’s activity
between the “observation” – “search” – “understanding” states depending on the
actual task [12].
For a
deeper understanding of many related cognitive and emotional factors
responsible for the results of visual interpretation of data, an analysis of
known solutions to a number of applied problems in the field of design –
industrial and digital – has been carried out. As a result, a hypothesis has
been formulated that explains a significant number of stable reactions of users
of design objects – fears, sympathies, interest or apathy, – as well as the
reasons for disappearance or transformation of these reactions. The meaning of
the assumption made is as follows: the achievement of the necessary user
reaction and control over it depends on the ability to influence the subject’s
own role in the system “user – space perceived by him”. It is easy to give
several examples illustrating the validity of the proposed hypothesis.
Fear
and similar emotional reactions in terms
of the hypothesis under discussion correspond to a situation where the user’s
activity or its planning is suppressed by external factors. In other words, if
the perceived space dominates the analyzed system, then this deprives the user
of the opportunity to understand and predict its development and, consequently,
causes formation of various negative reactions. Within the framework of the
semiotic visualization model, this is an analogue of the state of passive
perception, i.e. visual informing.
Curiosity and cunning.
From the point of view of the proposed
hypothesis, in this case, elements of the system retain activity parity, i.e.
have mutual influence on each other. The activity aspect in the user’s behavior
is aimed at obtaining new information about the perceived space and clarifying
already existing knowledge. The user’s emotional state is considered “conditionally
positive”, because the knowledge gained compensates for the effort to obtain
it. In the semiotic model, these processes include two of its states at once – learning
and research.
Possession and manipulation.
This is the most productive situation for
the user, because it correlates with the possibilities of full-fledged planning
of activities and self-realization. The perceived space is considered a place
for realizing one’s own interests, forming hypotheses and receiving dividends
from their practical verification. Thus, it should be considered that the
emotional state is exclusively positive and constructive because it is defined
as directed use of existing experience and achievement of new results. From the
point of view of the semiotic model, this is the control task.
An
important consequence of the hypothesis under consideration is another
assumption that can be used in visual analytics systems. The main idea is that
the emotional states described above can be primary in relation to visual
analysis tasks. In other words, visualization tools, their perceived
characteristics and control subsystems, can form necessary emotional factors
for the user to control his activities. Thus, design of visualization tools is
considered as a complex parameter for managing their purpose and efficiency.
When
studying the applicability of the design of visual analytics tools, including
correspondence of a visualization metaphor to its purpose, two extreme cases
should be distinguished on the basis of the hypothesis under discussion: a
specific object space image and its opposite – the most abstract environment.
In the first case, interpretation of visualized data through the symbolism of
the object metaphor becomes the most accurate and unambiguous. At the same
time, the adequacy of interpretation is determined not by the metaphor but
rather by its correspondence to the user’s visual thinking. This situation is
the most popular among developers of visualization systems, because it
simplifies information communication and increases its speed.
Figure 1
– Example of an
abstract visual model
In the
opposite situation, the initial data are compared with an image that does not
have an associative connection with the user’s subjective experience and,
therefore, is considered abstract (Fig. 1). It is easy to assume that a
cognitive effort aimed at interpreting such an image can lead to the emergence
of a new subjective hypothesis and its subsequent verification. Designing a visualization
tool oriented towards this kind of result is associated with several
complexities at the same time. Firstly, personal user experience in today’s
digital world is extremely extensive and diverse, so choosing a metaphor that
does not use known associations is not an easy task. Secondly, the reverse side
of the refusal to use stable associative links is the difficulty in developing
controls for such visualization systems. And thirdly, as the results of
experimental assessments show, construction of a cognitive model of visual
information is quite fast, it is determined by characteristics of the user’s
perception, by the convenience of the controls and by information richness of
visualization.
Overcoming each of these difficulties is an
independent direction of research into the future possibilities of visual
analytics. In the generalized case, they can be defined as the development of a
transformable visual communication language, the definition of its syntax and
its own expressive means. A separate task is the study of ergonomics of visual
space. This term refers, firstly, to the correspondence between the user’s
perception capabilities and visual space implementation, and secondly, to the
consistency between the purpose of visual analytics tools, determined by the
developer, and their use, depending on the user and his goals.
To determine the functional approach to the
study of ergonomics of visual digital space, a number of experiments have been
carried out using various versions of such a space. The purpose of the study
was to identify factors which, when their values are changed, can change or
significantly affect the perception or interpretation of visual information. To
achieve this goal, an interactive visual environment has been designed that
allows evaluating the observer’s reaction and decision-making while interacting
with the environment.
Among the results obtained, several factors
should be noted that can be used to create a controlled influence on the user’s
perception:
Direction.
In a visually perceived space limited, for example, by the plane of the screen,
it is easy to distort the subjective sense of direction (Fig. 2). To do this,
it is enough to violate the main spatial landmarks that characterize the view direction
– top-bottom, left-right. In the simplest case, this is achieved by eliminating
the horizon line or by building a homogeneous environment. It takes 3.0–5.0
seconds for a change in view direction to completely disorientate the user.
Figure
2
–
Space
with direction distortion
Time.
Focusing on details of an image or
focusing on individual changes alters the idea of natural speed of these
processes. As a result, changes can occur in the visual space simultaneously
with different, also changeable, time scales. Meanwhile, time scale control
becomes a convenient parameter for manipulating the visual image, i.e. creates
conditions for increasing its cognitive significance.
Structure (Rhythm).
Visually perceived space structure, as
well as appearance of a rhythmic pattern that is not related to the initial
data, can be a template that determines the content of interpretation
hypotheses (Fig. 3). Naturally, use of a template always reduces time for searching
a solution, but only in those tasks where the template matches the task. In the
event that a visualization tool is needed for cognitive search for non-standard
answers, the imposed rhythm can become a tangible hindrance. For example, in a model
of a connected system using the curvilinear relationship metaphor, the
relationships between two nodes expressed as a straight line will look like an
error and can be excluded.
Figure
3 –
Perceived
visualization rhythm
Perspective.
In three-dimensional space, perspective
control and distortion become another way to influence the observer. The result
of such an effect is a change in the information richness of the image (Fig.
4), establishment of the proportionality of the system visible elements,
management of the viewer’s expectations and preferences. Wide-angle distortions
can change the significance of individual information elements and therefore
become one of the reasons for the emergence of new hypotheses.
Figure 4
–
Adjustable
information richness
The task of developing visual analysis tools for the study of informal
data can be exemplified by a system of visualizing social data. The primary
difficulty preventing meaningful conclusions is a significant amount of data
(10
–
200 people) that characterizes communication between
members of a social group.
To analyze the initial data, an interactive visual model is proposed
that combines data on several types of communication in one image. A spherical
surface has been chosen as the main object necessary for the formation of a
common visual space. This has made it possible to place data without considering
perceived proximity to boundaries or artificially selected points (poles).
Thus, the researcher, having received the opportunity to observe data in a
three-dimensional representation from an arbitrary perspective, actively uses
the possibility of a subjective choice of the most informative direction (Fig.
5).
Figure 5
– Visual model
of social data. Informative perspective indicating heterogeneity of
communication processes in a social group
The location of nodes, each of which symbolizes a member of the group
under study, on the surface of the sphere is arbitrary and can be changed
interactively. In one of visualization options, the nodes were placed at a
distance from each other, corresponding to the activity of their face-to-face
communication. Thus, an image was formed that allowed visualizing the internal
division of the group under study. Then the model was supplemented with
information about the interaction between the representatives of the group in
work or educational projects. The final model made it possible to draw
conclusions about the complementarity of various types of communication and to
predict the dynamics of changes in interaction (Fig. 6).
In different options for setting up the model, the researcher has the
opportunity to form subjective preliminary conclusions about the connectedness
or disunity of the tested group (Fig. 6a), the volume and complexity of working
communication (Fig. 6b, the red line is the largest working group), the
significance of connections that go beyond close interpersonal communication
(Fig. 6c, curved lines correspond to distant connections).
Figure 6
– Visualization
of interpersonal and work communication in a group
The model makes it easy to imagine the
relationship between face-to-face and work communications, as well as their
possible mutual influence. From the point of view of studying the capabilities
of visualization tools for their use in the study of systems similar to the test
one, it should be noted that the instrument takes little time to master for an
unprepared specialist, as well as provides the possibility to simulate new,
modified system states that correspond, for example, to the optimality criteria
for the parameters chosen by the researcher. An essential circumstance that
determines results of visual research is the active involvement by the user of
subjective experience that complements analyzed data.
All examples of visual models presented in
the work, are built with the author's algorithms implemented in the languages
MaxScript and Python in the three-dimensional modeling environment Autodesk 3ds
Max, using internal visualization tools.
In the
course of the study, there has been proposed a classification of visualization
tools differing in the ways of influencing interpretation and cognitive search
for an answer to the task assigned to the user. The developed classification is
consistent with the semiotic visualization model and is its consequence. The
use of visual models based on the abstract space metaphor is proposed as a
means of visual analytics with the greatest functional flexibility. Examples of
features of user interaction with such models and options for their useful
application are given.
[1] A. Zakharova, A.
Shklyar,
è
E. Vekhter, «Semiotic Assessment of Visualization
Tools»,
â
Proceedings of the 31st International Conference
on Computer Graphics and Vision (GraphiCon 2021),
Nizhny Novgorod, Russia,
ñåí. 2021,
ò. 3027,
ññ. 288–295.
Ïðîñìîòðåíî: 7 èþëü 2022 ã. [Îíëàéí]. Äîñòóïíî íà:
http://ceur-ws.org/Vol-3027/#paper28
[2] A. Zakharova A.
Shklyar,
è
E. Vekhter, «Semiotic Approach in the Development of
Interactive Visual Analytics Systems»,
Proceedings of the 30th International
Conference on Computer Graphics and Machine Vision (GraphiCon 2020). Part 2,
ññ. short12-1-short12-9,
äåê. 2020, doi:
10.51130/graphicon-2020-2-4-12.
[3] G. K. L. Tam, V.
Kothari,
è
M. Chen, «An Analysis of Machine- and Human-Analytics
in Classification»,
IEEE Transactions on Visualization and Computer Graphics,
ò. 23,
âûï. 1,
ññ. 71–80, 2017, doi:
10.1109/TVCG.2016.2598829.
[4] D. Sacha
è
äð.,
«Human-centered
machine learning through interactive visualization»,
24th European Symposium
on Artificial Neural Networks, Computational Intelligence and Machine Learning,
âûï. April,
ññ. 27–29, 2016.
[5] A. Zakharova, E.
Vekhter,
è
A. Shklyar, «Adaptable Visualization»,
Scientific
Visualization,
ò. 13,
âûï. 2,
ññ. 67–78, 2021, doi: 10.26583/sv.13.2.05.
[6] N. J. Pizzi, A. Demko,
è
R. Vivanco, «Discrimination of software quality in a
biomedical data analysis system»,
â
Proceedings Joint 9th IFSA World
Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569),
ò. 3,
ññ. 1702–1707. doi:
10.1109/NAFIPS.2001.943808.
[7] M. Matsushita
è
T.
Kato, «Interactive visualization method for exploratory data analysis»,
â
Proceedings Fifth International Conference on Information
Visualisation,
ññ.
671–676. doi: 10.1109/IV.2001.942128.
[8] A. Batch
è
N.
Elmqvist, «The Interactive Visualization Gap in Initial Exploratory Data
Analysis.»,
IEEE transactions on visualization and computer graphics,
ò.
24,
âûï. 1,
ññ. 278–287,
ÿíâ. 2018, doi:
10.1109/TVCG.2017.2743990.
[9] D. Schofield,
«Courting the visual image: The ability of digital graphics and interfaces to alter
the memory and behaviour of the viewer»,
Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes
in Bioinformatics),
ò. 10901 LNCS,
ññ.
325–344, 2018, doi: 10.1007/978-3-319-91238-7_27.
[10] R. J. Crouser,
L. Franklin, A. Endert,
è
K. Cook, «Toward Theoretical Techniques
for Measuring the Use of Human Effort in Visual Analytic Systems»,
IEEE
Transactions on Visualization and Computer Graphics, 2017, doi: 10.1109/TVCG.2016.2598460.
[11] A. Dasgupta
è
äð.,
«Familiarity Vs
Trust: A Comparative Study of Domain Scientists’ Trust in Visual Analytics and
Conventional Analysis Methods.»,
IEEE transactions on visualization and computer
graphics,
ò. 23,
âûï. 1,
ññ. 271–280,
ÿíâ. 2017, doi:
10.1109/TVCG.2016.2598544.
[12] C. Vieira, P.
Parsons, è V. Byrd, «Visual learning analytics of educational data: A
systematic literature review and research agenda»,
Computers and Education,
2018, doi: 10.1016/j.compedu.2018.03.018.
