Nowadays, stereo
animation begins to play an important role in displaying the obtained
scientific results in various applied scientific research and industries. This
is largely due to the fact that a situation often arises when a flat image of
an object does not have the entirety of information to achieve its goals and
limits the analysis to a schematic image or a truncated viewing angle.
Unlike a
two-dimensional flat image, three-dimensional stereo allows you to more
effectively explore and model objects with a developed spatial structure or
interpret photographs taken during the experiment from different angles at
given times and combined into animation.
A lot of work
has been devoted to this topic. They discuss methods for constructing images
for various types of stereo devices, both passive and active types. The
difference between these concepts is that passive ones allow one or another
material to be demonstrated according to a predetermined scenario, however, the
viewer is not able to influence the display process. Active (or interactive)
installations mean that the viewer has the opportunity to work with the
demonstration material and directly influence the demonstration process.
General aspects of image construction for various types of stereo units are
presented in [1].
The specific
problems that arise when using a computer system to generate and visualize a
composite multi-screen stereo frame, and methods for solving such problems are
described in sufficient detail in [1-3].
The gradual
spread of modern stereo systems has generated great interest in the development
of practical algorithms for stereo presentation of static and animated images,
which is reflected in [4,5 - 10]. These works present the tasks of constructing
stereo images in various fields of research: from the explosion of a Supernova
to the processes of oil displacement from porous media.
The spread of
stereo animations and virtual reality also revealed a rather important
particular structural subproblem. The standard procedure for a report,
presentation of something or a virtual walk implies the presence of not only a
visual type of information in them, but also a symbolic one - letters, numbers,
symbols, metric quantities, names, numbers, state standards, without which the
perception and assimilation of information by the viewer is significantly
complicated.
The problems of
constructing stereo images for geometries and fields of physical quantities
have been developed in sufficient detail, but quite a few works have been
devoted to the development of specific practical approaches and algorithms for
constructing stereo representations of texts and formulas with sufficient
expressiveness and the necessary stereo effect. One of them, which cannot be
ignored, is the work [11] describing a study conducted in Japan of the
perception of stereo images of fonts on the screens of stereoscopic mobile
devices.
This paper
continues the research [1-3,7,12-14] carried out at the Keldysh Institute of
Applied Mathematics RAS. The aim of research is to create effective technology
for stereopresentations of scientific results. The research is based on the
available stereo units of two types.
The first type
of device is a 3D projection stereo system for displaying stereo presentations,
educational applications, graphics and films. It is an example of a classical
stereoscopic system using two projectors, a screen and linear polarization
glasses. The articles [12–13] describe in detail studies on the presentation of
textual information by this type of stereo unit using a linear stereo base. In
these works, test inscriptions were constructed with variations of the font,
background, font embossment, the whole angle of rotation of the inscription,
and shift along the linear stereo base. The parameters that provided the maximum
effect were found and the basic requirements for the fonts used and a number of
conditions were identified, the satisfaction of which is necessary to achieve
the optimal result.
The second type
of device is a Dimenco DM654MAS autostereoscopic monitor. Autostereoscopic
monitors provide stereo images without having to track the position of the
observer. Typically, such monitors make it possible to observe stereo images,
providing several fixed segments in the space for observation. The viewer can
move between segments, getting the opportunity to view the displayed object in
3D from different angles. The principle of operation of the autostereoscopic
monitor is the use of parallax partitions or Fresnel lenses installed behind
the protective glass of the screen, which gives it one of the most important
advantages: displaying the image does not require special glasses or other
devices from the viewer.
An
autostereoscopic monitor is capable of demonstrating a visualization object
using two methods: either using a composite frame containing views of the
visualization object at different angles that form a certain viewing sector -
this is called a multi-view, or using depth maps.
In multi-view
image construction, nine frames are combined into one image according to the
principle of a 3 × 3 matrix. In the first and last (ninth) frames, the
desired object, for example, the inscription, is in its extreme positions, and
in the intermediate frames it is rotated sequentially by a given angle.
When using the
method of depth maps, a certain range of depth is selected, which is manually set
by the user by fixing the nearest and far planes. The nearest plane is assigned
a value of 255, and the farthest - 0. Information describing the depth is
explicitly placed in the image and recorded in the form of gray gradations. The
intensity value is thus encoded with eight bits in the range [0, 255]. Further,
objects located in a given range fully use the stereo effect depth range that
the monitor simulates. This method is described in detail in [9].
In previous
works devoted to stereo imaging of textual information [12-14], some
computational experiments were described. These experiments were carried out
for stereo systems of both types — classical stereo installations using special
glasses and an autostereoscopic monitor. For the autostereoscopic monitor,
experiments on constructing stereo images of texts were carried out using depth
maps as well as using a multi-view presentation.
One of the tasks
when conducting experiments using a multi-view was that the viewer did not see the
transition between each of the stereopairs when changing the viewing angle, and
due to this the most powerful reality effect would be created. If the linear
and angular shift parameters are too large, artifacts arise: the observer has a
feeling that the image in front of him is blurred at the transitions from one
stereo pair to another, and with the slightest change of position, the observer
notices the glare and boundaries of each of the stereo pairs. Such phenomena
can cause the observer a certain discomfort, which interferes with working with
stereo images and an adequate perception of visual information on a stereo
frame. The problems of visual discomfort in virtual and mixed reality systems
are described in detail in [17].
In order to
create the most comfortable stereo image for the viewer with the maximum stereo
effect, during the computational experiments it was necessary to study the
effect of various parameters on the quality of the stereo image.
During the
experiments, when constructing the inscription, various parameters varied: font
size, rotation angle on each frame in multi-view, and the distance between
frames for a linear stereo image.
Due to the lack
of a theoretical part in this area, it was necessary to search empirically with
numerous tests for the necessary linear and angular shift parameters for a
multi-view method. The optimal parameters were found, in which the transitions
between stereo pairs became invisible to the viewer, and the inscription itself
acquired volume and became optimal for the viewer to perceive in
three-dimensional form.
At this stage of
the work, when constructing, the combination of linear and angular displacement
was first used. This made it possible to achieve a volume effect comparable to
the effect achieved in a classic stereo setup.
It was found
that, in principle, these parameters remain the same for very different font
sizes. A similar effect was obtained for the inscription, where the font size
was 32, and the most successful version of the inscription, where the font size
was 66 (Fig. 1).
Fig. 1.
The
construction of stereo images of the inscription with a font size of 66.
In the case of
constructing a depth map for the inscription, it was revealed that the desired
object should have clearly expressed depth and volume. The algorithm for
constructing a depth map, involving the alignment of pictures — rectification —
and the search for the corresponding pairs of points is described in detail in
[14–15] and implemented using a special software package [16]. For example, a
text variant suitable for a multi-view representation (Fig. 1) turned out to be
unsuitable for constructing a depth map due to insufficient volume of letters.
Figure 2 shows a depth
map for an inscription with pronounced depth and volume.
Fig. 2.
Construction a depth map for the whole inscription.
As a result of all
experiments, a clearly visible stereo effect was achieved and the conditions
necessary for constructing depth maps were identified.
However, in the
process, the following problem was discovered - the presence of artifacts
clearly visible on the autostereoscopic monitor. In the above image they are
also visible, only on depth maps. This problem arises due to the lack of
information for constructing an image for areas that are not visible in the
original image, but are observed in a shifted angle.
The reasons are that
the algorithms for smoothing the constructed depth map used in the software
package used in the study work well mainly with real stereo photographs when
the visualization object does not contain sharp edges with a contrasting color
change. In our case, the object of visualization is text, most often possessing
such properties. Thus, further studies suggest the selection and application of
the most effective smoothing algorithms.
After solving the
problem of creating stereo labels on their own, as separate frames in a stereo
presentation or stereo film, an equally important sub-task arises - combining
the image and text information in one frame. Currently, in the case of
demonstration of scientific results to observers, very many objects require
accompanying information located directly on the same frame as the image. In
many cases, the signature and the object cannot be separated into different
frames, since they make up a single logical display of information. For
example, when depicting a coordinate system, one cannot fail to mark a
designation for each of the coordinates. An example is shown in Figure 3. This
figure is not informative, like a graph, because it does not carry accurate
data and is not bound to coordinates. This is a simple three-dimensional model,
which is a schematic three-dimensional graph.
Fig. 3.
Three-dimensional model of the coordinate system in volume.
The constructed
nine-view image gave a stereo effect, which was recognized by observers as
satisfactory. However, the presented figure does not carry an informative load,
since it is not accompanied by the necessary additional information, and the
viewer does not even have an idea about the coordinate system.
Figure 4 shows a more
complete image. The coordinates are signed on it, and the letters indicating
the coordinates also provide a stereo effect.
Fig. 4.
Three-dimensional model of the coordinate system in volume with signed
coordinates.
To construct this
stereo image, an image matrix was used (Fig. 3), to which volume letters were
added and sequentially shifted and rotated on each frame by the same interval
experimentally calculated earlier. As a result, a stereo image with a volume
signature was obtained, the stereo effect of which was pronounced. Along with
this, an additional effect was revealed when the viewer from different
positions saw that the letters are on different planes each time. For example,
the location of the letter Z was perceived by observers either behind the
horizontal axis or in front of it, depending on the viewer's location in a
particular observation sector in front of the screen of the autostereoscopic
monitor.
For an integral label,
completely located on top of the image, such problems do not arise. Figure 5
shows the image of the simulation results of a supersonic flow around a cone at
an angle of attack with the corresponding inscription.
Fig. 5.
Image of simulation results of a supersonic flow around a cone with the
corresponding signature.
As a result, the
inscription was located on top of the cone, but behind its tip, which in turn
was perceived by the audience as protruding from the screen by several tens of
centimeters. As a result, another important parameter was revealed - the
remoteness of the object from the plane of the screen and its correlation with
other objects in the frame. Further experimental studies are planned to be
directed at controlling the remoteness of an object from the plane of the
screen and other objects depending on the stereo base.
This work is a
continuation of a works series devoted to the implementation of the project to
build stereo presentations of the results solving mathematical modeling
problems. The results of numerical experiments on the presentation of textual
information on an autostereoscopic monitor that allows the construction of
stereo images using depth and multi-view maps are presented. The problem of combining
images and textual information in one frame for a multi-view presentation is
considered. As a result of practical experiments, the most clear and expressive
stereo effect was achieved.
Further research is
planned to be directed to the development of a practical technology for
controlling the remoteness of an object from the plane of the screen and other
objects depending on the stereo base and its correlation with other objects in
the frame.
[1] Generation of
Stereo-Presentations in Photorealistic Rendering and Scientific Visualization
/Andreev S.V.[et al] // Keldysh Institute preprints, 2010. ¹ 61. 16 p.
http://library.keldysh.ru/preprint.asp?id=2010-61[inRussian].
[2] Synthesis of
photorealistic three-dimensional images in modern presentation systems /
Andreev S.V.[et al] // Software&Systems 2007. ¹ 3. p. 37-40. [in
Russian].
[3] Andreev S., Filina
A. Using stereo presentations for visualization of scientific calculations
results / Scientific Visualization. 2012. v.4. ¹ 1. p.12-21.
[4] Hardware-software
complex of 3D presentations based on a virtual studio and virtual environment /
Vandanov V.G. [et al] // Proceedings of the 1-st international conference «3D
visualization of scientific, technical and social reality. Cluster technologies
of modeling» Izhevsk. 2009. P.73-77. [in Russian].
[5] Mezhenin A.V.,
Tozik V.G. 3D Visualization using the stereo image effect // Proceedings of the
2-nd international conference «3D visualization of scientific, technical and
social reality. Cluster technologies of modeling» Izhevsk. 2010. [in Russian].
[6] Mikhaylyuk M.V.,
Huraskin I.A. Synthesis of stereo images for virtual reality systems using an
optical tracking system / Software&Systems 2006. ¹ 3. p. 10-14. [in Russian].
[7] Andreev S.V. [et al]
Modelling and visualisation of blade assembly with complicated shape for power
turbine / Scientific Visualization. 2015. v.7. ¹ 4. p.1-12
[8] Torgashev M.A.,
P.Y. Timokhin. The technology of stereo video files’ synthesis for the system
of 3D real-time visualization / Software Products and Systems, 2012, ¹ 3, pp.
74-80. [In Russian]
[9] Torgashev M.A. Implementation
of stereo mode for various devices for real time displaying / Software Products
and Systems, 2010, ¹ 2, pp. 23-29. [In Russian]
[10] Mikhaylyuk M.V.,
Maltsev A.V. Timokhin P.Yu. The methods of 3D stereo visualizationof data
obtainedin simulation of unstable oil displacement from porous media /
Proceedings of Scientific Research Institute for System Analysis RAS, 2018,
v.8, N 2, p. 125-129. [in Russian].
[11] Visibility
Experiment and Evaluation of 3D Character Representation on Mobile Displays /
Hiromu Ishio [et al.] // C. Stephanidis (Ed.): Posters, Part II, HCII 2011,
CCIS 174, pp. 46–51, 2011.
[12] Andreev S.V.,
Bondareva N.A. Constructing a representation of textual information in stereo
presentations // Proceedings of the 28-th International Conference of Computer
Graphics and Vision GraphiCon-2018, Tomsk, Russia, 23-28 September 2018, p.
86-89. [in Russian].
[13] S.V.Andreev,
A.E.Bondarev, V.A.Galaktionov, N.A.Bondareva (2018) The problems of stereo
animations construction on modern stereo devices. Scientific Visualization
10.4: 40 - 52, DOI: 10.26583/sv.10.4.04
[14] S.V.
Andreev, N.A. Bondareva, E.Yu. Denisov. Stereo Presentations
Problems of Textual information on an Autostereoscopic Monitor (2019). Scientific
Visualization 11.4: 90 - 101, DOI: 10.26583/sv.11.4.08
[15] Kalinichenko A.V.,
Sveshnikova N.V., Yurin D.V. Epipolar geometry and evaluation of its
reliability from the results of the restoration of the three-dimensional scene
by factorization algorithms // Proceedings of the 16-th International
Conference of Computer Graphics and Vision GraphiCon-2006, 2006. [in Russian].
[16] 3D Stereoscopic
Photography, http://3dstereophoto.blogspot.com/ last access February 23 2020.
[17]. A.D. Zhdanov,
D.D. Zhdanov, N.N. Bogdanov, I.S. Potemin,V.A. Galaktionov, and M.I. Sorokin.
Discomfort of Visual Perception in Virtual and Mixed Reality Systems // Programming
and Computer Software, 2019, Vol. 45, No. 4, pp. 147–155. DOI:
10.1134/S036176881904011X