The emergence, development and gradual spread of stereo
equipment provides an opportunity for research in the field of stereo-animation
technologies and their application in scientific developments. The construction
of stereo-animation representations of scientific research results using such stereo
devices opens up completely new possibilities the significance and relevance of
which can hardly be underestimated.
When working with a flat image of an object, a situation
often arises when such image does not have the completeness of information required
to achieve its goals and limits the analysis to a schematic image or a truncated
viewing angle. Unlike a two-dimensional flat image, three-dimensional stereo allows
exploring more effectively and modeling objects with a developed spatial structure
or interpreting pictures, made in the course of the experiment from different angles
at specific points in time and combined into an animation.
The representation of numerical simulation results that
display the flow of complex physical processes over time or show the operation of
complex technical installations in three-dimensional representation allows implementing
a wide range of features. Firstly, a three-dimensional representation of the process
development over time helps to verify the computational model and applied algorithms.
Secondly, it provides the observer with a full understanding of the simulated phenomenon.
Finally, it popularizes the research and its results for society, including for
decision makers. All of the above makes theoretical and practical developments in
the field of building stereo representations very important.
With the advent and development of stereo installations,
more and more works devoted to the construction of stereo images began to appear.
Specific problems that arise when using a computer system to generate and visualize
a composite multi-screen stereo frame, and methods for solving such problems are
described in sufficient detail in [1-3]. At first, most research in the field of
stereo images related to the tasks of building virtual reality, simulators and trainers,
as well as training systems. However, later the issue of building presentation complexes
became very relevant, which also allowed demonstrating the results of scientific
research in stereoscopic mode. These works include works [3-10]. In [3], the results
of displaying a Supernova explosion in stereo mode are presented. The work [7] is
devoted to the creation of a computational technology for modeling the operation
of a three-dimensional node of power plant blades when the flow of a viscous compressible
heat-conducting gas flows around. Currently, the problems of constructing stereo
images are considered in a wide variety of research areas [3-9].
However, when constructing a stereo image of the scientific
calculations’ results, a rather important particular structural subproblem arises.
As a rule, the standard presentation of a scientific report includes not only visualization
of complex geometry or fields of physical quantities, but also explanations, usually
represented by text and formulas. If for geometries and physical quantities the
issues of constructing stereo representations are sufficiently developed, then the
tasks of combining them with stereo images of texts and formulas with sufficient
expressiveness and the necessary stereo effect are still open.
As for the development of specific practical approaches
in the field of combining images and textual information to it, quite a few works
have been devoted to this topic. Only the work [10] can be mentioned here describing
a study conducted in Japan on the perception of stereo images of fonts on the screens
of stereoscopic mobile devices. However, this research was more social in nature
rather than providing recipes for constructing stereo images of texts and formulas.
Construction of stereo images is possible using a fairly
wide range of different technologies: from classical stereoscopy to the use of depth
maps. The choice of technology depends on the type of affordable stereo system,
the field of application, and directly on the goals of creating the stereo image.
Unlike the film industry in scientific research topics,
the given area is still developing and adapting to its needs and specific requirements.
Precisely depending on the transmission accuracy of the original images, values
of the required stereo effect, correspondence to the research topic, one or another
technology for creating stereo images is defined and selected.
This work is a continuation of the cycle of research
works [1-3,7,11-14] carried out at the Keldysh Institute of Applied Mathematics
on the basis of existing stereo devices. The main type of device, that the stereo
imaging software system described in this paper is designed to work with, is the
Dimenco DM654MAS autostereoscopic monitor. Autostereoscopic monitors provide the
display of stereo images without having to track the observer's position. Usually
these monitors allow observing stereo images, providing a few fixed segments in
the observation space. The spectator can move between segments, getting the opportunity
to view the displayed object in 3D from different angles of view. The operation
principle of auto-stereoscopic monitor is the use of parallax partitions or Fresnel
lenses installed behind the protective glass of the screen, which gives it one of
the most important advantages: the image display does not require the viewer to
have special glasses or other devices.
An autostereoscopic monitor is capable of demonstrating
the visualization object in two ways: either using a composite frame containing
views of the visualization object from different angles that form a certain sector
of the review– this method is called multi-view - or using depth maps.
In [12,13], the step-by-step process of developing
such a technology for creating stereo images matched with stereo text was considered
in detail . These stereo images were constructed on the base of multi-view technology.
This technology allows achieving the highest stereo effect to visualize the results
of mathematical modeling which is engaged at the Keldysh Institute of Applied Mathematics[7,14,15].
However, it is worth noting that until now, the construction
of stereo frames was carried out almost manually, even if using a well-established
technology, by applying various programs and sequential processing of each frame
with setting and configuring all the necessary parameters. This process required
a large amount of time and a high degree of accuracy and care.
This work is devoted to the development of the task
of automating this process and providing automatic frame generation where multiple
objects or inscriptions could be placed, each of them has its own stereo base.
Autostereoscopic monitor has the ability to demonstrate
the visualized object
using a composite frame containing views
of the visualization object at different angles,
which
together form a certain sector of the review.
Usually,
there are nine types.
These nine species form eight stereo
pairs ([1/2], [2/3], [3/4],..., [8/9]),
however,
the viewer can only watch one of the stereo pairs in one position,
depending
being in a certain corner sector of observation.
When
the observer moves from sector to sector,
he
gets
information about the object using all nine angles,
that
is, as if inspecting at the visualized object from different sides.
When building a multi-view stereo image, nine frames
are connected into one image
based on the 3×3 matrix principle.
In
the first and last (ninth) frames,
the represented object is in its extreme
positions.
In
the first frame, the object is in its rightmost position,
in
the last one - in the far left.
Next in the middle frames,
respectively
from the second to the eighth,
with certain equal intervals, it shifts
to the left,
until
it reaches the leftmost position.
Herewith, in each frame, the object
turns progressively at the same angle and it is shifted by the specified interval.
This
allows achieving the most effective result when building a stereo image of an object.
With too large parameters of the angular shift, artifacts
occur: the observer has a feeling that the image in front of him is blurring at
the transitions from one stereo pair to another,
and
at the slightest change of position, the observer notices the highlights and borders
of each of the stereopairs.
Similar phenomena can cause a certain
discomfort in the observer,
which interferes with work
with stereo images
and adequate perception of visual information on a stereo
frame. The problems of visual perception discomfort in virtual and mixed reality
systems are described in detail in [17].
After
solving the problem of creating a multi-view representation of an object, in the
investigated
case of
stereo
labels, in the quality separate frames in a stereo presentation or stereo film,
an equally important sub-task was considered- matching the image and text information
in one frame.
Currently, in the case of demonstrating scientific results
to observers, very many objects require accompanying information situated directly
on the same frame as the image.
In many cases, the caption and the object
can
not be
distributed
into different frames,
because they constitute a single logical
display of information.
To construct such a stereo image in which both the object
and the caption to it would be combined,
an
image matrix is used (Fig. 1),
to
which three-dimensional letters were added and sequentially shifted and rotated
at each frame by the same interval (angular stereobase of the object), previously
calculated experimentally.
The result was a stereo image with a
three-dimensional
caption,
where
the stereo effect was pronounced.
Figure 1 presents an image of the results of modeling
supersonic flow around a cone at an angle of attack with the corresponding caption. This is one of the results of previous research- a multi-view image of the results
of modeling the supersonic flow around the cone at an angle of attack with the corresponding
inscription [16].
Here, the very image of the modeled cone
and separate inscriptions to it are matched.
Each
of them is rotated to its own experimentally determined angle.
As
shown in the figure,
then a matrix of images is compiled,
which
in turn make up a single stereo image.
Ultimately, the inscription
was placed over the cone, but behind its point, which in turn, was perceived by
the audience as protruding from the screen by several tens of centimeters.
Fig.
1. Image of the results of modeling supersonic flow around a cone with the corresponding
caption [16].
In
the process of research, the following technology for constructing a similar stereo
frame was worked out,
which is
performed
manually by the user:
1)
The
rotation of the object (the result of visualization of scientific computations)
on the Z axis in the Tecplot program by a given degree and subsequent saving of
the frame with each rotation;
2)
Creating
9 frames for every
rotation:
1st
frame -
extreme
left position, 9 -
the extreme right;
3)
Adding
the required text (having volume and clarity) for each frame;
4)
Consecutive
rotation of the text by the required fraction of a degree
in
each frame;
5)
Stitching
all frames into an image matrix in certain sequence.
Building
manually nine versions of such a stereo frame, which there are present several objects
in, each of which has its own stereo base, takes a lot of time. It is necessary
to build nine views for each object, and then match them, without violating the
integrity of the final stereo image. The main difficulty in building a stereo image
with matching several stereo objects is the need for multiple repetitions of the
same operation while generating the same stereo image.Multi-view StereoMaker software
package was developed to automate this process.
Multi-view
StereoMaker program represents a software package,
allowing
to create a set of nine images / representations of rotation of three-dimensional
objects on a given angular stereo base,
and
then get their union, matched into one image.
Multi-view
StereoMaker was implemented in Python 3.6.
In view of
the fact that there were no restrictions on the speed of the software package,
when choosing given language, the following criteria as
cross-platform, simplicity of the language and its broad support for optional ready
functionality were taken into account.
The
program is designed to work with arbitrary 3D objects with specified parameters.
As
such 3D objects can be the results of scientific calculations and accompanying information
to them, such as accompanying captions, formulas, additional images. Because of
the visualization of the scientific calculations used was carried out in the Tecplot
software package,
then Multi-view StereoMaker additionally
organizes the implementation of interaction with this software package. The program
Blender was chosen as a software package for generating arbitrary 3D images, which
is open source and free software with extensive support for the python language.
A screenshot of the Multi-view StereoMaker interface is shown in figure 2.
Among the main functions of the program, the particularly
important are selection of the stereo image generation algorithm for each object,
the ability to preview the generated images according to the specified parameters
for each object, setting the parameters of
generated
images
on top of each other by the user- specified coordinates and also the
selection
of the way to save the final result.
Among the algorithms for generating stereo images,
there is a category of "custom images", where the user can respectively
specify the directory where his images are. When choosing the Tecplot category,
the user must specify file in plt format with necessary model and when setting the
necessary parameters
specify the size and width of the output
image, the initial rotation angle,
the number of rotations,
the object rotation angle, the axis of rotation.
The last category - the Blender algorithm -
assumes
generating a 3D object either from the original user image in the .svg format, or
by using internal text generation, as well as indication of additional parameters
such as the color and depth of the 3D object, its scale, size and width of the output
image, starting angle of rotation, number of turns, axis of rotation, etc.
Fig.
2. Screenshot of the interface of the Multi-view StereoMaker software package.
The
program includes the following components, shown in the diagram (Fig. 3.). The user
must enter the parameters using the provided graphical interface (GUI).The graphical
interface allows referring to the generated images preview subsystem for each object.
The parameters entered by the user are transmitted to the preliminary data subsystem,
which creates a directory system for saving the results of algorithms. Then the
parameters are passed to the algorithm generator.
Fig.
3. Organization diagram of the Multi-view StereoMaker software package.
The algorithm generator generates the appropriate algorithms
for the external programs’ operation (Tecplot, Blender), after which the obtained
algorithms and parameters are transferred to the subsystem of interaction with external
programs and data. In the case of transferring custom images, given subsystem will
copy the custom images to the resulting directory and transfer them to the image
gluing subsystem. In all other cases, the given subsystem will launch external programs
with the algorithms specified for them. Then there is a call to the subsystem of
gluing images, which performs multiple superposition of a number of images formed
for two objects. The result of the program’s work is a directory containing algorithms
formed for a specific case for external programs, directories with generated images
for objects with user-specified parameters, and a directory with the final result.
Figure 4 presents the results of the operation of the
Multi-view StereoMaker software package, demonstrating three-dimensional visualization
of multicriteria alternatives using the histograms described in [18].
Fig.
4. 3D visualization of three multi-criteria alternatives using histograms.
This
project is not completed. Only the first version of the developed program is presented.
It is planned to further develop and improve the program for combining multiple
objects and interactively placing them in a stereo frame with the simultaneous generation
of nine views for each. The additional options for recording statistical data will
also be developed.
The results of constructing a stereo image for an autostereoscopic
monitor
by
the method of multi-view representation using the Multi-view StereoMaker software
package
are
presented.
In
the course of practical experiments, an optimal technology for constructing multi-view
stereo images of the results of solving mathematical modeling problems has been
developed.
The goal of this work is to automate manual processes
and development of a program for constructing stereo images with textual accompaniment
for creating stereo presentations.
In the long term, further
debugging and testing of the program and automatic conversion to the required autostereoscopic
monitor
format,
as well as solving the problem of interactively placing objects in one stereo frame
with the simultaneous generation of nine views for each are expected.
[1] Synthesis
of photorealistic three-dimensional images in modern presentation systems / Andreev
S.V.[et al] // Software&Systems 2007. ¹ 3. p. 37-40. [in Russian]. Hardware-software
complex of 3D presentations based on a virtual studio and virtual environment /
Vandanov V.G. [et al] // Proceedings of the 1-st international conference «3D visualization
of scientific, technical and social reality. Cluster technologies of modeling» Izhevsk.
2009. P.73-77. [in Russian].
[2] Generation
of Stereo-Presentations in Photorealistic Rendering and Scientific Visualization
/Andreev S.V.[et al] // Keldysh Institute preprints, 2010. ¹ 61. 16 p. http://library.keldysh.ru/preprint.asp?id=2010-61[in
Russian].
[3] Andreev
S., Filina A. Using stereo presentations for visualization of scientific calculations
results / Scientific Visualization. 2012. v.4. ¹ 1. p.12-21
[4] Hardware-software
complex of 3D presentations based on a virtual studio and virtual environment /
Vandanov V.G. [et al] // Proceedings of the 1-st international conference «3D visualization
of scientific, technical and social reality. Cluster technologies of model-ing»
Izhevsk. 2009. P.73-77. [in Russian].
[5] Mezhenin
A.V., Tozik V.G. 3D Visualization using the stereo image effect // Proceedings of
the 2-nd international conference «3D visualization of scientific, technical and
social reality. Cluster technologies of modeling» Izhevsk. 2010. [in Russian].
[6] Mikhaylyuk
M.V., Huraskin I.A. Synthesis of stereo images for virtual reality systems using
an optical tracking system / Software&Systems 2006. ¹ 3. p. 10-14. [in Russian].
[7] Andreev
S.V. [et al] Modelling and visualisation of blade assembly with complicated shape
for power turbine / Scientific Visualization. 2015. v.7. ¹ 4. p.1-12
[8] Torgashev
M.A., P.Y. Timokhin. The technology of stereo video files’ synthesis for the system
of 3D real-time visualization. Software Products and Systems, 2012, ¹ 3, pp. 74-80.
[In Russian]
[9] Mikhaylyuk
M.V., Maltsev A.V. Timokhin P.Yu. The methods of 3D stereo visualizationof data
obtainedin simulation of unstable oil displacement from porous media / Proceedings
of Scientific Research Institute for System Analysis RAS, 2018, v.8, N 2, p. 125-129.
[in Russian].
[10] Visibility
Experiment and Evaluation of 3D Character Representation on Mobile Displays / Hiromu
Ishio [et al.] // C. Stephanidis (Ed.): Posters, Part II, HCII 2011, CCIS 174, pp.
46–51, 2011.
[11] Andreev
S.V., Bondareva N.A. Constructing a representation of textual information in stereo
presentations // Proceedings of the 28-th International Conference of Computer Graphics
and Vision GraphiCon-2018, Tomsk, Russia, 23-28 September 2018, p. 86-89. [in Russian].
[12] S.V.Andreev,
A.E.Bondarev, V.A.Galaktionov, N.A.Bondareva (2018) The problems of stereo animations
construction on modern stereo devices. Scientific Visualization 10.4: 40 - 52, DOI:
10.26583/sv.10.4.04
[13] S.V.
Andreev, N.A. Bondareva, E.Yu. Denisov. Stereo Presentations Problems of Textual
information on an Autostereoscopic Monitor (2019). Scientific Visualization 11.4:
90 - 101, DOI: 10.26583/sv.11.4.08
[14] S.V.
Andreev, A.E. Bondarev, N.A. Bondareva. Stereo images of error surfaces in problems
of numerical methods verification (2020). Íàó÷íàÿ âèçóàëèçàöèÿ 12.2: 151 - 157,
DOI: 10.26583/sv.12.2.12
[15]
Stereo animation "Modeling and visualization of the work of a power plant having
complex shape in the stereo animation mode". https://yadi.sk/i/3CcYhlMB3TCH4r
(Accessed 01.11.2020).
[16] S.V.
Andreev, A.E. Bondarev, N.A. Bondareva. Stereoscopic construction of textual information
in presentations of research results on an autostereoscopic monitor (2020). Scientific
Visualization 12.1: 132 - 139, DOI: 10.26583/sv.12.1.12
[17] A.D.
Zhdanov, D.D. Zhdanov, N.N. Bogdanov, I.S. Potemin,V.A. Galaktionov, and M.I. Sorokin.
Discomfort of Visual Perception in Virtual and Mixed Reality Systems // Programming
and Computer Software, 2019, Vol. 45, No. 4, pp. 147–155. DOI: 10.1134/S036176881904011X
[18] A.A.
Zakharova, D.A. Korostelyov. Application of Visual Analytics Methods to Reduce the
Dimensionality of Decision-making Problems (2020). Scientific Visualization 12.4:
23 - 32, DOI: 10.26583/sv.12.4.03
