With the gradual spread of stereo devices
there is a possibility of research and application of stereo-animation
technologies in scientific researches. There is also an active development of
creation of different types of stereo demonstration complexes. At the moment
there are more than thousands of virtual environment devices and tens of
thousands of presentation complexes in the world, which are successfully
applied in different fields. Serious results have been achieved in aviation and
automobile simulators, crew training systems for merchant ships and warships,
car design tasks, research and development of nano-technologies, surgical
operations training and other fields. Stereo technology is being actively
explored and applied in media services as well. The introduction of
three-dimensional visualization into these areas successfully contributes to
the development and application of the latest achievements of technology in
various fields of activity.
Stereo image construction has two main
directions: presentation and research. Presentation direction allows to present
the results of scientific or design works to expert groups and decision makers
in the most informative and accessible way, as three-dimensional representation
promotes popularization of carried out researches. Stereo image allows them to
get a complete picture of the modeled object.
The research direction of stereo images
allows researchers to see an object or physical phenomenon in volume and gain a
deep and clear understanding of the object, phenomenon or process being
studied. With the help of stereo images, researchers can gain a new perspective
and a better understanding of their research. A three-dimensional stereo model
helps to verify computational models or complex designs, and provides a more
complete understanding of the simulated phenomena to the observer.
In the scientific and technical sphere,
stereo image construction is a universal tool that can be effectively applied
in various fields. It finds its application in engineering and design
developments, mathematical modeling of complex objects and, in particular, in
medical technologies. It is important to note that the joint work of scientists
and specialists in various fields of science, such as medicine, computer
graphics and mathematical modeling, is of great importance and has wide
application prospects.
Currently, the direction of stereoscopy and
stereoimage creation is being actively researched and developed in application
to various fields of science. This was preceded by a large cycle of works
devoted to the research and development of methods for creating stereo images
of various objects under study [1-10]. For example, specific problems arising
when using a system of computers for generation and visualization of a
composite multiscreen stereo frame and methods of solving such problems are
described in [1-3].
The papers [3-6] describe the results of
studies made at Keldysh IPM RAS on the basis of available stereo devices of the
main two types. One of them is the Dimenco DM654MAS autostereoscopic monitor
capable of displaying stereo images without the need to track the observer's
position. The second type is a classic type of stereo device. Typically,
autostereoscopic monitors provide an opportunity to observe stereo images by
providing several fixed segments in space for observation, between which the
viewer can move, viewing the object under study in 3D from different angles of
view.
Autostereoscopic monitor is capable of
demonstrating the visualization object using two methods: either using a
composite frame containing views of the visualization object from different
angles (Fig.1), which form a certain viewing sector - this method is called
multi-view - or using depth maps. When constructing a complex stereo frame, the
stereo scene construction itself and object placement on it, volume, object
depth and even color play a great role. In works [4, 5] the step-by-step
process of developing such technology of constructing stereo images combined
with stereo text as multi-view representation was considered in detail. This
technology allows to achieve the highest stereo effect for visualization of
results for mathematical modeling calculations, which are performed by Keldysh
Institute of Applied Mathematics of the Russian Academy of Sciences [6, 9, 19].
Figure 1 shows an image of the simulation
results of supersonic cone flow at angle of attack with the corresponding
inscription. This is one of the results of earlier studies - a multiview image
of the results of modeling the supersonic flow of a cone at the angle of attack
with the corresponding inscription [6]. Here the image of the modeled cone
itself and separately the inscriptions to it are combined. Each of them is
rotated to a different experimentally detected angle. As shown in the figure, a
matrix of images is further compiled, which in turn form a single stereo image.
In the end, the inscription was located on top of the cone, but behind its tip,
which in turn was perceived by viewers as protruding from the screen by several
tens of centimeters.
Figure
1. Image of the simulation results of supersonic cone streamline with the
corresponding inscription
Currently, the problems of stereo image
construction are considered in various research areas [3-18]. For example, paper
[3] presents the results of displaying a SuperNova explosion in stereo mode.
The work [9] is devoted to the creation of computational technology for
modeling the operation and visualization of a three-dimensional assembly of
blades of a power plant in the flow of viscous compressible heat-conducting
gas. Many works are devoted to the issues of searching for the optimal stereo
effect [13-18], but universal recipes for obtaining the most optimal stereo
effect have not been created yet.
Stereoscopy can also be used in medical
research. For example, in [19], computer tomography data were visualized,
including stereo images. Such visualization allows specialists in medicine to
detect pathologies in patients more effectively. The possibility of using
stereo images in maxillofacial surgery to demonstrate to patients future models
of changes in the oral cavity and patient's appearance and more accurate
prognosis on the prescribed treatment is also envisioned. This will allow for
more accurate correction of the physician's actions and achieve
physician-patient rapport. Figure 2 shows the results of craniofacial computed
tomography of a patient, visualized in volumetric form on the autostereoscopic
monitor screen and accompanied by an appropriate caption.
Figure
2. A frame constructed for an autostereoscopic monitor for visualizing CT scan
results.
Stereo images become no less demanded in
the field of biological research, in particular, in the study and analysis of
electric fields of the human body. In this work, stereo imaging technology was
used to visualize functional tomograms of the brain and the results are
presented.
As the data of biological studies, we used
the data of tasks, which have been dealt with by the team of IMPB RAS (a branch
of the Keldysh Institute of Mathematics and Mathematics of the Russian Academy
of Sciences) for quite a long time. An example of such a task is the research
devoted to the analysis of functional tomograms of complex systems and
consideration of methods of filtering of registered signals to obtain a
reliable spatial configuration, with visual display of which users can work
[20].
Figure 3 shows a functional tomogram in
different frequency ranges [20]. The results are shown as three-dimensional
images presented from different angles. Similar three-dimensional images can be
visualized in stereo mode.
Fig.
3. Functional tomogram in different frequency bands. Panel A shows a functional
tomogram in the 8–13 Hz band (alpha-rhythm). Panel B shows a functional
tomogram in the 30–100 Hz band (muscle activity) [20].
A functional brain tomogram was chosen for
demonstration. The method for calculating such functional tomograms is
described in detail in [21].
Magnetic encephalogram (MEG) recordings
were obtained from a healthy adult male subject, 32 years of age, at the Center
for Neuromagnetism at New York University School of Medicine. The subject was
asked to relax but remain awake during the 5-minute recording. The recording
was made in the “eyes closed” state. Three reference markers were used to
determine head position during recording (one each on the right and left
preauricular points and one on the bridge of the nose). Magnetic encephalogram
(MEG) measurements were taken in a magnetically shielded mu-metal room on a
275-channel magnetic encephalograph (CTF Systems), with the subject sitting
upright, and the sampling rate was 1200 Hz. A 3rd order synthetic gradiometer
was used to suppress artifacts and distant noise. The instrument's own noise
and distant noise were recorded before each measurement session.
The magnetic encephalogram was analyzed
using the functional tomography method, based on the Fourier transform and
solving the inverse problem for all frequencies. In this method, each frequency
component is assigned one spatial position.
The next stage of the analysis was
segmentation of magnetic resonance imaging (MRI), the result of which is an
annotated three-dimensional map of the brain, in which each elementary cell
(voxel) of the tomogram has a sign of belonging to one or another part of the
brain.
A voxel mask of the brain was constructed
and its intersection with the full functional tomogram was plotted. The result
was a list of all sources related to the brain. In the next step, all unique
source positions were found, and the sums of powers were calculated for them.
To graphically display the obtained power values, a LUT table was built that
connects power and color.
The Blender software package was used to
construct the three-dimensional representation. The coordinates and color
values of the elements of the functional tomogram were transferred to it, from
which a three-dimensional scene was constructed using the blender-plots library
[22].
When using 3D scene data as initial data
for constructing a stereo image, the following curious circumstance was
revealed. Existing proprietary libraries for constructing stereo images were
primarily focused on ordered data in certain areas. Similar data is provided by
data visualization systems such as Tecplot. However, these systems are very
poor at visualizing a disordered set of points. It was precisely this data,
visualized in the Blender visualization system, that needed to be used to
construct a stereo image in this task. This problem was solved by reorienting
the author's libraries for constructing stereo images to work with data
obtained using Blender. On the one hand, this circumstance brought certain
difficulties to the work, but on the other hand, it significantly expanded the
capabilities of existing proprietary libraries for constructing stereo images
and the proprietary software package StereoMaker 2.0, designed for constructing
stereo images with accompanying objects (labels, additional icons, etc.).
As a result, an image was constructed that
was adapted to be presented in multi-view mode on an autostereoscopic monitor.
Figure
4. Frame of an autostereoscopic monitor for visualizing the results of a
functional tomogram
To construct the multi-view image presented
in Figure 5, a special composite frame was created, when demonstrated on an
autostereoscopic monitor, the user could see the modeling results in a
three-dimensional and informative form.
Figure
5. The resulting composite frame of an autostereoscopic monitor for visualizing
the results of modeling a functional brain tomogram
This work is a continuation of a series of
works devoted to the implementation of a project to study and construct stereo
images of the results of solving mathematical modeling problems. The
construction of stereo frames was carried out in a previously developed mode of
combining the main image object and the corresponding text inscriptions and
designations in one stereo frame. The constructed stereo frames provide the
researcher with the opportunity for a deep and thorough visual analysis of the
results obtained. The results of work on constructing a functional tomogram of
the brain on an autostereoscopic monitor are presented.
A number of computations were carried out
using the K-100 hybrid supercomputer installed at the Center for Collective Use
of Keldysh Institute of Applied Mathematics RAS
[1]
Andreev S.V. [et al]. Synthesis of photorealistic
three-dimensional images in modern presentation systems //
Software&Systems, 2007. N 3. P. 37-40.
[2]
Vandanov V.G. [et al]. Hardware-software complex
of 3D presentations based on a virtual studio and virtual environment //
Proceedings of the 1-st international conference «3D visualization of
scientific, technical and social reality. Cluster technologies of modeling».
Izhevsk, 2009, P.73-77. GraphiCon 2022 Íàó÷íàÿ âèçóàëèçàöèÿ è âèçóàëüíàÿ
àíàëèòèêà 19–22 ñåíòÿáðÿ 2022, Ðÿçàíü, Ðîññèÿ 439
[3]
Andreev S., Filina A. Using stereo presentations for
visualization of scientific calculations results // Scientific Visualization.
2012.
v.4. ¹ 1.
P.12-21.
[4]
Andreev S.V., Bondareva N.A. Constructing a
representation of textual information in stereo presentations // Proceedings of
the 28-th International Conference of Computer Graphics and Vision
GraphiCon-2018. Tomsk, 2018. P. 86-89
[5]
Andreev S.V., Bondarev A.E., Bondareva N.A. Stereoscopic
construction of textual information in presentations of research results on an
autostereoscopic monitor // Scientific Visualization 2020 12.1: 132 - 139, DOI:
10.26583/sv.12.1.12. Scientific Visualization and Visual Analytics GraphiCon
2022 440 19–22 September 2022, Ryazan, Russia
[6]
Andreev S.V., Bondareva N.A., Bondarev A.E.
Expansion of the Functions of the Multi-View Stereomaker Software for Automatic
Construction of Complex Stereo Images // Scientific Visualization. 2021. Vol.
13. N 2. P. 149-156. DOI: 10.26583/sv.13.2.10
[7]
Mezhenin A.V., Tozik V.G. 3D Visualization using
the stereo image effect // Proceedings of the 2-nd international conference «3D
visualization of scientific, technical and social reality. Cluster technologies
of modeling» — Izhevsk, 2010.
[8]
Mikhaylyuk M.V., Huraskin I.A. Synthesis of
stereo images for virtual reality systems using an optical tracking system //
Software&Systems. 2006. N 3. P. 10-14.
[9]
Andreev S.V. [et al] Modelling and visualisation
of blade assembly with complicated shape for power turbine / Scientific
Visualization. 2015. Vol. 7. N 4. P. 1-12.
[10]
Torgashev M.A., P.Y. Timokhin. The technology of
stereo video files’ synthesis for the system of 3D real-time visualization. //
Software Products and Systems. 2012. N 3. P. 74-80.
[11]
Hiromu Ishio [et al]. Visibility Experiment and
Evaluation of 3D Character Representation on Mobile Displays // HCII, CCIS 174,
2011, P. 46-51.
[12]
Zhdanov A.D., Zhdanov D.D., Bogdanov N.N.,
Potemin I.S., Galaktionov V.A., Sorokin M.I. Discomfort of Visual Perception in
Virtual and Mixed Reality Systems // Programming and Computer Software. 2019.
Vol. 45. N 4. P. 147-155. DOI: 10.1134/S036176881904011X.
[13]
Zhou J., Jiang G., Mao X., Yu M., Shao F., Peng
Z. [et al]. Subjective quality analyses of stereoscopic images in 3DTV system.
// Visual Communications and Image Processing VCIP. IEEE, 2011. P. 1-4.
[14]
Lee J.S., Goldmann L., Ebrahimi T. Paired
comparison-based subjective quality assessment of stereoscopic images //
Multimedia tools and applications. 2013. Vol. 67. N 1. P. 31-48. [13]
IJsselsteijn W.A., de Ridder H., Vliegen J. Subjective evaluation of
stereoscopic images: effects of camera parameters and display duration //
Circuits and Systems for Video Technology, IEEE Transactions. 2000. Vol. 10. N
2. P. 225-233.
[15]
Wöpking M. Viewing comfort with
stereoscopic pictures: An experimental study on the subjective effects of
disparity magnitude and depth of focus // Journal of the society for
information display. 1995. Vol. 3. N 3. P. 101-103.
[16]
Akhter R., Sazzad Z.P., Horita Y., Baltes J.
No-reference stereoscopic image quality assessment. // IS&T/SPIE Electronic
Imaging. International Society for Optics and Photonics. 2010. P.
75240T-75240T.
[17]
Boev A., Gotchev A., Egiazarian K., Aksay A.,
Aka G.B. Towards compound stereo-video quality metric: a specific encoder-based
framework // Image Analysis and Interpretation, IEEE Southwest Symposium. 2006.
P. 218-222.
[18]
You J., Xing L., Perkis A., Wang X. Perceptual
quality assessment for stereoscopic images based on 2D image quality metrics
and disparity analysis // Proc. of International Workshop on Video Processing
and Quality Metrics for Consumer Electronics. Scottsdale, AZ, USA, 2010.
[19]
Kobets K.K., Andreev S.V., Bondarev A.E.
Stereoanimation in Maxillofacial Surgery // Scientific Visualization. 2022.
Vol. 14. N 1. P. 121-133. DOI: 10.26583/sv.14.1.10.
[20]
Rykunov S.D., Oplachko E.S., Ustinin M.N.
FTViewer Application for Analysis and Visualization of Functional Tomograms of
Complex Systems // ISSN 1054-6618, Pattern Recognition and Image Analysis,
2020, Vol. 30, No. 4, pp. 727–736. DOI: 10.1134/S1054661820040227
[21]
Llinás RR, Rykunov S., Walton KD, Boyko A
and Ustinin M. Splitting of the magnetic encephalogram into «brain» and
«non-brain» physiological signals based on the joint analysis of
frequency-pattern functional tomograms and magnetic resonance images // Front.
Neural Circuits 2022, 16:834434. DOI: 10.3389/fncir.2022.834434()
[22]
Linusnie blender-plots URL:
https://github.com/Linusnie/blender-plots
