%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
\chapter{Wake-Vortex Simulations}\label{chapter:wake}

\section{Wake-Vortex Theory}
\myfigureW{figs/vortexdiag.ps}{vortexdiag}{12cm}
{Schematic of a wake-vortex pair}
\myfigureW{figs/inducedvel.ps}{iv}{10cm}
{Nomenclature used for the velocity induced by a 3D, straight vortex segment (Courtesy Katz and Plotkin~\cite{inducedvel})}

For the wake-vortex simulations described in this thesis, aerodynamic
potential flow theory is used to predict the strength of the wake-vortex
elements~\cite{vortexsimulation}. The circulation produced by the wing
is assumed to be contained in two vortices of opposite signs trailing
from the tips. Figure~\ref{fig:vortexdiag} shows the schematic of a
wake-vortex pair. The wake is assumed to consist of a pair of vortices
which are parallel and the longitudinal axis of the tracked airplane is
assumed to be parallel to the vortex pair. The centers of the vortices
are on a horizontal line separated by a distance of $b_s = \frac{\pi}{4}
b_g$, a result of assuming an elliptic distribution, where $b_s$ is the
separation of the vortices in the wake-vortex pair, and $b_g$ is the
span of the airplane wing generating the wake vortex~\cite{nvortex1}.
The magnitude of the circulation of each vortex element is approximately
\begin{equation}\label{eq:init} |\Gamma| = \frac{4}{\pi}\frac{L_g}{\rho
V_g b_g}, \end{equation} where $L_g$ and $V_g$ are the lift and the
velocity of the aircraft, respectively. References~\cite{tass1,tass2}
deal with more details on the numerical simulation of these
aircraft vortices. The velocity $\vec{V}_{1,2}$ induced by a 3D,
straight vortex segment $(1,2)$ of strength $\Gamma$ at point
$P(x,y,z)$ outside the vortex core as shown in Fig.~\ref{fig:iv} is:
\begin{equation}\label{eq:iv} \vec{V}_{P_{(1,2)}} = \frac{\Gamma}{4\pi}
\frac{\vec{r_1} \times \vec{r_2}}{{|\vec{r_1} \times \vec{r_2}|}^2}
(\vec{r_1}-\vec{r_2}) \cdot \left( \frac{\vec{r_1}}{|\vec{r_1}|} -
\frac{\vec{r_2}}{|\vec{r_2}|} \right). \end{equation} If the point $P$
is within the vortex core of radius $\epsilon$, the induced velocity is
\begin{equation}\label{eq:iv2} \vec{V}_{P_{(1,2)}} = \frac{\Gamma}{2\pi
\epsilon^2} \frac{\vec{r_1} \times \vec{r_2}}{|\vec{r_1}-\vec{r_2}|}.
\end{equation} $\epsilon$ is typically $10\%$ of the chord length of the
airplane wing.

After the strength of these vortices are computed, the effect due
to the prevailing weather data is applied to the prediction. The
vortex filaments propagate with the freestream wind conditions and
the induced velocity due to the other vortex elements. A standard
turbulent atmospheric boundary layer profile is used to compute the wind
velocities close to the ground. The typical height of the atmospheric
boundary layer is taken to be $400$ meters above the ground. Now, in
the wake-vortex pair, the velocity induced by the rotating flow from
one vortex pushes its neighbor down. And the velocity induced by the
rotation from the other vortex, in turn, pushes its neighbor down.
Consequently, the vortices descend toward the ground. Near the ground,
a boundary condition has to be implemented to ensure that the downward
velocity of the vortices goes to zero at the ground plane. This
boundary condition is satisfied by imagining a pair of virtual vortices
under the ground with equal but opposite circulation to the real
vortices. Typically, this boundary condition is enforced only when the
wake-vortices are within $300$ meters of the ground. These boundary
conditions ensure that both the wind velocities and the induced
velocities smoothly go to zero as the vortices approach the ground.

The decay of the vortex strength is based on a simplified version of the
model suggested by Greene~\cite{greene}:
\begin{equation}\label{eq:decay}
\Gamma_{t+\Delta t} = \Gamma_{t}  ( 1  -  \frac{\Delta t V_{t}}{8 b_g}),
\end{equation}
where $V_t$ is the vortex velocity at time $t$ and is given by
$$
V_{t} = \frac{\Gamma_{t}}{2 \pi b_g}.
$$
Here $\Gamma_t$ represents the strength of the vortex element at time $t$
and $\Gamma_{t+\Delta t}$ represents the strength of the vortex at time
$t+\Delta t$ (next time-step).


\section{Simulation Complexity}
%\vspace{-1.7cm}
The wake-vortex prediction for an entire fleet of aircraft taking-off
and landing at a busy airport is an extremely computationally intensive
problem. Not only does the wake-vortex simulation have to run fast
enough to mimic the actual physical phenomena in real-time, but also
the time required to communicate the computed data to the remote
visualization client has to be kept to a minimum. As such, a POSSE-based
parallel solution for the same is required to maintain a real-time
response of the simulation. For example, a typical metropolitan airport
in the USA is extremely busy with several take-offs and landings
occurring every few minutes.

Dallas/Fort Worth, USA's third busiest airport, has seven
runways that handle nearly $2,300$ take-offs and landings every day.
For the wake-vortex code to track the vortices shed by an aircraft for
$5$ miles after take-off, assuming that a vortex core is stored every
5 meters, $5 \times1,600/5\times2=3,200$ vortex filaments have to be
tracked. For $2,300$ take-offs and landings every day, it implies that
$3,200\times2,300/24=306,667$ vortex filaments have to be tracked
every hour. Since the vortices may take as long as 15 minutes to decay
significantly, vortices due to typically half the take-offs and landings
every hour need to be tracked at any given time. This amounts to roughly
$153,333$ vortex filaments. While this may not seem to be a very large
number on its own, the problem gets complicated by the presence of an
$O(N^2)$ calculation for the induced velocity of every vortex element
on every other vortex element, where $N$ represents the number of
vortex elements. Even if the induced velocity effect due to vortices
from the other aircraft are ignored, this still amounts to as much as
$3,200\times3,200=10.24$ million calculations for each airplane at every
timestep.

For $2,300$ planes/day, this comes out to $10.24\times2,300/24/2=490.7$
million calculations per timestep for the induced velocity, a very
large number indeed for a conventional uniprocessor system. And with
each timestep being, say $0.2$ seconds, this amounts to $2.45$ billion
calculations per second. This number can be reduced by as much as a
factor of $100$ by making a simplifying assumption for the induced
velocity calculations. It is assumed that any vortex element is only
affected by a fixed number of neighboring elements, say $k$, on each
stream of the wake-vortex pair rather than all the $N$ elements in each
stream. This reduces the simulation complexity by a factor of $N/4k$.
Here, $k$ is chosen as the smallest acceptable value such that the error
introduced by the approximation on a sample set of runs is a maximum of
$1\%$ when compared to the original $O(N^2)$ solution. The typical value
of $k$ used in these simulations is $10$, which for an $N$ of $4,000$,
is a factor of $100$ lower. However, even a $100$-fold reduction still
amounts to a large computation considering that each induced velocity
calculation consists of $200-300$ floating point operations. This takes
the net computational requirement to approximately $5-8$ Gigaflops,
necessitating the need of a parallel computer. To address the problem of
maintaining real-time response, the wake-vortex prediction code, based
on the potential flow theory described above, is written in C++ with MPI
for parallelization.

\section{Pseudocode}

In Fig.~\ref{fig:pseudocode}, we present the pseudocode for the
wake-vortex simulation. Every continuous wake-vortex is represented by
several discrete ``vortex elements'' within the computation. Each vortex
element has two main properties associated with it, strength, position
and orientation. The initial strength ($\Gamma$) is calculated based on
the potential flow theory (Eq.~\ref{eq:init}) and the initial position
is based on the the position of the aircraft. The vortex strength then
decays as a function of time (Eq.~\ref{eq:decay}) and the prevailing
weather conditions, and the position and orientation changes because of
the velocity induced by neighboring vortex elements (Eq.~\ref{eq:iv})
and the prevailing wind velocity. Fig.~\ref{fig:bigpicture} depicts a
diagram of the complete client/server simulation system. The simulation
system consists of the {\em Wake-Vortex Server}, {\em Airport Data
Server} and the {\em Sound Server}. The Wake-Vortex Server is the
simulation code enhanced using POSSE. This code can be found in
Appendix~\ref{ch:appendixServer}.

The Airport Data Server is another POSSE server that serves the
positions of the aircraft in the vicinity of the airport as well as the
prevailing weather conditions. The Sound Server is an optional component
in the the system for simulating the noise-level at the airport. The
wake-vortex code has been parallelized to track vortex elements from
each aircraft on a different processor in such a way that (if it is
possible) nearly real-time solution to this problem is obtained with
tolerable lag no more than the time-step $\Delta t$ in the simulation.

In the parallel simulation, one processor (usually the first) acts as
the master doing a round-robin scheduling of any new aircraft to be
tracked among the available processors (including itself). The master,
therefore, does the additional work of distributing and collecting vortex
data from the slave nodes. Hence, it is ensured that the master is always
running on a Symmetric Multi-Processor (SMP) node with at least two
processors so that the POSSE server thread runs on an idle processor and
does not slow down the master node because of the constant monitoring of
the vortex data by the visualization client.

\begin{figure} [htb]
\begin{tabbing}
123 \= 123 \= 123 \= 123 \= 123 \= 123 \= 123 \= 123 \+ \kill
$V \leftarrow \emptyset$ \\
$t \leftarrow 0$ \\
{\bf Foreach} aircraft $A$ on a different processor \\
\>      {\bf While} ($A$ in specified range from airport) {\bf do} \\
\>\>       {\bf read} updated aircraft position from {\bf airport data server}\\
\>\>       {\bf read} updated weather condition from {\bf airport data server}\\
\>\>       $V \leftarrow V + \{$newly created vortex element from wing using potential theory$\}$ \\
\>\>      {\bf Foreach} vortex element ($v_i \in V$) \\
\>\>\>              $v_i.inducedvel \leftarrow 0$ \\
\>\>\>          {\bf Foreach} vortex element ($(v_j \in V) \neq v_i$ \& $|j-i| \leq k$) \\
\>\>\>\>              $v_i.inducedvel \leftarrow v_i.inducedvel + \textrm{InducedVelocity}(v_i, v_j)$ \\
\>\>\>          {\bf Endforeach} \\
\>\>\>          $v_{i}.position \leftarrow v_{i}.position + \Delta t \times v_i.inducedvel$ \\
\>\>\>          $v_{i}.position \leftarrow v_{i}.position + \Delta t \times$ (prevailing wind velocity) \\
\>\>\>          $v_{i}.strength \leftarrow v_{i}.strength - \textrm{DecayFunction}(\Delta t, \textrm{Weather Conditions})$ \\
\>\>\>          {\bf If} ($v_{i}.strength < threshold$) {\bf then} \\
\>\>\>\>          $V \leftarrow V - v_i$ \\
\>\>\>          {\bf Endif} \\
\>\>     {\bf Endforeach} \\
\>\>	 $t \leftarrow t + \Delta t$ \\
\>       {\bf Endwhile} \\
{\bf Endforeach} 
\end{tabbing}
\caption{Algorithm for Wake-Vortex prediction}
\label{fig:pseudocode}
\end{figure}

\myfigureW{figs/bigpicture.ps}{bigpicture}{14cm}
{Wake-Vortex Simulation System}

\myfigureW{figs/uml_vortex.ps}{uml_vortex}{16cm}
{UML Diagram for Wake-vortex classes}

\section{Implementation}

To tackle the issues arising when dealing with such a complex
simulation, the wake-vortex application is run as four separate
components: {\em Airport Data Server}, {\em Sound Server}, {\em
Wake-Vortex Server} and {\em Visualization Client}. The airport data
server is written using POSSE and provides the current weather data
and aircraft trajectory to any wake-vortex client that requests it. It
can either read the data from a file or generate random data. For the
sound server, an already existing application, the {\em Bergen sound
server}~\cite{bergen} is used. Like POSSE, Bergen too is written in
C++, is based on UNIX sockets and is portable. It comes with a C++
library with functions to communicate with the server. The third component,
the wake-vortex server is broken into three separate parts:
\begin{enumerate}

\item {\em Numerical}: The numerical portion of the wake-vortex
simulation code is written as a separate library which is independent
of POSSE calls. A UML diagram for this library can be seen in
Fig.~\ref{fig:uml_vortex}. It consists of two main classes: {\tt Wing}
and {\tt Vortex}. The {\tt Wing} class manages all the simulation data
for any single aircraft which includes their physical attributes, their
trajectory information and all the wake-vortex simulation data. The {\tt
Vortex} class consists of a collection of {\tt Wing} objects (using the
C++ STL\footnote{Standard Template Library} vector) and contains methods
for performing all the wake-vortex computations, as well as for the
addition and deletion of aircraft from the structure as and when they
appear or disappear around an airport. In the absence of the optional
airport data-server, an input file specifies the weather conditions, the
number of aircraft in the simulation, their physical attributes, their
location and their trajectories. Appendix~\ref{ch:vortexinp} contains
a sample input and trajectory file.

\item {\em Parallel}: An MPI application is written
using the wake-vortex library to parallelize the simulation 
based on the pseudocode described in Fig.~\ref{fig:pseudocode}.

\item {\em Steering}: The POSSE server component (DataServerMPI) is then
added and relevant simulation data required by the visualization client
is registered. Any locking of registered data is done as required to
avoid data coherence problems. A POSSE client component (DataClient) is
also added to periodically retrieve the trajectory and weather data from
the {\em airport data server}.

\end{enumerate}
The visualization client is broken into three separate parts:
\begin{enumerate}

\item {\em Steering}: The POSSE client component (DataClient) is
used to periodically receive the updated {\tt Vortex} structure from
the wake-vortex server. The {\tt Vortex} structure contains the updated
trajectory and wake-vortex data for all the aircraft being tracked. This
component is run as a separate thread to maintain reasonable response
times for the graphics component of the application, as the receipt of
wake-vortex data is dependent on the amount of data to be communicated
and the network speed, and could thus vary from a few milliseconds to
several seconds.

\item {\em Graphics}: This component deals solely with the rendering of
the display and the user-interaction. The graphics calls are written
using OpenGL and user-interaction is managed by GLUT when running on a
commodity (non-VR) hardware and by CAVELib when running on the RAVE.
The CAVELib support adds the stereoscopic mode to the application and
also the ability to navigate using the wand. The data for the display
is obtained from the {\tt Vortex} structure which is interpreted using
the functions provided by the wake-vortex library. The vortex elements
are connected using a 4th-order B-Spline curve~\cite{foley} to give
a smooth connectivity while maintaining the local control of the
individual vortex elements. The aircraft models are stored as
Wavefront~\cite{wavefront} .OBJ files and are loaded at run-time
using a public domain OBJ file loader written in C. The land, sea
and the sky are drawn using OpenGL mipmaps.

The airport is modeled as a separate class with its own file format.
An example airport input file for San Francisco airport is provided in
Appendix~\ref{ch:airportinp}. The airport class consists of functions
to simulate some intelligence for automatic landing and take-off of
aircraft by queueing requests from aircraft in the vicinity of the
airport and providing them with pre-computed take-off and landing paths
in First-In First-Out (FIFO) order.

The user-interface consists of the arrow-keys and/or the mouse movements
for navigation of the airport. The mouse mode can be used to fly around
the airport at varying speeds. The viewer can also optionally attach
oneself to the aircraft frame of reference by pressing a hot-key. A
compass on the top-right portion of the screen depicts the direction
of the viewer. All important information is displayed on the screen
as text and can be toggled using a hot-key. The top-left corner of the
application window displays the name of the wake-vortex server along
with the number of processors it is running on. It also provides
information on the prevailing wind velocity, the location of the
viewer, the number of aircraft being tracked and their distance from
the center of the airport and the jet noise due to each of them. It
also gives the type and location of each aircraft and the strength
of the vortex-element being formed at its wing-tips. The bottom-left
portion of the screen displays the information on the performance of the
application. It displays the frame-rate for the graphics, the current
time-step of the simulation, the elapsed simulation time, the number of
iterations performed, the computational time required per iteration, and
the effective network bandwidth for the monitoring of wake-vortex data.

\item {\em Sound}: This component calculates the sound pressure
level due to the jet-noise from the aircraft engines at the viewer
location. This is calculated by a simple empirical relationship
which is inexpensive to compute. Figure~\ref{fig:sound_angle}
depicts the distribution around a commercial
jet that is used to calculate the noise level based on the
angle from the jet nozzle~\cite{sound}.
Figure~\ref{fig:sound_distance} depicts the behavior of noise amplitude
as the distance from the jet increases. This is based on the following relation:
$$
dB(\textrm{distance}) = 120-20\log_{10} \left( \frac{\textrm{distance in feet}}{100} \right).
$$
According to the above relation, the noise level is $120$
dB\footnote{decibel: a logarithmic unit of sound intensity} at a
distance of $100$ feet from the jet and decreases by $6$ dB ($\approx
20\log_{10} 2$) every time the distance doubles. The final noise level in
decibels is then sent to the Bergen sound server using Bergen library
calls which then outputs the corresponding audio via the speakers.
Figure~\ref{fig:noise} lists the client code in C++ used to
initialize and communicate with the Bergen sound server.

\myfigureBW{figs/sound_angle.ps}{figs/sound_distance.ps}{sound_angle}{sound_distance}{11cm}
{Sound dB level as a function of angle from the jet axis~\cite{sound}}
{Sound dB level as a function of distance from the jet nozzle}

\begin{figure}[p] 
\begin{small}
\begin{verbatim} 
//****************************************************************************
#include "bergen.h"
#include "vortex.h"
//****************************************************************************
bergenSample *sound;  // Global variable "sound"
//****************************************************************************
// Initialize the connection to the Bergen Sound Server
void InitSound(char *SoundServer)
{
  char s[100];
  sprintf(s, "BERGEN_SERVER=%s", SoundServer);
  putenv(s);  // Set the name of the sound server in the environment

  // Connect to the Bergen sound server specified by "BERGEN_SERVER" above.
  bergenServer *server = new bergenServer;

  // Play the "jet.wav" file in an infinite loop.
  sound = new bergenSample("jet.wav", server);
  sound->setLoop(1);
  sound->play();
}
//****************************************************************************
// Compute and send the current jet-noise level to the Bergen server
void PlaySound(Vortex * V)
{
  double SPL = 0.0;            // Sound Pressure Level

  // Loop over all aircraft present in the structure
  for(int i = V->Begin(); i < V->nwings; i = V->NextWing(i))
    {
      // Get angle and distance of observer from the "i"th aircraft
      AngleAndDistFromPlane(V, i, &angle, &dist);
      SPL += dB2Pressure(SoundDB_distance(dist) * SoundDB_angle(angle));
    }

  // Send the modified amplitude to the Bergen sound server
  sound->setAmplitude(Pressure2dB(SPL));
}
//****************************************************************************
\end{verbatim}
\end{small}
\caption{Bergen client code for simulating noise level}
\label{fig:noise}
\end{figure}

\end{enumerate}

For the real-time simulation, the parallel wake-vortex code runs on COCOA-2
and the visualization code runs on the RAVE. A screenshot of the program
depicting the wake-vortex simulation for a single aircraft is shown in
Fig.~\ref{fig:screenshot}. 
%Looking closely at the figure reveals the Crow
%instability~\cite{crow}, a non-linear feature of the wake-vortex interaction. 
Another screenshot (Fig.~\ref{fig:airport}) shows several
aircraft flying above a model of the San Francisco International airport
(SFO). Another screenshot (Fig.~\ref{fig:airport_jfk}) shows aircraft
flying over a model of the New York--John F. Kennedy International
airport (JFK). The colors of the wake-vortices represent the relative
strength of the vortices with respect to a base value, with red being
maximum and blue being minimum.

\myfigureW{figs/screenshot3.ps}{screenshot}{10cm}
{Screenshot of the Wake-Vortex simulation for a single aircraft from a visualization client}

\myfigureW{figs/screenshot2.ps}{airport}{10cm}
{Screenshot of the Wake-Vortex simulation for several aircraft flying above the San Francisco (SFO) airport}

\myfigureW{figs/jfk.ps}{airport_jfk}{10cm}
{Screenshot of the Wake-Vortex simulation for several aircraft flying above the New York--John F. Kennedy (JFK) airport}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
