mirror of
https://gitlab.com/libvirt/libvirt.git
synced 2024-09-09 09:14:57 +00:00
Auditing all callers of virCommandRun and virCommandWait that passed a non-NULL pointer for exit status turned up some interesting observations. Many callers were merely passing a pointer to avoid the overall command dying, but without caring what the exit status was - but these callers would be better off treating a child death by signal as an abnormal exit. Other callers were actually acting on the status, but not all of them remembered to filter by WIFEXITED and convert with WEXITSTATUS; depending on the platform, this can result in a status being reported as 256 times too big. And among those that correctly parse the output, it gets rather verbose. Finally, there were the callers that explicitly checked that the status was 0, and gave their own message, but with fewer details than what virCommand gives for free. So the best idea is to move the complexity out of callers and into virCommand - by default, we return the actual exit status already cleaned through WEXITSTATUS and treat signals as a failed command; but the few callers that care can ask for raw status and act on it themselves. * src/util/vircommand.h (virCommandRawStatus): New prototype. * src/libvirt_private.syms (util/command.h): Export it. * docs/internals/command.html.in: Document it. * src/util/vircommand.c (virCommandRawStatus): New function. (virCommandWait): Adjust semantics. * tests/commandtest.c (test1): Test it. * daemon/remote.c (remoteDispatchAuthPolkit): Adjust callers. * src/access/viraccessdriverpolkit.c (virAccessDriverPolkitCheck): Likewise. * src/fdstream.c (virFDStreamCloseInt): Likewise. * src/lxc/lxc_process.c (virLXCProcessStart): Likewise. * src/qemu/qemu_command.c (qemuCreateInBridgePortWithHelper): Likewise. * src/xen/xen_driver.c (xenUnifiedXendProbe): Simplify. * tests/reconnect.c (mymain): Likewise. * tests/statstest.c (mymain): Likewise. * src/bhyve/bhyve_process.c (virBhyveProcessStart) (virBhyveProcessStop): Don't overwrite virCommand error. * src/libvirt.c (virConnectAuthGainPolkit): Likewise. * src/openvz/openvz_driver.c (openvzDomainGetBarrierLimit) (openvzDomainSetBarrierLimit): Likewise. * src/util/virebtables.c (virEbTablesOnceInit): Likewise. * src/util/viriptables.c (virIpTablesOnceInit): Likewise. * src/util/virnetdevveth.c (virNetDevVethCreate): Fix debug message. * src/qemu/qemu_capabilities.c (virQEMUCapsInitQMP): Add comment. * src/storage/storage_backend_iscsi.c (virStorageBackendISCSINodeUpdate): Likewise. Signed-off-by: Eric Blake <eblake@redhat.com>
606 lines
19 KiB
XML
606 lines
19 KiB
XML
<?xml version="1.0" encoding="UTF-8"?>
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<body>
|
|
<h1>Spawning processes / commands from libvirt drivers</h1>
|
|
|
|
<ul id="toc"></ul>
|
|
|
|
<p>
|
|
This page describes the usage of libvirt APIs for
|
|
spawning processes / commands from libvirt drivers.
|
|
All code is required to use these APIs
|
|
</p>
|
|
|
|
<h2><a name="posix">Problems with standard POSIX APIs</a></h2>
|
|
|
|
<p>
|
|
The POSIX specification includes a number of APIs for
|
|
spawning processes / commands, but they suffer from
|
|
a number of flaws
|
|
</p>
|
|
|
|
<ul>
|
|
<li><code>fork+exec</code>: The lowest & most flexible
|
|
level, but very hard to use correctly / safely. It
|
|
is easy to leak file descriptors, have unexpected
|
|
signal handler behaviour and not handle edge cases.
|
|
Furthermore, it is not portable to mingw.
|
|
</li>
|
|
<li><code>system</code>: Convenient if you don't care
|
|
about capturing command output, but has the serious
|
|
downside that the command string is interpreted by
|
|
the shell. This makes it very dangerous to use, because
|
|
improperly validated user input can lead to exploits
|
|
via shell meta characters.
|
|
</li>
|
|
<li><code>popen</code>: Inherits the flaws of
|
|
<code>system</code>, and has no option for bi-directional
|
|
communication.
|
|
</li>
|
|
<li><code>posix_spawn</code>: A half-way house between
|
|
simplicity of system() and the flexibility of fork+exec.
|
|
It does not allow for a couple of important features
|
|
though, such as running a hook between the fork+exec
|
|
stage, or closing all open file descriptors.</li>
|
|
</ul>
|
|
|
|
<p>
|
|
Due to the problems mentioned with each of these,
|
|
libvirt driver code <strong>must not use</strong> any
|
|
of the above APIs. Historically libvirt provided a
|
|
higher level API known as virExec. This was wrapper
|
|
around fork+exec, in a similar style to posix_spawn,
|
|
but with a few more features.
|
|
</p>
|
|
|
|
<p>
|
|
This wrapper still suffered from a number of problems.
|
|
Handling command cleanup via waitpid() is overly
|
|
complex & error prone for most usage. Building up the
|
|
argv[] + env[] string arrays is quite cumbersome and
|
|
error prone, particularly wrt memory leak / OOM handling.
|
|
</p>
|
|
|
|
<h2><a name="api">The libvirt command execution API</a></h2>
|
|
|
|
<p>
|
|
There is now a high level API that provides a safe and
|
|
flexible way to spawn commands, which prevents the most
|
|
common errors & is easy to code against. This
|
|
code is provided in the <code>src/util/command.h</code>
|
|
header which can be imported using <code>#include "command.h"</code>
|
|
</p>
|
|
|
|
<h3><a name="initial">Defining commands in libvirt</a></h3>
|
|
|
|
<p>
|
|
The first step is to declare what command is to be
|
|
executed. The command name can be either a fully
|
|
qualified path, or a bare command name. In the latter
|
|
case it will be resolved wrt the <code>$PATH</code>
|
|
environment variable.
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandPtr cmd = virCommandNew("/usr/bin/dnsmasq");
|
|
</pre>
|
|
|
|
<p>
|
|
There is no need to check for allocation failure after
|
|
<code>virCommandNew</code>. This will be detected and
|
|
reported at a later time.
|
|
</p>
|
|
|
|
<h3><a name="args">Adding arguments to the command</a></h3>
|
|
|
|
<p>
|
|
There are a number of APIs for adding arguments to a
|
|
command. To add a direct string arg
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddArg(cmd, "-strict-order");
|
|
</pre>
|
|
|
|
<p>
|
|
If an argument takes an attached value of the form
|
|
<code>-arg=val</code>, then this can be done using
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddArgPair(cmd, "--conf-file", "/etc/dnsmasq.conf");
|
|
</pre>
|
|
|
|
<p>
|
|
If an argument needs to be formatted as if by
|
|
<code>printf</code>:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddArgFormat(cmd, "%d", count);
|
|
</pre>
|
|
|
|
<p>
|
|
To add a entire NULL terminated array of arguments in one go,
|
|
there are two options.
|
|
</p>
|
|
|
|
<pre>
|
|
const char *const args[] = {
|
|
"--strict-order", "--except-interface", "lo", NULL
|
|
};
|
|
virCommandAddArgSet(cmd, args);
|
|
virCommandAddArgList(cmd, "--domain", "localdomain", NULL);
|
|
</pre>
|
|
|
|
<p>
|
|
This can also be done at the time of initial construction of
|
|
the <code>virCommandPtr</code> object:
|
|
</p>
|
|
|
|
<pre>
|
|
const char *const args[] = {
|
|
"/usr/bin/dnsmasq",
|
|
"--strict-order", "--except-interface",
|
|
"lo", "--domain", "localdomain", NULL
|
|
};
|
|
virCommandPtr cmd1 = virCommandNewArgs(cmd, args);
|
|
virCommandPtr cmd2 = virCommandNewArgList("/usr/bin/dnsmasq",
|
|
"--domain", "localdomain", NULL);
|
|
</pre>
|
|
|
|
<h3><a name="env">Setting up the environment</a></h3>
|
|
|
|
<p>
|
|
By default a command will inherit all environment variables
|
|
from the current process. Generally this is not desirable
|
|
and a customized environment will be more suitable. Any
|
|
customization done via the following APIs will prevent
|
|
inheritance of any existing environment variables unless
|
|
explicitly allowed. The first step is usually to pass through
|
|
a small number of variables from the current process.
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddEnvPassCommon(cmd);
|
|
</pre>
|
|
|
|
<p>
|
|
This has now set up a clean environment for the child, passing
|
|
through <code>PATH</code>, <code>LD_PRELOAD</code>,
|
|
<code>LD_LIBRARY_PATH</code>, <code>HOME</code>,
|
|
<code>USER</code>, <code>LOGNAME</code> and <code>TMPDIR</code>.
|
|
Furthermore it will explicitly set <code>LC_ALL=C</code> to
|
|
avoid unexpected localization of command output. Further
|
|
variables can be passed through from parent explicitly:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddEnvPass(cmd, "DISPLAY");
|
|
virCommandAddEnvPass(cmd, "XAUTHORITY");
|
|
</pre>
|
|
|
|
<p>
|
|
To define an environment variable in the child with an
|
|
separate key / value:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddEnvPair(cmd, "TERM", "xterm");
|
|
</pre>
|
|
|
|
<p>
|
|
If the key/value pair is pre-formatted in the right
|
|
format, it can be set directly
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandAddEnvString(cmd, "TERM=xterm");
|
|
</pre>
|
|
|
|
<h3><a name="misc">Miscellaneous other options</a></h3>
|
|
|
|
<p>
|
|
Normally the spawned command will retain the current
|
|
process and process group as its parent. If the current
|
|
process dies, the child will then (usually) be terminated
|
|
too. If this cleanup is not desired, then the command
|
|
should be marked as daemonized:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandDaemonize(cmd);
|
|
</pre>
|
|
|
|
<p>
|
|
When daemonizing a command, the PID visible from the
|
|
caller will be that of the intermediate process, not
|
|
the actual damonized command. If the PID of the real
|
|
command is required then a pidfile can be requested
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandSetPidFile(cmd, "/var/run/dnsmasq.pid");
|
|
</pre>
|
|
|
|
<p>
|
|
This PID file is guaranteed to be written before
|
|
the intermediate process exits.
|
|
</p>
|
|
|
|
<h3><a name="privs">Reducing command privileges</a></h3>
|
|
|
|
<p>
|
|
Normally a command will inherit all privileges of
|
|
the current process. To restrict what a command can
|
|
do, it is possible to request that all its capabilities
|
|
are cleared. With this done it will only be able to
|
|
access resources for which it has explicit DAC permissions
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandClearCaps(cmd);
|
|
</pre>
|
|
|
|
<h3><a name="fds">Managing file handles</a></h3>
|
|
|
|
<p>
|
|
To prevent unintended resource leaks to child processes, the
|
|
child defaults to closing all open file handles, and setting
|
|
stdin/out/err to <code>/dev/null</code>. It is possible to
|
|
allow an open file handle to be passed into the child, while
|
|
controlling whether that handle remains open in the parent or
|
|
guaranteeing that the handle will be closed in the parent after
|
|
virCommandRun, virCommandRunAsync, or virCommandFree.
|
|
</p>
|
|
|
|
<pre>
|
|
int sharedfd = open("cmd.log", "w+");
|
|
int childfd = open("conf.txt", "r");
|
|
virCommandPassFD(cmd, sharedfd, 0);
|
|
virCommandPassFD(cmd, childfd,
|
|
VIR_COMMAND_PASS_FD_CLOSE_PARENT);
|
|
if (VIR_CLOSE(sharedfd) < 0)
|
|
goto cleanup;
|
|
</pre>
|
|
|
|
<p>
|
|
With this, both file descriptors sharedfd and childfd in the
|
|
current process remain open as the same file descriptors in the
|
|
child. Meanwhile, after the child is spawned, sharedfd remains
|
|
open in the parent, while childfd is closed.
|
|
</p>
|
|
|
|
<p>
|
|
For stdin/out/err it is sometimes necessary to map a file
|
|
handle. If a mapped file handle is a pipe fed or consumed by
|
|
the caller, then the caller should use virCommandDaemonize or
|
|
virCommandRunAsync rather than virCommandRun to avoid deadlock
|
|
(mapping a regular file is okay with virCommandRun). To attach
|
|
file descriptor 7 in the current process to stdin in the child:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandSetInputFD(cmd, 7);
|
|
</pre>
|
|
|
|
<p>
|
|
Equivalently to redirect stdout or stderr in the child,
|
|
pass in a pointer to the desired handle
|
|
</p>
|
|
|
|
<pre>
|
|
int outfd = open("out.log", "w+");
|
|
int errfd = open("err.log", "w+");
|
|
virCommandSetOutputFD(cmd, &outfd);
|
|
virCommandSetErrorFD(cmd, &errfd);
|
|
</pre>
|
|
|
|
<p>
|
|
Alternatively it is possible to request that a pipe be
|
|
created to fetch stdout/err in the parent, by initializing
|
|
the FD to -1.
|
|
</p>
|
|
|
|
<pre>
|
|
int outfd = -1;
|
|
int errfd = -1
|
|
virCommandSetOutputFD(cmd, &outfd);
|
|
virCommandSetErrorFD(cmd, &errfd);
|
|
</pre>
|
|
|
|
<p>
|
|
Once the command is running, <code>outfd</code>
|
|
and <code>errfd</code> will be initialized with
|
|
valid file handles that can be read from. It is
|
|
permissible to pass the same pointer for both outfd
|
|
and errfd, in which case both standard streams in
|
|
the child will share the same fd in the parent.
|
|
</p>
|
|
|
|
<p>
|
|
Normally, file descriptors opened to collect output from a child
|
|
process perform blocking I/O, but the parent process can request
|
|
non-blocking mode:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandNonblockingFDs(cmd);
|
|
</pre>
|
|
|
|
<h3><a name="buffers">Feeding & capturing strings to/from the child</a></h3>
|
|
|
|
<p>
|
|
Often dealing with file handles for stdin/out/err is
|
|
unnecessarily complex; an alternative is to let virCommandRun
|
|
perform the I/O and interact via string buffers. Use of a buffer
|
|
only works with virCommandRun, and cannot be mixed with pipe
|
|
file descriptors. That is, the choice is generally between
|
|
managing all I/O in the caller (any fds not specified are tied
|
|
to /dev/null), or letting virCommandRun manage all I/O via
|
|
strings (unspecified stdin is tied to /dev/null, and unspecified
|
|
output streams get logged but are otherwise discarded).
|
|
</p>
|
|
|
|
<p>
|
|
It is possible to specify a string buffer to act as the data
|
|
source for the child's stdin, if there are no embedded NUL
|
|
bytes, and if the command will be run with virCommandRun:
|
|
</p>
|
|
|
|
<pre>
|
|
const char *input = "Hello World\n";
|
|
virCommandSetInputBuffer(cmd, input);
|
|
</pre>
|
|
|
|
<p>
|
|
Similarly it is possible to request that the child's
|
|
stdout/err be redirected into a string buffer, if the
|
|
output is not expected to contain NUL bytes, and if
|
|
the command will be run with virCommandRun:
|
|
</p>
|
|
|
|
<pre>
|
|
char *output = NULL, *errors = NULL;
|
|
virCommandSetOutputBuffer(cmd, &output);
|
|
virCommandSetErrorBuffer(cmd, &errors);
|
|
</pre>
|
|
|
|
<p>
|
|
Once the command has finished executing, these buffers will
|
|
contain the output. Allocation is guaranteed if virCommandRun
|
|
or virCommandWait succeed (if there was no output, then the
|
|
buffer will contain an allocated empty string); if the command
|
|
failed, then the buffers usually contain a best-effort
|
|
allocation of collected information (however, on an
|
|
out-of-memory condition, the buffer may still be NULL). The
|
|
caller is responsible for freeing registered buffers, since the
|
|
buffers are designed to persist beyond virCommandFree. It
|
|
is possible to pass the same pointer to both
|
|
virCommandSetOutputBuffer and virCommandSetErrorBuffer, in which
|
|
case the child process interleaves output into a single string.
|
|
</p>
|
|
|
|
<h3><a name="directory">Setting working directory</a></h3>
|
|
|
|
<p>
|
|
Daemonized commands are always run with "/" as the current
|
|
working directory. All other commands default to running in the
|
|
same working directory as the parent process, but an alternate
|
|
directory can be specified:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandSetWorkingDirectory(cmd, LOCALSTATEDIR);
|
|
</pre>
|
|
|
|
<h3><a name="hooks">Any additional hooks</a></h3>
|
|
|
|
<p>
|
|
If anything else is needed, it is possible to request a hook
|
|
function that is called in the child after the fork, as the
|
|
last thing before changing directories, dropping capabilities,
|
|
and executing the new process. If hook(opaque) returns
|
|
non-zero, then the child process will not be run.
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandSetPreExecHook(cmd, hook, opaque);
|
|
</pre>
|
|
|
|
<h3><a name="logging">Logging commands</a></h3>
|
|
|
|
<p>
|
|
Sometimes, it is desirable to log what command will be run, or
|
|
even to use virCommand solely for creation of a single
|
|
consolidated string without running anything.
|
|
</p>
|
|
|
|
<pre>
|
|
int logfd = ...;
|
|
char *timestamp = virTimestamp();
|
|
char *string = NULL;
|
|
|
|
dprintf(logfd, "%s: ", timestamp);
|
|
VIR_FREE(timestamp);
|
|
virCommandWriteArgLog(cmd, logfd);
|
|
|
|
string = virCommandToString(cmd);
|
|
if (string)
|
|
VIR_DEBUG("about to run %s", string);
|
|
VIR_FREE(string);
|
|
if (virCommandRun(cmd, NULL) < 0)
|
|
return -1;
|
|
</pre>
|
|
|
|
<h3><a name="sync">Running commands synchronously</a></h3>
|
|
|
|
<p>
|
|
For most commands, the desired behaviour is to spawn
|
|
the command, wait for it to complete & exit and then
|
|
check that its exit status is zero
|
|
</p>
|
|
|
|
<pre>
|
|
if (virCommandRun(cmd, NULL) < 0)
|
|
return -1;
|
|
</pre>
|
|
|
|
<p>
|
|
<strong>Note:</strong> if the command has been daemonized
|
|
this will only block & wait for the intermediate process,
|
|
not the real command. <code>virCommandRun</code> will
|
|
report on any errors that have occurred upon this point
|
|
with all previous API calls. If the command fails to
|
|
run, or exits with non-zero status an error will be
|
|
reported via normal libvirt error infrastructure. If a
|
|
non-zero exit status can represent a success condition,
|
|
it is possible to request the exit status and perform
|
|
that check manually instead of letting <code>virCommandRun</code>
|
|
raise the error. By default, the captured status is only
|
|
for a normal exit (death from a signal is treated as an error),
|
|
but a caller can use <code>virCommandRawStatus</code> to get
|
|
encoded status that includes any terminating signals.
|
|
</p>
|
|
|
|
<pre>
|
|
int status;
|
|
if (virCommandRun(cmd, &status) < 0)
|
|
return -1;
|
|
if (status == 1) {
|
|
...do stuff...
|
|
}
|
|
|
|
virCommandRawStatus(cmd2);
|
|
if (virCommandRun(cmd2, &status) < 0)
|
|
return -1;
|
|
if (WIFEXITED(status) && WEXITSTATUS(status) == 1) {
|
|
...do stuff...
|
|
}
|
|
</pre>
|
|
|
|
<h3><a name="async">Running commands asynchronously</a></h3>
|
|
|
|
<p>
|
|
In certain complex scenarios, particularly special
|
|
I/O handling is required for the child's stdin/err/out
|
|
it will be necessary to run the command asynchronously
|
|
and wait for completion separately.
|
|
</p>
|
|
|
|
<pre>
|
|
pid_t pid;
|
|
if (virCommandRunAsync(cmd, &pid) < 0)
|
|
return -1;
|
|
|
|
... do something while pid is running ...
|
|
|
|
int status;
|
|
if (virCommandWait(cmd, &status) < 0)
|
|
return -1;
|
|
|
|
if (WEXITSTATUS(status)...) {
|
|
..do stuff..
|
|
}
|
|
</pre>
|
|
|
|
<p>
|
|
As with <code>virCommandRun</code>, the <code>status</code>
|
|
arg for <code>virCommandWait</code> can be omitted, in which
|
|
case it will validate that exit status is zero and raise an
|
|
error if not.
|
|
</p>
|
|
|
|
<p>
|
|
There are two approaches to child process cleanup, determined by
|
|
how long you want to keep the virCommand object in scope.
|
|
</p>
|
|
|
|
<p>1. If the virCommand object will outlast the child process,
|
|
then pass NULL for the pid argument, and the child process will
|
|
automatically be reaped at virCommandFree, unless you reap it
|
|
sooner via virCommandWait or virCommandAbort.
|
|
</p>
|
|
|
|
<p>2. If the child process must exist on at least one code path
|
|
after virCommandFree, then pass a pointer for the pid argument.
|
|
Later, to clean up the child, call virPidWait or virPidAbort.
|
|
Before virCommandFree, you can still use virCommandWait or
|
|
virCommandAbort to reap the process.
|
|
</p>
|
|
|
|
<h3><a name="release">Releasing resources</a></h3>
|
|
|
|
<p>
|
|
Once the command has been executed, or if execution
|
|
has been abandoned, it is necessary to release
|
|
resources associated with the <code>virCommandPtr</code>
|
|
object. This is done with:
|
|
</p>
|
|
|
|
<pre>
|
|
virCommandFree(cmd);
|
|
</pre>
|
|
|
|
<p>
|
|
There is no need to check if <code>cmd</code> is NULL
|
|
before calling <code>virCommandFree</code>. This scenario
|
|
is handled automatically. If the command is still running,
|
|
it will be forcably killed and cleaned up (via waitpid).
|
|
</p>
|
|
|
|
<h2><a name="example">Complete examples</a></h2>
|
|
|
|
<p>
|
|
This shows a complete example usage of the APIs roughly
|
|
using the libvirt source src/util/hooks.c
|
|
</p>
|
|
|
|
<pre>
|
|
int runhook(const char *drvstr, const char *id,
|
|
const char *opstr, const char *subopstr,
|
|
const char *extra) {
|
|
int ret;
|
|
char *path;
|
|
virCommandPtr cmd;
|
|
|
|
ret = virBuildPath(&path, LIBVIRT_HOOK_DIR, drvstr);
|
|
if ((ret < 0) || (path == NULL)) {
|
|
virHookReportError(VIR_ERR_INTERNAL_ERROR,
|
|
_("Failed to build path for %s hook"),
|
|
drvstr);
|
|
return -1;
|
|
}
|
|
|
|
cmd = virCommandNew(path);
|
|
VIR_FREE(path);
|
|
|
|
virCommandAddEnvPassCommon(cmd);
|
|
|
|
virCommandAddArgList(cmd, id, opstr, subopstr, extra, NULL);
|
|
|
|
virCommandSetInputBuffer(cmd, input);
|
|
|
|
ret = virCommandRun(cmd, NULL);
|
|
|
|
virCommandFree(cmd);
|
|
|
|
return ret;
|
|
}
|
|
</pre>
|
|
|
|
<p>
|
|
In this example, the command is being run synchronously.
|
|
A pre-formatted string is being fed to the command as
|
|
its stdin. The command takes four arguments, and has a
|
|
minimal set of environment variables passed down. In
|
|
this example, the code does not require any error checking.
|
|
All errors are reported by the <code>virCommandRun</code>
|
|
method, and the exit status from this is returned to
|
|
the caller to handle as desired.
|
|
</p>
|
|
|
|
</body>
|
|
</html>
|