Commit Graph

1036 Commits

Author SHA1 Message Date
Jiri Denemark
a231016b69 Use virTimeMs when appropriate 2011-06-13 11:24:58 +02:00
Wen Congyang
9f8baf646e support multifunction PCI device
If qemu supports multi function PCI device, the format of the PCI address passed
to qemu is "bus=pci.0,multifunction=on,addr=slot.function".

If qemu does not support multi function PCI device, the format of the PCI address
passed to qemu is "bus=pci.0,addr=slot".
2011-06-13 09:27:22 +08:00
Wen Congyang
6fe678cdaf the hotplugged PCI device should use the whole slot
Hot pluging/unpluging multi PCI device is not supported now. So the function
of hotplugged PCI device must be 0. When we hot unplug it, we should set release
all functions in the slot.
2011-06-13 09:27:22 +08:00
Wen Congyang
f12f34fe2f assign the whole slot to the PCI device that has no address
If user does not specify the PCI address, we should auto assign an unused slot.
2011-06-13 09:27:22 +08:00
Wen Congyang
38912f3107 Reimplement qemuDomainPCIAddressReserveSlot(): reserve all functions in the slot
We will support multi function PCI device. So we should reserve all functions in
the slot if we want to reserve a slot.
2011-06-13 09:27:22 +08:00
Wen Congyang
d8f24f75a8 the key of hash table should include the function value
We save all used PCI address in the hash table. The key is generated by domain,
bus and slot now. We will support multi function PCI device, so the key should
be generated by domain, bus, slot and function.
2011-06-13 09:27:21 +08:00
Wen Congyang
cb6ea85b78 prevent hot unplugging multi function PCI device
We do not support to hot unplug multi function PCI device now. If the device is
one function of multi function PCI device, we shoul not allow to hot unplugg
it.
2011-06-13 09:27:21 +08:00
Wen Congyang
d58c382e93 check whether qemu supports multi function PCI device
qemu supports multi function PCI device at least version 0.13.0.
2011-06-13 09:27:21 +08:00
Osier Yang
08a6826a14 qemu: Fix one type in the error prompt string
s/hostdevwork/hostdev/
2011-06-10 11:30:59 +08:00
Eric Blake
d7814b21a2 build: break some long lines
As long as I was already touching the function...

* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphics): Line wrap.
2011-06-08 05:30:56 -06:00
Eric Blake
ddc5b158d7 qemu: add missing break statement
Detected by Coverity.  Bug introduced in commit 9d73efd (v0.8.8).

* src/qemu/qemu_hotplug.c (qemuDomainChangeGraphics): Don't report
error on success.
2011-06-08 05:30:56 -06:00
Eric Blake
4eb17d642e qemu: reorder checks for safety
Detected by Coverity.  All existing callers happen to be in
range, so this isn't too serious.

* src/qemu/qemu_cgroup.c (qemuCgroupControllerActive): Check
bounds before dereference.
2011-06-08 05:28:20 -06:00
Cole Robinson
1ba75cf9aa qemu: Convert virExec usage to virCommand
v2:
    Have virCommand cleanup intermediate process for us

v3:
    Preserve original FD closing behavior

Signed-off-by: Cole Robinson <crobinso@redhat.com>
2011-06-07 11:11:02 -04:00
Osier Yang
ebf6b11ac1 Use VIR_USE_CPU instead of new wheel 2011-06-07 16:51:51 +08:00
Daniel P. Berrange
3e87a3901a Fix QEMU p2p v2 migration when run from a v3 client
When peer-2-peer migration was invoked by a client supporting
v3, but where the target server only supported v2, we'd not
correctly shutdown the guest.

* src/qemu/qemu_migration.c: Ensure guest is shutdown in
  v2 peer 2 peer migration
2011-06-06 11:05:34 +08:00
Daniel P. Berrange
a50f5f6faa Don't raise an error if the migration cookie is NULL
The v2 migration protocol doesn't use cookies, so we should not
be raising an error if the cookie parameters are NULL.

* src/qemu/qemu_migration.c: Don't raise error if cookie is NULL
2011-06-04 07:26:32 -04:00
Daniel P. Berrange
a018c0b910 Fix check of virKillProcess return status
The error code for virKillProcess is returned in the errno variable
not the return value. THis mistake caused the logs to be filled with
errors when shutting down QEMU processes

* src/qemu/qemu_process.c: Fix process kill check.
2011-06-04 07:26:07 -04:00
Eric Blake
33d90bafe7 API: consolidate common unreleased enums
This commit is safe precisely because there has been no release
for any of the enum values being deleted (they were added post-0.9.1).

After the 0.9.2 release, we can then take advantage of
virDomainModificationImpact in more places.

* include/libvirt/libvirt.h.in (virDomainModificationImpact): New
enum.
(virDomainSchedParameterFlags, virMemoryParamFlags): Delete, since
these were never released, and the new enum works fine here.
* src/libvirt.c	(virDomainGetMemoryParameters)
(virDomainSetMemoryParameters)
(virDomainGetSchedulerParametersFlags)
(virDomainSetSchedulerParametersFlags): Update documentation.
* src/qemu/qemu_driver.c (qemuDomainSetMemoryParameters)
(qemuDomainGetMemoryParameters, qemuSetSchedulerParametersFlags)
(qemuSetSchedulerParameters, qemuGetSchedulerParametersFlags)
(qemuGetSchedulerParameters): Adjust clients.
* tools/virsh.c (cmdSchedinfo, cmdMemtune): Likewise.
Based on ideas by Daniel Veillard and Hu Tao.
2011-06-04 09:58:53 +02:00
Jiri Denemark
aeed51f775 qemu: Avoid use after free in qemuCaps parsing 2011-06-03 19:58:43 +02:00
Eric Blake
2834d57175 qemu: avoid memory leak on vcpupin
Detected by Coverity.  This leaked a cpumap on every iteration
of the loop.  Leak introduced in commit 1cc4d02 (v0.9.0).

* src/qemu/qemu_process.c (qemuProcessSetVcpuAffinites): Plug
leak, and hoist allocation outside loop.
2011-06-03 08:11:43 -06:00
Daniel P. Berrange
6d68e0df72 Explicitly set VM state to paused after migration completes
In v3 migration, once migration is completed, the VM needs
to be left in a paused state until after Finish3 has been
executed on the target. Only then will the VM be killed
off. When using non-JSON QEMU monitor though, we don't
receive any 'STOP' event from QEMU, so we need to manually
set our state offline & thus release lock manager leases.
It doesn't hurt to run this on the JSON case too, just in
case the event gets lost somehow

* src/qemu/qemu_migration.c: Explicitly set VM state to
  paused when migration completes
2011-06-03 09:59:47 -04:00
Daniel P. Berrange
2027e184c5 Fix regressions BlockStats/Info APIs in QEMU driver
The change 18c2a59206 caused
some regressions in behaviour of virDomainBlockStats
and virDomainBlockInfo in the QEMU driver.

The virDomainBlockInfo API stopped working for inactive
guests if querying a block device.

The virDomainBlockStats API did not promptly report
an error if the guest was not running in some cases.

* src/qemu/qemu_driver.c: Fix inactive guest handling
  in BlockStats/Info APIs
2011-06-03 14:55:11 +01:00
Daniel P. Berrange
020342e68d Fix auditing of disk hotunplug operations
The qemuAuditDisk calls in disk hotunplug operations were being
passed 'ret >= 0', but the code which sets ret to 0 was not yet
executed, and the error path had already jumped to the 'cleanup'
label. This meant hotunplug failures were never audited, and
hotunplug success was audited as a failure

* src/qemu/qemu_hotplug.c: Fix auditing of hotunplug
2011-06-02 17:23:45 +01:00
Daniel P. Berrange
43917db3f9 Fix handling of VIR_EVENT_HANDLE_ERROR in QEMU monitor
Commit 4454a9efc7 introduced bad
behaviour on the VIR_EVENT_HANDLE_ERROR condition. This condition
is only hit when an invalid FD is used in poll() (typically due
to a double-close bug). The QEMU monitor code was treating this
condition as non-fatal, and thus libvirt would poll() in a fast
loop forever burning 100% CPU. VIR_EVENT_HANDLE_ERROR must be
handled in the same way as VIR_EVENT_HANDLE_HANGUP, killing the
QEMU instance.

* src/qemu/qemu_monitor.c: Treat VIR_EVENT_HANDLE_ERROR as EOF
2011-06-02 17:23:44 +01:00
Daniel P. Berrange
30ffe7bceb Allow leases to be hotpluged with QEMU guests
* src/conf/domain_conf.c, src/conf/domain_conf.h: APIs for
  inserting/finding/removing virDomainLeaseDefPtr instances
* src/qemu/qemu_driver.c: Wire up hotplug/unplug for leases
* src/qemu/qemu_hotplug.h, src/qemu/qemu_hotplug.c: Support
  for hotplug and unplug of leases
2011-06-02 10:54:01 +01:00
Daniel P. Berrange
7474560bef Support passing QEMU lock state to dest during migration
Some lock managers associate state with leases, allowing a process
to temporarily release its leases, and re-acquire them later, safe
in the knowledge that no other process has acquired + released the
leases in between.

This is already used between suspend/resume operations, and must
also be used across migration. This passes the lockstate in the
migration cookie. If the lock manager uses lockstate, then it
becomes compulsory to use the migration v3 protocol to get the
cookie support.

* src/qemu/qemu_driver.c: Validate that migration v2 protocol is
  not used if lock manager needs state transfer
* src/qemu/qemu_migration.c: Transfer lock state in migration
  cookie XML
2011-06-02 10:54:01 +01:00
Daniel P. Berrange
12317957ec Integrate the QEMU driver with the lock manager infrastructure
The QEMU integrates with the lock manager instructure in a number
of key places

 * During startup, a lock is acquired in between the fork & exec
 * During startup, the libvirtd process acquires a lock before
   setting file labelling
 * During shutdown, the libvirtd process acquires a lock
   before restoring file labelling
 * During hotplug, unplug & media change the libvirtd process
   holds a lock while setting/restoring labels

The main content lock is only ever held by the QEMU child process,
or libvirtd during VM shutdown. The rest of the operations only
require libvirtd to hold the metadata locks, relying on the active
QEMU still holding the content lock.

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h,
  src/qemu/libvirtd_qemu.aug, src/qemu/test_libvirtd_qemu.aug:
  Add config parameter for configuring lock managers
* src/qemu/qemu_driver.c: Add calls to the lock manager
2011-06-02 10:54:01 +01:00
Daniel P. Berrange
44924615ce Allow custom XML to be passed in during migration
Update the qemuDomainMigrateBegin method so that it accepts
an optional incoming XML document. This will be validated
for ABI compatibility against the current domain config,
and if this check passes, will be passed back out for use
by the qemuDomainMigratePrepare method on the target

* src/qemu/qemu_domain.c, src/qemu/qemu_domain.h,
  src/qemu/qemu_migration.c: Allow custom XML to be passed
2011-05-31 12:14:57 +01:00
Daniel P. Berrange
80d2038d58 Issue full error messages when processing QEMU monitor I/O
Currently the QEMU monitor I/O handler code uses errno values
to report errors. This results in a sub-optimal error messages
on certain conditions, in particular when parsing JSON strings
malformed data simply results in 'EINVAL'.

This changes the code to use the standard libvirt error reporting
APIs. The virError is stored against the qemuMonitorPtr struct,
and when a monitor API is run, any existing stored error is copied
into that thread's error local

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_text.c: Use
  virError APIs for all monitor I/O handling code
2011-05-29 20:51:08 +08:00
Daniel P. Berrange
4454a9efc7 Don't kill QEMU process when a monitor I/O parsing error occurs
Currently whenever there is any failure with parsing the monitor,
this is treated in the same was as end-of-file (ie QEMU quit).
The domain is terminated, if not already dead.

With this change, failures in parsing the monitor stream do not
result in the death of QEMU. The guest continues running unchanged,
but all further use of the monitor will be disabled.

The VMM_FAILURE event will be emitted, and the mgmt application
can decide when to kill/restart the guest to re-gain control

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Run a
  different callback for monitor EOF vs error conditions.
* src/qemu/qemu_process.c: Emit VMM_FAILURE event when monitor
  fails
2011-05-29 20:37:29 +08:00
Eric Blake
8432170f6f qemu: introduce qemuGetSchedulerParametersFlags
* src/qemu/qemu_driver.c (qemuGetSchedulerParameters): Move
guts...
(qemuGetSchedulerParametersFlags): ...to new callback, and honor
flags more accurately.
2011-05-29 18:40:06 +08:00
Eric Blake
163e5f04ef maint: prefer newer API names internally
Rather mechanical in nature.

* src/driver.h: Use newer virTypedParameter API names.
* src/libvirt.c: Likewise.
* daemon/remote.c: Likewise.
* src/esx/esx_driver.c: Likewise.
* src/libxl/libxl_driver.c: Likewise.
* src/lxc/lxc_driver.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/remote/remote_driver.c: Likewise.
* src/test/test_driver.c: Likewise.
* src/xen/xen_driver.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
* src/xen/xen_hypervisor.h: Likewise.
* src/xen/xend_internal.c: Likewise.
* tools/virsh.c: Likewise.
2011-05-29 18:26:31 +08:00
Hu Tao
4b997daaf5 qemu: implement the new flags for setting memory parameters
* src/qemu/qemu_driver.c: update qemuDomainSetMemoryParameters to
  look at the flag parameter and depending on it save the config
  or the live amount
2011-05-27 15:49:07 +08:00
Hu Tao
7639d1ef31 qemu: implement the new flags for getting memory parameters
* src/qemu/qemu_driver.c: update qemuDomainGetMemoryParameters to
  look at the flag parameter and depending on it read the config
  or the live amount
2011-05-27 15:48:55 +08:00
KAMEZAWA Hiroyuki
9c26d6f09e qemu : support persistent add/delete network interface
This patch allows to modify interfaces of domain(qemu)
* src/conf/domain_conf.c src/conf/domain_conf.h src/libvirt_private.syms:
  (virDomainNetInsert)     : Insert a network device to domain definition.
  (virDomainNetIndexByMac) : Returns an index of net device in array.
  (virDomainNetRemoveByMac): Remove a NIC of passed MAC address.
* src/qemu/qemu_driver.c
  (qemuDomainAttachDeviceConfig): add codes for NIC.
  (qemuDomainDetachDeviceConfig): add codes for NIC.
2011-05-27 15:01:47 +08:00
Federico Simoncelli
18c2a59206 qemu: allow blkstat/blkinfo calls during migration
Originally most of libvirt domain-specific calls were blocking
during a migration.
A new mechanism to allow specific calls (blkstat/blkinfo) to be
executed in such condition has been implemented.
In the long term it'd be desirable to get a more general
solution to mark further APIs as migration safe, without needing
special case code.

 * src/qemu/qemu_migration.c: add some additional job signal
   flags for doing blkstat/blkinfo during a migration
 * src/qemu/qemu_domain.c: add a condition variable that can be
   used to efficiently wait for the migration code to clear the
   signal flag
 * src/qemu/qemu_driver.c: execute blkstat/blkinfo using the
   job signal flags during migration
2011-05-26 11:05:04 -06:00
Markus Groß
23e5393c40 Fix modifying disk devices in qemu driver
When modifying the disk devices of a live domain and the domain
configuration, the function qemuDomainAttachDeviceConfig
first sets dev->data->disk to NULL. Later qemuDomainAttachDeviceLive
accesses dev->data.disk and causes a segfault.
* src/qemu/qemu_driver.c: fix qemuDomainModifyDeviceFlags() accordingly
2011-05-26 22:32:51 +08:00
Richard W.M. Jones
78eb8b60d5 json: Avoid passing large positive 64 bit integers to QMP.
http://lists.gnu.org/archive/html/qemu-devel/2011-05/threads.html#02162

Currently, qemu silently clips any JSON integer in the range
0x8000000000000000 - 0xffffffffffffffff (all numbers in this range
will be clipped to 0x7fffffffffffffff == LLONG_MAX).

To avoid this, pass these as signed 64 bit integers in the QMP
request.
2011-05-25 21:30:39 +01:00
Daniel P. Berrange
2593f9692d Fix the signature of virDomainMigrateFinish3 for error reporting
The current virDomainMigrateFinish3 method signature attempts to
distinguish two types of errors, by allowing return with ret== 0,
but ddomain == NULL, to indicate a failure to start the guest.
This is flawed, because when ret == 0, there is no way for the
virErrorPtr details to be sent back to the client.

Change the signature of virDomainMigrateFinish3 so it simply
returns a virDomainPtr, in the same way as virDomainMigrateFinish2
The disk locking code will protect against the only possible
failure mode this doesn't account for (loosing conenctivity to
libvirtd after Finish3 starts the CPUs, but before the client
sees the reply for Finish3).

* src/driver.h, src/libvirt.c, src/libvirt_internal.h: Change
  virDomainMigrateFinish3 to return a virDomainPtr instead of int
* src/remote/remote_driver.c, src/remote/remote_protocol.x,
  daemon/remote.c, src/qemu/qemu_driver.c, src/qemu/qemu_migration.c:
  Update for API change
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
5e31df9335 Fix preservation of errors across migration steps
When doing migration, if an error occurs in Perform, it must not
be overwritten during Finish/Confirm steps. If an error occurs
in Finish, it must not be overwritten in Confirm.

Previous commit a9d12c2444 added
code to qemudDomainMigrateFinish2 to preserve the error. This
is not the right place, because it is not applicable in non-p2p
migration. The src/libvirt.c virDomainMigrateV2/3 methods need
code to preserve errors for non-p2p migration, while the
doPeer2PeerMigrate2 and doPeer2PeerMigrate3 methods contain
code to preverse errors for p2p migration.

Remove the bogus error preservation from qemudDomainMigrateFinish2
and qemudDomainMigrateFinish3.

Fix virDomainMigrateV3 and doPeer2PeerMigrate3 so that they
preserve any error hit during the Finish3 step, before invoking
Confirm3.

Finally if qemuMigrationFinish fails to resume the CPUs, it must
preserve the error before tearing down the VM, so that VM cleanup
doesn't overwrite it.

* src/libvirt.c: Preserve error before invoking Confirm3
* src/qemu/qemu_driver.c: Remove bogus error preservation
  code in qemudDomainMigrateFinish2/qemudDomainMigrateFinish3
* src/qemu/qemu_migration.c: Preserve error before invoking Confirm3
  and after resume fails in qemuMigrationFinish.
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
03547eee92 Improve debug logging of migration APIs
* src/libvirt.c: Add further debug lines in helper APIs for
  migration
* src/qemu/qemu_migration.c: Add debug lines for all internal
  migration API parameters
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
1853bd9d6e Fix error propagation in finish method for v3 migration
Even when failing to start CPUs, the finish method was returning
a success result. Fix this so that the QEMU process is killed
off when finish fails under v3 protocol. Also rename the
killOnFinish boolean to 'v3proto' to make it clearer that this
is a tunable based on the migration protocol version

* src/qemu/qemu_driver.c: Update for API change
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Kill
  VM in qemuMigrationFinish if failing to start CPUs
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
b336710383 Fix SPICE seamless migration hostname
The SPICE seamless migration process requires data to be passed
back from the target host, to the source host via a cookie.
The cookie includes the target host's hostname, but this was not
stored, merely validated. This patch explicitly records the
remote hostname after parsing the cookie, and uses it when
initiating the SPICE migration

* qemu/qemu_migration.c: Fix SPICE seamless migration hostname
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
73b3e0b70b Fix resume on destination when doing non-live tunnelled migration
Before running perform in peer-2-peer migration, the current
guest state must be recorded, so that non-live migration can
currently unpause a running guest on completion.

* src/qemu/qemu_migration.c: Move check for offline guest
  to fix non-live migration
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
f9f2d4e147 Add a second URI parameter to virDomainMigratePerform3 method
The virDomainMigratePerform3 currently has a single URI parameter
whose meaning varies. It is either

 - A QEMU migration URI (normal migration)
 - A libvirtd connection URI (peer2peer migration)

Unfortunately when using peer2peer migration, without also
using tunnelled migration, it is possible that both URIs are
required.

This adds a second URI parameter to the virDomainMigratePerform3
method, to cope with this scenario. Each parameter how has a fixed
meaning.

NB, there is no way to actually take advantage of this yet,
since virDomainMigrate/virDomainMigrateToURI do not have any
way to provide the 2 separate URIs

* daemon/remote.c, src/remote/remote_driver.c,
  src/remote/remote_protocol.x, src/remote_protocol-structs: Add
  the second URI parameter to perform3 message
* src/driver.h, src/libvirt.c, src/libvirt_internal.h: Add
  the second URI parameter to Perform3 method
* src/libvirt_internal.h, src/qemu/qemu_migration.c,
  src/qemu/qemu_migration.h: Update to handle URIs correctly
2011-05-25 11:47:48 -04:00
Daniel P. Berrange
7ad4b6b9cc Extend v3 migration protocol to allow app supplied XML for target
This extends the v3 migration protocol such that the
virDomainMigrateBegin3 and virDomainMigratePerform3
methods accept an application supplied XML config for
the target VM.

If the 'xmlin' parameter is NULL, then Begin3 uses the
current guest XML as normal. A driver implementing the
Begin3 method should either reject all non-NULL 'xmlin'
parameters, or strictly validate that the app supplied
XML does not change guest ABI.

The Perform3 method also needed the xmlin parameter to
cope with the Peer2Peer migration sequence.

NB it is not yet possible to use this capability since
neither of the public virDomainMigrate/virDomainMigrateToURI
methods have a way to pass in XML.

* daemon/remote.c, src/remote/remote_driver.c,
  src/remote/remote_protocol.x, src/remote_protocol-structs:
  Add 'remote_string xmlin' parameter to begin3/perform3
  RPC messages
* src/libvirt.c, src/driver.h, src/libvirt_internal.h: Add
  'const char *xmlin' parameter to Begin3/Perform3 methods
* src/qemu/qemu_driver.c, src/qemu/qemu_migration.c,
  src/qemu/qemu_migration.h: Pass xmlin parameter around
  migration methods
2011-05-25 11:47:47 -04:00
Michal Privoznik
764c282cdf qemu: Don't change ownership of file when appending to it
Saving domain to previously created file changes also its ownership.
This is certainly not what users want if some conditions are met:
it is a regular, local file and dynamic_ownership is off.
2011-05-24 17:41:34 +02:00
Laine Stump
51781b82f7 qemu: fix typo in name - should be VHOST_NET, not VNET_HOST
NB: the enum that uses the string vnet-host (now changed to vhost-net)
is used in XML, but fortunately that hasn't been in an official
release yet, so it can still be fixed.
2011-05-24 10:45:55 -04:00
Daniel P. Berrange
58765b58a3 Fix QEMU -vnc arg generation with raw IPv6 addresses
Since -vnc uses ':' to separate the address from the port, raw
IPv6 addresses need to be escaped like [addr]:port

* src/qemu/qemu_command.c: Escape raw IPv6 addresses with []
* tests/qemuxml2argvdata/qemuxml2argv-graphics-vnc.args,
  tests/qemuxml2argvdata/qemuxml2argv-graphics-vnc.xml: Tweak
  to test Ipv6 escaping
* docs/schemas/domain.rng: Allow Ipv6 addresses, or hostnames
  in <graphics> listen attributes
2011-05-24 10:33:53 -04:00
Daniel P. Berrange
65e1acad80 Fix peer2peer migration with transient VMs
The qemuMigrationConfirm method shouldn't deal with final VM
cleanup, since it can be called from the peer2peer migration,
which expects to still use the 'vm' object afterwards.

Push the cleanup code out of qemuMigrationConfirm, into its
caller, qemuDomainMigrateConfirm3

* src/qemu/qemu_driver.c: Add VM cleanup code to
  qemuDomainMigrateConfirm3
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Remove
  job handling cleanup from qemuMigrationConfirm
2011-05-24 05:41:23 -04:00