Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/management/hosts-pools.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,8 @@ Lost the root password? See the [reset procedure](../troubleshooting/common-prob

Correct and consistent clocks across the pool matter more than on ordinary servers: XAPI coordination, live migration, HA heartbeats, logs correlation and Windows guests (which get their initial time from the host) all rely on it. NTP servers are normally configured at [installation time](../installation/install-xcp-ng.md).

See [Time synchronization](time-synchronization.md) for what a wrong clock breaks, how to check whether `chronyd` has any time sources, and how to correct a clock that is already wrong.

To change them afterwards on XCP-ng 8.3, edit `/etc/chrony.conf`, then:

<Terminal shell title="Time synchronization (NTP)">{`
Expand Down
167 changes: 167 additions & 0 deletions docs/management/time-synchronization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
---
sidebar_position: 8
---

# Time synchronization

XCP-ng keeps the dom0 clock synchronized using `chronyd`. A host whose date is wrong keeps
working normally until something needs to trust that date. When that happens, the resulting
failures rarely point to the clock as the cause. This page explains why the clock matters, how
to configure NTP, and how to check and correct it.

## ⏰ Why the clock must be correct {#why-the-clock-must-be-correct}

- **Access to the repositories.** Updates are fetched over HTTPS, and TLS validates the
server's certificate against the host's own date. If the host's clock is set too far in the
past, the TLS handshake fails because, from the host's perspective, the mirror's certificate
is not yet valid. The error `yum` prints in that situation does not mention time.
- **Pool membership.** A host joining a pool must have its clock synchronized with the pool
master. See [Pool Requirements](../../installation/requirements#pool-requirements).
- **Certificates.** Certificates are only valid inside a validity window. A host whose date
falls outside that window will reject, or be rejected by, connections that would otherwise
succeed.
- **Logs and scheduled operations.** Correlating events across the hosts of a pool, and
anything that runs on a schedule such as backups, depend on the hosts agreeing on what
time it is.

## πŸ•°οΈ How XCP-ng keeps time {#how-xcp-ng-keeps-time}

The time daemon on dom0 is `chronyd`, configured through `/etc/chrony.conf`.

The default configuration enables `rtcsync`, allowing chronyd to keep the hardware clock
synchronized with the system clock automatically. There is no need to manually update the
hardware clock after correcting the system time, and the correction persists across reboots.

## πŸ”§ Configuring the time sources {#configuring-the-time-sources}

### During installation

At step 11, the installer prompts you to enter the time, select a time zone and either
configure NTP servers or set the time manually. Always configure at least one NTP server. See
[Install XCP-ng](../../installation/install-xcp-ng).

:::warning
If you set the time manually at this step, the host can end up with `chronyd` running but no
time source configured at all. It will keep whatever date it was installed with. See
[Checking the time sources](#checking-the-time-sources) below.
:::

### During an automated installation

The installation answer file accepts one or more NTP servers through the `<ntp-server>`
element. See [Answer file](../../appendix/answerfile).

### On an installed host

You can configure NTP from `xsconsole`, or by editing `/etc/chrony.conf` directly and
restarting the service.

## πŸ” Checking the time sources {#checking-the-time-sources}

Check that the service is running, and that it actually has something to synchronize with:

```bash
systemctl status chronyd
chronyc sources
```

Both matter, and the second is the one that gets skipped. `chronyd` can be running and
enabled while having **no time sources configured at all**:

```
# chronyc sources
210 Number of sources = 0
```

Then, the service behaves exactly as configured, and does nothing. Every other check an
operator would normally run looks healthy, which is what makes this one worth running early
rather than late.

When sources are configured, `chronyc sources` lists them. In the `MS` column, the second
character is the state of that source:

- `*` marks the one currently being used.
- `+` marks another acceptable source being combined with it.
- `?` means the source is unreachable.

At least one source should reach the `*` state.

To see the offset chrony believes it has, and whether it has settled:

```bash
chronyc tracking
```

## πŸ› οΈ Correcting a wrong clock {#correcting-a-wrong-clock}

Start with `chronyc sources`, because the answer decides which of the two paths below
applies.

### If sources are listed

Correct the clock and check the result:

```bash
chronyc makestep
date
```

`chronyc makestep` is what does the work here. By default, chrony corrects time offsets by
gradually slewing the system clock, which never converges for an offset of months or years.

### If `Number of sources = 0`

Do not start with `makestep`. It will report success and move nothing, because chrony has no
measured offset to step to.

1. Add time sources to `/etc/chrony.conf`:

```
server 0.pool.ntp.org iburst
server 1.pool.ntp.org iburst
server 2.pool.ntp.org iburst
server 3.pool.ntp.org iburst
```

2. Restart the service and confirm that a source has become reachable, as described in
[Checking the time sources](#checking-the-time-sources):

```bash
systemctl restart chronyd
chronyc sources
```

3. Once a source is reachable, correct the clock:

```bash
chronyc makestep
date
```

:::note
Correcting the date does not invalidate the host's own certificate. XAPI issues it with a
ten-year validity, so a host installed with a wrong date still holds a certificate that
covers the corrected date. `xe host-refresh-server-certificate` is not needed for this.
:::

:::tip
If `date` is wrong again after every power cycle, the motherboard's RTC battery is probably
dead and should be replaced. `rtcsync` can only keep the hardware clock in step while the
host is running; it cannot help a clock that loses its value when the power goes.
:::

## 🌐 Networks without internet access {#networks-without-internet-access}

Hosts without Internet access cannot reach the public NTP pool. The result is the same as
having no sources at all: `chronyc sources` lists servers that never leave the `?` state.

On such networks, point `/etc/chrony.conf` to a time source that hosts can actually reach,
such as an appliance on the same network, or a local server that is itself synchronized and
acts as the reference time for the network.

## 🎱 Pools {#pools}

Keep every host in a pool synchronized, ideally against the same sources. A host whose clock
differs from the pool master is not just inconvenient: clock synchronization is one of
the requirements for joining a pool, along with the other requirements listed in
[Pool Requirements](../../installation/requirements#pool-requirements).
4 changes: 4 additions & 0 deletions docs/management/updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ If your version is lower than `8.3`, it will not receive updates anymore. To kee

Your dom0 system must either have access to the internet, or to a local mirror. In the second case, make sure to update the `baseurl` values in `/etc/yum.repos.d/xcp-ng.repo` to make them point at the local mirror, and keep the mirror regularly synced.

:::warning
The host clock must also be correct. Repository access uses HTTPS, and a host whose date is wrong cannot validate the mirrors' TLS certificates. The error `yum` reports in that case is misleading, because it names a redirect instead of the certificate: see [yum fails with "HTTPS Error 301 - Moved Permanently"](../../troubleshooting/common-problems#yum-fails-with-https-error-301---moved-permanently).
:::

#### Proxy configuration

If your hosts require a proxy to access the repositories, you have several options depending on your needs:
Expand Down
54 changes: 54 additions & 0 deletions docs/troubleshooting/common-problems.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,60 @@ chmod +x /etc/rc.d/rc.local

---

## πŸ“¦ yum fails with "HTTPS Error 301 - Moved Permanently" {#yum-fails-with-https-error-301---moved-permanently}

Every `yum` command on dom0 fails, most often on a freshly installed host:

```
Loaded plugins: fastestmirror
* xcp-ng-base: mirrors.xcp-ng.org
http://mirrors.xcp-ng.org/8/8.3/base/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 301 - Moved Permanently
Trying other mirror.
...
failure: repodata/repomd.xml from xcp-ng-base: [Errno 256] No more mirrors to try.
```

### Cause

In almost every reported case, **the system clock on dom0 is wrong**, usually set far in the past.

`mirrors.xcp-ng.org` redirects HTTP to HTTPS, then to a mirror close to you. That redirect is normal and is not the problem. But if the host's date is earlier than the start of the mirror's TLS certificate validity period, the HTTPS connection cannot be established: from the host's point of view, the certificate is not yet valid.

The message names the redirect rather than the certificate because of the way `yum` reports errors. When a transfer fails *after* a redirect, it prints the redirect's status code and discards the underlying error. The `301` is genuine, and it is the last thing that succeeded.

To confirm the diagnosis before changing anything, you can temporarily change the repository URL in `/etc/yum.repos.d/xcp-ng.repo` from `http://` to `https://`. This does not fix the download, but it removes the redirect, so `yum` reports the real error:

```
https://mirrors.xcp-ng.org/8/8.3/base/x86_64/repodata/repomd.xml: [Errno 14] curl#60 - "SSL certificate problem: certificate is not yet valid"
```

### Solution

Check the date on dom0:

<Terminal shell title="root@xcp-ng-host β€” Check the date">{`
date
`}</Terminal>

If it is wrong, fix the time synchronization: see [Time synchronization](../management/time-synchronization.md#correcting-a-wrong-clock) for how to check the chrony sources and step the clock. On a host whose date was set manually at installation, `chronyd` is often running with no time source at all, and correcting the clock then takes an extra step.

Once the clock is correct:

<Terminal shell title="root@xcp-ng-host β€” Solution">{`
yum clean all
yum check-update
`}</Terminal>

:::warning
Revert any change you made to `/etc/yum.repos.d/xcp-ng.repo` while investigating. With a correct clock, the default `http://mirrors.xcp-ng.org` URL works. Pinning a single mirror by hand takes the host out of the automatic mirror selection, and out of failover if that mirror becomes unavailable. See [Mirrors](../project/mirrors.md).
:::

:::note
Correcting the date does not invalidate the host's own certificate. XAPI issues it with a ten-year validity, so a host installed with a wrong date still holds a certificate that covers the corrected date. `xe host-refresh-server-certificate` is not needed here.
:::

---

## 🐌 Async Tasks/Commands Hang or Execute Extremely Slowly {#async-taskscommands-hang-or-execute-extremely-slowly}

### Cause
Expand Down