Release notes for Uyuni Server Version 2021.02 2021-03-01 11:47:13 +0100 Table of Contents * Version Revision History * Stay informed * Support * Release model * Major changes since Uyuni Server 4.0.0 + Features and changes o Version 2021.02 # Recent Salt CVEs remediation # Prometheus exporters' reverse proxy formula Ubuntu support o Version 2021.01 # Vendor change for some Java dependencies # Fix version comparison algorithm for deb packages (Ubuntu) # New products enabled # SAP content # CPU mitigations formula # Vendor change on SP migration # Autoinstallation of older operating systems # Oracle Linux ULN repositories # CentOS 6 repositories # New countries and timezones # Cluster management: upgrade plan # Yomi refresh # Uyuni Server connections are always and only secure # Monitoring updates o Version 2020.11 # Recent Salt CVEs remediation # CentOS 7/8 ppc64le support # Prometheus Exporter Exporter for CentOS, Oracle and RHEL 7 and 8 # Node Exporter updated to version 1.0.1 for most operating systems # Web UI themes # Prometheus Exporter Exporter # XML-RPC power management API # Third-party errata information on vendor channels # Bootstrap repositories no longer flushed by default # DNSSEC enabled by default by bind update # Virtualization: Creation of virtual machines with Yomi, KickStart or AutoYaST profiles # Japanese translation o Version 2020.09 # Uyuni Hub XML-RPC API is now supported # Formula for peripheral server management (Technology Preview) # Maintenance windows # Monitoring reverse proxies # Added new type of "Virtual Host Manager": Nutanix AHV # Grafana 7.1.5 # New products enabled o Version 2020.07 # Upgrade notes # Salt 3000.0 # New "mgrcompat.module_run" custom compatibility state for Salt is available for registered systems. # PostgreSQL 12 # Base System Upgrade # New products enabled # Ubuntu 20.04 LTS # hwdata vendor change for openSUSE Leap 15.1 clients o Version 2020.06 # Oracle Linux # Third-party GPG keys now included # Cluster Management # Dropped feature: Unpublished patches o Version 2020.05 # Repository syncing performance improvements # Image profiles key-value pairs supported as arguments for Docker build # Service pack migrations: run a real migration after a successful dry-run o Version 2020.04 # Recurring actions # Bootstrapping Salt Clients with a Private SSH key (from API) # CentOS8 Content Lifecycle Management: Better Feedback with Appstreams # Automated Schema Database Upgrades and Failure Security Mechanism # Large Deployments Guide (draft) # Uyuni Hub documentation # Public Cloud QuickStart Guide (draft) # CaaSP Grafana Dashboads # Prometheus Federation Support in Formulas with Forms # Pre-configured default alerting rules # Prometheus Exporters for CentOS8 x86_64 # Node Exporter Updated to 0.18.1 # Virtualization: Management of storage pools o Version 2020.03 # Debian client tools # SUSE Container as a Service Platform v4 nodes: action filtering # Subscription matching in public cloud: BYOS vs PAYG # Automatic generation of bootstrap repositories # Salt clients: provisioning API # Recurring highstate scheduling # Content Lifecycle Filters for AppStreams # New products enabled # Ubuntu enhancements # Yomi (Technology Preview) # Uyuni Hub XML-RPC API (Technology Preview) # spacewalk-utils # EFI HTTP booting # Subscription matching enhancements # Single Sign-On (SSO) is now stable # Single Page Application UI (SPA) is now stable # Red Hat Enterprise Linux 8 onboarding simplified o Version 2020.01 # Version format change # Adjust your repository at the Server system # Remove current Uyuni Proxy 4.0 channel and repository from the Server and add the new ones # Remove current Uyuni Server 4.0 channel and repository from the Server and add the new ones # CentOS8, RHEL 8 and SLES ES 8 support # Monitoring # Package Hub # Formulas # New Content Lifecycle Management filters # Enhanced support for Debian and Ubuntu # New Prometheus exporters and formulas # Subscription matching in Public Cloud # Preventive shutdown of Server when running out of disk space # Single Page Application UI # Grafana # Prometheus service autodiscovery # CPU mitigation formula # Updated documentation # Enhanced support for Ubuntu and Debian clients # New products enabled (from SCC) # SUSE Container as a Service Platform v4 support # Other changes o Version 4.0.2 # Migrating the Server from 4.0.1 to 4.0.2 # Salt 2019.2.0 # Base system upgrade # Prometheus Monitoring # Content lifecycle management # Virtualization management for Salt minions # Updated Documentation Structure # Improved logging for Salt Remote Command Page # Support for more Distributions as Clients # EoL for openSUSE Leap 42.3 clients # Salt Rate Limiting (Batching) # Product Information Loaded from SCC # Image build host with SLES 12 SP4 # Updated backend for communicating with SCC # XMLRPC API changes # Support for Ubuntu Clients # Change behavior on token refresh # New option to force regeneration of channel metadata # New products supported # Package download endpoint override # Technical preview: Single Sign-On (SSO) o Version 4.0.1 # Support for PostgreSQL 10 # Migrating from PostgreSQL 9.6 to PostgreSQL 10 # spacecmd: Support state channels # New API calls # spacewalk-common-channels: Support for Uyuni, Fedora 29 and cleanup # Support for more Distributions as Clients # New products added to SCC syncing * Known issues + Single Sign On, API and CLI tools + EPEL and Salt packages + RHEL 6, CentOS 6 and Oracle Linux 6 minimal installations + RHEL native clients + Registering Spacewalk 2.x/Red Hat Satellite 5.x clients to Uyuni as Salt minions + CentOS * Client Tools Notes + Supported clients + Untested clients + Known limitations * Documentation * Installation + Requirements + Installing the Server + Update from previous versions of Uyuni Server + Update from previous versions of Uyuni Proxy * Other information + Red Hat Channels + SUSE Channels * Providing feedback * Legal Notices Version Revision History * 2021/03/01: 2021.02 release * 2021/02/05: 2021.01 release * 2020/11/26: 2020.11 release * 2020/09/22: 2020.09 release * 2020/07/24: 2020.07 release * 2020/06/15: 2020.06 release * 2020/05/21: 2020.05 release * 2020/04/16: 2020.04 release * 2020/03/19: 2020.03 release * 2020/01/31: 2020.01 release * 2019/08/02: 4.0.2 release * 2018/12/19: 4.0.1 release * 2018/10/26: 4.0.0 release Stay informed You can stay up-to-date regarding information about Uyuni: Check the home site https://www.uyuni-project.org Support Uyuni is a community-supported project. The ways or contacting the community are available at the home site. Release model Uyuni uses a rolling release model (meaning there will be no bugfixing for given Uyuni version, but new frequent versions that will include bugfixes and features) Check the home site get in contact with the community. Major changes since Uyuni Server 4.0.0 Features and changes Version 2021.02 Recent Salt CVEs remediation This release includes the fixes for CVE-2020-28243, CVE-2020-28972, CVE-2021-3148, CVE-2021-25281, CVE-2021-25282, CVE-2021-25283, CVE-2021-3144, CVE-2021-25284, CVE-2021-3197 and CVE-2020-35662 The fixes affect your Uyuni Server, Proxy, Retail Branch Server and Salt minions, so we recommend appling the fixes as soon as possible. Prometheus exporters' reverse proxy formula Ubuntu support The formula for Prometheus exporter?s formula can now be used with Ubuntu clients. Version 2021.01 Vendor change for some Java dependencies Besides the regular update, you will need to execute the following command to change the vendor for some Java dependencies: zypper install --allow-downgrade --allow-vendor-change -f apache-commons-cli-1.4-lp152.1.3.noarch apache-commons-jexl-2.1.1-lp152.1.1.noarch apache-commons-el-1.0-lp152.2.3.1.noarch If you do not do this, the WebUI will not start and you will get a HTTP 404 erroor. Fix version comparison algorithm for deb packages (Ubuntu) In some rare cases, Uyuni suggested that users upgrade Ubuntu packages with an older version than the one currently installed (for example, suggesting installation of libtre5-0.8.0-3+deb7u1ubuntu1 instead of libtre5-0.8.0-3ubuntu1). Starting with Uyuni 2021.02, the algorithm used for comparing package versions has been separated for RPM and deb packages. Having two algorithms for comparing packages means that deb packages in Ubuntu are now correctly ordered, and work as successfully as the RPM package algorithm. This means that the rare case explained above no longer occurs, and any proposed update is correct and should be performed. This update also fixes problems syncing Ubuntu and Debian channels and repositories. IMPORTANT: You need to plan this update. The database changes require updating the EVR information (epoch, version, release) for all packages. Depending on the specifications of your Uyuni installation, the number of channels, and onboarded instances, the services will take between 30 minutes and several hours while the schema is migrated. New products enabled * SUSE Linux Enterprise 15 SP3 family (beta) * SLE 15 SP1 LTSS * SUSE Linux Enterprise HPC 15 SP2 LTSS * SUSE Container as a Service Platform 4.5 (x86_64 and aarch64) SAP content SUSE Linux Enterprise Server for SAP applications is the best operating system to run your SAP workloads. Tthis release of Uyuni includes content which provides added value to SLES for SAP users: * Documentation: New Quick Start: SAP * Formulas: + saphanabootstrap-formula: SAP HANA deployment Salt formula. This formula can install SAP HANA nodes, enable system replication and configure SLE-HA cluster with the SAPHanaSR resource agent, using standalone Salt or via Uyuni formulas with forms. + sapnwbootstrap-formula: SAP Netweaver deployment Salt formula. This formula can install SAP Netweaver instances (ASCS, ERS, PAS, AAS) and perform some basic actions to optimize their usage. + drbd-formula: DRBD deployment Salt formula (requires drbd-utils) + habootstrap-formula: HA cluster salt deployment formula. This formula can boostrap an HA cluster ((init, join, remove) using standalone Salt or via Uyuni formulas with forms. * Salt state modules: + salt-shaptools: Salt modules and states for SAP Applications and SLE-HA components management * Grafana dashboards: + grafana-sap-hana-dashboards: Grafana Dashboards displaying metrics about SAP HANA databases. + grafana-sap-netweaver-dashboards: Grafana Dashboards displaying metrics about a SAP NetWeaver landscape. + grafana-ha-cluster-dashboards: Grafana Dashboards displaying metrics about a Pacemaker/Corosync High Availability Cluster. + grafana-sap-providers: Automated configuration provisioners used by other packages to enable zero-config installation of Grafana dashboards. The formulas and Salt state modules are included in the Uyuni Server channel. The Grafana dashboards are included in the Uyuni Client Tools for SLE 12 and SLE 15 channels. CPU mitigations formula Unsupported clients are now handled gracefully and mitigations have been added for the Xen hypervisor. Vendor change on SP migration Vendor change (changing the repository where a package comes from) can now optionally be enabled during service pack migration. This feature is useful where the client system is using unofficial packages and you want to move back to official packages, or to switch from an official package to a third-party version of a package. Instead of performing the SP migration within the same vendor and then manually installing the package from the new vendor, you can now do everything in a single action. This feature is available for SUSE Linux Enterprise 12 or newer, and can also be used to migrate from openSUSE Leap 15 to SUSE Linux Enterprise 15. Autoinstallation of older operating systems Autoinstallation provisioning is now compatible with GRUB and ELILO in addition to GRUB2 only, which is useful when provisioning SLES 11 SP4 and RHEL 6 (and clones) systems. Oracle Linux ULN repositories Oracle Unbreakable Linux Network repositories are now supported in Software > Manage > Repositories. Oracle Linux users with a subscription from Oracle can use this to manually add the repositories for KSplice and others. CentOS 6 repositories CentOS 6 reached end-of-life on November 30th, 2020, and the CentOS Project moved its repositories to the vault archive. URLs at spacewalk-common-channels for new Uyuni Servers. For existing Uyuni servers the database migration will take care of updating the URLs at the database. Other operating systems in the same class also reached end-of-life but require no change, since they will continue to work as-is: Oracle Linux 6 (URLs not changed) and Red Hat Enterprise Linux 6 (URLs are provided by users). New countries and timezones The countries and timezones list have been refreshed, adapting to the latest timezone and geopolitical changes. Cluster management: upgrade plan When upgrading a cluster, the upgrade plan is now shown in the WebUI. This makes it easier to verify that an upgrade will be conducted as expected. Yomi refresh The formulas that make autoinstallation of SLES and openSUSE systems simpler have been upgraded to the latest version provided by the Yomi project. The updated formulas are more intuitive, harder to misuse, and allow you to specify additional advanced options. Uyuni Server connections are always and only secure The WebUI and the CLI commands no longer provide an option to disable SSL. The option was in fact already obsolete and not working. Monitoring updates Grafana 7.3.1 Grafana has been updated from version 7.1.5 to version 7.3.1 which brings a number of bugfixes and improvements. Notable improvements: * Add monitoring mixing for Grafana. * New Cloudwatch metrics * Elasticsearch: Support multiple pipeline aggregations for a query. * Support request cancellation properly for PostgreSQL, Loki and Prometheus * Postgres: Support Unix socket for host * Loki: Re-introduce running of instant queries * Prometheus: Support request cancellation properly. Add $__rate_interval variable * API improvements * Variables: enables cancel for slow query variables queries * Table: Adds column filtering Breaking changes: * CloudWatch: The AWS CloudWatch data source?s authentication scheme has changed. See the upgrade notes for details and how this may affect you. * Units: The date time units YYYY-MM-DD HH:mm:ss and MM/DD/YYYY h:mm:ss a have been renamed to Datetime ISO and Datetime US respectively. A detailed changelog is available at upstream. Prometheus 2.22.1 The core of our monitoring solution, Prometheus, has been updated from version 2.18.0 to version 2.22.1, which brings a number of bugfixes and improvement. Notable improvements: * Web: Remove APIv2. * React UI: Implement missing TSDB head stats section. * UI: Add Collapse all button to targets page. * UI: Clarify alert state toggle via checkbox icon. * Gracefully handle unknown WAL record types. * Issue a warning for 64-bit systems running 32-bit binaries. * TSDB: Memory-map full chunks of Head (in-memory) block from disk. This reduces memory footprint and makes restarts faster. * TSDB: Reduced contention in isolation for high load. * Discovery: Added discovery support for Triton global zones. * Remote Read: Added prometheus_remote_storage_remote_read_queries_total counter to count the total number of remote read queries. * Added time range parameters for label names and label values API. Detail changelogs for each version between 2.18.0 and 2.22.1 can be found at: * https://github.com/prometheus/prometheus/releases/tag/v2.22.1 * https:// github.com/prometheus/prometheus/releases/tag/v2.22.0 * https://github.com/ prometheus/prometheus/releases/tag/v2.19.3 * https://github.com/prometheus/ prometheus/releases/tag/v2.19.2 * https://github.com/prometheus/prometheus/ releases/tag/v2.19.1 * https://github.com/prometheus/prometheus/releases/tag/ v2.19.0 * https://github.com/prometheus/prometheus/releases/tag/v2.18.2 * https://github.com/prometheus/prometheus/releases/tag/v2.18.1 Prometheus Exporter Exporter for Ubuntu 18.04/20.04 and Debian 9/10 The Reverse-proxy Exporter Exporter, which allows you to expose a single port no matter how many exporters are running on the client, is now available for 18.04/20.04 and Debian 9/10. Version 2020.11 Recent Salt CVEs remediation This release includes the fixes for CVE-2020-16846, CVE-2020-17490 and CVE-2020-25592 that we already released on November 16th for Uyuni 2020.09. If you did not apply the patch already, update your Uyuni Server, Proxy, Retail Branch Server and Salt minions as soon as possible. CentOS 7/8 ppc64le support Uyuni can now manage CentOS7 and CentOS8 ppc64le clients. Supported features are the same available for x86_64 clients. Prometheus Exporter Exporter for CentOS, Oracle and RHEL 7 and 8 The reverse-proxy Exporter Exporter, which allows you to expose a single port no matter how many exporters are running on the client, is now available for CentOS, Oracle and RHEL 7 and 8 for both x86_64 and ppc64le. Node Exporter updated to version 1.0.1 for most operating systems The following operating systems will receive version 1.0.1: * openSUSE Leap 15.1 and 15.2 * SLE12 (all service packs) * SLE15 (all service packs) * Ubuntu 20.04 * CentOS/Oracle/RHEL 8 * CentOS/Oracle/RHEL 7 All the changes can be found at the changelog for the package, or at https:// github.com/prometheus/node_exporter/releases/tag/v1.0.0 and https://github.com/ prometheus/node_exporter/releases/tag/v1.0.1 Keep in mind this new version includes some breaking changes: * The netdev collector CLI argument --collector.netdev.ignored-devices was renamed to --collector.netdev.device-blacklist in order to conform with the systemd collector. * The label named state on node_systemd_service_restart_total metrics was changed to name to better describe the metric. * Refactoring of the mdadm collector changes several metrics: * node_md_disks_active is removed * node_md_disks now has a state label for "fail", "spare", "active" disks. * node_md_is_active is replaced by node_md_state with a state set of "active", "inactive", "recovering", "resync". * Additional label mountaddr added to NFS device metrics to distinguish mounts from the same URL, but different IP addresses. * Metrics node_cpu_scaling_frequency_min_hrts and node_cpu_scaling_frequency_max_hrts of the cpufreq collector were renamed to node_cpu_scaling_frequency_min_hertz and node_cpu_scaling_frequency_max_hertz. * Collectors that are enabled, but are unable to find data to collect, now return 0 for node_scrape_collector_success. Web UI themes Uyuni now supports themes. Users can select what theme they want to use in the User Preferences page in the Web UI. Initially, we are providing three themes: * SUSE Manager light: default light, low-contrast theme * SUSE Manager dark: high-contrast theme based on the light theme * Uyuni: SUSE Manager 4.0 and Uyuni theme. Also high-contrast. Administrators can globally disable themes in /etc/rhn/rhn.conf by listing which themes they want to allow: # susemanager-light,susemanager-dark,uyuni web.themes = susemanager-light,susemanager-dark,uyuni web.theme_default = susemanager-light Prometheus Exporter Exporter The reverse-proxy Exporter Exporter, which allows you to expose a single port no matter how many exporters are running on the client, is now available for Ubuntu 20.04 LTS. XML-RPC power management API New APIs have been added to do IPMI power management. Redfish power management will be included in a future maintenance update. Third-party errata information on vendor channels It is now possible to add third-party errata information to CentOS and Ubuntu 20.04 LTS channels without cloning them, as described at the CentOS Clients section of the Client Configuration Guide. The known issue present in previous releases of Uyuni has been fixed. Bootstrap repositories no longer flushed by default In Uyuni 2020.03, we automated the generation of bootstrap repositories on channel sync. Bootstrap repositories were not only autogenerated but also autoflushed, which caused disappearing packages problems to some users (e. g., in the case of multi-architecture bootstrap repositories). Starting with Uyuni 2020.11, bootstrap repositories are not flushed by default. If you want to save some disk space, you can manually flush them using mgr-create-bootstrap-repo --flush. DNSSEC enabled by default by bind update With the update of ISC bind to version 9.16.6 on openSUSE Leap 15.1 and openSUSE Leap 15.2, DNSSEC is now enabled by default, which may cause DNS resolution to fail unless there are fallback DNS servers. The Retail Branch Server formula has been modified to disable DNSSEC, and will be updated to support DNSSEC in a future release of Uyuni. For existing Retail Branch Servers, you can disable DNSSEC to retain the same behaviour ISC bind showed until version 9.11.2. To do that, edit /etc/bind and set: dnssec-enable no; dnssec-validation no; Virtualization: Creation of virtual machines with Yomi, KickStart or AutoYaST profiles Creating a virtual machine using the Web UI and the Salt virt states can now use a defined Autoinstallation profile, any defined cobbler profile like the Yomi one. The virtual machine can also be created using PXE or by adding a CDROM device with an attached ISO image. Japanese translation The Uyuni Web UI and command-line tools are now available in Japanese thanks to the upstream Uyuni Community. Since this is a community translation, it is not enabled by default. In order to allow users to select Japanese in their User Preferences in the Web UI, add the following line to /etc/rhn/rhn.conf: java.supported_locales=en_US,ja A restart of Tomcat is required. Version 2020.09 Uyuni Hub XML-RPC API is now supported Starting with Uyuni 2020.09, the Uyuni Hub XML-RPC API is no longer considered a tech preview, but a fully supported feature. This means that multiple peripheral servers (other Uyuni Servers) can be managed from a single Hub node, as a supported feature. Formula for peripheral server management (Technology Preview) This version of Uyuni includes formulas that can be installed on a Hub node to manage the following on peripheral servers: * Organizations * System groups in organizations * Users in organizations * Access to system groups * Access to software channels To use the formula, run zypper in uyuni-config-formula on the Hub node, and then enable the formula for the peripheral servers, and use it to manage them. This feature will be documented at the Large Deployments Guide in a future Uyuni release. Maintenance windows The new maintenance windows feature allows you to schedule sensitive actions (like package installation or upgrade) to occur during a scheduled one-time or recurrent maintenance window period on selected systems. These actions are forbidden to be executed outside of the specified period. Maintenance windows are defined using iCalendar data, which can be exported from your favorite calendaring tool (Microsoft Outlook, KDE Organizer, Google Calendar??). For more information about Maintenance windows check the Administration Guide Monitoring reverse proxies Prometheus fetches metrics using a pull mechanism, so the server must be able to establish TCP connections to each exporter on the monitored clients. The new monitoring reverse proxies feature allows you to simplify your firewall configuration. By installing the reverse proxy on the clients you can get all the metrics for all the exporters on a single TCP port. Check the Monitoring section of the Administration Guide for information about how to set up. Monitoring reverse proxies are only available for SLE12, SLE15, and openSUSE Leap 15 families of products, and not yet available for other operating system platforms, including Red Hat Enterprise Linux and Ubuntu. Support for other operating system platforms will come in future releases of Uyuni Added new type of "Virtual Host Manager": Nutanix AHV In Uyuni 2020.09, we have added a new type of Virtual Host Manager in order to gather virtual machines from Nutanix AHV infrastructure. Creating a VHM to gather virtual instances from the Nutanix AHV enables the subscription matcher to match 1-2 virtual machines subscriptions for those instances that are running on the same virtualization host. For more information about how to setup this new type, see the Client Configuration Guide Note that this feature requires the virtual-host-gatherer-Nutanix package. Grafana 7.1.5 Grafana has been updated from version 7.0.3 to 7.1.5 which brings a number of bugfixes and improvements. Notable improvements: * Stats: Stop counting the same user multiple times. * Field overrides: Filter by field name using regex. * AzureMonitor: map more units. * Explore: Don?t run queries on datasource change. * Graph: Support setting field unit & override data source (automatic) unit. * Explore: Unification of logs/metrics/traces user interface * Table: JSON Cell should try to convert strings to JSON * Variables: enables cancel for slow query variables queries. * TimeZone: unify the time zone pickers to one that can rule them all. * Search: support URL query params. * Grafana-UI: Add FileUpload. * TablePanel: Sort numbers correctly. A detailed changelog is available upstream. New products enabled * SUSE Linux Enterprise Real Time 15 SP2 Version 2020.07 Upgrade notes WARNING: Check "Update from previous versions of Uyuni Server" section below for details, as this release updates the base OS from openSUSE Leap 15.1 to openSUSE Leap 15.2, and there are special steps required. The migration will be performed allowing vendor changes, so this upgrade will fix the issues with python3-psycopg2 mentioned at Uyuni Server 2020.05 release notes. Therefore you will not need to perform the manual steps mentioned there. Salt 3000.0 Salt has been upgraded to upstream version 3000, plus a number of patches, backports and enhancements by SUSE, for the Uyuni Server, Proxy and Client Tools. In particular, CVE-2020-11651 and CVE-2020-11652 fixes are included in our release. As part of this upgrade, cryptography is now managed by the Python-M2Crypto library (which is itself based on the well-known OpenSSL library). We intend to regularly upgrade Salt to more recent versions. For more details about changes in your manually-created Salt states, see the Salt 3000 upstream release notes. Please note Salt 3000 is the last version of Salt which will support the old syntax of the module.run module. New "mgrcompat.module_run" custom compatibility state for Salt is available for registered systems. WARNING - POSSIBLE ACTION REQUIRED: The syntax for Salt module.run state has changed starting in next Salt 3001 (Sodium) release. This means, any custom SLS file or "Configuration State Channel" that is using module.run state needs to be adapted to fit into the new syntax. This turns even more problematic when you have minions with different Salt versions, because some minions would accept the new syntax but others would fail with it, so the SLS files would require extra logic to handle the different Salt versions & configurations. To make this process much easier, we have introduced this new mgrcompat.module_run compatibility state, which is essentially a wrapper of module.run which accept the deprecated syntax and takes care of tailoring the parameters for the actual module.run if necesasary according to the particular minion version and configuration. The only thing to do would be to change module.run to `mgrcompat.module_run in your SLS files and "Configuration State Channels". As an example of this, a non-migrated state like this: my_module_run_state: module.run: - name: mymodule.func - m_name: foobar - other: 1234 would be adapted to: my_module_run_state: mgrcompat.module_run: - name: mymodule.func - m_name: foobar - other: 1234 We really encourage users and customer to start migrating their Salt States to use mgrcompat.module_run now before Salt 3001 (Sodium) release. Once Salt 3001 comes, those states will simply fail. PostgreSQL 12 The database engine has been updated from PostgreSQL 10 to PostgreSQL 12, which brings a number of performance and reliability improvements. A detailed changelog is available upstream. To prevent inconsistent configurations and data on upgrade or update, Uyuni 2020.06 will refuse to start until the database migration from PostgreSQL 10 to PostgreSQL 12 has completed successfully. Base System Upgrade The base system was upgraded to openSUSE Leap 15.2. New products enabled * Ubuntu 20.04 LTS Ubuntu 20.04 LTS Starting with Uyuni 2020.07, Ubuntu 2020.04 LTS is supported as a client. hwdata vendor change for openSUSE Leap 15.1 clients package hwdata now comes from from openSUSE Leap 15.1 and not from the client tools. In oder to get updated versions, the following command must be executed on the clients: zypper in --allow-vendor-change hwdata It is recommended to execute this as a remote command. This change is mandatory if you intend to use the openSUSE Leap 15.1 as a KVM virtualization host. This does not affect openSUSE Leap 15.2 as it will always have hwdata from the distribution. Version 2020.06 Oracle Linux Oracle Linux 6, 7 and 8 can now be managed with salt and it will support the same features CentOS 6, 7 and 8 support. The channels can be managed using spacewalk-common-channels. Third-party GPG keys now included Enabling verification of non-SUSE product metadata used to require manual acceptance, and sometimes even manual installation, of the third-party keys for products available from the product tree. Alternatively, an option to not verify the GPG key signature was there. Uyuni 2020.06 now includes the GPG keys used to sign packages and/or metadata by other the following vendors: * CentOS * Oracle Linux * Ubuntu * MicroFocus Open Enterprise Server Manual acceptance of those keys is no longer required for GPG signature verification for those products to work. Manual acceptance of GPG keys for any other product or repository is still required for security reasons. Cluster Management As you modernize your IT landscape and make use of Software Defined Infrastructure stacks based on technologies like Kubernetes and Ceph, your focus of managing the IT infrastructure has to move from managing individual Linux servers and VMs to managing infrastructure clusters. Multiple cluster types will be supported in coming releases, with Uyuni 2020.06 initially providing support for SUSE CaaSP. Computing is increasingly being a more complex architecure: redundant servers, scale out, high-availability, etc where you deploy different kinds of clusters, such as SUSE CaaS Platform, SUSE Enterprise Storage or SAP. Managing those as a whole piece of infrastructure instead of as discrete nodes puts you in charge. Uyuni 2020.06 implements cluster management of SUSE CaaS Platform clusters. Uyuni works hand-in-hand with CaaS Platform to make sure that all cluster operations are issued properly. The following actions are currently supported: * Register an existing cluster to Uyuni * Add or remove nodes to the cluster * Promote SLES system to managing node * Upgrade the cluster Deployment of CaaS Platform clusters from scratch will be supported in an upcoming version of Uyuni. Dropped feature: Unpublished patches The Unpublished Patches feature has been dropped in Uyuni 2020.05. This was a very old feature which originated more than 15 years ago when Spacewalk was used internally by vendors to manage patches before making them available to their customers. This functionality has been superseded a long (more than 10 years) time ago by other features in Uyuni for sysadmins, and by tools such as the Open Build Service for operating system vendors. After a consultation period with users both in the upstream Uyuni community and the SUSE Manager community, we received no feedback against the removal and executed on it. This will help us realize even further performance improvements in several areas, including the commonly-used Content Lifecycle Management build and promotion operations. If you still have any unpublished patches, make sure you publish them with Uyuni 2020.05 before migrating to Uyuni 2020.06. API breakage With the removal of the unpublished patches feature, the API specification changed as follows: * Method errata.listUnpublishedErrata was removed * Method errata.create has one less parameter (the publish boolean, now always true) and it is now mandatory to specify at least one channel label in the last parameter (channelLabels). Previously specifying at least one channel label was mandatory only if publish was set to true. Therefore some API calls that worked in Uyuni 2020.05 and earlier may need changes for Uyuni 2020.06 and later. Version 2020.05 Repository syncing performance improvements Repository synchronization has been optimized to perform faster than in previous versions. This applies to if the synchronization is triggered in the WebUI, or from the command prompt using the spacewalk-repo-sync command. It also applies whether the synchronization is invoked manually, or automatically as part of product or custom channel synchronizations. The performance improvement is up six times faster than previous versions, but the improvement depends mostly on your hardware setup, especially the number of CPUs, and how many packages are being synchronized. IMPORTANT: This requires a vendor change for the package python3-psycopg2-2.8.4-2.1.uyuni.x86_64. After running zypper update you will need force the vendor change with zypper in python3-psycopg2-2.8.4-2.1.uyuni.x86_64 Then update again again, so the spacewalk-backend subpackages are updated: zypper update As soon as python3-psycopg2-2.8.4 is part of openSUSE Leap 15.1 we will provide instructions use the openSUSE version again. Image profiles key-value pairs supported as arguments for Docker build Custom info key-value pairs defined in image profiles are now passed to the Docker build command as build arguments. They can be accessed in Dockerfiles using the ARG command. Service pack migrations: run a real migration after a successful dry-run After a Service Pack migration dry-run, if the result is a success you will get a "Run migration" button in the event history to retrieve the "dry-run" settings and confirm the migration with these settings. Version 2020.04 Recurring actions Scheduling recurring actions allows you to manage schedules for automated recurring highstate execution on client, group, and organization level depending on the frequency you choose. This is useful, for example, to apply highstates on a regular schedule and ensure configurations are enforced. For more information, see the Administration Guide. Bootstrapping Salt Clients with a Private SSH key (from API) Before this release, only password authentication was available for bootstrapping Salt clients from the Server. Now SSH private key authentication is available, including use of a passphrase on the private key. For Uyuni 2020.04 this is only available from the API. It will be made available from the WebUI in a future release. For security reasons, the private key is stored at the Uyuni Server only for the bootstrap procedure, and removed after bootstrapping is complete. The private key must be provided for each bootstrap. The new method bootstrapWithPrivateSshKey in the namespace system is documented in the API Documentation. You can use this example by adjusting the client, keyfile, passphrase, MANAGER_URL, MANAGER_LOGIN and MANAGER_PASSWORD according to your environment: #!/usr/bin/python import xmlrpclib client = '192.168.1.2' keyfile = '/path/to/priv/key' passphrase = '' # empty string = no passphrase conn = xmlrpclib.Server(MANAGER_URL, verbose=0) key = conn.auth.login(MANAGER_LOGIN, MANAGER_PASSWORD) with open(keyfile, 'r') as file: data = file.read() conn.system.bootstrapWithPrivateSshKey(key, server, 22, 'root', data, passphrase, '', False); conn.auth.logout(key) CentOS8 Content Lifecycle Management: Better Feedback with Appstreams The content lifecycle project page in the WebUI now has improved feedback messages about module filters, including missing or conflicting modules, and dependency resolution problems. The messages are in the form of errors that require the user to fix configurations, or warnings about potential problems. Automated Schema Database Upgrades and Failure Security Mechanism Database schema upgrades are now applied automatically during services startup, so there is no need to call spacewalk-schema-upgrade manually. A security mechanism has been implemented that prevents Uyuni Services from starting if the schema upgrade has failed. When this occurs: 1. When you run spacewalk-service start, it will fail and show an output with information about the error. 2. All services, including the Apache service, will not start. This will also cause the WebUI to be unavailable. Large Deployments Guide (draft) Uyuni is designed by default to work on small and medium scale installations. For installations with more than 1000 clients per Uyuni Server, adequate hardware sizing and parameter tuning must be performed, and the new guide provides information about how to do it. Keep in mind there is no hard maximum number of supported systems. Many factors can affect how many clients can reliably be used in a particular installation. Factors can include which features are used, and how the hardware and systems are configured. Uyuni Hub documentation The Uyuni Hub announced for 2020.03 has now documentation available as part of the Large Deployments Guide (section Multiple Servers with Hub). This is a draft release, so please provide feedback using the Resources menu in the online documentation Public Cloud QuickStart Guide (draft) This new draft guide shows you the fastest way to get Uyuni up and running in a public cloud. It includes instructions for Amazon Web Services, Microsoft Azure, and Google Cloud Engine. This is a draft release, so please provide feedback using the Resources menu in the online documentation CaaSP Grafana Dashboads CaaSP specific Grafana dashboards have been integrated and can be deployed via the UI. Prometheus Federation Support in Formulas with Forms The new version of the Prometheus formula allows configuring federation and pulling relevant metrics from Prometheus instances to provide a global monitoring view. Note that suitable recording rules have to be configured on the Prometheus instances (for example at CaaSP Prometheus instances). For more information about Prometheus federation, check the official documentation. Pre-configured default alerting rules A default set of alerting rules have been added to monitor the Prometheus instances themselves (meta-monitoring) and the availability of configured targets. The rules can be disabled in the WebUI. Prometheus Exporters for CentOS8 x86_64 We now provide these Prometheus exporters as packages for CentOS8 x86_64 (compatible also with similar systems such as RHEL8): * Node exporter - Hardware and operating system metrics * PostgreSQL exporter - PostgreSQL database metrics * Apache exporter - Apache HTTP server metrics Node Exporter Updated to 0.18.1 All the changes can be found at the changelog for the package, or at https:// github.com/prometheus/node_exporter/releases/tag/v0.18.0 and https://github.com /prometheus/node_exporter/releases/tag/v0.18.1 Keep in mind this new version includes some breaking changes: * Renamed interface label to device in netclass collector for consistency with other network metrics * The cpufreq metrics now separate the cpufreq and scaling data based on what the driver provides * The labels for the network_up metric have changed * Bonding collector now uses mii_status instead of operstatus * Several systemd metrics have been turned off by default to improve performance. These include unit_tasks_current, unit_tasks_max, service_restart_total, and unit_start_time_seconds * The systemd collector blacklist now includes automount, device, mount, and slice units by default Virtualization: Management of storage pools Until now users could list the storage pools, which is where the virtual machines disks are stored. Storage pools are where virtual machine disks are stored. In previous versions, you could only list the pools. With this update, you can create, edit, start, stop, refresh, and delete storage pools. This is available from the WebUI, or through Salt states. Version 2020.03 Debian client tools We now offer Debian client tools that allow for easy onboarding of Debian as salt minions, as well as running spacecmd from them. Check the Client Configuration Guide for information about how to configure Uyuni Server to work with Debian clients. For now the following architectures are supported: x86_64, aarch64, armv7l, i586 We plan to continue improving Debian support in the future, including support for ppc64le and s390x Debian 10 clients. SUSE Container as a Service Platform v4 nodes: action filtering Nodes in a SUSE Container as a Service Platforms should be patched, rebooted, etc following CaaSP recommendations to avoid breaking cluster availability and software compability. In Uyuni 2020.03, we have introduced node locking and action filtering to prevent uninteded operations. * When CaaSP nodes are added to Uyuni, the registered systems will be locked automatically: * When a system is locked, the web UI shows a warning and you can schedule actions using the web UI or the API, but the action will fail. You can enable or disable the system lock using the System Lock formula. When the system lock is disabled, all operations are permitted. Subscription matching in public cloud: BYOS vs PAYG In Uyuni 4.0.1, we introduced virtual host gatherers for Amazon Web Services, Microsoft Azure and Google Cloud Engine. With these gatherers, our subscription matcher gained the ability to also include virtual machines running on the cloud in its calculations. We have now enhanced the subscription matcher to exclude pay-as-you-go (PAYG) instances. Those do not require a subscription, as the agreement between the Cloud Service Provider and the Customer covers them. Automatic generation of bootstrap repositories A bootstrap repository contains packages for installing Salt on clients, as well as the required packages for registering Salt or traditional clients during bootstrapping. In Uyuni 2020.01 and earlier, bootstrap repository creation was a manual step, by using the mgr-create-bootstrap-repo tool. In Uyuni 2020.03, bootstrap repositories are automatically created and regenerated on the Uyuni Server after a product is synchronized (i. e. all mandatory channels are fully mirrored). More details, including how to revert to manual invokation, are available from the Client Configuration Guide. Salt clients: provisioning API Enable provisioning API with Salt and bootstrap entitled systems. Previously, this only worked for traditional clients. Recurring highstate scheduling You can schedule automated recurring highstate actions for Salt clients. Recurring highstate actions apply the highstate to clients on a specified schedule. You can apply recurring action to individual clients, to all clients in a system group, or to an entire organization. The Recurring Actions section in the Administration Guide contains all the details for this feature. More improvements in regards to automation will be coming in subsequent releases of Uyuni: maintenance windows and patch automation. Content Lifecycle Filters for AppStreams RHEL, SLES ES, CentOS and Oracle Linux 8 appstreams can now be mixed and converted to flat repositories using a new type of CLM filter. New products enabled * SUSE Linux Enterprise Real Time 12 SP5 * SUSE Linux Enterprise 15 SP2 family * MicroFocus Open Enterprise Server 2018 SP2 (product GA in Q2 2020) * Oracle Linux 8 (using spacewalk-common-channels) Ubuntu enhancements Each Uyuni release and maintenance update brings better Ubuntu support. In Uyuni 2020.03, we have include two small but valuable improvements: * Support package pre-downloading, to ensure all content (.deb packages) is downloaded before patching. This should be very useful for large Ubuntu deployments managed by Uyuni. * Display additional information in the UI for .deb packages (dependencies and more headers) Yomi (Technology Preview) Yomi (yet one more installer) is a Salt-based installer for SUSE and openSUSE operating systems. In Uyuni, Yomi can be used as part of provisioning new clients, as an alternative to AutoYaST. Yomi consists of two components: * The Yomi formula, which contains the Salt states and modules required to perform the installation. * The operating system image, which includes the pre-configured salt-minion service. Detailed information on how to use Yomi is available from the Salt Guide. Yomi is work in progress and more operating systems and features will be added in coming releases. Uyuni Hub XML-RPC API (Technology Preview) The Uyuni Hub is a new multi-server architecture we are introducing as a technology preview in Uyuni 2020.03. Multiple Uyuni Servers can be managed from a single Hub node. The Hub is a Salt master itself and the managed Uyuni Server servers are both a minion (to the hub) and a master (to their own minions). Uyuni Hub Architecture The Hub covers a number of use cases, such as: * Scalability: when a single Uyuni Server will no longer be enough * Intermittently connected and bandwidth-limited sites, which can now be managed with their own schedule thanks to the Hub * Multi-tenancy with individual Uyuni Servers. While Uyuni is multi-organization itself, in some scenarios, an even stronger separation is required. The Hub provides a way to manage and aggregate back information for all those Uyuni Server servers. The Hub comprises a number of components that we will be releasing and enhancing in the future. The first component of the Hub we are now introducing as a Technology Preview is the Hub XML-RPC API, which provides an extended version of the Uyuni Server XML-RPC API, targeted for the multi-server case. Installation and usage Install Uyuni Server and then install the hub-xmlrpc-api package. That Uyuni Server is now the Hub Server. Configuration of hub-xmlrpc-api is specified in a JSON file like the following: { "type": "json", "hub": { "manager_api_url": "http://localhost/rpc/api" }, "connect_timeout": 10, "read_write_timeout": 10, } Set the HUB_CONFIG_FILE environment variable to point to the configuration file. hub-xmlrpc-api is a daemon, currently to be launched from the command line. Once running, you can connect to the hub-xmlrpc-api at port 8888 via any XMLRPC compliant client libraries (see examples below). API endpoints, namespaces and examples Details about usage with Python script examples are available at the Uyuni project site: https://github.com/uyuni-project/hub-xmlrpc-api spacewalk-utils In Uyuni 2020.01 and earlier, the spacewalk-utils package contained a mix tested and untested tools. In Uyuni 2020.03, we have split spacewalk-utils in two packages: * spacewalk-utils contains only fully-tested tools: + spacewalk-common-channels + spacewalk-hostname-rename + spacewalk-clone-by-date + spacewalk-sync-setup + spacewalk-manage-channel-lifecycle * spacewalk-utils-extras contains the tools that untested or not completely tested: + apply_errata + delete-old-systems-interactive + migrate-system-profile + spacewalk-api + spacewalk-export + spacewalk-export-channels + spacewalk-final-archive + spacewalk-manage-snapshots + sw-ldap-user-sync + sw-system-snapshot + taskotop + spacewalk-manage-channel-lifecycle Tools in spacewalk-utils-extras are valuable but they are so specific, or require additional customization for each user, that it is not possible for us to test for every use case. If you were using these scripts in spacewalk-utils in Uyuni 2020.01 or earlier, you will need to install spacewalk-utils-extras in Uyuni 2020.03. EFI HTTP booting The dhcp formula, branch network formula and pxe formula have been updated to support booting EFI terminals (systems) via HTTP in addition to TFTP. Subscription matching enhancements On public cloud providers, the subscription matcher will identify pay-as-you-go instances, whose subscription is provided by the Cloud Service Provider, and will not ask for additional subscriptions. Also, stackable subscriptions with the same parameters will be aggregated. Single Sign-On (SSO) is now stable Uyuni supports Single Sign-On authentication by implementing the Security Assertion Markup Language (SAML) 2 protocol. This feature, introduced in 4.0.2 as a Technology Preview, is now declared stable Uyuni must be reconfigured to use the IdP as the source of authentication and post-login mapped users must be already created before enabling SSO. For more on configuring SSO, see the Authentication Methods chapter in the Administration guide. Single Page Application UI (SPA) is now stable In an effort to provide our web UI users with a smoother navigation, we have implemented large parts of the user interface as a single page application. This enhancement was started in Uyuni 2020.01 as an opt-in feature and now becomes the default in Uyuni 2020.03 Red Hat Enterprise Linux 8 onboarding simplified It is no longer necessary to have Python 3 on RHEL8 systems for the onboarding to work. With this enhancement, even plain-text RHEL machines can be onboarded directly. Version 2020.01 Version format change Uyuni is now changing from X.Y version format to YYYY.MM format, and the URLs for the repositories remove the X.Y part. This will allow easier releases, no need to change URLs at all in the future, and less confussion regarding the relationship between Uyuni and SUSE Manager (Uyuni is always ahead). Adjust your repository at the Server system Because of the version format change, you need to adapt your zypper repository at the server before updating. If you followed the instructions for installation, this command will do it for you: sed -i -e 's/Uyuni-Server-4.0-POOL-x86_64-Media1/ Uyuni-Server-POOL-x86_64-Media1/' /etc/zypp/repos.d/uyuni-server-stable.repo Otherwise, find the Uyuni Server Stable repository and replace: baseurl=https://download.opensuse.org/repositories/systemsmanagement:/Uyuni:/ Stable/images/repo/Uyuni-Server-4.0-POOL-x86_64-Media1/ with: baseurl=https://download.opensuse.org/repositories/systemsmanagement:/Uyuni:/ Stable/images/repo/Uyuni-Server-POOL-x86_64-Media1/ Remove current Uyuni Proxy 4.0 channel and repository from the Server and add the new ones If you are currently syncing Uyuni Proxy 4.0 (usually because you have proxies), you need to: 1. Add the new channel with spacewalk-common-channel uyuni-proxy-stable-leap-151 2. Sync the new channel (and configure autosync if required) 3. See what instances are using the channel Uyuni Proxy 4.0 for openSUSE Leap 15.1 4. Adjust the channels assigned instances from previous step (tip: You can use "System Set Manager") to remove the old one and add the new one. 5. See what activations key are using the channel Uyuni Proxy 4.0 for openSUSE Leap 15.1 6. Adjust the activation keys from previous set to remove the old channel and add the new one. 7. Remove the channel Uyuni Proxy 4.0 for openSUSE Leap 15.1 8. Remove the repository External - Uyuni Proxy 4.0 for openSUSE Leap 15.1 (x86_64) Remove current Uyuni Server 4.0 channel and repository from the Server and add the new ones Most users will not require this unless, but if you have the Uyuni Server 4.0 channel at your server: 1. Add the new channel with spacewalk-common-channel uyuni-server-stable-leap-151 2. Sync the new channel (and configure autosync if required) 3. See what instances are using the channel Uyuni Server 4.0 for openSUSE Leap 15.1 4. Adjust the channels assigned instances from previous step (tip: You can use "System Set Manager" at the WebUI) to remove the old one and add the new one. 5. See what activations key are using the channel Uyuni Server 4.0 for openSUSE Leap 15.1 6. Adjust the activation keys from previous set to remove the old channel and add the new one. 7. Remove the channel Uyuni Server 4.0 for openSUSE Leap 15.1 8. Remove the repository External - Uyuni Server 4.0 for openSUSE Leap 15.1 (x86_64) CentOS8, RHEL 8 and SLES ES 8 support CentOS 8, Red Hat Enterprise Linux 8 and SUSE Linux Enterprise Server Expanded Support 8 are now supported clients as Salt minions. The traditional stack will not be supported on these operating systems. With the new application streams concept introduced in these operating systems, you will need to import both the BaseOS and the AppStream directories from the ISO image for the bootstrap repository to be created correctly. If the AppStream directory is not imported, you will receive an error about missing Python 3 packages. AppStream awareness in the UI and Content Lifecycle Management will be available in an upcoming version of Uyuni. Monitoring This version of Uyuni includes formulas to install Prometheus and Grafana, and makes the Apache exporter available for Ubuntu 18.04, CentOS6, CentOS7 and Proxy. Additionally, self-monitoring capabilities have been implemented in the Admin Monitoring UI. Package Hub SUSE Package Hub is now supported on the Server, since the problems with the search that were caused by PackageHub-provided packages have been solved. If you were using Package Hub as a source of packages for you clients, it is recommended that you re-generate all package metadata. The reason for this is in the Package Hub repositories there may exist multiple packages with the same NEVRA but different checksums. This might result in checksum errors when repositories are used on the clients as Uyuni randomly selected any of those packages. After this update, Uyuni will generate the checksum into the package path to ensure the right package is used. If you use also Uyuni Proxy please update all of them before you re-generate the metadata. Formulas The Formulas with Forms screen has an enhanced layout that folds vertically instead of nesting deep inside, making if cleaner. Besides this, validators are now possible in formulas using the JEXL expression language. The cpu-mitigations-formula is now installed by default. The Retail branch network formula now works all SUSE and openSUSE based distros, using SuSEfirewall or firewalld as appropriate. New Content Lifecycle Management filters In Uyuni 4.0.2 we introduced Content Lifecycle Management with a filter to exclude packages and patches based on their name. Feedback for this feature was very positive and many proposals for enhancement were received. In this release, we are introducing a lot of new possibilities for Content Lifecycle Management: * New filters: by date, by keyword (e. g. "reboot needed" or "package manager restart required"), by type (security, recommended or optional), by synopsis and "patch contains package". * New ALLOW mode, which in addition to the existing DENY mode, makes possible to filter out packages, and then include them again into the resulting set. * New matchers: in addition to the existing greater than, lesser than, equals, etc, we have now added a regular expression matcher for package names, patch names, patch synopsis and package names in patches. * Better visualization of the filters attached to a CLM project, with ALLOW and DENY now shown on each side of the screen. We have documented two typical use cases: a monthly patch cycle and live patching. More enhancements to Content Lifecycle Management will come in future releases of Uyuni. Enhanced support for Debian and Ubuntu With each release of Uyuni, we continue to enhance our Debian and Ubuntu support. Uyuni 2020.01 greatly improves our compatibility thanks to: * Support for all of the headers in .deb packages, including custom ones, when syncing Debian/Ubuntu repositories. You can use the new script mgr-update-pkg-extra-tags to update extra fields in DB without recreating all Debian/Ubuntu channels. * Support for .deb packages with hyphens in the package name or version. There remain a very small percentage (<0.1%) of packages for which our version comparison algorithm fails; we will fix this known issue in a coming release. New Prometheus exporters and formulas A new set of client tool packages now includes Prometheus exporters for more clients: CentOS 6, CentOS 7, RHEL 6, RHEL 7, SLES ES 6, SLES ES 7 and Ubuntu 18.04. Both the Prometheus node exporter and the PostgreSQL exporter are provided for those operating systems. The prometheus-exporters-formula formula makes easy to deploy them. Subscription matching in Public Cloud We?ve added new types of Virtual Host Managers in order to gather virtual instances from Public Cloud providers. Azure, AWS and Google Cloud are now supported, in addition to the existing VMware and generic (file-based, manually-maintained, useful for any cloud provider) gatherer modules. Creating VHM to gather virtual instances from the Public Cloud will enable the subscription matcher to match "1-2 virtual machines" subscriptions for those instances that are running on the same Public Cloud zone. Please take into account the following considerations in this version. They will be addressed in upcoming versions of Uyuni: * This functionality will only work with Salt clients. * Manual installation of the virtual-host-gatherer-libcloud package is required. * The public cloud gatherers will report and try to match all instances, no matter if they are BYOS or PAYG, leading to an incorrect calculation of the required subscriptions if you combine BYOS and PAYG. Preventive shutdown of Server when running out of disk space Some users have hit in the past a database corruption problem when PostgreSQL ran out of space. In order to prevent that from happening in the future, we have added a diskchecker to Uyuni Server. This feature will send a warning mail when the most common and important Uyuni directories are below 10% of free disk space, and will shut down the Uyuni Server when those directories are below 5% of free disk space. This new feature is only enabled by defult in new installations. For existing installations, the administrator can enable the tool manually after updating to the latest maintenance update by running: systemctl --quiet enable spacewalk-diskcheck.timer systemctl start spacewalk-diskcheck.timer Full details on the parameterization of this new feature are available in the Managing disk space documentation page. Single Page Application UI In an effort to provide our web UI users with a smoother navigation, we have implemented large parts of the user interface as a single page application. This feature is optional for this release and is disabled by default. To enable it, users can now add web.spa.enable = true to /etc/rhn/rhn.conf, and then restart Tomcat. Grafana Grafana is a tool for data visualization, monitoring, and analysis. It is used to create dashboards with panels representing specific metrics over a set period of time. Grafana is commonly used together with Prometheus, but also supports other data sources such as ElasticSearch, MySQL, PostgreSQL, and Influx DB. This version of Uyuni includes Grafana in the client tools repositories. An Uyuni Grafana dashboard is provided as an example. Monitoring section of the Administration Guide contains full detail on how to configure Grafana together with Uyuni. Prometheus service autodiscovery Prometheus is a monitoring tool used to record real-time metrics in a time-series database. Metrics are collected using HTTP pulls, allowing for higher performance and scalability. We have updated the Prometheus package with a new version that include a built-in service discovery mechanism that will allow users to more easily configure monitoring on their Uyuni systems. Previously, after configuring the exporters on managed clients, users had to manually configure their Prometheus servers to start scrapping metrics from those systems. With this update, it will be possible to use a "service discovery" mechanism that will automate this part of the configuration. The configuration options are simple: it is only required to provide a Uyuni Server URL and valid API credentials. Under the hood, what this mechanism does is letting Prometheus poll the Uyuni API, asking for a list of systems that have monitoring enabled, and automatically configuring Prometheus to collect metrics from those systems. In this version, the autodiscovery functionality is provided as a Technology Preview. More information about configuring Prometheus can be found in the Monitoring section of the Administration Guide. CPU mitigation formula CPU mitigations have been introduced to improve security on CPUs affected by vulnerabilities such as Meltdown and Spectre. The mitigations are available in SUSE Linux Enterprise 12 SP3 and later in the cpu-mitigations-formula package, which is not installed by default. The new CPU Mitigation formula allows you to control which mitigations are enabled. Updated documentation The Uyuni documentation has received improvements in all of the books, with small clarifications and enhancements all around: content lifecycle management filters, public cloud, JeOS, formulas, etc Of particular interest for users with large installations will be the new Large Scale Deployment and Salt Tuning sections in the Salt Guide. Given that modifying advanced parameters can cause catastrophic failure, we recommend making a backup and being conservative doing changes. Additionally, the search functionality in the documentation now works offline. Enhanced support for Ubuntu and Debian clients The Multi-Arch and Pre-depends headers are now supported for .deb repositories, hence avoiding installation problems that could arise in some cases when deploying packages from the UI. Also, Ubuntu and Debian channels now come preconfigured in spacewalk-common-channels. The Debian CDN is used to provide the best mirror at each moment. For Ubuntu, you may want to replace the default mirror with a closer geo-mirror. Keep in mind SUSE does not provide support for the spacewalk-common-channels tool form the spacewalk-utils package. New products enabled (from SCC) * SLES12 SP3 LTSS * SUSE Linux Enterprise Real Time 12 SP4 * SLES12 SP5 * RHEL 8 and SLES ES 8 * CaaSP 4 * openSUSE Leap 15.1 SUSE Container as a Service Platform v4 support The Virtual Host Manager functionality has been extended to support SUSE Container as a Service Platform v4. You can register each CaaSP node to Uyuni using the same method as you would a Salt client. After doing this, you will be able to see the patch level status of each node, perform configuration management on the nodes and assign channels o clusters. We strongly recommend to check the documentation on the scope and extent of the CaaSPv4 integration in Uyuni: https://www.uyuni-project.org/uyuni-docs/ suse-manager/client-configuration/vhm-caasp.html Upcoming versions of Uyuni will enhance CaaSP integration. Other changes * Since this version, as part of a bugfix, it is no longer allowed to delete a channel when there are cloned channels based on it. * Taskomatic now takes a maximum of 4 GB of RAM (it used to be 2 GB), which better matches the current average use case. * Salt clients can now be re-provisioned from Uyuni. This allows major version OS updates for SLES and Uyuni Proxy. * Normalize date formats for actions, notifications and CLM Version 4.0.2 Migrating the Server from 4.0.1 to 4.0.2 If you are using DHCP addresses and you do not use DHCP reservations, migrating from openSUSE Leap 42.3 to Leap 15.0 can change the IP address of your NICs. If using DHCP, make sure your instances have reserved IP addresses. Before starting, make sure you have a backup of your server, as it will be hard to recover from failures during the migration. 4.0.2 is now based on openSUSE Leap 15.1, so a base OS system is required. To help administrators with the migration, a new script is provided by the susemanager package at /usr/lib/susemanager/bin/server-migrator.sh Then, update susemanager package only: zypper ref zypper in susemanager And finally run the script: /usr/lib/susemanager/bin/server-migrator.sh After the migration is complete, you will be requested to reboot your server Uyuni Server 4.0.2 works with SUSE Uyuni Proxy 4.0.1. When upgrading, upgrade the Server first, followed by the Proxies. Salt 2019.2.0 Salt has been upgraded to the 2019.2.0 release. We intend to regularly upgrade Salt to more recent versions. For more detail about changes in your manually-created Salt states, see the Salt upstream release notes 2019.2.0. Base system upgrade The base system was upgraded to openSUSE 15.1. As a result, all code was ported to run with Python 3 and OpenJDK 11. Prometheus Monitoring We now include packages for the latest version of Prometheus, as well as self-monitoring capabilities for Uyuni. Prometheus is a monitoring tool that is used to record real-time metrics in a time-series database. For more information about Prometheus, see the Administration Guide Exporters Exporters convert existing metrics into the format Prometheus requires. We are now providing the following Prometheus Exporters as packages, for SLE12 and SLE15 as well as openSUSE Leap 15.1: * Node exporter - Hardware and operating system metrics * PostgreSQL exporter - PostgreSQL database metrics * Squid exporter - Squid Proxy metrics * Apache exporter - Apache HTTP server metrics In addition we provide JMX exporter on Uyuni Server. Monitoring is not yet available for other operating system platforms like Red Hat Enterprise Linux or Ubuntu. Self-monitoring features in Uyuni Uyuni provides metrics about its health to Prometheus. Both Server and Proxy can expose metrics. Self-monitoring can be enabled via the Web UI. For that purpose, some Prometheus exporters are pre-installed on Uyuni Server and Proxy. A new formula is also included, to install and manage Node and PostgreSQL exporters on clients managed by Salt. This formula can be configured in the Uyuni Web UI. Content lifecycle management The content lifecycle management feature allows you to clone software channels through a lifecycle of several environments. You are able to create content projects, select a custom set of software channels as sources, and promote software channels through a pre-defined lifecycle of environments. You can define filters to exclude specific packages and patches. More filters will be added in a later release. Once you have selected your sources you can build the selected set which will populate the first environment. After the first environment is built, you can promote it through the environment lifecycle to the next environment in the loop. You can see the status of the build at any time throughout the process. The result of the build, and the content of every environment, is a channel tree made of cloned software channels of the selected sources, to which systems can be assigned. Virtualization management for Salt minions The existing virtualization features have been enhanced for Salt-based systems. This is a technology preview and will require an additional Virtualization Management entitlement. Pricing will be announced soon. Salt-based virtualization host systems can also create virtual machines using a pre-built disk image. These features have been added: * Deleting virtual machines. * Editing virtual machines to add or remove network interfaces or disk, change CPU and memory allocation or the display type. * Quick update of the list and state of virtual machines. * Displaying virtual machines graphical display in a new tab. Updated Documentation Structure In this release, we have reorganized our documentation and updated our tooling to make it clearer where information is, and make it easier for you to find the content you need, when you need it. Old Naming Format * Getting Started * Best Practices * Reference * Advanced Topics New Naming Format * Installation Guide (Requirements, supported platforms, installation methods, etc) * Client Configuration Guide (Configuring and connecting clients to Uyuni) * Upgrade Guide (Migrate and update clients and Uyuni) * Reference Guide (Comprehensive guide to the Web UI) * Administration Guide (Maintenance and administration tasks in Uyuni) * Salt Guide (A comprehensive guide to Salt for system administrators) * Retail Guide (A guide to using Uyuni for Retail) Improved logging for Salt Remote Command Page The Salt Remote Command Page log now every command executed in a separate logfile (/var/log/rhn/rhn_salt_remote_commands.log). In addition to this, an entry in the System History is generated for every minion where the command was executed. Support for more Distributions as Clients openSUSE Leap 15.1 and SLE15 SP1 can now be managed. EoL for openSUSE Leap 42.3 clients openSUSE Leap 42.3 is now End of Life since July 1st, as announced at the openSUSE Mailing lists While the repositories for Leap 42.3 are still available, no support is provided aymore. Salt Rate Limiting (Batching) Any action scheduled on multiple Salt minions has now an upper limit on the number of systems that will process it simultaneously. This is referred to as batch size in Salt jargon, and defaults to 100 minions. Please check the documentation for performance considerations in large installations (more than 1000 minions). Product Information Loaded from SCC In the past information about product channels were shipped via maintenance updates. Now these information will be downloaded from SUSE Customer Center (SCC) like the other product and repository information. In case of using the fromdir configuration with SMT or RMT, please check if they support already downloading this file. You can get the file with the following command: curl -O https://scc.suse.com/suma/product_tree.json Image build host with SLES 12 SP4 Using SLES 12 SP4 as the base OS for an image build host is now supported. Also building SLES 12 SP4 OS Images is supported. Updated backend for communicating with SCC This update contains a new backend to communicate with the SUSE Customer Center (SCC). This requires to run a mgr-sync refresh at the end of the update procedure. The whole update procedure: $> spacewalk-service stop $> zypper patch $> spacewalk-schema-upgrade $> spacewalk-service start $> mgr-sync refresh In case of Inter Server Sync (ISS) the master needs to be updated first, then the slave. This change show products like they are setup in the SUSE Customer Center. As a consequence of this some older products show no architecture anymore and mirror all available architectures when such a product is selected for mirroring. With this change also some invalid product combinations were removed. Please check /var/log/rhn/rhn_web_ui.log for error messages. Invalid channels can be removed using spacewalk-remove-channel command. XMLRPC API changes Due to the changes in the backend for communicating with SCC corresponding XMLRPC API has changed: Deprecated calls: synchronizeChannels() synchronizeProductChannels() New call: synchronizeRepositories() For a refresh the XMLRPC API should be called in the following order: synchronizeChannelFamilies synchronizeProducts synchronizeRepositories synchronizeSubscriptions Support for Ubuntu Clients Management of Ubuntu clients is now supported. We provide a repository with salt packages that can easily be added with spacewalk-common-channels or manually. The following new features were added: * Bootstrapping and performing initial state runs such as setting repositories and performing profile updates * Assigning .deb channels to minions * Information displayed in System details pages * Package install, update, and remove * Package install using Package States * Configuration and state channels * Support Ubuntu products and Debian architectures in mgr-sync * Support creating bootstrap repositories for Ubuntu 18.04 and 16.04 * Add support for Ubuntu in the bootstrap script * Generate InRelease file for Debian/Ubuntu repos when metadata signing is enabled * Trust SUSE GPG key for client tools channels on Ubuntu systems However, the root user on Ubuntu is disabled by default, so in order to use bootstrapping, you will require an existing user with sudo privileges for Python. Change behavior on token refresh Channel authentication tokens are valid by default for about 1 year. The renew of tokens happens automatically some time before they expire but they are not deployed automatically to the clients. As the renew happens mostly without noticing by the administrator that behavior has changed to autodeploy renewed tokens to the clients automatically. This old behavior can be preserved by setting token_refresh_auto_deploy = false in /etc/rhn/rhn.conf and restarting the services. In case of a token renew without autodeployment enabled a log message will inform the administrator about it. New option to force regeneration of channel metadata A new option --force was added to spacecmd softwarechannel_regenerateyumcache to force a regeneration of the metadata files. New products supported * openSUSE Leap 15.1 * SLES11 SP4 LTSS * SLES12 SP3 LTSS * SLES 15 SP1 product family * CaaSP 4 Toolchain Package download endpoint override It is now possible to set a custom protocol, host and path for minions to download packages at installation time. This will override the default setting of the Uyuni Server or Uyuni Proxy used at registration time. Technical preview: Single Sign-On (SSO) Uyuni supports Single Sign-On authentication by implementing the Security Assertion Markup Language (SAML) 2 protocol. Mandatory requirement: an already existing and configured SAML Identity Service Provider (IdP). Uyuni must be reconfigured to use the IdP as the source of authentication and post-login mapped users must be already created before enabling SSO. For more on configuring SSO, see the Administration Guide Version 4.0.1 Support for PostgreSQL 10 A new version of the PostgreSQL database is available in openSUSE Leap 42.3 and can be used for Uyuni Server. New installations of Uyuni Server based on openSUSE Leap 42.3 will automatically pick up this version. PostgreSQL 10 needs a new version of smdba to initiate backups. This version is part of Uyuni Server 4.0.1. Migrating from PostgreSQL 9.6 to PostgreSQL 10 You should have an up-to-date database backup before attempting the migration. Existing installations of Uyuni Server will need to run /usr/lib/susemanager/bin/pg-migrate-96-to-10.sh to migrate from PostgreSQL 9.6 to PostgreSQL 10 Your Uyuni Server installation will not be accessible during the migration. Note The migration will create a copy of the database under /var/lib/pgsql and thus needs sufficient disk space to hold two copies (9.6 and 10) of the database. Since it does a full copy of the database, it also needs considerable time depending on the size of the database and the IO speed of the storage. If your system is scarce on disk space you can do an fast, in-place migration by running /usr/lib/susemanager/bin/pg-migrate-96-to-10.sh fast The fast migration usually only takes minutes and no additional disk space. However, in case of failure you need to restore the database from a backup. This wiki page contains additional information about the database migration. spacecmd: Support state channels spacecmd, the command line access to the Uyuni API, has been adapted to support state channels (aka Salt Minion config channels) with the following changes: * system_scheduleapplyconfigchannels + new call to schedule application of the assigned config channels to the system (minion only) * configchannel_updateinitsls + new call to update the init.sls file * configchannel_create + adapted call, now has a -t option to specify the channel type (normal or state) * configchannel_import + adapted call, honors channel type Please use the help functionality of spacecmd for detailed option descriptions for each mentioned call. New API calls Functions softwarechannel_mergepackages and softwarechannel_errata_merge to merge packages and errata through spacecmd were added. spacewalk-common-channels: Support for Uyuni, Fedora 29 and cleanup Added: * Uyuni Server, Uyuni Proxy, Uyuni Client Tools, both stable and development version. * Fedora 29 Removed: * Fedora 26 * Spacewalk 2.6 Server and Client Tools * Spacewalk 2.7 Server and Client Tools * Spacewalk 2.8 Server * Spacewalk nightly * OpenSUSE 13.2 and openSUSE 13.2 Client Tools Support for more Distributions as Clients openSUSE Leap 15.0, openSUSE Leap 42.3, SLE12, SLE15, CentOS6 and CentOS7 are now verified to bootstrap as both salt minions and traditional clients. New products added to SCC syncing * SUSE OpenStack Cloud 9 Known issues Single Sign On, API and CLI tools Single Sign On can be used to authenticate in the Web UI but not with the API or CLI tools. This will be fixed in a future release of Uyuni. EPEL and Salt packages Using the Extra Packages for Enterprise Linux directly on RHEL clients (or compatible: SLES ES, CentOS, Oracle Linux, etc) will install the Salt packages from EPEL, which miss some features in the Uyuni-provided Salt packages. This is an unsupported scenario. If you need to enable the EPEL repository, make sure you filter out the Salt packages from EPEL (for instance, by creating a new channel using Content Lifecycle Management). RHEL 6, CentOS 6 and Oracle Linux 6 minimal installations In the case of RHEL 6, CentOS 6 and Oracle Linux 6, the "Minimal" installation set is missing some packages required for the onboarding to work. It is recommented to install at least a "Basic Server". Alternatively, if using a minimal installation, you must install the perl and openssh-clients packages before onboarding. RHEL native clients When autogenerating bootstrap repositories for native RHEL clients, some errors may be logged from the moment the official Red Hat channels are added until the moment those channels are fully synchronized for the first time. This does not affect SLES Expanded Support, CentOS or Oracle Linux. Registering Spacewalk 2.x/Red Hat Satellite 5.x clients to Uyuni as Salt minions If a client machine is running the Red Hat Satellite 5.x agent, registering it to Uyuni as a Salt minion will fail due to package conflicts. Registering a RH Satellite 5.x client as a Uyuni traditional client works fine. Registering a Uyuni traditional client as a Uyuni Salt minion will also work. Works Fails RH Satellite 5.x ? Uyuni traditional RH Satellite 5.x ? Uyuni Salt minion Uyuni traditional ? Uyuni Salt minion In order to register Red Hat Satellite 5.x clients to Uyuni as Salt minions, you will need to modify the bootstrap script to remove the Satellite agent packages first. Spacewalk 2.x and Oracle Spacewalk 2.x clients will show the same behavior as Red Hat Satellite 5.x clients CentOS When mirroring CentOS AppStreams, only the most up-to-date packages can be synchronized. If a package was previously synchronized it will remain available but old versions cannot be synchronized if they never were earlier. This will be fixed in the next Uyuni release. Client Tools Notes URLs of the Client Tools are: * openSUSE Leap 15.* (x86_64): https://download.opensuse.org/repositories/ systemsmanagement:/Uyuni:/Stable:/openSUSE_Leap_15-Uyuni-Client-Tools/ openSUSE_Leap_15.0/ * SLE12 (x86_64, pcc64le. s390x, aarch64): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/SLE12-Uyuni-Client-Tools/ SLE_12/ * SLE15 (x86_64, pcc64le. s390x, aarch64): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/SLE15-Uyuni-Client-Tools/ SLE_15/ * CentOS6, Oracle Linux 6 (i686, x86_64): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/CentOS6-Uyuni-Client-Tools/ CentOS_6/ * CentOS7, Oracle Linux 7 (x86_64, ppc64le): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/CentOS7-Uyuni-Client-Tools/ CentOS_7/ * CentOS8, Oracle Linux 8 (x86_64, ppc64le): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/CentOS8-Uyuni-Client-Tools/ CentOS_8/ * Ubuntu 16.04 (x86_64): https://download.opensuse.org/repositories/ systemsmanagement:/Uyuni:/Stable:/Ubuntu1604-Uyuni-Client-Tools/ xUbuntu_16.04/ * Ubuntu 18.04 (x86_64): https://download.opensuse.org/repositories/ systemsmanagement:/Uyuni:/Stable:/Ubuntu1804-Uyuni-Client-Tools/ xUbuntu_18.04/ * Ubuntu 20.04 (x86_64): https://download.opensuse.org/repositories/ systemsmanagement:/Uyuni:/Stable:/Ubuntu2004-Uyuni-Client-Tools/ xUbuntu_20.04/ * Debian 9 (x86_64, aarch64, armv7l, i586): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/Debian9-Uyuni-Client-Tools/ Debian_9/ * Debian 10 (x86_64, aarch64, armv7l, i586): https://download.opensuse.org/ repositories/systemsmanagement:/Uyuni:/Stable:/Debian10-Uyuni-Client-Tools/ Debian_10/ Keep in mind you should manage the client tools using the command spacewalk-common-channels on the server, that will also allow you to add the required channels for all those operating systems that are freely available. Supported clients At the moment the status is the following: Distribution Salt bootstrap Salt SSH bootstrap Salt bootstrap Traditional from server from server from client openSUSE Leap 15 SLE12 SLE15 CentOS6 CentOS7 CentOS8 Oracle Linux 6 Oracle Linux 7 Oracle Linux 8 Ubuntu16.04 Ubuntu18.04 Ubuntu20.04 Debian9 Debian10 = Working, = Not working, = Untested With the exception of RHEL/CentOS and Oracle Linux, all maintained SPs and subversions are supported. Untested clients Distribution Salt bootstrap Salt SSH bootstrap Salt bootstrap Traditional from server from server from client RHEL6 RHEL7 RHEL8 RHEL6, RHEL7 and RHEL8 are expected to work in the same way CentOS6, CentOS7 and CentOS8 respectively. Client Tools repositories for a CentOS version should work at the respective RHEL version. CentOS8 (and therefore RHEL8) does not have support for the traditional client tools, only salt. Known limitations The GPG key for Uyuni Client Tools is not trusted by default by the respective package management tools for each OS. The systems will bootstrap without the GPG key being trusted, but will not be able to install new client tool packages or updated them. This can be fixed by adding the key uyuni-gpg-pubkey-0d20833e.key to all the bootscrap scripts at variable ORG_GPG_KEY=. If you already have other keys there, you can keep them. For systems bootstrapped from WebUI, a salt state should be created to trust the key, then the state can be assigned to the organization, and finally it can be used using an Activation Key and the Configuration Channels to deploy the change to the clients. Documentation It is usable but you can still find some issues, such references to SUSE Manager that are scheduled to be fixed on subsequent versions. Installation Requirements * OS: openSUSE Leap 15.2 x86_64, fully updated * Main memory: Minimum 16 GB for base installation * Disk space: Minimum 100 GB for root partition, Minimum 50 GB for /var/lib/ pgsql, Minimum 50 GB per SUSE product + 100 GB per RHEL product (/var/ spacewalk) See the Getting Started manual for more details on the system requirements. Installing the Server Add the Stable repository: zypper ar https://download.opensuse.org/repositories/systemsmanagement:/ Uyuni:/Stable/images/repo/Uyuni-Server-POOL-x86_64-Media1/ uyuni-server Install the pattern: zypper in patterns-uyuni_server Run Yast2 and go to Network Services > Uyuni Setup Follow the setup assistant. Update from previous versions of Uyuni Server WARNING: Make sure you check the documentation this time. Because of the change from openSUSE Leap 15.1 to openSUSE Leap 15.2, some special steps are required! You can update from previous Uyuni Server Stable versions. See the "Upgrade Guide" for detailed instructions on how to upgrade. You will need to follow the "Upgrade the Server" > "Major Upgrade" section. All connected clients will continue to run and are manageable unchanged. Update from previous versions of Uyuni Proxy When updating, always start with the server first and then continue with the proxies. See the release notes for the proxy and the "Upgrade Guide" for detailed upgrade instructions. Other information Red Hat Channels Managing RHEL clients requires availability of appropriate Red Hat packages. SUSE Channels Managing SUSE Linux clients requires availability of appropriate SUSE channels. Your licensed SUSE products can be used with Uyuni by following the setup Wizard. Check the manuals for more information. Providing feedback In case of encountering a bug please report it at https://github.com/ uyuni-project/uyuni/issues Legal Notices Copyright ? 2018 ? 2020 The Uyuni Project This work is licensed under the Creative Commons Attribution-ShareAlike 3.0 License. To view a copy of this license, visit http://creativecommons.org/ licenses/by-sa/3.0/es/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. For SUSE trademarks, see http://www.suse.com/company/legal/. All other third-party trademarks are the property of their respective owners. Trademark symbols (?, ? etc.) denote trademarks of SUSE and its affiliates. Asterisks (*) denote third-party trademarks. All information found in this document has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Last updated 2021-03-01 11:47:13 +0100