sos-4.0

The sos team is pleased to announce the relase of sos-4.0. This is a major version release that represents a significant change to the sos project, new features, and bug fixes.

Major changes

Sos has been redesigned to provide functionality beyond the well known report data collection usage. It has been updated to provide
more functionality via sub-commands.

• A new sos binary has replaced the former sosreport binary as the main entry point for the utility.

  • sos report is now used to generate sosreport tarballs. A sosreport binary is maintained as a redirection point and will now invoke sos report.
  • sos collect formally brings sos-collector into the main sos project, and is used to collect sosreports from multiple nodes simultaneously. A sos-collector binary is maintained as a redirection point and will invoke sos collect.
    • This means the standalone sos-collector utility will no longer be independently developed.
  • sos clean formally brings soscleaner-like functionality into the main sos project. This sub command will perform further data obfuscation on reports, such as scrubbing IP addresses, domain names, and user-provided keywords. See below for more information.

• /etc/sos.conf has been moved to /etc/sos/sos.conf, and the layout of the config file has changed:

  • The general section has been renamed to global, and may be used to specify options that are available to all sos commands and sub-commands.
  • Each sub-command will have its own section, e.g. sos report will load options from global and from report.

• Sos is now a Python3-only utility. Python2 is no longer supported in any capacity.

• Dropped use of make, building/installing sos from source should now exclusively be done via setuptools

Report

• sos will now generate metadata and save it in sos_reports/manifest.json.

• 7 new plugins: nvmetcli, drbd, openstack_designate, pmem, containers_common, hyperv, freeipmi

• The nfsserver plugin has been merged into the nfs plugin

• sos may now be used to collect data from within a container, rather than aborting if that container was not configured to allow sos to collect information from the host

• Added support for Container-Optimized OS (COS)

• Dropped Mac OSX support

• Dropped bzip2 compression support

• Users may now use the --clean or --mask option to process a report-being-generated through sos clean at runtime.

• Size limits will now apply to add_copy_spec() calls that target a directory

• The openshift plugin has been re-written to be used for Openshift Container Platform 4

• Significantly expanded the amount of API resources the openstack_octavia plugin will collect

• The networking plugin will no longer execute ethtool -e against NICs using the bnx2x driver

• The logs plugin will now capture journal information correctly when logs are stored in-memory only

• The systemd plugin will no longer collect systemd-resolve if the service is not running

• The openvswitch plugin has been significantly updated to pull more meaningful data, and now supports OpenFlow 1.4 and 1.5

Plugin API changes

• The command execution/collection methods have been overhauled:

  • add_cmd_output() should continue to be used to specify commands that should be executed during the collection phase
  • exec_cmd() should now be used to execute commands and retrieve output during setup(), but will not save that output to the archive
  • collect_cmd_output() should be used to execute commands, retrieve output during setup() and will save that output to the archive
  • get_command_output() has been removed

• A new add_device_cmd() method is available to facilitate easier iteration of commands over a set of devices

  • add_blockdev_cmd() has been added to facilitate iteration of commands of storage devices

• A container runtime abstraction has been added that aims to standardize the discovery of a container runtime in use (e.g. docker, podman) and the retrieval of data from the runtime across plugins

Collect

sos collect is a new sub command in this release, and is an integration of the standalone sos-collector project, with the aim being to collect sosreports from multiple systems simultaneously. Note that this sub-command requires python3-pexpect to be available. If the module is not available, sos collect will abort with an appropriate error message

Compared to the standalone project, enhancements include:

• collect is now supported on all distributions that sos report supports (i.e. any distribution with a Policy defined)
• The --insecure-sudo option has been renamed to --nopasswd-sudo
• --threads in the context of the number of nodes to simultaneously connect to has been renamed to jobs
• Fixed a bug where a local node would be displayed for collection even when --no-local was used

Cleaner

sos clean, also available as sos mask, is a newly added sub-command in this release and is an implementation of the standalone soscleaner project. Its aim is to scrub potentially sensitive information from sosreports in a consistent manner, beyond the obfuscation done by plugins already.

• Support for ipv4 address/network obfuscation. Note that this will attempt to preserve topological relationships between discovered addresses
• Support for hostname, and domain name obfuscation.
• Support for user-provided keyword obfuscations
• Users may either use the --clean or --mask flag to sos report to obfuscate a report being generated, or may use sos (clean|mask) $archive to obfuscate an already existing report.
  • Using the former will result in a single obfuscated report archive, while the latter approach will result in two; an obfuscated archive and the un-obfuscated original.

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.0

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-3.9.1

The sos team is pleased to announce the release of 3.9.1. This is a maintenance release tag designed to aide downstreams with 3.9 maintenance by signalling the end of 3.x development, while including all merged changes following the 3.9 release to date.

This maintenance release includes:

• New plugins: sos_extras, ovirt_engine_backup, console, validation_framework

• lxd plugin collections have been overhauled

• Fixed handling of the namespace pattern for the networking plugin

• A basic path is now defined in Policy for all subclasses

• Plugin API Enhancements:

  • Enablement checks have been extended to include architecture constraints
  • SoSPredicate has been extended to include architecture constraints, as well as negative constraints for all elements
  • Plugins will now capture service status information for all services defined in the services class attr

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/3.9.1

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-3.9

The sos team is pleased to announce the release of sos-3.9. This is a significant release containing a large number of enhancements, new features, and bug fixes, including:

• Improved human-readable archive naming and support for archive labels

• Improved reporting of archive output and properties

• Support for automatic uploading of report archives via FTP and HTTPS

  • Policy or command line specified URL and authentication
    • Policy specified URLs for Red Hat and Canonical uploads

• Automatic PATH support on Ubuntu distributions

• Improved policy performance

• Improved service status collection API

• 9 new plugins:
  cloud_init, convert2rhel, ebpf, fwupd, login, nginx, nvidia, openstack_tripleo

• 6 obsolete plugins removed or merged into other plugins:
  katello, last, mrggrid, mrgmessg, satellite

• Significant updates to 14 plugins:
  dlm, dnf, ceph, foreman, gluster, gnocchi, juju, kubernetes, logs, maas, networking,
  openvswitch, python, plugins

• The openswan plugin was renamed to libreswan to reflect the active upstream project name

• Updated Red Hat presets and new Cloud Forms preset

• Updates to networking plugin namespace handling

• Updates to the OVN plugins (ovn_central, ovn_host)

• Kernel eBPF data consolidated in a single plugin

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/3.9

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-3.8

The sos team is pleased to announce the release of sos-3.8. This release contains a number of enhancements, new features, and bug fixes including:

• 6 new plugins: container_log, frr, leapp, openstack_placement, qt, and vdsm (both backported from downstream)

• The kubernetes plugin can now optionally grab logs only for certain pods

• The kdump plugin will now collect initramfs content

• The pulp and foreman plugins now support collecting from an external database

• The sar plugin will now collect the full sar log dir

• The vdsm and ovirt plugins will now collect host certificates

• The openvswitch plugin will now enable on openvswitch2.* packages

• Added support for only capturing logs after a specific date, see the new --since option

• Fixed an issue causing high CPU utilization which slowed journal collection

• Fixed an issue where plugins could continue executing commands after their timeout was hit

• sosreport will no longer abort execution on Red Hat family systems when the package manager fails to query a file list

• Plugin API enhancements

  • Plugins may now capture environment variables, which will be written to /environment in the sos archive root

  • Plugins may now write command output to subdirs within their own sos_commands/plugin directory

    • The container plugins have been updated to make use of this functionality

  • SoSPredicate usage may now be match either any or all of the provided elements, and may mix requirements of kmods and services to determine if a command should be collected

• Significant changes to the reporting system of sos

  • HTML reports replaced by a Report subclass
  • HTML report creation time is significantly improved
  • Added a JSON formatted report option

• Allow system changes option

  • A new --allow-system-changes option has been added that will allow users to collect certain data, even if it means the host system would be changed. For example, the networking plugin will not run certain commands if specific kernel modules are not loaded, unless --allow-system-changes is specified, in which case the missing kernel modules will be loaded when the command(s) run.

For full information on the changes contained in this release, please refer to the Git commit logs.

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-3.7

The sos team is pleased to announce the release of sos-3.7. This is a significant release containing a large number of enhancements, new features, and bug fixes, including:

• New distribution policies for CentOS and Amazon Linux

• 19 new plugins:
  candlepin, cifs, cockpit, composer, crio, gssproxy, katello, openstack_novajoin, ovirt_node, peripety, podman, pulp, rasdaemon, rhcos, rhv_analyzer, rpmostree, ruby, stratis, sudo

• Obsolete IPSec plugin removed (in favour of OpenSwan)

• Support for passphrase and key based encryption of the report archive

  • Ability to encrypt the archive using GPG, with either a key or passphrase
  • New --encrypt-key and --encrypt-pass arguments to sosreport

• Improved handling of paths containing directory symbolic links (for e.g. /sys)

  • Previous versions of sos would replace intermediate path components that contain a symbolic link to a directory in the host file system with an actual directory in the report archive. The host file system structure is now reflected properly in the report directory structure.

• New InitSystem abstraction

  • Allows plugin and collection enablement based on the presence of a service, and methods to test whether a given service is currently running

• LVM2 plugin enhancements

  • Locking fixes for LVM2 metadata and reporting output capture
  • Additional LVM2 logical volume manager report data

• Append plugin exceptions to sos_logs/*-plugin-exception.txt

  • Previous versions of sos would overwrite earlier exceptions if more than one exception occurred while running a plugin (for example, when an exception occurs in both setup() and postproc() phases).

• Dry run mode (--dry-run)

  • Allows sos to run without collecting data, or executing commands, and proving a log of actions that would have been taken by a normal run on the current system configuration.

• Plugin API enhancements

  • SoSPredicates for gating collection on service and kernel module presence, and during dry-run mode

• Significant enhancements to core features and existing plugins

  • Fixes to threaded exception handling, and interactive debugging with --debug

  • Support for OpenShift 3.10 deployments

  • Improved multipath data collection

  • Fixed RHEL Atomic default command line preset

  • Support for PowerPC DLPAR and LPM logs

  • Additional FIPS and crypto-policies data collection

  • Test suite and CI support for Python-3.7 final

  • Additional systemd listings and statuses

  • Support for user-controlled per-plugin timeouts

  • Do not leave report artefacts in TMP when executing list commands

  • Improvements to command termination in the event of plugin timeouts

  • Policy support for Red Hat Enterprise Linux 8.0

  • Improved STONITH and watchdog data collection for Pacemaker clusters

  • Support for Debian journald logging in the logs plugin

  • New built-in 'cantboot' preset for collecting information relevant to failed boots

  • Ability to disable default presets on the command line (--preset=none)

  • Support for setting all sosreport command line options (including global and plugin options) in the sos.conf configuration file

  • The deprecated XML reporting module has been removed

  • Continuous integration with the LGTM static analyser (rated 'A')

  • Apache plugin fixed to support --log-size global option

  • Native support for collecting foreman-debug equivalent data in sos

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/3.7

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-3.6

The sos team is pleased to announce the release of sos-3.6. This is a
significant release containing a number of enhancements, new features,
and bug fixes, including:

• 29 new plugins:
  alternatives, ansible, btrfs, buildah, clear_containers, date,
  elastic, fibrechannel, host, kata_containers, lustre, memcached,
  mssql, networkmanager, nvme, omnipath_client, omnipath_manager,
  opendaylight, openstack_octavia, ovirt_provider_ovn, ovn_central,
  ovn_host, rear, release, runc, skydive, unpackaged, watchdog, wireless

• User and policy defined command line presets

  • The ability to save and recall specific combinations of command
    line parameters
  • Policy authors may define presets for specific situations, products
    or other uses (e.g. "cantboot", "rhel", "openshift" etc.).

• Size limits for external commands

  • Certain commands produce large volumes of data, inflating report
    size (e.g. journalctl): the command collection interface now allows
    an arbitrary size limit to be applied, which includes memory used
    during the run (reducing sosreport's peak memory usage).

• Automatic file and command size limits

  • Plugins that do not specify an explicit size limit for files or
    commands are now subject to the default value (specified with the
    --log-size command line option).
  • Plugin authors may override this behaviour if needed

• Concurrent plugin execution

  • Plugins are now run in parallel using a thread pool
  • Reduces runtime by up to 50% (workload dependent)
  • Command line --threads option to set the number of threads to
    use, or to disable parallel execution

• New profiles (including containers and the Apache webserver)

• major enhancements to core features and existing plugins:

  • better package manager version information
  • policy support for detecting package managed files
  • fixed exit status propagation
  • deprecated optparse replaced with argparse
  • simplified and improved SoSOptions interface
  • better error handling during interactive prompting
  • allow journal collection by identifier
  • allow collection of journal message catalogs
  • support for collecting binary file data
  • more fine-grained system plugins (date etc.)
  • policy defined report file name patterns
    • more human-readable report file names by default
  • increased default log size (25MiB vs. 10MiB)
  • support for forbidden path lists and forbid logging
  • support for enabling plugins by kernel module name
  • support for enabling plugins by executable name
  • support for collecting eBPF (bpftool) data
  • support for device information via add_udev_info()
  • support for detecting and reporting unpackaged binaries
  • optional collection of the RPMDB
  • improved archive compression level and multithreading
  • default log size increased from 10MiB to 25MiB
  • improved debug logging and ENOSPC handling
  • major updates to the IPA plugin
  • major updates to the Docker plugin
  • string decoding fixes
  • DNF and Yum module support
  • OpenShift 3.10 support
  • Python3 fixes

Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/3.6

Please report any problems to the sos-devel mailing list, or the
GitHub issue tracker:

https://github.com/sosreport/sos/issues/

I'd like to thank everyone who contributed fixes, new features,
testing, and feedback for this release.

sos-3.5.1

The sos team is pleased to announce the release of sos-3.5.1. This is a maintenance release containing a number of enhancements, new features, and bug fixes, including:

• 22 new plugins:

  • alternatives, ansible, btrfs, buildah, clear_containers, date, fibrechannel, host, kata_containers, lustre, memcached, networkmanager, nvme, opendaylight, openstack_octavia, ovirt_provider_ovn, ovn_central, ovn_host, rear, release
    runc, wireless

• New profiles (including containers and the Apache webserver)

• major enhancements to core features and existing plugins:

  • better package manager version information
  • fixed exit status propagation
  • deprecated optparse replaced with argparse
  • better error handling during interactive prompting
  • allow journal collection by identifier
  • allow collection of journal message catalogs
  • support for collecting binary file data
  • more fine-grained system plugins (date etc.)
  • policy defined report file name patterns
    • more human-readable report file names by default
  • support for forbidden path lists and forbid logging
  • support for enabling plugins by kernel module name
  • support for enabling plugins by executable name
  • support for collecting eBPF (bpftool) data
  • support for device information via add_udev_info()
  • optional collection of the RPMDB
  • default log size increased from 10MiB to 25MiB
  • string decoding fixes
  • improved debug logging and ENOSPC handling
  • OpenShift 3.10 support
  • Python3 fixes

This release allows distribution packagers to update to a new upstream release before the final release of 3.6. The 3.6 release will include further enhancements in core sosreport functionality and is planned for late June 2018.

sos-3.5

The sos team is pleased to announce the release of sos-3.5. This release includes a number of enhancements, new features, and bug fixes, including:

• New plugins for perl, boom, vdo, os_net_config, conntrackd, ovirt_imageio, nss, sas3ircu, openstack_aodh, docker_distribution, gluster_block, snappy
• Plugin API enhancements
  • Plugin triggers by executable name
  • Improved log size limit handling
  • Better handling of compressed log files
  • Per-plugin package verification lists
• Updates to 74 plugins

Thanks to everyone who submitted patches, testing feedback, and filed issues that helped to make this a quality release with many new features and improvements.

sos-3.4

The sos team is pleased to announce the release of sos-3.4. This release includes a number of enhancements, new features, and bug fixes, including:

• New plugins for virt-who, nodejs, npm, dracut, juju 2.0, grafana, nfsganesha, collectd, canonical_livepatch, jars, salt, saltmaster, storageconsole, gnocchi, crypto, azure/Red Hat, zfs
• Plugin API enhancements
• Internationalisation updates

Thanks to everyone who submitted patches, testing feedback, and filed issues that helped to make this a quality release with many new features and improvements.

3.3

The sos team is pleased to announce the release of sos-3.3. This release includes a number of enhancements, new features, and bug fixes, including:

• Support for OpenShift Enterprise 3.x
• Improved and expanded OpenStack plugins
• Enhanced support for Open vSwitch
• Enhanced Kubernetes data collection
• Improved support for systemd journal collection
• Policy support for the SuSE family of Linux distributions
• Policy support for the IBMKvm hypervisor distribution
• Enhanced display manager and 3D acceleration data capture
• Improved support for Linux clusters, including Pacemaker
• Expanded CPU and NUMA topology collection
• Expanded mainframe (s390x) coverage
• Collection of multipath topology

Thanks to everyone who submitted patches, testing feedback, and filed issues that helped to make this a quality release with many new features and improvements.