|
| 1 | +======== |
| 2 | +Autospec |
| 3 | +======== |
| 4 | + |
| 5 | +autospec is a tool to assist in the automated creation and maintainence |
| 6 | +of RPM packaging. It will continuously run updated builds based on new |
| 7 | +information discovered from build failures, until it has a complete and |
| 8 | +valid .spec file. The tool makes use of mock to achieve this. |
| 9 | + |
| 10 | +License |
| 11 | +======= |
| 12 | +autospec is available under the terms of the GPL, version 3.0 |
| 13 | + |
| 14 | +Copyright (C) 2015 Intel Corporation |
| 15 | + |
| 16 | + |
| 17 | +Configuration of autospec |
| 18 | +========================= |
| 19 | +autospec is configured by means of a simple INI-style configuration file. |
| 20 | +The default location of this file is assumed to be ``common/autospec.conf``, |
| 21 | +relative to the directory in which autospec is executed. |
| 22 | + |
| 23 | +Example ``autospec.conf`` file:: |
| 24 | + |
| 25 | + [autospec] |
| 26 | + |
| 27 | + license_fetch = http://yourhost/hash.php |
| 28 | + license_show = http://yourhost/showone.php?hash=%(HASH)s |
| 29 | + upstream = http://yourhost/tarballs/%(HASH)s/%(NAME)s |
| 30 | + |
| 31 | + |
| 32 | +**git** |
| 33 | + The upstream git repository URL base |
| 34 | + |
| 35 | +**license_fetch** |
| 36 | + Optional URL to use for scanning license files |
| 37 | + |
| 38 | +**license_show** |
| 39 | + Optional URL to interact with online license checker |
| 40 | + |
| 41 | +**upstream** |
| 42 | + Base URL for stored upstream tarballs |
| 43 | + |
| 44 | +Synopsis |
| 45 | +======== |
| 46 | + |
| 47 | +Usage: ``python3 autospec.py [options] URL`` |
| 48 | + |
| 49 | + |
| 50 | +-h, --help show help message and exit |
| 51 | +-g, --skip-git Don't commit result to git |
| 52 | +-n NAME, --name NAME Override the package name |
| 53 | +-a ARCHIVES, --archives ARCHIVES |
| 54 | + tarball URLs for additional source archives and a |
| 55 | + location for the sources to be extacted to (e.g. |
| 56 | + http://example.com/downloads/dependency.tar.gz |
| 57 | + /directory/relative/to/extract/root ) |
| 58 | +-l, --license-only Only scan for license files |
| 59 | +-b, --skip-bump Don't bump release number |
| 60 | +-c CONFIG, --config CONFIG Set configuration file to use |
| 61 | + |
| 62 | + |
| 63 | +Requirements |
| 64 | +============= |
| 65 | + |
| 66 | +In order to run correctly, ``autospec`` requires the following components: |
| 67 | + |
| 68 | + * python3 |
| 69 | + * correctly configured mock |
| 70 | + |
| 71 | +If ``autospec`` is not configured to use a license server, then you will |
| 72 | +need a ``common/licenses`` file - which should be an up to date list of |
| 73 | +licenses to facilitate automatic license detection during the scan of a |
| 74 | +tarball. For correctness, license names should be in the SPDX identifier |
| 75 | +format. Each line in the file constitutes a license definition, for example:: |
| 76 | + |
| 77 | + 750b9d9cc986bfc80b47c9672c48ca615cac0c87 | BSD-3-Clause |
| 78 | + 175e59be229a5bedc6be93e958a970385bb04a62 | Apache-2.0 |
| 79 | + 794a893e510ca5c15c9c97a609ce47b0df74fc1a | BSD-2-Clause |
| 80 | + |
| 81 | + |
| 82 | +Control files |
| 83 | +============== |
| 84 | + |
| 85 | +It is possible to influence precisely how autospec will behave in order to |
| 86 | +gain fine control over the build itself. These files may be used to alter |
| 87 | +the default behaviour of the configure routine, to blacklist build dependencies |
| 88 | +from being automatically added, and such. |
| 89 | + |
| 90 | +These files are expected to live in same directory that the resulting ``.spec`` |
| 91 | +will live. |
| 92 | + |
| 93 | +Common files |
| 94 | +------------ |
| 95 | + |
| 96 | +**release** |
| 97 | + |
| 98 | + This file contains the current release number that will be used in the |
| 99 | + ``.spec``. This is also bumped and generated on existing and new packages, |
| 100 | + respectively. This results in less manual work via automatic management. |
| 101 | + |
| 102 | +**$package.license** |
| 103 | + |
| 104 | + In certain cases, the package license may not be automatically discovered. |
| 105 | + In this instance, ``autospec`` will exit with an error. Update this file |
| 106 | + to contain the valid SPDX identifier for any license(s) for the package, |
| 107 | + replacing ``$package`` in the filename with the actual package name. |
| 108 | + |
| 109 | +Controlling dependencies |
| 110 | +------------------------- |
| 111 | + |
| 112 | +**buildreq_add** |
| 113 | + |
| 114 | + Each line in the file provides the name of a package to add |
| 115 | + as a build dependency to the ``.spec``. |
| 116 | + |
| 117 | +**pkgconfig_add** |
| 118 | + |
| 119 | + Each line in the file is assumed to be a pkgconfig() build dependency. |
| 120 | + Add the pkg-config names here, as ``autospec`` will automatically transform |
| 121 | + the names into their ``pkgconfig($name)`` style when generating the ``.spec``. |
| 122 | + |
| 123 | +**buildreq_ban** |
| 124 | + |
| 125 | + Each line in the file is a build dependency that under no circumstance |
| 126 | + should be automatically added to the build dependencies. This is useful |
| 127 | + to block automatic configuration routines adding undesired functionality, |
| 128 | + or to omit any automatically discovered dependencies during tarball scanning. |
| 129 | + |
| 130 | +**pkgconfig_ban** |
| 131 | + |
| 132 | + Each line in this file is a pkgconfig() build dependency that should not |
| 133 | + be added automatically to the build, much the same as ``buildreq_ban``. |
| 134 | + As with ``pkgconfig_add``, these names are automatically transformed by |
| 135 | + ``autospec`` into their correct ``pkgconfig($name))`` style. |
| 136 | + |
| 137 | + |
| 138 | +Controlling the build process |
| 139 | +------------------------------ |
| 140 | + |
| 141 | +**configure** |
| 142 | + |
| 143 | + This file contains configuration flags to pass to the ``%configure`` |
| 144 | + macro for autotools based tarballs. As an example, adding ``--disable-static`` |
| 145 | + to ``./configure`` for an autootools based tarball would result in |
| 146 | + ``%configure --disable-static`` being emitted in the ``.spec``. |
| 147 | + |
| 148 | +**cmake_args** |
| 149 | + |
| 150 | + This file contains arguments that should be passed to the ``%cmake`` |
| 151 | + macro for CMake based tarballs. As an example, adding ``-DUSE_LIB64=ON`` to |
| 152 | + ``./cmake_args`` would result in ``%cmake -DUSE_LIB64=ON`` being emitted |
| 153 | + in the ``.spec``. |
| 154 | + |
| 155 | +**broken_parallel_build** |
| 156 | + |
| 157 | + If this file exists, then parallelisation will be disabled in the build. |
| 158 | + This usually means that ``%{?_smp_mflags}`` will not be passed to ``make`` |
| 159 | + |
| 160 | +**make_args** |
| 161 | + |
| 162 | + The contents of this file are appended to the ``make`` invocation. This |
| 163 | + may be useful for passing arguments to ``make``, i.e. ``make TOOLDIR=/usr`` |
| 164 | + |
| 165 | +**make_install_args** |
| 166 | + |
| 167 | + Much like ``make_args``, this will pass arguments to the ``make install`` |
| 168 | + macro in the ``.spec`` |
| 169 | + |
| 170 | +**install_macro** |
| 171 | + |
| 172 | + The contents of this file be used instead of the automatically detected |
| 173 | + ``install`` routine, i.e. use this if ``%make_install`` is insufficient. |
| 174 | + |
| 175 | +**subdir** |
| 176 | + |
| 177 | + Not all packages have their ``Makefile``'s available in the root of the tarball. |
| 178 | + An example of this may be cross-platform projects that split Makefile's into |
| 179 | + the ``unix`` subdirectory. Set the name in this file and the ``.spec`` will |
| 180 | + emit the correct ``pushd`` and ``popd`` lines to utilise these directories |
| 181 | + for each step in the build. |
| 182 | + |
| 183 | +**build_pattern** |
| 184 | + |
| 185 | + In certain situations, the automatically detected build pattern may not |
| 186 | + work for the given package. This one line file allows you to override the |
| 187 | + build pattern that ``autospec`` will use. The supported build_pattern types are: |
| 188 | + |
| 189 | + - configure: Traditional ``%configure`` autotools route |
| 190 | + - configure_ac: Like ``configure, but performs ``%reconfigure`` to regenerate ``./configure`` |
| 191 | + - autogen: Similar to ``configure_ac`` but uses the existing ``./autogen.sh`` instead of ``%reconfigure`` |
| 192 | + - distutils: Only build the Pythonic package with Python 2 |
| 193 | + - distutils23: Build the Pythonic package using both Python 2 and Python 3 |
| 194 | + |
| 195 | +**series** |
| 196 | + |
| 197 | + This file contains a list of patches to apply during the build, using the ``%patch`` |
| 198 | + macro. As such it is affected by ``-p1`` style modifiers. |
| 199 | + |
| 200 | +Controlling files and subpackages |
| 201 | +--------------------------------- |
| 202 | + |
| 203 | +**excludes** |
| 204 | + |
| 205 | + This file is used to generate ``%exclude`` lines in the ``.spec``. This |
| 206 | + is useful for omitting files from being included in the resulting package. |
| 207 | + Each line in the file should be a full path name. |
| 208 | + |
| 209 | +**keepstatic** |
| 210 | + |
| 211 | + If this file exists, then ``%define keepstatic 1`` is emitted in the ``.spec``. |
| 212 | + As a result, any static archive (``.a``) files will not be removed by rpmbuild. |
| 213 | + |
| 214 | +**extras** |
| 215 | + |
| 216 | + Each line in the file should be a full path within the resulting package, |
| 217 | + that you wish to be placed into an automatic ``-extras`` subpackage. This |
| 218 | + allows one to keep the main package slim and split out optional functionality |
| 219 | + or files. |
| 220 | + |
| 221 | +**setuid** |
| 222 | + |
| 223 | + Each line in this file should contain the full path to a binary in the resulting |
| 224 | + build that should have the ``setuid`` attribute set with the ``%attr`` macro. |
| 225 | + |
| 226 | +**attrs** |
| 227 | + |
| 228 | + Each line in this file should be a full ``%attr`` macro line that will be included |
| 229 | + in the ``.spec`` to have fine-grained control over the permissions and ownership |
| 230 | + of files in the package. |
| 231 | + |
| 232 | + |
| 233 | +Controlling test suites |
| 234 | +----------------------- |
| 235 | + |
| 236 | +By default, ``autospec`` will attempt to detect potential test suites that |
| 237 | +can be run in the ``%check`` portion of the ``.spec``. |
| 238 | + |
| 239 | +**skip_test_suite** |
| 240 | + |
| 241 | + If this file exists, ``autospec`` will not emit any ``%check`` functionality. |
| 242 | + |
| 243 | +**unit_tests_must_pass** |
| 244 | + |
| 245 | + This file is automatically created upon successful completion of a package build. |
| 246 | + This allows one to identify regressions in test failures when updating or |
| 247 | + altering a package. |
| 248 | + ``autospec`` will fail a package that does not pass it's test suite if this file |
| 249 | + exists. |
| 250 | + |
| 251 | +**make_check_command** |
| 252 | + |
| 253 | + Override or set the command to use in the ``%check`` portion of the ``.spec``. |
| 254 | + This may be useful when a package uses a custom test suite, or requires |
| 255 | + additional work/parameters, to work correctly. |
| 256 | + |
| 257 | +**allow_test_failures** |
| 258 | + |
| 259 | + The existence of this file will allow test failures, and will still emit |
| 260 | + the ``%check`` code in a way that allows the build to continue. |
| 261 | + |
| 262 | + |
| 263 | +Controlling flags and optimisation |
| 264 | +----------------------------------- |
| 265 | + |
| 266 | +**asneeded** |
| 267 | + |
| 268 | + If this file exists, the ``.spec`` will disable the LD_AS_NEEDED variable. |
| 269 | + Supporting binutils (such as found in Clear Linux Project for Intel Architecture) |
| 270 | + will then revert to their normal behaviour, instead of enforcing ``-Wl,-as-needed`` |
| 271 | + in the most correct sense. |
| 272 | + |
| 273 | +**optimize_size** |
| 274 | + |
| 275 | + If this file exists, the ``CFLAGS/LDFLAGS`` will be extended to build |
| 276 | + the package optimised for *size*, and not for *speed*. Use this when |
| 277 | + size is more critical than performance. |
| 278 | + |
| 279 | +**funroll-loops** |
| 280 | + |
| 281 | + If this file exists, the ``CFLAGS/LDFLAGS`` will be extended to build |
| 282 | + the package optimised for *speed*. In short this where speed is of |
| 283 | + paramount importance, and will use ``-03`` by default. |
| 284 | + |
| 285 | +**insecure_build** |
| 286 | + |
| 287 | + If this file exists, the ``CFLAGS/LDFLAGS`` will be **replaced**, using |
| 288 | + the smallest ``-02`` based generic flags possible. This is useful for |
| 289 | + operating systems employing heavy optimisations or full RELRO by default. |
| 290 | + |
| 291 | + |
| 292 | +Name and version resolution |
| 293 | +=========================== |
| 294 | + |
| 295 | +``autospec`` will attempt to use a number of patterns to determine the name |
| 296 | +and version of the package by examining the URL. For most tarballs this is |
| 297 | +simple, if they are of the format ``$name-$version.tar.$compression``. |
| 298 | + |
| 299 | +For websites such as ``bitbucket`` or ``GitHub``, using ``get$`` and ``v$.tar.*`` |
| 300 | +style links, the project name itself is used from the URL and the version is |
| 301 | +determined by stripping down the tag. |
| 302 | + |
| 303 | +CPAN Perl packages, R packages, and rubygems.org rubygems are automatically |
| 304 | +prefixed with their language name: ``perl-``, ``R-`` and ``rubygem-`` respectively. |
| 305 | + |
| 306 | +When these automated detections are not desirable, it is possible to override |
| 307 | +these with the ``--name`` flag when invoking ``autospec`` |
| 308 | + |
| 309 | + |
| 310 | +Automatic license server support |
| 311 | +================================ |
| 312 | +``autospec`` can optionally talk to a license server instead of checking |
| 313 | +local hashsum files, which enables greater coverage for license detection. |
| 314 | +The URL set in ``license_fetch`` is expected to be a simple script that |
| 315 | +talks HTTP. |
| 316 | + |
| 317 | +This URL should accept ``POST`` requests with the following keys: |
| 318 | + |
| 319 | +**hash** |
| 320 | + Contains the SHA-1 hash of the potential license file being checked. |
| 321 | + |
| 322 | +**package** |
| 323 | + The name of the package being examined |
| 324 | + |
| 325 | +**text** |
| 326 | + The contents of the potential license file |
| 327 | + |
| 328 | +Implementations return a *plain text* response with the SPDX identifier |
| 329 | +of the license, if known. An empty response is assumed to mean that this |
| 330 | +license is unknown, in which case ``autospec`` will emit the ``license_show`` |
| 331 | +URL. The implementation should show the now-stored license file via a |
| 332 | +web page, and enable a human to make a decision on the license. This is |
| 333 | +then stored internally, allowing future requests to automatically know |
| 334 | +the license type when this hash is encounted again. |
0 commit comments