summaryrefslogtreecommitdiff
path: root/meta/classes
AgeCommit message (Collapse)AuthorFiles
2017-02-05kern-tools: re-enable scc merge commandBruce Ashfield1
The ability to merge two branches directly from a .scc file was dropped during the streamlining of the tools. As was pointed out by David Vincent <freesilicon@gmail.com>, there is once again a valid use case for this functionality, so we restore the capability. Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-02-02uninative: Make patchelf modified files sparseRichard Purdie1
When we switched to recipe specific sysroots (rss), performance took a nose dive. Its easy to blame rss but it turns out not to be entirely at fault. Three configurations are compared here: a) Pre-RSS (revision 45df694a9f472ac2f684aadac4d864c3dfdc48a7) b) Post-RSS (revision 226a508da955439b881b2f0a544a3aee76e59919) c) as b) with this change Overall build times: a) 22794.25user 2687.88system 30:32.84elapsed 1390%CPU (0avgtext+0avgdata 919056maxresident)k b) 22677.25user 3238.79system 36:16.68elapsed 1190%CPU (0avgtext+0avgdata 918896maxresident)k c) 23571.84user 3383.65system 31:36.83elapsed 1421%CPU (0avgtext+0avgdata 919068maxresident)k For the overall build and sstate directories, du -s shows: a) 3992588 build-pre-rss/sstate-cache 30804484 build-pre-rss/tmp b) 4013272 build-with-rss/sstate-cache 36519084 build-with-rss/tmp c) 4014744 build-with-rss2/sstate-cache 35336960 build-with-rss2/tmp However more worryingly: $ du -s build-pre-rss/tmp/sysroots/ 2506092 build-pre-rss/tmp/sysroots/ $ du -s build-with-rss/tmp/sysroots-components/ 3790712 build-with-rss/tmp/sysroots-components/ $ du -s build-with-rss2/tmp/sysroots-components/ 2467544 build-with-rss2/tmp/sysroots-components/ These numbers *should* be equivalent but as you can see, b) is ~1.2GB larger. The reason turned out to be patchelf. Taking a specific binary from a specific recipe, bc from bc-native, in a) its 82kb (stripped) yet in b) its 2.17MB. $ ./patchelf --set-interpreter /bin/rp bc warning: working around a Linux kernel bug by creating a hole of 2084864 bytes in ‘bc’ https://github.com/NixOS/patchelf/blob/master/src/patchelf.cc#L710 shows that this "hole" is just padded zeros using memset, its not a proper sparse hole. This patch copies files with cp --sparse=always after modifying them with patchelf, then replacing the original file. The better fix will be to fix this in patchself itself and seek() there when writing the new file but that means new uninative tarballs and will take a bit of work so I'm proposing this workaround in the meantime. Also, this patch drops error handling since subprocess check_output() tracebacks will print this information if the command fails so we can simplify the code.
2017-02-01staging: Reduce the number of mkdirs callsRichard Purdie1
The number of mkdir calls was showing up high on the profile charts since it was getting called once per file which is excessive. Each call results in one or more syscalls which is bad for performance. Cache which directories we've seen to reduce the calls to a more reasonable number and speed up recipe specific sysroots. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-01sstate/staging: Batch log messages for performanceRichard Purdie2
According to profile data, repeated calls to bb.debug and bb.note in the extend_recipe_sysroot() codepath were accounting for 75% of the time (1.5s) in calls from tasks like do_image_complete. This batches up the log messages into one call into the logging system which gives similar behaviour to disabling the logging but retains the debug information. Since setscene_depvalid is also called from bitbake's setscene code, we have to be a little creative with the function parameters and leave the other debug output mechanism in place. This should hopefully speed up recipe specific sysroots. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31relocatable: Make native .pc files relocatableRichard Purdie1
The native .pc files currently have hardcoded paths in them meaning each has to be relocated at final install time. pkg-config has built in functionality to avoid this, namely the pcfiledir variable. This function translates .pc files to use the variable meaning further relocation later is unnecessary. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31utils: Avoid need to relocate wrapper scriptsRichard Purdie1
Whilst the path to the executable is dynamically determined, the passed in environment variables or parameters are not relocatable and rely on the sstate remapping code. In the recipe specific sysroot case this has become more costly. This patch translates such paths into relocatable entries which means that a sed replacement at final installation isn't needed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31oeqa/runtime/cases: Rename syslog module to oe_syslogAníbal Limón1
Debian based distros has a builtin syslog module so when try to load tests using unittest it references the builtin module instead of runtime/cases. [YOCTO #10964] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31testimage.bbclass: Fixes about suites and dependenciesMariano Lopez1
There are some missing suites on sato-sdk and lsb-sdk images so add it. The createrepo-native needs to be built before run testimage in order to create a repository from install packages. The DL_DIR data also needs to be updated from the current bitbake environment instead of use the value from testdata file. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31populate_sdk_ext: fix == bashismPatrick Ohly1
Found via verify-bashisms. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31rootfs-postcommands.bbclass: drop insert_feed_uris functionMing Liu1
The source makes no reference to insert_feed_uris any more, simply drop the unused reference, to avoid confusing the developers. Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31meta: remove remaining True option to getVar callsMing Liu5
This is a complementary fix to commit 7c552996: [ meta: remove True option to getVar calls ] it intended to remove all True option to getVar calls, but there are still some remaining. Search made with the following regex: getVar ?\((.*), True\) Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31testimage.bbclass: Allow to run tests on autobuilder's imagesMariano Lopez1
With the change to the new framework data store dependecy was removed, instead a new file is generated and used in testimage. When testing builds from the autobuilders the test data values are from the autobuilder, including the paths. Some tests require paths to current environment in order to run, this commit will update such paths and fix the error of running images donwloaded from autobuilders. [YOCTO #10964] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31externalsrc: Hide created symlinks with .git/info/excludeOla x Nilsson1
Add created symlinks to the exclude file. This will both make them less distracting and hide them from the srctree_hash_files function. Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31sstate: Make SSTATE_SCAN_CMD vars configurable using weak defaults.Kristian Amlie4
For this we move them out of the python section so they can be overridden on a per-recipe basis. The motivation for this change is that not all tool chains need the path modifications provided by the command, and these will provide alternative or empty commands. The Go compiler is such an example. Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31toolchain-shar-extract: compare SDK and host gcc versionsEd Bartosh1
If ext sdk is built by gcc version higher than host gcc version and host gcc version is 4.8 or 4.9 the installation is known to fail due to the way uninative sstate package is built. It's a known issue and we don't have a way to fix it for above mentioned combinations of build and host gcc versions. Detected non-installable combinations of gcc versions and print an installation error. [YOCTO #10881] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31package_deb: handle allarch when mapping tune to dpkg architecturesRoss Burton1
It was discovered that buildtools-tarball can't be built with Debian packaging: E: Unable to locate package nativesdk-buildtools-perl-dummy The package exists but dpkg doesn't see it because the Architecture field isn't valid: for this package the Architecture field is 'allarch' which isn't in the list of valid architectures we define. package_deb already has a mapping of OE architectures (PACKAGE_ARCH) to dpkg architectures (DPKG_ARCH), for example our x86_64 is amd64 in dpkg. However allarch.bbclass sets an invalid TARGET_ARCH of 'allarch' (to break anything which attempts to use the target architecture) and package_deb relied on the PACKAGE_ARCH check later turning the Architecture field into 'all'. This usually works, but nativesdk-buildtools-perl-dummy inherits allarch but then changes PACKAGE_ARCH to buildtools-dummy-nativesdk to isolate the packages in a separate feed. We can handle this by explictly checking for allarch in the TARGET_ARCH check, as that everything that inherits allarch will want to have Architecture: all in their packages. [ YOCTO #10700 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31distrodata: add DATETIME to vardepsexclude for do_distro_check.Ross Burton1
This task uses DATETIME so add it to vardepsexclude so bitbake doesn't refuse to execute the task when the time changes between the controller and the worker calculating the hash. [ YOCTO #10960 ] Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31grub-efi.bbclass: use 'grub-efi-' prefixEd Bartosh1
grub-efi recipe added 'grub-efi-' prefix to the file name of efi binary. Changed grub-efi.bbclass accordingly. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCHEd Bartosh1
isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image. It doesn't work for all machines, for example for quemux86-64 machine MACHINE_ARCH is quemux86_64 and initrd name is core-image-minimal-initramfs-qemux86-64.cpio.gz Used TRANSLATED_TARGET_ARCH variable to get the initrd image name. Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to make it available from <image>.env file. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31image-wic: remove HDDDIR from WICVARSEd Bartosh1
Removed HDDDIR as it's not used by wic anymore. Stopped usage of HDDDIR in wic test suite. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: use INITRD_LIVE in isoimage-isohybridEd Bartosh1
INITRD variable is not set if hddimg is disabled. isoimage-isohybrid can't get correct name for initrd if INITRD variable is not set. Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31image-wic: move wic code to image-wic.bbclassEd Bartosh3
There is a lot of wic code in image.bbclass and image_types.bbclass Having all code separated in one file should make it more readable and easier to maintain. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
2017-01-31wic: change location of .env filesEd Bartosh2
Current location of .env files $STAGING_DIR/imagedata. It doesn't depend on machine and be rewritten by the builds for different machines. Changed location to $STAGING_DIR/$MACHINE/imagedata to avoid .env files to be rewritten. Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-31image_types: use correct output directoryEd Bartosh1
Wic put result images into <output dir>/build, which was confusing. Now it's fixed in wic code and images are put into output directory. Changed code in image_types to reflect this. [YOCTO #10783] Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28staging: Handle stale sysroot rebuild issueRichard Purdie1
If a stale sysroot object was found the complete stamp wasn't removed, meaning later code could fail when the new link creation was attempted. Ensure the stale complete stamp is also removed if present. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28staging: Fix missing nativesdk do_populate_sysroot depends on binutilsRichard Purdie1
Recent changes exposed the fact this dependency was missing in nativesdk cases, fix this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28staging: Allow removal of stale sysroot objectsRichard Purdie1
The main sysroot components of unreachable build targets will be removed by the core code. This currently doesn't trigger a removal in the individual workdirs. This adds in symlinking between the complete stamps and the component sysroot meaning we can detect when someting was removed and hence remove it from the sysroot. This fixes cases where DISTRO_FEATURES like systemd are changed amongst other things and makes builds more robust against configuration changes. If a dependency is rebuild, that is caught by checksum comparision code elsewhere in this function as before. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28staging: Drop common prefix WORKDIR from manifest filesRichard Purdie1
Manifest files containing the same duplicated prefix are wasteful on space and ultimately this costs build time. Drop the WORKDIR prefix from the manifest files since this small change mounts up a lot. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28sstate: Allow sstate_clean_manifest to take a prefixRichard Purdie1
Manifest files containing the same duplicated prefix are wasteful on space and ultimately this costs build time. Add support for manifest files with common prefixes removed and use the prefix if the path isn't absolute. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28gzip/pigz/expat/image: Simplify gzip-native/pigz-nativeRichard Purdie3
With recipe specific sysroots, the gzip-replacement-native dance/class is obsolete, simplify the code accordingly. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28allarch: Drop STAGING_DIR_HOST expansionRichard Purdie1
Now that STAGINGDIR_HOST doesn't contain MACHINE, we no longer need to expand the value. Pre-expansion can mean components like PV can be expanded too early and cause problems for certain use cases. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28image/kernelsrc/packagegroups/recipes: Remove uneeded noexec tasksRichard Purdie3
We used to have issues removing tasks like do_fetch due to implications for targets like world and universe. These have now been resolved. Removing uneeded tasks has advantages compared to noexec since it means that accidentally left in dependencies are no longer needed/processed (e.g. do_patch depends on quilt-native). This cleans up a number of cases which local analysis highlighted as being unneeded leading to slightly cleaner task graphs. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-28image_types.bbclass: rebuild .wks file when .wks.in changesPatrick Ohly1
WKS_FILE(S) can refer to .wks.in files which get expanded during the build by do_write_wks_template. The actual content of the .wks.in file gets added to the recipe meta data during parsing, and thus we need to ensure that the recipe gets re-parsed when the file changes. This fixes two related problems: - editing the .wks.in file and rebuilding an image did not recreate the image unless something else changed or "bitbake -c clean" was used explicitly - when forcing a rebuild, the cached meta data and the actual one do not match, leading to "ERROR: Taskhash mismatch ... for ....bb.do_write_wks_template" Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26testimage.bbclass: Add systemd test automaticallyMariano Lopez1
This adds systemd test automatically when an image is build with systemd DISTRO_FEATURE [YOCTO #10935] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26useradd_base: keep group if it still has usersIoan-Adrian Ratiu1
perform_groupdel() tries to delete a group irrespective if other users have it as their primary group, thus the call to groupdel will fail with the following error error: groupdel: cannot remove the primary group of user '<user>' Add a check to perform_groupdel() to determine if there are other users and keep the group, printing a warning. This is called right after a user is deleted to delete it's group. If the last user is deleted, only then the group is also deleted. Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26useradd-staticids: Fix groupaddDavid Vincent1
When a group name is duplicated between USERADD_PARAM and GROUPADD_PARAM, this class tries to add it twice which results in failure of the groupadd command. Signed-off-by: David Vincent <freesilicon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26toolchain-scripts: remove CCACHE_PATH from environment scriptJoshua Lock1
CCACHE_PATH was added to the toolchain-scripts in commit 1d31ddb856a80ba9da1a64ed5d701dc0f7351ef7 without a detailed explanation as to why. This commit removes that environment variable as it's causing failures since the introduction of host_gcc_version() to oe.utils in d36f41e5658bbbb6080ee833027879c119edf3e0, as used by uninative to determine NATIVELSBSTRING. This causes errors when the host has ccache available in in PATH - host_gcc_version() fails because ccache has been told to only look for the compiler in CCACHE_PATH and can't find gcc in order to check its version. Toolchain compilers aren't detected due to the binaries being prefixed with the triplet of the target. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26autotools.bbclass: Change acpaths defaults to be a weak assignmentKhem Raj1
We currently have to override the default setting using a particular assignment order in the recipe, setting it weakly helps to get rid of this problem. Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26staging: Handle overlapping files for build-sysrootRichard Purdie1
The use of bb.fatal means build-sysroots fails as soon as multiple gdb's are built with overlapping files, or multiple recipes with overlapping headers exist. Change the fatal call into an exception which we can then trap. Also avoid trying to call readlink on something with isn't a symlink. This allows build-sysroots to work better under various scenarios. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26image/packagegroup/populate_sdk: Drop do_populate_sysroot task properlyRichard Purdie3
Setting do_populate_sysroot as noexec means the code keeps thinking it can find a manifest file for it. It also complicates sstate installtion since the code would believe there is an sstate object there it should look for. Instead, delete the task. This causes sdk failures as the dependencies are wrong so fix those as well. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26cross: Update MULTIMACH_TARGET_SYS to reflect modern recipesRichard Purdie1
Mixing PACKAGE_ARCH with BUILD_* has always seemed somewhat odd. This comes into play when building different SDKMACHINES as the PN space for the recipes did once overlap. With the modern build process we use, this overlap is a thing of the past since PN has SDK_SYS appended to it. cross.bbclass sets PACKAGE_ARCH == BUILD_ARCH so this change only affects crosssdk.bbclass where PACKAGE_ARCH == SDK_ARCH. The current structure is quite confusing and this removes this artefact of history, allowing other gcc cleanups rather than meaning we futher complicate the code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-26staging/allarch: Remove hardcoded PACKAGE_ARCHS from the classRichard Purdie2
The code was making an assumption that the only PACKAGE_ARCH in use was TUNE_PKGARCH. This is incorrect so iterate over the list from PACKAGE_EXTRA_ARCH instead. We also need to change allarch to preserve this variable, else the staging code doesn't function. We do this in a way which clears the variable history so that the task hashes remain unaffected. [Thanks to Andrew Goodbody <andrew.goodbody@cambrionix.com> for testing/fixing] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23classes/oeqa: Replace subprocess.check_call() with check_output()Richard Purdie3
If you use subprocess.check_output() the traceback will contain the output when the command fails which is very useful for debugging. There is no good reason not to use this everywhere. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23sanity/abi-version: Force a clean TMPDIR after recipe specific sysroot changesRichard Purdie1
With the recipe specific sysroots, we need a clean tmpdir, else pseudo-native, openssl-native, subversion-native and serf-native need to be manually cleaned. After these there are probably more places where software doesn't rebuild correctly even if we pass in new parameters to it. The simplest solution is to force people to start from a clean TMPDIR since everything would rebuild anyway. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23kernel-fitimage: Use compressed ramdisks in FIT images if availableRick Altherr1
kernel-fitimage:fitimage_assemble() was calling copy_initramfs from kernel.bbclass which decompresses the initramfs cpio. Assume that if INITRAMFS_FSTYPES includes a compressed cpio, that is what it desired in the FIT image. Signed-off-by: Rick Altherr <raltherr@google.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
2017-01-23qemuboot.bbclass: add blank lines in commentsRobert Yang1
Add blank lines in comments to make it easy for readind and updating. Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2017-01-23runqemu: fixes for slirp, network device and hostfwdRobert Yang1
Fixed: - Add QB_NETWORK_DEVICE to set network device, it will be used by both slirp and tap. - Set QB_NETWORK_DEVICE to "-device virtio-net-pci" in qemuboot.bbclass but runqemu will default to "-device e1000" when QB_NETWORK_DEVICE is not set, this is because oe-core's qemu targets support virtio-net-pci, but the one outside of oe-core may not, "-device e1000" is more common. - Set hostfwd by default: 2222 -> 22, 2323 -> 23, and it will choose a usable port when the one like 222 is being used. This can avoid conflicts when multilib slirp qemus are running. We can forward more ports by default if needed, and bsp.conf can custom it. - Use different mac sections for slirp and tap to fix conflicts when running both of them on the same host. [YOCTO #7887] CC: Nathan Rossi <nathan@nathanrossi.com> CC: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
2017-01-23testexport.bbclass: Migrate testexport to use new frameworkMariano Lopez1
This migrates current testexport implmentation to use the new OEQA framework. [YOCTO #10686] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23testimage.bbclass: Add package install featureMariano Lopez1
This allows to use the package install feature with the new OEQA framework. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2017-01-23testimage.bbclass: Add support for package extractionMariano Lopez1
testimage support the installation of packages without a package manager in the target. This adds support for package extraction required to support the installation feature. [YOCTO #10234] Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-18 21:22:12 +0000