summaryrefslogtreecommitdiff
path: root/meta/classes/sstate.bbclass
AgeCommit message (Collapse)AuthorFiles
2012-07-19Convert tab indentation in python functions into four-spaceRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-16sstate.bbclass: Move crosssdk before cross to ensure correct variables are setRichard Purdie1
There was a bug showing up where the crosssdk recipes were being installed as machine specific in the sstate-control directory. This turned out to be due to the architecture fields used by sstate being set incorrectly. The problem is that the crosssdk inherits the cross class. It therefore needs to be listed in this if statement block before the cross check, not after. This should resolve some package architecture issue of crosssdk sstate files. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Saul Wold <sgw@linux.intel.com>
2012-05-30meta: replace os.system with subprocess.callRobert Yang1
Replace os.system with subprocess.call since the older function would fail (more or less) silently if the executed program cannot be found More info: http://docs.python.org/library/subprocess.html#subprocess-replacements [YOCTO #2454] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-18sstate.bbclass: Make sure we don't have an empty fixmepath fileMark Hatle1
Jason Wessel noticed that a package without any fixmepath entries would generate a sed warning about no input files. This patch resolves that by ensuring that an empty fixmepath file never gets written into the sstate archive. Also we avoid a second message by only doing xargs if we got input. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-17meta/classes/sstate.bbclass: ensure sstate files have group R/WJoshua Lock1
In order to make sharing sstate files easier chmod them with 0664 permissions so that they are readable and writable by the user and any other members of their group. Signed-off-by: Joshua Lock <josh@linux.intel.com>
2012-05-15sstate.bbclass: Optimize the generation and install path fixupsMark Hatle1
The fixmepath file that is generated contains a list of all of the files that need their paths fixed. In the previous version the fixmepath was generated to include all of the files that sed may have changed. In the new version, we first grep the files to see if they contain a path that needs to be changed, only then do we perform the sed operation on those files. This results in a modest performance increate in the creation of the sstate file. The following numbers include the do_package and do_populate_sysroot tasks on the perl recipe. Before the change: real 4m23.018s user 1m57.067s sys 1m33.327s After the change: real 4m13.083s user 1m54.062s sys 1m26.064s However, a more significnt performance gain is felt during the extraction/install of sstate cache files, as the fixmepaths file now has a significantly smaller list of files to modify. Before the change: real 0m39.798s user 0m11.158s sys 0m12.642s After the change: real 0m25.511s user 0m8.408s sys 0m5.077s (All numbers above were recorded with a cold filesystem cache on a machine with 12 GB of ram.) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-11sstate.bbclass: Improve sstate_installpkg performanceMark Hatle1
In a pathological case, lots of files to process, the sstate_installpkg performance was very poor. It interated over each file and ran 3 individual sed commands per file. Changing this to keep iterating but running only a single command took about 1/3 time time. However, when looking at the corresponding sstate_hardcode_path function, it was clear we could optimize this further. Using the same encoding logic to specify only the minimumal sed operation necessary, and using xargs to avoid the os.system call the install step was able to be performed in 13% of the original time. Example timing numbers for perl: 3m7s original code 1m20s single sed, but interating 0m26s using xargs and limited sed Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-23Revert "bitbake.conf/sstate.bbclass: Change PATH when installing sstate ↵Richard Purdie1
files to avoid issues" Reluctantly revert this since it breaks the tar-native workaround we have for old versions of tar :( This reverts commit 01218e29f963120c77885964702198d9bce77e11.
2012-03-23sstate.bbclass: Remove possibility of file corruption and make package ↵Richard Purdie1
writing atomic There is currently a race window when creating sstate packages since we don't atomically write the files to SSTATE_DIR. This change ensures we do so by writing to a temporary file and then doing an atomic move. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-21bitbake.conf/sstate.bbclass: Change PATH when installing sstate files to ↵Richard Purdie1
avoid issues This resolves issues related to pigz-native when installing from sstate that people have been seeing. It also gives us a way to solve issues like the gzip-native race during sstate package creation covered in Yocto #1774. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-05meta: Replace bb.data.expand(xxx, d) -> d.expand(xxx)Richard Purdie1
sed \ -e 's:bb.data.\(expand([^,()]*\), *\([^) ]*\) *):\2.\1):g' \ -i `grep -ril bb.data.expand *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-05sstate: Remove unecessary bb.data.expand()Richard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-03-05meta/classes: Convert to use appendVar and appendVarFlagsRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-24sstate.bbclass: improve performance of sstate package creationMartin Jansa1
* also fixes replacing paths for perl where cmd line was probably too long for os.system(cmd) (it had 560410 characters because a lot of files from sstate_scan_cmd). * also print those 2 commands so we can find them in log.do_package Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
2012-02-23sstate.bbclass: add *_config to SSTATE_SCAN_FILESMartin Jansa1
* e.g. mysql5 has mysql_config not mysql-config Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-10sstate.bbclass: Optimise sstate_hardcode_pathRichard Purdie1
The sstate_hardcode_path() function triggered large numbers of exec() calls when processing packages with large numbers of file relocations (e.g. perl). This patch optimises those calls into longer single commands which make the code significantly more efficient. This reduced the do_package time for perl by 2 minutes (from 4.75 minutes) for me. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-27sstate.bbclass: Add support for sstate preinst functionsRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-12-15sstate: Add SSTATE_SCAN_FILESRichard Purdie1
We process all files in the native/cross cases for finding and fixing relocation issues. In the target case we've only processed .la and binconfig files. Since there are other files which are in need of this processing, this change allows recipes to specify files that may be outside the normal set. This means hardcoded paths that need to be fixmepathed to work correctly are handled and addresses some sstate relocation bugs that have been seen. Based on a patch from Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-26getVar/setVar cleanupsRichard Purdie1
Complete the bb.data.getVar/setVar replacements with accesses directly to the data store object. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-21sstate.bbclass: Ensure we expand stamp-extra-infoRichard Purdie1
Without this change we can end up looking for <stamp>.${MACHINE} instead of the expected expanded value. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-10Convert to use direct access to the data store (instead of bb.data.*Var*())Richard Purdie1
This is the result of running the following over the metadata: sed \ -e 's:bb.data.\(setVar([^,()]*,[^,()]*\), *\([^ )]*\) *):\2.\1):g' \ -e 's:bb.data.\(setVarFlag([^,()]*,[^,()]*,[^,()]*\), *\([^) ]*\) *):\2.\1):g' \ -e 's:bb.data.\(getVar([^,()]*\), *\([^(), ]*\) *,\([^)]*\)):\2.\1,\3):g' \ -e 's:bb.data.\(getVarFlag([^,()]*,[^,()]*\), *\([^(), ]*\) *,\([^)]*\)):\2.\1,\3):g' \ -e 's:bb.data.\(getVarFlag([^,()]*,[^,()]*\), *\([^() ]*\) *):\2.\1):g' \ -e 's:bb.data.\(getVar([^,()]*\), *\([^) ]*\) *):\2.\1):g' \ -i `grep -ril bb.data *` Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-10-06sstate.bbclass: Ensure machine specific stamps are only wiped for the ↵Richard Purdie1
current task sstate was being a little too ethusiastic about removing stamp files and was removing stamp files for other machines when it shouldn't have been. This patch teaches sstate about machine specific stamp extensions and allows it to only remove the current task's stampfiles. Based on a patch from Phil Blundell <philb@gnu.org> with some tweaks from me. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-09-26sstate: No need to spew out a debug message per file, summarise insteadRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-09-07sstate: Avoid contention between sstate packagesRichard Purdie1
If several sstate packages are decompressing at the same time, they can conflict over the "fixmepath". If two packages try to write to this at the same time it results in an error. To avoid this we remove the file once we've finished processing it. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-08-11sstate: Add level 2 debug so its possible to see what sstate is doingRichard Purdie1
Currently its hard to figure out if/when sstate is checking for possible packages to speed up builds. This patch adds level 2 debug output which better indicates what files are being searched for an why. [YOCTO #1259] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-25bitbake.conf/classes: Variable cleanupRichard Purdie1
This patch removes the variables BASE_PACKAGE_ARCH, BASEPKG_HOST_SYS, BASEPKG_TARGET_SYS and also removes the immediate assignments in several core classes as these are no longer required. This should make it clearer what some of the core variables do and simplfy some overly complex and confusing class code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-12sstate: Improve performance by moving files rather than using copy and deleteRichard Purdie1
Acked-by: Joshua Lock <josh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-29sstate.bbclass: Fix an issue if the config changesMark Hatle1
We need to check if we know of the task type, before we attempt to process it. In order to reproduce the problem build with: PACKAGE_CLASSES = "package_ipk" Then change it to: PACKAGE_CLASSES = "package_rpm" Build again -- and then try bitbake -c cleansstate <recipe> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2011-06-28classes/conf: Drop MULTIMACH_ARCH variable, it adds unused complexity and ↵Richard Purdie1
serves no useful purpose Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09sstate.bbclass: When removing files, take an exclusive lock, even for shared ↵Richard Purdie1
lockfiles Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-04-28sstate.bbclass: Reduce SSTATE_MANIFESTS assignment to be weakKhem Raj1
Distributions can then override this variable if needed useful e.g. when building uclibc/eglibc both using same tmpdir Signed-off-by: Khem Raj <raj.khem@gmail.com>
2011-04-08sstate: ensure an ordered mapping between SSTATETASKS and SSTATETASKNAMESDexuan Cui1
Fix [YOCTO #964] A recent commit 25a6e5f9(sstate: use only unique set of SSTATETASK) breaks the ordered mapping between SSTATETASKS and SSTATETASKNAMES. As a result, in sstate_cleanall, the line taskname = tasks[namemap.index(name)] gets an incorrect result, and "bitbake -c cleanall" doesn't really remove the files populalted by do_populate_sysroot. Signed-off-by: Dexuan Cui <dexuan.cui@intel.com>
2011-03-29sstate: Add support for taking shared lockfilesRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-24sstate: use only unique set of SSTATETASKMartin Jansa1
* otherwise strange error like this: ERROR: Logfile of failure stored in: /OE/shr-core/tmp/work/armv7a-oe-linux-gnueabi/libtool-cross-2.4-r1/temp/log.do_package_write_ipk.25551 Log data follows: | ERROR: Package already staged (/OE/shr-core/tmp/sstate-control/manifest-nokia900-libtool-cross.deploy-ipk)?! | ERROR: Function 'sstate_task_postfunc' failed NOTE: package libtool-cross-2.4-r1: task do_package_write_ipk: Failed ERROR: Task 11 (/OE/shr-core/openembedded-core/meta/recipes-devtools/libtool/libtool-cross_2.4.bb, do_package_write_ipk) failed with exit code '1' is shown in this case with package_ipk twice in INHERIT * Thanks to Richard for fix Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-19sstate: Ensure a clean removes setscene stamps as well as the main task stampsRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-18sstate.bbclass: Turn absolute symbolic links into relative ones for sstate ↵Richard Purdie1
packages Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-15sstate.bbclass: always delete stamp files in sstate_cleanPaul Eggleton1
For safety, always delete the stamp files in sstate_clean regardless of whether the manifest file exists or not. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-03-15sstate.bbclass: avoid deleting unrelated stamp filesPaul Eggleton1
Avoid deleting stamp files whose names contain the current task's name as a substring. This will be especially important for example if do_package_write is ever made an sstate task (as it would previously have deleted the stamps here for do_package_write_ipk etc.) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-03-08sstate: Ensure the SRCURI fetcher cache is not used for sstateRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23sstate.bbclass: add SSTATEPOSTINSTFUNCSScott Garman1
When doing builds using sstate cache, there was no way to run the equivalent of a pkg_postinst function. This is needed by the SGML-related documentation recipes to properly update the catalog files when new DTDs and stylesheets are installed. SSTATEPOSTINSTFUNCS is a new variable you can set to function(s) in your recipe to run after install is completed from sstate. Thanks to Richard Purdie for suggesting this solution. Signed-off-by: Scott Garman <scott.a.garman@intel.com>
2011-02-11sstate: Fix bugs after new fetcherZhai Edwin1
Current sstate's fetch code doesn't reflect latest fetcher changes, so old fetch style cause exception and fail silently. [BUGID #708] got fixed. Another issue is "import xxx" in python function from sstate.bbclass can only sit in the head of the function, else have UnboundLocalError: "local variable XXX referenced before assignment". Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>
2011-02-07meta/classes: Update classes to use new fetcher APIRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03license.bbclass: Add coverage by sstate code (also use cleandirs and dirs ↵Richard Purdie1
task attribures for directory handling) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-28staging.bbclass: Only clean out the sysroot if we rerun configure, not at ↵Richard Purdie1
setscene time Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-28sstate: Ensure stale stamp files are removed as part of sstate_clean()Richard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27sstate.bbclass: Fix stupid typoRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27sstate.bbclass: No need to dump the contents of the sstate package to stdoutRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27sstate.bbclass: Handle whitespace in the list of files to 'fixup' to resolve ↵Richard Purdie1
log warning Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-27sstate.bbclass: Optimise to check if the manifest exists without taking locksRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-26sstate.bbclass: Correctly handle empty sstate packagesRichard Purdie1
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-26 13:44:33 +0000