summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-02-10makerepropkg: take makepkg.conf from devtools pkgdatadir locationLevente Polyak1-2/+1
This ensures we use the same configuration for reproducing packages as we use for building them via devtools. One example of why we care about this are the COMPRESS* settings that may differ from the guest's pacman shipped makepkg.conf that affect the reproducibility of packages.
2019-12-27makerepropkg: correctly reproduce a pkgfile with any compression typeEli Schwartz1-1/+2
We don't want the default PKGEXT in the current version of devtools, we want the PKGEXT we *know* the input file used. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-12-27Version 2019122720191227Levente Polyak1-1/+1
2019-12-27makerepopkg: fix zstd extension (.zstd -> .zst)Robin Broda1-1/+1
2019-12-27makepkg.conf: change default compression method to zstdRobin Broda1-2/+2
2019-12-12Version 2019121220191212Levente Polyak1-1/+1
2019-12-08lib: support arbitrary compressed package filesLevente Polyak1-2/+2
We need to modify the matching of valid package files to support formats like zstd. Let's try to use an eager approach instead of a simple whitelist in order to be functional for arbitrary formats that may be introduced in the future without the need to adjust any code. Allow any single fragment word as compression type but filter out known non-package content like signature files. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-12-05offload-build: return failure if no package files were foundEli Schwartz1-0/+2
This means that the remote command died at some stage earlier than the printing of created files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-12-05makechrootpkg: sync databases for checkpkg off-siteLevente Polyak1-2/+13
Use pacman's --dbpath feature to sync fresh databases inside an isolated location and split up the database sync and package location calls to remove the need of weird grep calls. It isn't nice of makechrootpkg to modify the host database state just by building packages. No foreign program shall automatically modify the host database other than by the explicit will of a system maintainer, which is the major reason this changes get incorporated. However, there is certain indoctrinated believe that using -Sy is the prime evil. In fact it has been declared as a social rule to a technical problem of not getting into potential partial upgrade states. This is not a proper loophole less solution as there are multiple ways and use cases that lead to such a state, like aborting a -Syu on the prompt for whatever reason, what really matters is that it is not a technically bullet proof solution to solve the problem. Databases shall have the freedom to be as up to date as databases or their owner wishes, allowing querying on latest database state without fear. The only loophole-less contract that _really_ is from importance is always using -Su instead of plain -S to install packages. Installing packages is what actually brings one into a potential partial upgrade state and by using -Su an outstanding upgrade is forced when installing a new package. This properly solves all edge cases in a technical manner instead of declaring people who abort the prompt of -Syu to be the problem. In fact, using this simple contract allows whatever system maintenance workflow a host owner wants to follow, which may still be to always use -Syu and deal with system upgrades explicitly instead of the time when installing new packages, but the -Su contract is the real safe guard to guarantee no edge case can ever slip in. This magically also opens up the freedom to people who wish to use -Sy to simply query on up to date data as the currently indoctrinated "never do -Sy" stone plates not only are not rock solid in technical terms but also make certain use cases simply impossible and hence cripple the functionality without at the very least being fully loophole free. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-30makechrootpkg: double quote array expansions for checkpkg locationsLevente Polyak1-1/+1
Avoid re-splitting remotepkg elements used for checkpkg conditions.
2019-11-30Version 2019113020191130Levente Polyak1-1/+1
2019-11-30makechrootpkg: sync database for checkpkg to avoid nonexistent targetsLevente Polyak1-1/+1
For build servers or similar infrastructure its relatively common to not sync/update the database regularly. This leads to problems properly running checkpkg duo to nonexistent target files that we try to download. As building on build servers is a very common use case, lets ensure we sync the local database before trying to resolve the package locations.
2019-11-30makechrootpkg: check local pkg versions before downloading for checkpkgLevente Polyak1-6/+12
Avoid always trying to download and output the according message. Add checks for packages either not being available in the repo or all variants have up to date versions stored in the local cache.
2019-11-30completion: add makerepropkg zsh completionLevente Polyak1-1/+8
2019-11-30completion: reflect new -U mkarchchroot optionLevente Polyak1-0/+1
2019-11-30lib/common.sh: do not use colors when running on a dumb terminalIvy Foster1-1/+1
2019-11-30doc: add manpage for the new makerepropkg toolEli Schwartz2-0/+39
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30makerepropkg: add new program to try to reproducibly build a packageEli Schwartz3-0/+192
This attempts to recreate a package that was probably created using makechrootpkg, and see if it conforms to the https://reproducible-builds.org/ specification. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30mkarchroot: support wrapping pacstrap -UEli Schwartz2-4/+12
Needed to support reproducible builds. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30archbuild: use better umaskEli Schwartz1-1/+1
In commit 40a90e2cab479cc64903a62b42eb617a8a7e5842 we tried to protect against system umasks resulting in unreadable chroots. However, we tried to do this in a targeted manner due to not wanting to fiddle with permissions for user-owned files. Unfortuantely, mkdir -p -m755 does not actually work that way -- the parent directory is created with broken permissions. We need umask. Run umask and mkdir in a subshell to prevent leakage. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30commitpkg: clearly delineate the commit message template and the rationaleEli Schwartz1-1/+1
In commit 75d23eec942e7160108ee194894b6b83ed3045d5 we moved to include commitpkg arguments as the first line of the svn commit message, but we simply dumped the result after the version number without separating the two, increasing the cognitive burden of parsing the rationale. Since the whole point of the change was to make it easier to see what happened when using git log --oneline (reducing the cognitive burden of parsing 'pretty' output with author/date info), it makes sense to also delineate the reason correctly. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30arch-nspawn: rely on deduplication to avoid making multiple host_mirror mountsEli Schwartz1-9/+7
Instead of comparing exact mirror urls to see if they are in host_mirrors in order to "skip" the official mirrors Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
2019-11-30offload-build: remove empty src dir when SRCDEST is setLevente Polyak1-0/+4
Right now there is a bug in makepkg that leaves back an empty src directory if SRCDEST is set. This is purely cosmetic, but lets just politely try to rmdir it and fail silently if its empty or non-existent. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-30offload-build: catch more signals to be sure SRCPKGDEST gets cleanedLevente Polyak1-1/+1
It may be not enough to just listen on EXIT depending on the shell used so lets make sure we clean up SRCPKGDEST by listening to more sigs. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07Makefile: sort program namesEli Schwartz1-6/+6
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07use libmakepkg to find and use variables in makepkg.confEli Schwartz2-29/+14
- drop homebrew function in makechrootpkg - use better mock to find invoking user's $HOME - make offload-build respect makepkg.conf to determine where to sync files, matching the behavior of makechrootpkg Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07arch-nspawn: simplify cachedir/host_mirror parsingEli Schwartz1-23/+13
consolidate logic flows in the same area for parsing and building arrays. Don't bother having a special function just to build the mount_args array, since we now use the same handling for adding any cachedir (including host mirrors) to the mount arguments, this becomes a trivial for loop -- and it really did not need to be delayed until after the sanity check, anyway. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07arch-nspawn: restore cachedir handling for host_mirrorsEli Schwartz1-0/+7
In commit 27ff286ee78eb2faac803e3ef67f3171ddfa0098, we moved from sourcing the primary cachedir via /etc/pacman.conf, to using the pacman.conf in the workdir. One unanticipated side effect of this was breaking the special host mirrors magic we used to turn a host mirror into a cachedir. It was still processed as a server, but we relied on it being in the host's cachedirs in order to be persisted, and this no longer occurred. Solve this by explicitly adding each host mirror root as a cachedir. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-07commitpkg: Include commit msg arg in first lineDaniel M. Capella1-2/+2
Commit messages belong on the first line, with optional "explanatory text" starting after a blank line: https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html Referencing commit ee970f0bde3c90a0dff909c366d4ab1a1bff9b9d Signed-off-by: Daniel M. Capella <polyzen@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-11-06conf: Sync with pacman 5.2.0-2Daniel M. Capella9-27/+11
Signed-off-by: Daniel M. Capella <polyzen@archlinux.org>
2019-10-16Version 2019101620191016Levente Polyak1-1/+1
2019-10-15ci: fix travis build using undocumented but reserved keywordLevente Polyak1-1/+1
https://github.com/mikkeloscar/arch-travis/issues/65
2019-10-15sogrep: redirect to destination mirror (#25)jelle van der Waa1-1/+1
Some mirrors redirect consumers to a near by mirror which isn't handled by sogrep.
2019-09-28zsh_completion: add offload-build completionLevente Polyak1-1/+10
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28zsh_completion: add sogrep completionsLevente Polyak5-10/+49
Transform sogrep into an in-prog so we can benefit from the m4 macro to specify valid repos in a single place of truth. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28commitpkg: prefer explicit signature+data parameters for gpg --verifyLevente Polyak1-2/+2
Lets prefer the explicit variant of gpg --verify by providing both, the signature and the data file as parameters. For the unlikely case there is a matching signature file already present that was created outside of the toolchain and has an embedded signature with data, we at least could detect it early with this check. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28commitpkg: disallow if PKGBUILD hash mismatches package's enclosed hashLevente Polyak1-1/+6
Several cases showed that we release packages that were built with different PKGBUILDs than the one commited to the source tree. This is bad for obvious reasons plus sploils reproducible builds. We, under no circumstances, want to allow using commitpkg to publish and release a packages whose PKGBUILD doesn't match the one to be commited. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28commitpkg: fix wrongly ordered find_cached_package callLevente Polyak1-1/+1
The unknown packager check didn't worked so far as the wrongly ordered call to find_cached_package lead to the enclosing block never being executed. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28make: add target to tag a new versionLevente Polyak1-1/+7
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28zsh_completion: overhaul all completions to match actual optionsLevente Polyak1-9/+23
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28checkpkg: add option to print a warning in case of differencesLevente Polyak4-4/+14
Sometimes its desired to be explicitly made aware of differences reporter by checkpkg via printing a warning instead of a regular message. Automatically use --warn for makechrootpkg builds so packagers are made visibly aware of a soname bump by simply looking out for colors indicating non success messages. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28checkpkg: add option to avoid keeping the tmp dirLevente Polyak4-3/+61
In some cases, like default makechrootpkg execution, the temporary directory used to assemble the differences is not required. Add an option to checkpkg that allows to get rid of that directory after run and call it automatically like that in makechrootpkg. Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-28makechrootpkg: run checkpkg automatically after buildLevente Polyak2-2/+20
Cache previous versions required for checkpkg via pacman to avoid multiple downloads when running multiple times. In case we can't download the packages, like while building out of repo packages, print a warning instead of running checkpkg Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-09-12Version 2019091220190912Levente Polyak1-1/+1
2019-09-12Revert "makechrootpkg: with -n, check if the package failed to install"Eli Schwartz1-8/+1
This reverts commit be44b9cde15f3228839253c0c0d7d56c124c4e26. This was a nice idea in theory, because it means that we can catch conflicting files before releasing them into the repos. In practice, there were unanticipated side effects: single-package installs which conflict against their own makedepends cannot be installed either. Examples include: - kernel modules which makedepend on their dkms equivalent - jack2, which makedepends/optdepends on portaudio, which requires jack... but jack2 is a drop-in provides/conflicts jack. We cannot reliably detect when makepkg --install will error out because of dependency conflicts vs. packages which are simply broken. So, back out this change for now. Revisit this once pacutils has a new release, because it will add the option --resolve-conflicts=all, allowing for much better scripted responses to "foo conflicts with bar, remove bar? [y/N]" than simply "--noconfirm and fail". Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-08-21Version 2019082120190821Levente Polyak1-1/+1
2019-08-09makechrootpkg: with -n, check if the package failed to installEli Schwartz1-1/+8
We previously whitelisted this return code because split packages can frequently conflict each other, so makepkg -i is *expected* to fail in such a case. However, there is no good reason to let this succeed if the pkgbase only builds one pkgname -- that will always be a severe issue. Add a check for how many split Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-08-09makechrootpkg: make the -U option work for the first time everEli Schwartz1-2/+2
The -U option was initially introduced in commit cda9cf436b2897b063c1e40efb144404aad8b821 in order to enable running makechrootpkg as root, delegating to another, manually selected, user to perform various non-root tasks (given that makepkg was modified to throw fatal errors when run as root without the option of --asroot to disable that). However, it was only ever implemented for the --verifysource option outside of the chroot, and the builduser inside the chroot is created with the same uid as the makechrootpkg invoker. It needs to run as the same uid, because it needs rw access to $startdir and $SRCDEST! Additionally this lets the invoking user more easily inspect the build directory in case of problems... The correct solution for this is to properly implement the initial intention of the -U option, and make it override the autodetection of the "invoking user" which is normally done by inspecting $SUDO_USER. This is then used as the single source of truth for "who am I pretending to be". Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-08-09archrelease: actually fail in failure conditionsEli Schwartz1-7/+8
When svn ls fails due to network timeouts, this currently results in archrelease deleting all files, then committing this as the changeset. This causes data loss... With bash 4.4 and using wait $! we can get return the return code of the last backgrounded command -- which process substitution qualifies as. Key off of this to make sure that `svn ls` actually succeeded. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
2019-08-09archbuild: prefer repo/arch-specific configs if they existEli Schwartz1-4/+13
When mixing and matching different repos and architectures not present in mainline archlinux, it is sometimes desirable to set up differing presets with more granularity than devtools currently allows. One example of this is when building for architectures that are only supported by another project -- in order to coexist on a mainline archlinux host, a different mirrorlist needs to be used. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
generated by cgit v1.2.3-24-g4f1b (git 2.32.0) at 2024-11-16 13:28:36 +0100