summaryrefslogtreecommitdiffstats
path: root/scripts/libmakepkg/source
AgeCommit message (Collapse)AuthorFilesLines
2020-06-18libmakepkg: fix regression in sending plain() output to stderrEli Schwartz5-19/+19
In commit 882e707e40bbade0111cf3bdedbdac4d4b70453b we changed message output to go to stdout by default, unless it was an error. The plain() function doesn't *look* like an error function, but in practice it was -- it's used to continue multiline messages, and all in-tree uses were for warning/error. This is a problem both because we're sending output to the wrong place, and because in some cases, we were performing error logging from a function which would otherwise return a value to be captured in a variable using command substution. Fix this and straighten out the API by providing two functions: one for continuing msg output, and one which wraps this by sending output to stderr, for continuing error output. Change all callers to use the second function. (cherry picked from commit bf458cced7c0845f7b6fabb887d3878ae4cd51b2)
2020-06-18makepkg: correctly handle missing download clientsEli Schwartz1-1/+1
This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, which changed 'plain()' messages to go to stdout, which was then captured as the download client in question: cmdline=("Aborting..."). The result was a very confusing error message e.g. /usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found or with makepkg --nocolor: /usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found The problem here is that we checked to see if an asynchronous subshell, in our case <(...), failed, by checking if its captured stdout is non-empty. Which is terrible, and also a limitation of old bash. But bash 4.4 can use wait $! to retrieve the return value of an asynchronous subshell. Now we target that as our minimum, we can sanely handle errors in such functions. Losing error messages on stdout by capturing them in a variable instead of printing them, continues to be a problem, but this will be fixed systematically in a later commit. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 381e11375569fa7588b1297e0e744749bdafe8f5)
2020-06-18makepkg: guard against undefined git pinned sourcesEli Schwartz1-1/+1
If something like source=(..."#commit=") is used, e.g. due to failed variable expansion, we try to check out an empty refspec as nothing at all, and end up just running "git checkout". This happens because we fail at variable expansion too -- so let's quote our variables properly and make sure git sees this as an empty refspec, so it can error out. Also make sure it is interpreted as a ref instead of a path. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 817f9fb715b4e90d28bc141dfccfc530b9e951dc)
2020-06-18build-aux/update-copyright 2019 2020Allan McRae6-6/+6
Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit e76ec94083235ddc5510ab57b7c2bc12a1d34e8a)
2020-01-14makepkg: fix regression that broke extraction of file:// sourcesEli Schwartz1-1/+2
In commit 9c817b654996249b8022e189ee7e2692f4668431 we made these sources extendable, and heuristically determined the correct extraction functions to use. But our fallback for protos that didn't have an exact extract_* function didn't take into account that 'extract_file' matches an actual proto... so we passed the netfile in while the function expected a file. Solution: the function should expect a netfile too, thereby allowing us to delay an attempted resolution of netfile -> file, to the one case where it is actually used. This makes us slightly more efficient in the non-file case, makes our functions a bit more consistent, and makes file:// extraction work again. Fixes FS#64648 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org> (cherry picked from commit 349c22d043290ccd0cce9f30981f5415e295442a)
2019-10-23Update copyright yearsAllan McRae6-6/+6
make update-copyright OLD=2018 NEW=2019 Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07Support file with seccomp enabledEli Schwartz1-1/+1
Not all compression types can be detected in the seccomp sandbox, so we need to disable it. This requires either configuring makepkg to know the sandbox is available, or checking for file >= 5.38 in which the sandbox option is a no-op even when seccomp is disabled. - Requires autoconf-archive for autotools version compare macro. - meson version comparison could be made a lot simpler using meson-git. Fixes FS#58626 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28Support application/gzip MIME type in extractionKevin Mihelich1-1/+1
file 5.37 changed the gzip MIME type from application/x-gzip to application/gzip, so support this when checking to extract source files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08scripts: protect against unintended glob matching in [[ ]] RHSEli Schwartz1-1/+1
The right-hand side of the [[ ... = ... ]] keyword is an exception to the general rule that quoting is unnecessary with [[ This is usually not a problem, e.g. in libmakepkg, lint_one_pkgname will already fail if pkgname has an asterisk, but it certainly doesn't hurt to be "more proper" and go with the spec; it is more dangerous in repo-add, which can get caught in an infinite loop instead of safely asserting there is no package named 'foo*'. Reported-by: Rafael Ascensão <rafa.almas@gmail.com> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08makepkg: correctly handle hg sources with updates on a non-default branchEli Schwartz1-1/+5
The "tip" ref actually signifies the most recently updated branch. hg does not support a default branch named anything other than "default", except by creating a "@" bookmark. The correct way to explicitly update to the default clone ref, is therefore to use one of these, rather than "tip". Fixes FS#62092 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19makepkg: use "shared" git clones when checking out sourcesEli Schwartz1-1/+1
In order to cache sources offline, makepkg creates *two* copies of every git repo. This is a useful tradeoff for network time, but comes at the cost of increased disk space. Normally, git can smooth this over automagically. Whenever possible, git objects are hardlinked to save space, but this does not work when SRCDEST and BUILDDIR are on separate filesystems. When the repo in question is both very large (linux.git for example is 2.2 GB) and crosses filesystem boundaries, this results in a lot of extra disk space being used; the most likely scenario is where BUILDDIR is a tmpfs for bonus ouch. git(1) has a builtin feature which serves this case handily: the --shared flag will create the info/alternates file instructing git to not copy or hardlink or create objects/packs at all, but merely look for them in an external location (that being the source of the clone). The downside of using shared clones, is that if you modify and drop commits from the original repo, or simply delete the whole repo altogether, you break the copy. But we don't care about that here, because 1) the BUILDDIR copy is meant to be a temporary copy strictly derived via PKGBUILD syntax from the SRCDEST, and must be able to be recreated at any time, 2) if the SRCDEST disappears, makepkg will redownload it, thus restoring the objects needed by the BUILDDIR clone, 3) if the user does non-default things like hacking on the BUILDDIR copy then deleting and re-cloning the SRCDEST may result in momentary breakage, but ultimately should be fine -- the unique objects they created will be stored in the BUILDDIR copy. While it's theoretically possible that upstream will force-push to overwrite the base tree from which makepkg is building (which they should not do), *and* the user deleted their SRCDEST which they should not do, *and* they saved work in makepkg's working directory which they should not do either... ... this is an unlikely chain of events for which we should not care. Using --shared is therefore helpful in immediately useful ways and IMHO has no actual downsides; we should use it. An alternative implementation would be to use worktrees. I've rejected this since it is essentially the same as shared clones, except adding additional restrictions on the branch namespace, and could potentially break existing use cases such as manually handling the SRCDEST in order to share repositories with normal working copies. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: implement extendable source protocolsEli Schwartz4-0/+20
Lookup the existence of matching functions for each protocol, and fallback on the generic file handler. New source protocols can then be added via thirdparty libmakepkg drop-ins without requiring modifications to source.sh Fixes FS#49076 Signed-off-by: Allan McRae <allan@archlinux.org>
2018-11-02Add meson.build files to build with mesonDave Reisner1-0/+22
Provide both build systems in parallel for now, to ensure that we work out all the differences between the two. Some time from now, we'll give up on autotools. Meson tends to be faster and probably easier to read/maintain. On my machine, the full meson configure+build+install takes a little under half as long as a similar autotools-based invocation. Building with meson is a two step process. First, configure the build: meson build Then, compile the project: ninja -C build There's some mild differences in functionality between meson and autotools. specifically: 1) No singular update-po target. meson only generates individual update-po targets for each textdomain (of which we have 3). To make this easier, there's a build-aux/update-po script which finds all update-po targets and runs them. 2) No 'make dist' equivalent. Just run 'git archive' to generate a suitable tarball for distribution.
2018-03-14Update coyrights for 2018Allan McRae6-6/+6
make update-copyright OLD=2017 NEW=201 Signed-off-by: Allan McRae <allan@archlinux.org>
2017-09-13libmakepkg: check for invalid tags in gitEli Schwartz1-1/+10
As per https://lists.archlinux.org/pipermail/arch-general/2017-July/043876.html git doesn't check that the tag name matches what an annotated tag object *thinks* it should be called. This is a bit of a theoretical attack and some would argue that we should always use commits since upstream can legitimately change a tag, but nevertheless this can result in a downgrade attack if the git download transport was manipulated or the upstream repository hacked. So, check the tag blob to make sure the tag actually matches the name we used for `git checkout`. This really should be fixed in git itself, rather than forcing all downstream users of git verify-tag to implement their own checks, but the git developers disagree, see the discussion surrounding https://public-inbox.org/git/xmqqk2hzldx8.fsf@gitster.mtv.corp.google.com/ Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-07-06libmakepkg: fix unsanitized source filenamesEli Schwartz1-2/+2
There were a couple places where filenames beginning with "-" were not properly guarded against by passing them after "--". Some PKGBUILD authors are crazy, but we still take those into account. Signed-off-by: Eli Schwartz <eschwartz93@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04makepkg: Verify git signaturesEli Schwartz1-8/+3
A git repository is marked as signed if it contains the query "signed" as defined by https://tools.ietf.org/html/rfc3986 Adds two utility functions in util/source.sh.in to extract fragments and queries, and modifies source/git.sh.in to use them. Signed-off-by: Eli Schwartz <eschwartz93@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04Update copyright yearsAllan McRae6-6/+6
Signed-off-by: Allan McRae <allan@archlinux.org>
2016-01-04Update copyright years for 2016Allan McRae6-6/+6
make update-copyright OLD=2015 NEW=2016 Signed-off-by: Allan McRae <allan@archlinux.org>
2015-12-15makepkg: Output full URL in case of download failureFlorian Pritz1-1/+1
Most entries in $sources contain variables so finding out why a URL fails to download is hard because one has to manually replace the variables when looking at the PKGBUILD. Simply output the full URL here so that it can be easily seen what is wrong. Old: ==> ERROR: Failure while downloading example-1.2.4.tar.gz New: ==> ERROR: Failure while downloading http://example.org/releases/1.1/example-1.2.4.tar.gz With the new format it is much more obvious that the directory name is the culprint (1.1 vs 1.2) while the old one would not display that information. Signed-off-by: Florian Pritz <bluewind@xinu.at> Signed-off-by: Allan McRae <allan@archlinux.org>
2015-10-19Remove space before ellipsesAllan McRae1-2/+2
Makes all use of ellipses consistent... Signed-off-by: Allan McRae <allan@archlinux.org>
2015-05-19libmakepkg: extract functions for source download and extractionAllan McRae6-0/+619
Signed-off-by: Allan McRae <allan@archlinux.org>