summaryrefslogtreecommitdiffstats
path: root/lib/libalpm
AgeCommit message (Collapse)AuthorFilesLines
2020-05-09Convert download packages logic to multiplexed APIAnatol Pomozov3-58/+25
Create a list of dload_payloads and pass it to the new _alpm_multi_* interface. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement multibar UIAnatol Pomozov3-30/+34
Multiplexed download requires ability to draw UI for multiple active progress bars. To implement it we use ANSI codes to move cursor up/down and then redraw the required progress bar. `pacman_multibar_ui.active_downloads` field represents the list of active downloads that correspond to progress bars. `struct pacman_progress_bar` is a data structure for a progress bar. In some cases (e.g. database downloads) we want to keep progress bars in order. In some other cases (package downloads) we want to move completed items to the top of the screen. Function `multibar_move_completed_up` allows to configure such behavior. Per discussion in the maillist we do not want to show download progress for signature files. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Extend download callback interface with start/complete eventsAnatol Pomozov2-7/+47
With the previous download interface the callback uses the first progress event as 'download has started' signal. Unfortunately it does not work with up-to-date files that never receive 'download progress' events. Up-to-date database messages are currently handled in sync_syncdbs() after the sequential download is completed and a result from ALPM is received. But this is not going to work with multiplexed download interface that returns the result only after all files are completed. Another problem with 'first progress event is the beginning of the download' is that such events time are unpredictable. Thus the UI progress bar order might differ from what has been passed by client to alpm_dbs_update() function. We actually want to keep the dbs progress bars in a strict order. To help to solve the given problems extend the download callback to allow 2 more events - download started and completed. 'Download started' events appear in the same order as in the list given by a client. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Introduce event types for start/end database list downloadAnatol Pomozov3-9/+22
Multiplexed database/files downloads will use multiple progress bars. The UI logic is quite complicated and printing error messages while handling multiple progress bars is going to be challenging. Instead we are going to save all ALPM error messages to a list and flush it at the end of the download process. Use on_progress variable that blocks error messages printing. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement multiplexed download using mCURLAnatol Pomozov2-10/+388
curl_multi_download_internal() is the main loop that creates up to 'ParallelDownloads' easy curl handles, adds them to mcurl and then performs curl execution. This is when the paralled downloads happens. Once any of the downloads complete the function checks its result. In case if the download fails it initiates retry with the next server from payload->servers list. At the download completion all the payload resources are cleaned up. curl_multi_check_finished_download() is essentially refactored version of curl_download_internal() adopted for multi_curl. Once mcurl porting is complete curl_download_internal() will be removed. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement _alpm_multi_downloadAnatol Pomozov1-4/+46
It is an equivalent of _alpm_download but accepts a list of payloads. curl_multi_download_internal() is a stub at this moment and will be implemented in the later commits of this patch series. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Inline dload_payload->curlerr field into a local variableAnatol Pomozov2-5/+5
dload_payload->curlerr is a field that is used inside curl_download_internal() function only. It can be converted to a local variable. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Add multi_curl handle to ALPM global contextAnatol Pomozov5-20/+17
To be able to run multiple download in parallel efficiently we need to use curl_multi interface [1]. It introduces a set of APIs over new type of handler 'CURLM'. Create CURLM object at the application start and set it to global ALPM context. The 'single-download' CURL handle moves to payload struct. A new CURL handle is created for each payload with intention to be processed by CURLM. Note that curl_download_internal() is not ported to CURLM interface due to the fact that the function will go away soon. [1] https://curl.haxx.se/libcurl/c/libcurl-multi.html Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Introduce alpm_dbs_update() function for parallel db updatesAnatol Pomozov4-0/+178
This is an equivalent of alpm_db_update but for multiplexed (parallel) download. The difference is that this function accepts list of databases to update. And then ALPM internals download it in parallel if possible. Add a stub for _alpm_multi_download the function that will do parallel payloads downloads in the future. Introduce dload_payload->filepath field that contains url path to the file we download. It is like fileurl field but does not contain protocol/server part. The rationale for having this field is that with the curl multidownload the server retry logic is going to move to a curl callback. And the callback needs to be able to reconstruct the 'next' fileurl. One will be able to do it by getting the next server url from 'servers' list and then concat with filepath. Once the 'parallel download' refactoring is over 'fileurl' field will go away. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Add config option to specify amount of parallel download streamsAnatol Pomozov3-0/+25
It includes pacman.conf new 'ParallelDownloads' option that specifies how many concurrent downloads cURL starts in parallel. Add alpm_option_set_parallel_downloads() ALPM function that allows to set this config option programmatically. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-04-29Constify some input pointersRikard Falkeborn3-3/+3
Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-04-13Use GOTO_ERR throughoutAllan McRae6-41/+22
The GOTO_ERR define was added in commit 80ae8014 for use in future commits. There are plenty of places in the code base it can be used, so convert them. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-04-13Use STRDUP for error checking in more placesAllan McRae3-5/+10
Use STRDUP() over strdup() to catch memory allocation errors. There are still some instances of strdup left, but these are in functions that currently have no error path and would require a larger rework. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-04-13Add REALLOC macro to simplify realloc error handlingRikard Falkeborn5-19/+7
realloc can fail just like the other memory allocation functions. Add a macro to simplify handling of realloc failures, similar to the already existing MALLOC, CALLOC, etc. Replace the existing realloc uses with the new macro, allowing us to move tedious error handling to the macro. Also, in be_package and be_sync, this fixes hypothetical memory leaks (and thereafter null pointer dereferences) in case realloc fails to shrink the allocated memory. Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-13Add file and line number to RET_ERR{,_VOID}Allan McRae1-2/+2
Following the example of the recently added GOTO_ERR, adding the file and line number in addition to the function name in our debug messages is potentially useful. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-09Add GOTO_ERR() macro to set error and then goto a labelAnatol Pomozov1-0/+5
This is a macro similar to RET_ERR but useful in the case when we need to record an error and then jump to some cleanup section. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-09Remove unneeded goto cleanupAllan McRae1-11/+5
Since commit 2ee7a8d8, there is no cleanup needed in this function. Just return instead of jumping to the cleanup label. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-05Do not download files if find_dl_candidates() failsAnatol Pomozov1-1/+1
One reason why the function returns an error is some repo does not have any servers. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-24Simplify construction of payloads in download_filesAnatol Pomozov1-21/+13
Currently, download_files() creates payloads for all packages then iterates over them, calling download_single_file. This can be simplified by looping over packages and constructing the payload as needed. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-10build-aux/update-copyright 2019 2020Allan McRae50-50/+50
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-30Eliminate extra loop over dbs_syncAnatol Pomozov1-9/+5
Current flow looks like loop dbs_sync { loop pkgs { if pkg.db == db then process(pkg, db) } } Package sync transaction always has a counterpart in the dbs_sync list (I cannot come up with a use-case when it is not true). So the loop can be simplified to: loop pkgs { process(pkg, pkg.db) } Tested: 'ninja test' & manually by using pacman with this patch for a week Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-28Docs docs docsmorganamilo17-298/+241
libalpm: move docs from .c files into alpm.h And fix/expand some along the way. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-28libalpm: fix alpm_option_set_assumeinstalledmorganamilo1-0/+1
It looks like this function has never actually worked. The current list is never set to NULL after being freed. So the new deps were just appended to the already freed list, leading to a segfault. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-27Fix "pacman -U <url>" operationsAllan McRae1-1/+1
Commit e6a6d307 detected complete part files by comparing a payload's max_size to initial_size. However, these values are also equal when we use pacman -U on a URL as max_size is set to 0 in that case. Add a further condition to avoid that. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-27Increase maximum database sizeAllan McRae1-2/+2
We previously has the maximum database size as 25MB. This was set in the days before repos had as many packages as they do now, and before we started distributing files databases. Increase this limit to 128MB. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-07Use c99 struct initialization to avoid memset callsDave Reisner6-38/+19
This is guaranteed less error prone than calling memset and hoping the human gets the argument order correct.
2019-11-26Fix documentation of alpm_mtree_next and remove libarchive exposureAllan McRae2-3/+17
The documentation of the return types of alpm_mtree_next was incorrect. This extended into the relevant function in be_local.c. Also, return explicit integer values, rather than the ARCHIVE_xxx values, to avoid unnecessarily exposing frontends to libarchive internals (even though it makes no functional difference). Original-work-by: morganamilo <morganamilo@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-22Improved documentation for alpm_db_search() parametersAllan McRae1-1/+2
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-21Fix leak in _alpm_db_search on error conditionAllan McRae1-1/+4
2019-11-16Dereference double pointer before assigning NULLDaniel T. Borelli1-2/+2
Daniel T. Borelli <danieltborelli@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-15Handle .part files that are the size of the correct packageAllan McRae2-2/+18
In rare cases, likely due to a well timed Ctrl+C, but possibly due to a broken mirror, a ".part" file may have size at least that of the correct package size. When encountering this issue, currently pacman fails in different ways depending on where the package falls in the list to download. If last, "wrong or NULL argument passed" error is reported, or a "invalid or corrupt package" issue if not. Capture these .part files, and remove the extension. This lets pacman either use the package if valid, or offer to remove it if it fails checksum or signature verification. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-11libalpm/sync.c: Do not download missing keys multiple timesAllan McRae1-2/+10
We now store key structs of our missing key info, so can not search the list for string matches. This caused missing keys to be downloaded once for every package they signed. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-08pacman+libalpm: handle search errorsmorganamilo3-14/+19
Previously, pacman treated no matches and an error during search the same. To fix this, alpm_db_search now returns its status as an int and instead takes the to be returned list as a param. Allowing front ends to easily differentiate between errors and no matches. Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-27Fix segfault importing PGP key for pacman -U operationsAllan McRae1-2/+2
Use after free. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-23Update copyright yearsAllan McRae50-50/+50
make update-copyright OLD=2018 NEW=2019 Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-21Final translation updates for pacman-5.2.0Allan McRae43-563/+565
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-15libalpm: set errno in signing dummy functionsmorganamilo1-3/+6
This allows pacman to print the correct error message when checking keys and libalpm has been compiled without gpgme support. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-15libalpm: fix segfault when -Qip'ing a packagemorganamilo1-1/+2
The dummy checksigs function never sets count to 0, leaving it unitialized. This caused the siglist cleanup to try and free the empty list. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-14Translation updatesAllan McRae43-2015/+1887
Pull all translations with >75% completion. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-12move wordsplit into common for sharingAndrew Gregory1-116/+3
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
2019-10-09signing: handle unknown uid in key importChristian Hesse1-0/+5
With unknown uid pacman crashed. Return with error from email_from_uid() if uid is NULL. Signed-off-by: Christian Hesse <mail@eworm.de> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07Pull and push translation changes in preparation for 5.2Allan McRae43-2694/+3002
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07libalpm: short circuit alpm_find_dbs_satisfiermorganamilo1-8/+7
when a satisfying package is already installed, we always pick it instead of prompting the user. So we can return that package as soon as we find it, instead of waiting until we've iterated through all the databases. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07libalpm: fix incorrect documentationmorganamilo1-5/+7
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07libalpm: resolvedep(): don't compare names twicemorganamilo1-1/+2
If we failed to get the pkg from pkgcache then we know no satisfying package exists by name. So only compare provides. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07signing: add ability to import keys using a WKDJonas Witschel1-13/+84
Currently pacman relies on the SKS keyserver network to fetch unknown PGP keys. These keyservers are vulnerable to signature spamming attacks, potentionally making it impossible to import the required keys. An alternative to keyservers is a so-called Web Key Directory (WKD), a well-known, trusted location on a server from where the keys can be fetched. This commit adds the ability to retrieve keys from a WKD. Due to the mentioned vulnerabilities, the WKD is tried first, falling back to the keyservers only if no appropriate key is found there. In contrast to keyservers, keys in a WKD are not looked up using their fingerprint, but by email address. Since the email address of the signing key is usually not included in the signature, we will use the packager email address to perform the lookup. Also see FS#63171. Signed-off-by: Jonas Witschel <diabonas@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07signing: move key import confirmation before key_searchJonas Witschel4-28/+58
Ask the user whether they want to import a missing key before even doing a search on the keyserver. This will be useful for getting Web Key Directory support in place: for a WKD, looking up and importing a key are a single action, so the current key_search -> QUESTION -> key_import workflow does not apply. Since only the ID of the package signing key is available before key_search, we display the packager variable in addition to the key ID for user convenience. Signed-off-by: Jonas Witschel <diabonas@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07dload: never return NULL from get_filenameDave Reisner1-2/+4
Downloads with a Content-Disposition header will typically not include slashes. When they do, we should most certainly only take the basename, but when they don't, we should treat the header value as the filename. Crash introduced in d197d8ab82cf when we started using get_filename in order to rightfully avoid an arbitrary file overwrite vulnerability. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-04autotools: distribute meson filesEli Schwartz1-0/+2
If we use make dist to create the official, signed release tarballs, those will not have meson build files by default since autotools doesn't know what they are. Also distribute all src/common/ files. We never strictly needed any of them to be distributed with autotools, because the dist tarball dereferences the symlinks (???), but only some of them were being distributed, and meson needs them to be in the right location as we only build libcommon from the primary files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-08-12Prepare translations for next releaseAllan McRae43-5025/+5289
Signed-off-by: Allan McRae <allan@archlinux.org>
generated by cgit v1.2.3-24-g4f1b (git 2.32.0) at 2024-11-19 05:13:51 +0100