summaryrefslogtreecommitdiffstats
path: root/lib/libalpm/dload.c
AgeCommit message (Collapse)AuthorFilesLines
2020-05-09Convert download packages logic to multiplexed APIAnatol Pomozov1-1/+0
Create a list of dload_payloads and pass it to the new _alpm_multi_* interface. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement multibar UIAnatol Pomozov1-29/+32
Multiplexed download requires ability to draw UI for multiple active progress bars. To implement it we use ANSI codes to move cursor up/down and then redraw the required progress bar. `pacman_multibar_ui.active_downloads` field represents the list of active downloads that correspond to progress bars. `struct pacman_progress_bar` is a data structure for a progress bar. In some cases (e.g. database downloads) we want to keep progress bars in order. In some other cases (package downloads) we want to move completed items to the top of the screen. Function `multibar_move_completed_up` allows to configure such behavior. Per discussion in the maillist we do not want to show download progress for signature files. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Extend download callback interface with start/complete eventsAnatol Pomozov1-4/+15
With the previous download interface the callback uses the first progress event as 'download has started' signal. Unfortunately it does not work with up-to-date files that never receive 'download progress' events. Up-to-date database messages are currently handled in sync_syncdbs() after the sequential download is completed and a result from ALPM is received. But this is not going to work with multiplexed download interface that returns the result only after all files are completed. Another problem with 'first progress event is the beginning of the download' is that such events time are unpredictable. Thus the UI progress bar order might differ from what has been passed by client to alpm_dbs_update() function. We actually want to keep the dbs progress bars in a strict order. To help to solve the given problems extend the download callback to allow 2 more events - download started and completed. 'Download started' events appear in the same order as in the list given by a client. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement multiplexed download using mCURLAnatol Pomozov1-10/+386
curl_multi_download_internal() is the main loop that creates up to 'ParallelDownloads' easy curl handles, adds them to mcurl and then performs curl execution. This is when the paralled downloads happens. Once any of the downloads complete the function checks its result. In case if the download fails it initiates retry with the next server from payload->servers list. At the download completion all the payload resources are cleaned up. curl_multi_check_finished_download() is essentially refactored version of curl_download_internal() adopted for multi_curl. Once mcurl porting is complete curl_download_internal() will be removed. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Implement _alpm_multi_downloadAnatol Pomozov1-4/+46
It is an equivalent of _alpm_download but accepts a list of payloads. curl_multi_download_internal() is a stub at this moment and will be implemented in the later commits of this patch series. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Inline dload_payload->curlerr field into a local variableAnatol Pomozov1-4/+5
dload_payload->curlerr is a field that is used inside curl_download_internal() function only. It can be converted to a local variable. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Add multi_curl handle to ALPM global contextAnatol Pomozov1-11/+6
To be able to run multiple download in parallel efficiently we need to use curl_multi interface [1]. It introduces a set of APIs over new type of handler 'CURLM'. Create CURLM object at the application start and set it to global ALPM context. The 'single-download' CURL handle moves to payload struct. A new CURL handle is created for each payload with intention to be processed by CURLM. Note that curl_download_internal() is not ported to CURLM interface due to the fact that the function will go away soon. [1] https://curl.haxx.se/libcurl/c/libcurl-multi.html Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09Introduce alpm_dbs_update() function for parallel db updatesAnatol Pomozov1-0/+12
This is an equivalent of alpm_db_update but for multiplexed (parallel) download. The difference is that this function accepts list of databases to update. And then ALPM internals download it in parallel if possible. Add a stub for _alpm_multi_download the function that will do parallel payloads downloads in the future. Introduce dload_payload->filepath field that contains url path to the file we download. It is like fileurl field but does not contain protocol/server part. The rationale for having this field is that with the curl multidownload the server retry logic is going to move to a curl callback. And the callback needs to be able to reconstruct the 'next' fileurl. One will be able to do it by getting the next server url from 'servers' list and then concat with filepath. Once the 'parallel download' refactoring is over 'fileurl' field will go away. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-04-13Use GOTO_ERR throughoutAllan McRae1-4/+2
The GOTO_ERR define was added in commit 80ae8014 for use in future commits. There are plenty of places in the code base it can be used, so convert them. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-10build-aux/update-copyright 2019 2020Allan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-28Docs docs docsmorganamilo1-1/+0
libalpm: move docs from .c files into alpm.h And fix/expand some along the way. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-27Fix "pacman -U <url>" operationsAllan McRae1-1/+1
Commit e6a6d307 detected complete part files by comparing a payload's max_size to initial_size. However, these values are also equal when we use pacman -U on a URL as max_size is set to 0 in that case. Add a further condition to avoid that. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-07Use c99 struct initialization to avoid memset callsDave Reisner1-7/+6
This is guaranteed less error prone than calling memset and hoping the human gets the argument order correct.
2019-11-15Handle .part files that are the size of the correct packageAllan McRae1-0/+6
In rare cases, likely due to a well timed Ctrl+C, but possibly due to a broken mirror, a ".part" file may have size at least that of the correct package size. When encountering this issue, currently pacman fails in different ways depending on where the package falls in the list to download. If last, "wrong or NULL argument passed" error is reported, or a "invalid or corrupt package" issue if not. Capture these .part files, and remove the extension. This lets pacman either use the package if valid, or offer to remove it if it fails checksum or signature verification. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-23Update copyright yearsAllan McRae1-1/+1
make update-copyright OLD=2018 NEW=2019 Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07dload: never return NULL from get_filenameDave Reisner1-2/+4
Downloads with a Content-Disposition header will typically not include slashes. When they do, we should most certainly only take the basename, but when they don't, we should treat the header value as the filename. Crash introduced in d197d8ab82cf when we started using get_filename in order to rightfully avoid an arbitrary file overwrite vulnerability. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-06-28Correctly report a download failiure for 404smorganamilo1-0/+1
Currently when caling alpm_trans_commit, if fetching a package restults in a 404 (or other non 400 response code), the function returns -1 but errno is never set. This patch sets errno to ALPM_ERR_RETRIEVE. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-01Sanitize file name received from Content-Disposition headerAndrew Gregory1-1/+2
When installing a remote package with "pacman -U <url>", pacman renames the downloaded package file to match the name given in the Content-Disposition header. However, pacman does not sanitize this name, which may contain slashes, before calling rename(). A malicious server (or a network MitM if downloading over HTTP) can send a content-disposition header to make pacman place the file anywhere in the filesystem, potentially leading to arbitrary root code execution. Notably, this bypasses pacman's package signature checking. For example, a malicious package-hosting server (or a network man-in-the-middle, if downloading over HTTP) could serve the following header: Content-Disposition: filename=../../../../../../usr/share/libalpm/hooks/evil.hook and pacman would move the downloaded file to /usr/share/libalpm/hooks/evil.hook. This invocation of "pacman -U" would later fail, unable to find the downloaded package in the cache directory, but the hook file would remain in place. The commands in the malicious hook would then be run (as root) the next time any package is installed. Discovered-by: Adam Suhl <asuhl@mit.edu> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-07libalpm: prevent 301 redirect loop from hanging the processMark Ulrich1-0/+1
If a mirror responds with a 301 redirect to itself, it will create an infinite redirect loop. This will cause pacman to hang, unresponsive to even a SIGINT. The result is pacman being unable to sync or download any package from a particular repo if its current mirror is stuck in a redirect loop. Setting libcurl's MAXREDIRS option effectively prevents a redirect loop from hanging the process. Signed-off-by: Mark Ulrich <mark.ulrich.86@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2018-10-18alpm: Fix SIGINT handling re: aborting downloadOlivier Brunel1-0/+1
Upon receiving SIGINT a flag is set to abort the (curl) download. However, since it was never reset/initialized, if a front-end doesn't actually exit on SIGINT, and later tries any operation that needs to perform a new download, said download would always get aborted right away due to the flag not having been reset.
2018-10-18alpm: Do not raise SIGINT when filesize goes over limitOlivier Brunel1-1/+1
Variable dload_interrupted is used both to abort a download because SIGINT was caught, and when a file limit is reached. But raising SIGINT is only meant to happen in the first case. Signed-off-by: Olivier Brunel <jjk@jjacky.com>
2018-08-10libalpm/dload.c: add case for CURLE_COULDNT_RESOLVE_HOSTMichael Straube1-0/+7
Add a case for curl error 'Could not resolve host'. An attempt to fix FS#48285. Signed-off-by: Michael Straube <straubem@gmx.de> Signed-off-by: Allan McRae <allan@archlinux.org>
2018-06-18libalpm/dload.c: fix filename in license headerMichael Straube1-1/+1
The filename in the license header did not match the actual filename as in the other files. Hopefully this is not too nit-picky. Signed-off-by: Michael Straube <straubem@gmx.de> Signed-off-by: Allan McRae <allan@archlinux.org>
2018-05-14Remove all modelines from the projectEli Schwartz1-2/+0
Many of these are pointless (e.g. there is no need to explicitly turn on spellchecking and language dictionaries for the manpages by default). The only useful modelines are the ones enforcing the project coding standards for indentation style (and "maybe" filetype/syntax, but everything except the asciidoc manpages and makepkg.conf is already autodetected), and indent style can be applied more easily with .editorconfig Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2018-03-14Update coyrights for 2018Allan McRae1-1/+1
make update-copyright OLD=2017 NEW=201 Signed-off-by: Allan McRae <allan@archlinux.org>
2018-01-06dload: ensure callback is always initialized onceAndrew Gregory1-4/+5
Frontends rely on an initialization call for setup between downloads. Checking for intialization after checking for a completed download can skip initialization in cases where files are small enough to be downloaded all at once (FS#56408). Relying on previous download size can result in multiple initializations if there are multiple non-transfer events prior to the download starting (fS#56468). Introduce a new cb_initialized variable to the payload struct and use it to ensure that the callback is initialized exactly once prior to any actual events. Fixes FS#56408, FS#56468 Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-13Introduce a 'disable-download-timeout' optionChristian Hesse1-2/+4
Add command line option ('--disable-download-timeout') and config file option ('DisableDownloadTimeout') to disable defaults for low speed limit and timeout on downloads. Use this if you have issues downloading files with proxy and/or security gateway. Signed-off-by: Christian Hesse <mail@eworm.de> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04alpm_fetch_pkgurl: fix memory leakAllan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04dload: s/CURLOPT_WRITEHEADER/CURLOPT_HEADERDATA/Dave Reisner1-1/+1
The former is really old, and should be avoided. Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04Update copyright yearsAllan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2016-12-05Parametrise the different ways in which the payload is resetMartin Kühne1-0/+10
In FS#43434, Downloads which fail and are restarted on a different server will resume and may display a negative download speed. The payload's progress in libalpm was not properly reset which ultimately caused terminal noise because the line width calculation assumes positive download speeds. This patch fixes the incomplete reset of the payload by mimicing what be_sync.c:alpm_db_update() does over in sync.c:download_single_file(). The new dload.c:_alpm_dload_payload_reset_for_retry() extends beyond the current behavior by updating initial_size and prevprogress for this case. This makes pacman reset the progress properly in the next invocation of the callback and display positive download speeds. Fixes FS#43434. Signed-off-by: Martin Kühne <mysatyre@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-12-05dload: use curl's keepalive mechanismDave Reisner1-43/+3
This does exactly the same thing as it code it replaces, but punt to curl to do it for brevity. Requires curl 7.25.0, which we already cover. Signed-off-by: Allan McRae <allan@archlinux.org>
2016-10-22Add ALPM_ERR_OK to _alpm_errno_tIvy Foster1-1/+1
This allows functions which return an _alpm_errno_t to always return a genuine _alpm_errno_t for consistency, even in cases where there are no errors. Since ALPM_ERR_OK = 0, their callers can still simply check 'err = some_fn(); if (!err) { ... }'. Signed-off-by: Ivy Foster <ivy.foster@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-08-31Replace CURLOPT_PROGRESSFUNCTION with CURLOPT_XFERINFOFUNCTIONIvy Foster1-9/+9
Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION, which deprecates CURLOPT_PROGRESSFUNCTION and means less casting doubles to size_ts for alpm. This change has no user-facing nor frontend-facing effects. Signed-off-by: Ivy Foster <ivy.foster@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-08-30Normalize alpm download callback's frontend cb argumentsIvy Foster1-6/+16
When curl calls alpm's dlcb, alpm calls the frontend's cb with the following (dlsize, totalsize) arguments: 0, -1: initialize 0, 0: no change since last call x {x>0, x<y}, y {y>0}: data downloaded, total size known x {x>0}, x: download finished If total size is not known, do not call frontend cb (no change to original behavior); alpm's callback shouldn't be called if there is a download error. See agregory's original spec here: https://wiki.archlinux.org/index.php/User:Apg#download_callback Signed-off-by: Allan McRae <allan@archlinux.org>
2016-01-04Update copyright years for 2016Allan McRae1-1/+1
make update-copyright OLD=2015 NEW=2016 Signed-off-by: Allan McRae <allan@archlinux.org>
2015-11-11Use correct format specifiersRikard Falkeborn1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2015-02-01Update copyright notices for 2015Allan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2014-12-24create_tempfile: fix memory leak on errorAllan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2014-10-19dload: mark final_url as constChristian Hesse1-4/+4
Signed-off-by: Allan McRae <allan@archlinux.org>
2014-10-16dload: unlink file on filesize exceeded errorChristian Hesse1-0/+1
On filesize exceeded error pacman leaves a .part file in cache dir, resulting in this error on next try: error: failed to commit transaction (wrong or NULL argument passed) Errors occurred, no packages were upgraded. Unlink the file on error to avoid this.
2014-10-16dload: use better error message on exceeded file sizeChristian Hesse1-4/+2
Signed-off-by: Allan McRae <allan@archlinux.org>
2014-10-02alpm: Fix wrong xferred/total sizes when resuming downloadsOlivier Brunel1-1/+3
When a package is already partially downloaded in the cache, its download size will only be of what's left to be downloaded. Since pkg->download_size is what's used when calculating the total download size for the totaldl callback, same thing apply. However, the download progress callback was including this initial size, which would thus lead to invalid values (and percentage) used in frontends. That is, the progress bar could e.g. go further than 100% In the case of pacman, there is a sanity check for different historical reason (44a57c89), so before the possible "overflow" was noticed, the total download size/progress reported was wrong. Once caught, the TotalDownload option was ignored and it would use individual file download values as fallback instead. Signed-off-by: Olivier Brunel <jjk@jjacky.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2014-08-04filecache_find_url: dereference the 'filebase' pointerRyo Munakata1-1/+1
Signed-off-by: Ryo Munakata <ryomnktml@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2014-05-22dload: avoid using CURLOPT_FAILONERRORDave Reisner1-7/+8
Use of this flag causes connections to be closed on 404s -- a common occurrence when your config sets DatabaseOptional. Handle the error gracefully, so that the connection can be reused. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2014-01-28Remove ts and sw from vim modeline when noet is setFlorian Pritz1-1/+1
Forcing vim users to view files with a tabstop of 2 seems really unnecessary when noet is set. I find it much easier to read code with ts=4 and I dislike having to override the modeline by hand. Command run: find . -type f -exec sed -i '/vim.* noet/s# ts=2 sw=2##' {} + Signed-off-by: Florian Pritz <bluewind@xinu.at> Signed-off-by: Allan McRae <allan@archlinux.org>
2014-01-06Update copyright years for 2014Allan McRae1-1/+1
Signed-off-by: Allan McRae <allan@archlinux.org>
2013-12-19dload: allow curl to response to any auth challengeDave Reisner1-0/+1
Previously, we only allowed the default of responding to basic auth challenges. Mirrors requiring authorization are far and away the edge case, but there's no sense in preventing access to them. Implements FS#38184. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2013-11-15Fix whitespace and other formatting issuesJason St. John1-1/+1
This commit: -- replaces space-based indents with tabs per the coding standards -- removes extraneous whitespace (e.g. extra spaces between function args) -- adds missing braces for a one-line if statement Signed-off-by: Jason St. John <jstjohn@purdue.edu>
2013-11-08Remove spaces between the opening "if" and the opening parenthesisJason St. John1-1/+1
Signed-off-by: Jason St. John <jstjohn@purdue.edu> Signed-off-by: Allan McRae <allan@archlinux.org>
generated by cgit v1.2.3-24-g4f1b (git 2.32.0) at 2024-06-03 09:14:46 +0200