summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2013-04-14Draw one mirror status graph per check locationDan McGee3-23/+60
Rather than lump it all together and have odd spikes depending on which side of the Atlantic checked a mirror in a given timeslot, draw a chart per check location. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-14Turn visualize-mirror into a CSS classDan McGee2-5/+5
2013-04-14Add a JSON view for retrieving mirror check locationsDan McGee2-2/+32
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-14Fix missing attribute error in replacment find codeDan McGee1-1/+2
Whoops. Just introduced this when ensuring we look for both the packgae in other repositories as well as any replacments. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-14Remove cache_function decorator from a few spotsDan McGee2-6/+1
The benefit of these storage operations might be outweighed by the cost, especially given how infrequently these functions are called. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-14Reduce mirror status query madnessDan McGee1-55/+91
Move completely to custom SQL for this logic. The Django ORM just doesn't play nice with the kind of query we are looking to do, so it is easier to do using raw SQL. The biggest pain factor here is in supporting sqlite as it doesn't have nearly the capabilities in handling datetime types directly in the database, as well as having some different type conversion necessities. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-13Show replacments for package if it has been removedDan McGee2-0/+11
This covers the case where we can't find the package in any other repositories, but it was removed recently enough that we have a found package update object. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-13Add additional pg_trgm indexes for quicker searchesDan McGee1-0/+3
This allows our normal keyword-based search to be index-optimized rather than always doing full table scans. It requires the pg_trgm extension which is shipped out of the box with any sane install of PostgreSQL. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-13Support only a single mirror ID in error/status retrievalDan McGee2-11/+11
This simplifies things and makes injecting this single mirror ID into custom SQL a whole lot easier. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-13Calculate average URL delay in the databaseDan McGee1-19/+30
Rather than doing this in the Python code and needing 12,000+ rows returned from the database, we can do it in the database and get fewer than 300 rows back. If I recall correctly, the reason this was not done originally was due to our usage of MySQL and some really bad date math/overflow stuff it did when the interval between last_sync and check_time were greater than about a week. Luckily, we have switched to using a more sane database. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-11Allow mirror rsync IPs to be IPv4/IPv6 addresses or networksDan McGee6-15/+138
This gives us a bunch more flexibility on this field, and now supports all the options that the rsync config file supports. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-04-08Bump psycopg2 requirements.txt versionDan McGee1-1/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-29Update some package requirementsDan McGee2-6/+6
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-29Remove old-style build date parsingDan McGee1-7/+3
This was added in 2010 in commit e95c4563e32 as a short-term fix. The short-term is up. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-29Add IP family lookup to CheckLocation modelDan McGee1-0/+9
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-13mirrorcheck: s/if/elif/ when determining what check function to runDan McGee1-1/+1
This was a silly thinko here; it caused the logs to fill up with a bunch of 'unknown url type: rsync' errors. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-13Don't check FTP + IPv6 combinationDan McGee1-1/+15
Very few, if any, FTP servers support connections over IPv6. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Remove function caching in packages/utilsDan McGee1-3/+1
We don't see these called enough to make caching the data worth it. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Update home page todo lists linkDan McGee1-1/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Only show incomplete todolists to unauthenticated usersDan McGee1-1/+2
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Make todolists fully publicDan McGee2-3/+5
Remove the login_required decorator from the index and detail views to allow everyone to see the same thing. Of course, when I say "same" here, unauthenticated users don't see the same links developers do to mark packages complete and incomplete. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Add some explanatory info back to todolist index pageDan McGee1-4/+7
This is similar to what we had before on the public list page; put it here too so the page explains at least a little to the general public. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Sort incomplete todolists firstDan McGee1-1/+1
And continue sorting by creation date after that. Turns out most people care about incomplete todolists more than complete ones. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-12Remove public todolists viewDan McGee3-80/+1
Replace this with a redirect to the developer todolist index page. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-11Don't link to a one-element listing page from recent updatesDan McGee1-3/+8
If a package is built as a split package where pkgname != pkgbase, but only one actual split package is produced, the link on the recent update screen requires an extra click to get to the single package. Fix this by linking directly to the package itself. (Examples in current repos: ntfs-3g, python2-south) Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-09Remove /jsi18n/ view mappingDan McGee1-2/+0
We removed the last non-admin usage of this in commit e65c78055474, so it can be removed. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Fix IntegrityError in corner case todolist updateDan McGee1-6/+19
We were seeing this in production: IntegrityError: duplicate key value violates unique constraint "todolists_todolistpackage_todolist_id_700d1b623414814c_uniq" DETAIL: Key (todolist_id, pkgname, arch_id)=(206, ruby-cairo, 2) already exists. This is due to a corner case where a package was originally on a todolist and the underlying package object disappeared, so the todolist entry was unlinked and pkg_id set to NULL. Later, this package came back, but our get_or_create tried to create an object that violated our unique constraint because of the missing pkg_id. Call get_or_create with the minimum necessary bits to find the todolist package object, and pass the rest of the values via defaults to avoid this problem. Additionally, relink any todolist entries up to a package in the repositories if one is available. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Don't add blank options to rsync command lineDan McGee1-1/+5
Rsync doesn't like this so much: Unexpected remote arg: rsync://mirror.example.com/archlinux/lastsync rsync error: syntax or usage error (code 1) at main.c(1214) [sender=3.0.9] Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Teach mirrorcheck management command about check locationsDan McGee1-17/+46
This adds the -l/--location argument to the command in order to pass in a check location that we are currently running from. This locks the IP address family to the one derived from the address on that location, and stores any check results tagged with a location ID. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Add location ID to mirror logsDan McGee2-0/+84
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Add family property to mirror check locationDan McGee1-0/+7
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Revert "mirrorcheck: Don't use bulk_create on sqlite3"Dan McGee1-6/+1
This reverts commit 3c4ceb16. We don't need this anymore as bulk_create gets automatic batching now on sqlite3 so it is safe to use. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Add mirror CheckLocationAdminDan McGee1-1/+17
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Add 'created' field to more mirror modelsDan McGee2-2/+104
We have been better about doing this to most of our models, but the ones here didn't have a created field. Add it where appropriate and set a reasonably old default value. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-07Add mirror check locations modelDan McGee2-1/+105
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-05Add link to get HTTPS-only mirrorlistDan McGee1-0/+1
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-03-05Add HTTPS mirror protocol fixtureDan McGee1-2/+11
For now, it is not included in the default selection, but we have a few existing mirrors that do support it. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-28Add django.http.response:Http404 to ignored NewRelic errorsDan McGee1-1/+1
This moved around in Django 1.5, so add it to the override list so it continues to be ignore like it was in older versions of Django. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Disable more @cache_function usageDan McGee1-2/+2
Seems the Django caching layer has changed quite a bit and is doing all sorts of funky s**t at this point. Yay for errors! Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Remove caching on reverse_conflictsDan McGee1-1/+1
We started seeing this in production with memcached usage and the upgrade to Django 1.5: PicklingError: Can't pickle <class 'devel.models.UserProfile_allowed_repos'>: attribute lookup devel.models.UserProfile_allowed_repos failed Without having time to investigate further, something changed, likely due to the whole user profile/configurable user model shenanigans done in Django 1.5. For now, simply don't cache this attribute to work around the problem. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Use user.userprofile rather than user.get_profile()Dan McGee2-3/+3
The get_profile() function is deprecated as of Django 1.5. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Don't blow up in download template if no releases existDan McGee1-3/+3
Protect a few more things in {% if %} block logic. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Fix ALPM unittest to run when ALPM isn't presentDan McGee1-2/+21
This gets some proper unittest skip decorator action now in addition to adding more testing around everything. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-27Merge branch 'django-1.5'Dan McGee23-65/+29
Conflicts: requirements.txt requirements_prod.txt
2013-02-27Bump Django requirement to 1.5Dan McGee2-2/+2
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-21Package sitemap adjustmentsDan McGee1-5/+15
Adjust changefreq and priority based on the repository the package is in. Testing and staging have quick turnover so set the changefreq to 'daily'. Additionally, give staging packages a super low priority of 0.1 and testing a slightly lowered priority of 0.4. We didn't include staging packages before, but since search engines are finding them anyway, we might as well. This gives us a chance to share how important the page is as well, which they could only guess before. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-20Fix some fallout with moving page to query paramsDan McGee1-1/+3
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-20Update Django versionDan McGee2-2/+2
Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-16Make page a query string parameter on package searchDan McGee3-3/+4
This is a bit silly to encode in the URL, or at least makes it much harder to screen out via robots.txt and other such things. Signed-off-by: Dan McGee <dan@archlinux.org>
2013-02-16Remove configurable pagination for package searchDan McGee2-40/+1
Switch it to a hardcoded value of 100 for all searches instead. It didn't make much sense having a page number be part of the URL and a limit value being part of the query string. Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3-24-g4f1b (git 2.32.0) at 2024-11-27 12:11:05 +0100