possible to make download resume?

[thyeun]

OS : Arch linux

been trying to completed the download for 2..0GB file size for 9 hours, but it keep on close-up the download connect around 1.5 -1.7GB, with the message below

curl: (18) transfer closed with 1529402169 bytes remaining to read

possible to make it resumeable? so that once the transfer closed up, it will be resume again

[ryanoasis]

@thyeun Hey really sorry to hear that. I know this repo is really large.

What is your method for downloading? You might be better off with an un-official AUR package or ad hoc Curl only download the fonts you want

possible to make it resumeable? so that once the transfer closed up, it will be resume again

I think this is beyond the scope of this project, however if you can show an example of how you are currently downloading the repo I would gladly like to update the docs for an examples of resuming a download 😃

[thyeun]

@ryanoasis, i'm using wget -c but still cant resume, how about using zstd for compression method?

as you know linux system not much of choice of downloading apps, mostly the best is curl or wget . since both given me the above message, i'm almost given up of using nerd font.

Anyway thanks for your feedback, and i know it is not related with nerd font, just a bit sad after 6 month of using it, i like it so much, but if i can't make an upgrade, than i have to move on.

[thamma]

I find myself with a similar problem. I want to patch some fonts myself and thus would like to clone the repository. However the master branch is huge. Is it possible to move the dev tools to a spearate branch (since it's too late to move the fonts themselves) so we can only clone what we need?

I don't think anyone appreciates downloading all the fonts when they just need the font-patcher.

[thyeun]

@thamma, are you using arch linux? for arch users, the maintainer guy already make the installation into 400mb, so it is more comfortable to finish the download with 400mb.

Hope it is helping you with above info. and thanks to arch maintainer guy for this installation, he really make it as small as possible for benefit to everyone that need it.

[thamma]

Conveniently, I am! Thanks for the heads up!

However I think this feature (a script-only branch) should be considered.

[rfvgyhn]

Just wanted to echo what @thamma said. I'd like to patch my own font without having to download the huge repo. Maybe you could include a zip of just the dev tools in the releases instead of creating a new branch for the dev tools?

I might be able to create a PR for that if that's something you agree with. Does font-patcher depend on only src/glyphs and optionally config.json?

[ryanoasis]

@rfvgyhn @thamma My thinking was to eventually do this: https://github.com/NerdFonts/patcher

Does font-patcher depend on only src/glyphs and optionally config.json?

Yes, that should be all that is required.

Related: #73

[oblitum]

The nerd-fonts-complete AUR package was recently updated to download repo tar file from GitHub and I'm hitting the same wall as this issue, it always fails at around ~1GB, and I can't resume it, tried curl used in yay, and aria2c. I'm on 30MB connection, download rate goes at ~4MiB and is expected to finish at around 10min, but it never finishes.

[thyeun]

@oblitum you can try using trizen instead of yay or aria2c, i'm now possible to finish it, even it fail, trizen will re-continue (as an example if you stop at 500mb/1.3GB, than it will continue back start from 501mb/800mb)

[oblitum]

@thyeun but this has never been an issue with curl and specially with aria2 which I've always used for downloads with re-continue. I'm not willing to replace AUR helper just for this, and I still ignore why other tools that support re-continue don't work.

[thyeun]

@oblitum Your only choice to download this complete font, is to try it using trizen, but since you dont want to change AUR, than nothing much i can help you here.

and for your info, curl/aria2/wget having the re-continue download issue on this (i dont know why but it happen)

[oblitum]

I'll just assume it's github issue, not client. That trizen happens to work for you is an outlier. Not having a command line downloader that works with this, and having to replace a manager of my packages is a bit too much...

[francoism90]

@oblitum You didn't had this issue before because it was using a different mirror and not this one.

Also I'm the maintainer for this PKG at the moment, please check the latest fixes.

[oblitum]

please check the latest fixes

Latest fixes did not change URL, so no deal for me, can't download it.

[francoism90]

@oblitum Please check the following link for the previous URL: https://aur.archlinux.org/cgit/aur.git/commit/?h=nerd-fonts-complete&id=c281aa0ee83f080bb50f02b8fd96e47059a0280f

Maybe a mirror can be provided, but like said before, I only want to use an official one.

[oblitum]

@francoism90

Please check the following link for the previous URL: https://aur.archlinux.org/cgit/aur.git/commit/?h=nerd-fonts-complete&id=c281aa0ee83f080bb50f02b8fd96e47059a0280f

These are not latest fixes, so not relevant. When I said latest fixes did not change URL, I meant from 2.1.0-1 to 2.1.0-2, since it's from 2.1.0-1 that it stopped working.

To give more detail that it's a server issue, when downloading with aria2c I don't get whole file size, so it keeps downloading without status, and when it fails at ~1GB, it doesn't have state to resume, add to the fact that URL and filename differs, but even with flags like --auto-file-renaming=false, it doesn't work.

[oblitum]

There was one occasion that server for some mysterious reason provided whole file size, but I was not using --auto-file-renaming=false at the attempt, so I missed the opportunity to resume after failing, since without the flag it simply started a new download.

[thyeun]

@oblitum lolz, why cant you try using trizen? it is not outlier, it working with resume.

almost two years ago, i open this issue because i had the same issue as you facing it now, and trizen is the one solving this.

And for the gitlab url (on 2.0.0-5), Dan Beste personally download it and zip it separately from the main stream. And that package are 700mb++, if you compare to the original one on here, it is 2.2GB.

So, just move on if you does't want to try trizen, and there are not much we can help you.

[oblitum]

@thyeun great that you opened the issue, now please unsubscribe if you're not interested anymore, afaict, it's still present, not everyone is willing to use, or even have trizen available.

[a-y-u-s-h]

@thyeun It's not working with trizen as well. It crashes once, I try again and it continues. It then crashes again after 2 GBs and then every time I retry I get the error:

curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
==> ERROR: Failure while downloading https://github.com/ryanoasis/nerd-fonts/archive/v2.1.0.tar.gz

Please reopen this issue.

[francoism90]

@a-y-u-s-h Could you please try to build without any helper, just download the PKG and run makepkg -sci.

I do have the seem issue with yay, and I think this is because GitHub simple doesn't support resume on the HTTP server and the helpers are instructed to use it.

Maybe switching to git cloning instead of download the release tarball will fix this issue.

[a-y-u-s-h]

Thanks, Git cloning worked. I was having issues with yay, trizen and other aur helpers.

[thyeun]

@thyeun It's not working with trizen as well. It crashes once, I try again and it continues. It then crashes again after 2 GBs and then every time I retry I get the error:

curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
==> ERROR: Failure while downloading https://github.com/ryanoasis/nerd-fonts/archive/v2.1.0.tar.gz

Please reopen this issue.

@a-y-u-s-h it is not crash, the message you getting it is because your resume are out of 20 times, and it is because you have slow internet connection.

[oblitum]

@francoism90 if you go about fixing the issue on AUR with git clone, I'm not sure whether it's automatically taken care or not, but just want to express about having a shallow clone, because this repository is LARGE.

[oblitum]

and it is because you have slow internet connection

Incorrect reason in my case, which have same behavior and I'm sure trizen would not make any diff, hence not tried. Download is simply aborted halfway on 30MB connection, expected to finish in less than 15min (but always failing before).

[a-y-u-s-h]

@thyeun I encountered this error earlier many times with yay, my internet connection is 30 Mbps, so it's not an issue. With yay, long ago when I encountered this error, I increased the size of buffer for git clone like so: git config --global http.postBuffer 6291456000 and it worked in the sense that it then showed me a different error http server doesn't seem to support byte ranges.

@oblitum Trizen did continue but showed me the same http server doesn't seem to support byte ranges error after 2 GBs and then I simply tried the git clone way (yes shallow clone) and it worked. Have been at it since yesterday now.

[toketin]

Hi, I'm facing the same issue, I've tried to download the tar.gz from yay, aria2, Firefox, trizen, and also by manual makepkg -sci. It always fails and kills the download.

[francoism90]

@toketin Why does it fail in Firefox? Do you use a download manager?

[toketin]

No I don't use a download manager on FF, it failed after about 800 MB downloaded, I don't know why.

…

On Sat, Mar 28, 2020, 15:46 François M. ***@***.***> wrote: @toketin <https://github.com/toketin> Why does it fail in Firefox? Do you use a download manager? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#274 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAD75HEAEKYWCJ2BOQH7U3RJYEUVANCNFSM4FC6RZCA> .

[francoism90]

@toketin Weird, maybe you should report this to GitHub?

I don't think git cloning is an alternative. I'm trying to make it to work, but so far it takes even more space as it pulls the history as well.

[oblitum]

I'm trying to make it to work, but so far it takes even more space as it pulls the history as well.

Even with shallow clone?

[francoism90]

@oblitum That isn't possible with PKGBUILD if I'm not mistaken.

[oblitum]

@francoism90 https://unix.stackexchange.com/questions/154919/how-to-modify-a-pkgbuild-which-uses-git-sources-to-pull-only-a-shallow-clone

[oblitum]

@francoism90 https://unix.stackexchange.com/questions/154919/how-to-modify-a-pkgbuild-which-uses-git-sources-to-pull-only-a-shallow-clone

hmm, looks like no solution there is self-contained to the package :/

[oblitum]

It looks like the only way out is to execute git clone --single-branch --depth 1 from prepare() step.

[francoism90]

@oblitum It seems /etc/makepkg.conf can be tweaked to use --depth 1, however this will be applied globally.

I would like to ask to first contact GitHub, maybe they can provide a solution for this.

[oblitum]

k, though I really don't expect github to fix this anytime soon, I'm not even trying.

[oblitum]

@francoism90

It looks like the only way out is to execute git clone --single-branch --depth 1 from prepare() step.

diff --git a/PKGBUILD b/PKGBUILD
index 8f26a66..240497a 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -15,13 +15,13 @@ depends=('fontconfig' 'xorg-font-utils')
 conflicts=('nerd-fonts-git' 'nerd-fonts-complete-mono-glyphs')
 source=(
   'fix-installer-font-dir.patch'
-  "${_gitname}-${pkgver}.tar.gz::https://github.com/ryanoasis/nerd-fonts/archive/v${pkgver}.tar.gz"
 )
-sha256sums=('ccf93b108044a87bfb29c3f836d2ce4d5bdb1829702e532a69ccb4ab4aecaceb'
-            'a084ca91a174b547bab4523507824c76aa91ebcf38f9256a4ffd181813f87bd8')
+sha256sums=('ccf93b108044a87bfb29c3f836d2ce4d5bdb1829702e532a69ccb4ab4aecaceb')

 prepare () {
-  cd "$srcdir/$_gitname-$pkgver"
+  cd "$srcdir"
+  git clone --branch v${pkgver} --depth 1 https://github.com/ryanoasis/nerd-fonts.git "$_gitname-$pkgver"
+  cd "$_gitname-$pkgver"

   patch -Np1 -i "$srcdir"/fix-installer-font-dir.patch
 }

[francoism90]

@oblitum Thanks for your patch. Could you please tell me the download size?
I'm a bit mixed on this workaround: it's not a clean solution as it doesn't use any of the inbuilt CVS makepkg hooks and may lead to some difficulties in clean-up later on. At the moment please submit your patch to the AUR package if other users want to use it.

My preferred solution would be using a different mirror (e.g. GitLab) to download the archive instead. I don't know if @ryanoasis would like to help us setup a mirror to this repo. The previous maintainer simple uploaded his own archive, which I really don't want to because it's not secure in anyway.

Thanks.

[oblitum]

Could you please tell me the download size?

I didn't track it in full, but git reported around 1GB for the shallow clone, after initial git setup, which took some time, so I'd say it's around 1.5GB. For sure it was faster than ~15min for the tar snapshot download.

It's up to you what to do, I just think expecting official solution from others will have no ETA, better fix where yourself is able to fix it now, for your AUR package, and wait for GitHub/upstream fix to come later eventually and pick it when available, if ever.

[oblitum]

My preferred solution would be using a different mirror (e.g. GitLab) to download the archive instead. I don't know if @ryanoasis would like to help us setup a mirror to this repo.

The GitHub release page provides individual zips for each font. I think @ ryanoasis isn't required to do any extra work here, since a packager can already package it up from all individual files (possibly even with parallel download) instead of picking it from a tar snapshot.

So there's two solutions, git shallow clone on prepare, or having the package based on all the zipped fonts provided on the release page assets.

[github-actions]

This issue has been automatically locked since there has not been any recent activity (i.e. last half year) after it was closed. It helps our maintainers focus on the active issues. If you have found a problem that seems similar, please open a new issue, complete the issue template with all the details necessary to reproduce, and mention this issue as reference.