Slow download of TrueNAS Scale ISO

[Dominick_Storj]

Appreciate the report and the detail. Escalating this internally now.

Our engineering teams continue to investigate. We are exploring solutions to route around the issue. Please see the two links below which likely explain what is going on.

The European Network Usage Fees proposal is about much more than a fight between Big Tech and Big European telcos

There’s an important debate happening in Europe that could affect the future of the Internet. The European Commission is considering new rules for how networks connect to each other on the Internet.

blog.cloudflare.com

Reddit - Dive into anything

www.reddit.com

 

[MrGuvernment]

Our engineering teams continue to investigate. We are exploring solutions to route around the issue. Please see the two links below which likely explain what is going on.

The European Network Usage Fees proposal is about much more than a fight between Big Tech and Big European telcos

There’s an important debate happening in Europe that could affect the future of the Internet. The European Commission is considering new rules for how networks connect to each other on the Internet.

blog.cloudflare.com

Reddit - Dive into anything

www.reddit.com

Bloody sad this, Telco's make 10 to 100's of millions in profits every year, do as little as they can to upgrade their networks and complain so loudly when people actually use them and they have to spend money (profits) to update their aginging infra.

Canada is the same here...Telco's try to do the least they can and then complain and want funding from governments to do upgrades...and then spent it on other things and complain again when it happens over and over..

 

[cismorov]

I used my mobile plan and could download the ISO without problems, stable at 5-6 MB/s.

Downloads over cable still very slow and mostly interrupts at about 10%.

Location: Vienna
Mobile provider: Drei
Cable provider: Magenta

 

[gtyovkov]

I am also based in Vienna and experiencing the same issue. Sadly, both my cable and mobile provider are Magenta (T-Mobile).

As a solution to ISPs limiting your means of distribution, wouldn't a PtP like BitTorrent be a good addition? Everyone would be able to verify integrity with the keys/hashes from your website, but we won't be relying solely on this one ISO link.

 

[Dominick_Storj]

We are actively working to mitigate these spot issues by deploying capacity in region. This is an ongoing effort, and I will have updates within the next few days.

 

[diwjjsvdnljsadsouh]

Came to this thread due to the same problems in northern Germany (Provider Deutsche Telekom).
To circumvent it I tried to download it via Azure Cloud Shell, but although it downloaded completely the image was incomplete.
Finally, it worked from a VPS in Frankfurt.

 

[CH4Pz]

FYI, downloading the new update through the UI was extremely slow too, around 80kb/s, so had to kill the update by restarting, connect to VPN, and download the manual update file, to update that way.

 

[Dominick_Storj]

We appreciate the reports. Fixes for ISP speed issues in locations like Europe have been deployed. We have additional capacity with an even greater mix of providers coming online soon that will further improve the consumer ISP connectivity. This will also benefit worldwide performance as there are additional routes/ISPs that can be used globally.

 

[nifoc]

Just found this thread because of DL speed issues (latest SCALE .update file) and I have to say that I cannot confirm the speed issue has been fixed.

My ISP is Deutsche Telekom and I'm getting 150 KB/s max with random aborts. Have not been able to download more than 100 MB before the download aborts.

 

[Dominick_Storj]

Just found this thread because of DL speed issues (latest SCALE .update file) and I have to say that I cannot confirm the speed issue has been fixed.

My ISP is Deutsche Telekom and I'm getting 150 KB/s max with random aborts. Have not been able to download more than 100 MB before the download aborts.

Appreciate the report, would you be able to provide us a traceroute to download.truenas.com? We appreciate the opportunity to observe routing via Deutsche Telekom.

 

[nifoc]

Appreciate the report, would you be able to provide us a traceroute to download.truenas.com? We appreciate the opportunity to observe routing via Deutsche Telekom.

Thanks for the quickly reply.

Sent a private message with a trace.

 

[Dominick_Storj]

Thanks for the quickly reply.

Sent a private message with a trace.

Thank you for the detail. Interesting that there are still peering issues via Deutsche Telekom to Hurricane Electric. We are researching options with Deutsche Telekom which has a history of peering issues.

 

[nifoc]

Thank you for the detail. Interesting that there are still peering issues via Deutsche Telekom to Hurricane Electric. We are researching options with Deutsche Telekom which has a history of peering issues.

I'm not a fan of what they're doing, but you have to pay DTAG and peer with them directly.
Or - depending in how you operate - get servers/colo somewhere that peers with them directly, like Hetzner or route traffic via Core-Backbone (also has direct DTAG peering).

 

[nifoc]

@jwilloughby Let's keep it in this thread. DL speed has not improved:

Code:

 3  f-ed14-i.f.de.net.dtag.de (217.0.203.22)  11.865 ms
    f-ed14-i.f.de.net.dtag.de (217.0.201.114)  11.797 ms
    f-ed14-i.f.de.net.dtag.de (217.0.203.22)  10.558 ms
 4  80.157.201.198 (80.157.201.198)  11.457 ms  16.089 ms  10.830 ms
 5  be3186.ccr41.fra03.atlas.cogentco.com (130.117.0.1)  11.434 ms  10.566 ms
    be3187.ccr42.fra03.atlas.cogentco.com (130.117.1.118)  11.455 ms
 6  be2814.ccr42.ams03.atlas.cogentco.com (130.117.0.141)  15.660 ms  14.847 ms
    be2813.ccr41.ams03.atlas.cogentco.com (130.117.0.121)  16.239 ms
 7  be2795.rcr21.b047839-0.ams03.atlas.cogentco.com (154.54.58.22)  24.592 ms
    be2796.rcr21.b047839-0.ams03.atlas.cogentco.com (154.54.58.126)  23.127 ms  27.092 ms
 8  * * *
 9  * * *

Fluctuates between 160 KB/s and 80 KB/s.

Going through DE-CIX will probably never really resolve these issues.

 

[justjasch]

Hi,

having the same Problem from Germany via DTAG Provider. (German Telekom)
(i have a Backup Line in my location from 1&1 (German Provider) wich works fine)
looks like is really a Telekom Problem


tracert download.truenas.com via DTAG Line -> Download slow with disconnects

Routenverfolgung zu link.storjshare.io [136.0.77.2]
über maximal 30 Hops:

1 <1 ms <1 ms <1 ms 192.168.1.254
2 11 ms 9 ms 9 ms p3e9bf7fe.dip0.t-ipconnect.de [62.155.247.254]
3 17 ms 16 ms 16 ms f-ed14-i.F.DE.NET.DTAG.DE [217.5.111.94]
4 31 ms 31 ms 30 ms 80.157.201.198
5 31 ms 30 ms 31 ms be3187.ccr42.fra03.atlas.cogentco.com [130.117.1.118]
6 30 ms * 30 ms be2814.ccr42.ams03.atlas.cogentco.com [130.117.0.141]
7 31 ms * 30 ms be2796.rcr21.b047839-0.ams03.atlas.cogentco.com [154.54.58.126]
8 55 ms 62 ms 64 ms fdcservers.demarc.cogentco.com [149.11.203.10]
9 * * 37 ms 136.0.77.2



tracert download.truenas.com via 1&1 Line -> Download ok

Routenverfolgung zu link.storjshare.io [136.0.77.2]
über maximal 30 Hops:
1 <1 ms 1 ms <1 ms 192.168.0.254
2 66 ms 66 ms 79 ms i5E86C70.versanet.de [94.134.199.0]
3 95 ms 78 ms 79 ms 62.214.42.187
4 122 ms 119 ms 131 ms core-backbone.versatel.de [5.56.18.222]
5 31 ms 30 ms 30 ms ae19-0.fra30.core-backbone.com [5.56.18.221]
6 37 ms 37 ms 37 ms ae5-2074.ams10.core-backbone.com [81.95.2.138]
7 35 ms 34 ms 34 ms coreb-ams.cdn77.com [169.150.195.66]
8 34 ms 34 ms 34 ms vl224.ams-eq6-dist-2.cdn77.com [185.229.188.215]
9 38 ms 38 ms 38 ms 136.0.77.2




wbr Alex

 

[IndieCoopz]

I feel that the whole TrueNAS experience is unbelievable slow sice the problems with DTAG. Installing Apps, loading preview images in the new "App Store" etc. It feels that every external server connection is slow. Hard to use since this problem appeared

 

[nifoc]

I feel that the whole TrueNAS experience is unbelievable slow sice the problems with DTAG. Installing Apps, loading preview images in the new "App Store" etc. It feels that every external server connection is slow. Hard to use since this problem appeared

I don't use Apps/Kubernetes, but from a quick look, this does appear to be the same issue.

I'm not sure where the catalog itself is fetched from, but preview images are accessed via media.sys.truenas.net, which is just a CNAME for link.storjshare.io.

@jwilloughby @Dominick_Storj Are you aware of the "recent" updates in this thread?

 

[Patrick M. Hausen]

I have one system running the 13.1 nightlies. Performing an update via UI is impossible. I have to activate a VPN connection to a FreeBSD VPS located in New Jersey that runs WireGuard, download the manual update tar file, then apply that.

Deutsche Telekom, business DSL line with fixed IP addresses, generally good performance.

 

[Dominick_Storj]

Wanted to update everyone that additional download capacity in Germany and Oceania will be coming online towards the end of the month. Key is the additional capacity in Germany will have more favorable peering with Deutsche Telekom.

@Patrick M. Hausen if you could provide me a traceroute from the system located in New Jersey I will promptly send that information over to the team for resolution. We should have fantastic performance from New Jersey.

 

[Patrick M. Hausen]

I use the VPN to a VPS in New Jersey to get good performance It's without when it is unusable.
Yesterday I found an even better solution: install tinyproxy in that VPS, configure TN to use proxy. No manual download necessary, anymore.