Upgrading to Snology DS1517+ (8GB RAM model) with 10GbE networking?

[Juraj]

My network drives are PCI-E Intel P3600 2TB each, so the 10gbit does actually bottleneck them in continual write/read speeds :- ) but I barely ever do that. It caps at 1200 MB/s +/-.

[3dwannab]

That's still good :)
Having these in these setup it's the CPU and RAM freq. that's the bottleneck. With overclocking to 4GHz you can get past 7GB/Sec read but similar writes.


My setup with this test was using stock threadripper 1950x [not overclocked, yet but will do eventually] and 128GB RAM @ 2,800MHz.

[Rhodesy]

Just hijacking this thread! I'm thinking about getting the DS1817+ with a 10gb card and 8 port 10gb unmanaged switch. I have recently bought 4x4gb iron wolf dives for my existing nas which is purely a back up drive with versioning. I thought I'd be able to use those drives and get 4 more of the same and run a raid 10 setup with 2 disk redundancy. But I have just checked them and they are only the 5900rpm flavour not 7200. Do you think this will make a big difference in that raid 10 setup for seak times which is important for tracking all those textures down? I would rather bite the bullet and buy some new larger capacity faster drives if it is going to make the difference but it's probably going to cost a good £600+ to do that on top of everything else. So don't want to do that if it's a waste.

Also am I likely to see speeds that are faster than my middle of the road ssd with this setup?

Finally do you know if it's possible to make a drive letter on the nas so it looks exactly like a local drive? I've got many materials all over the shop which have direct paths set to my texture ssd and it would be a nightmare to lose all those?

Thanks for any help.

[3dwannab]

I've ended up getting a 10g switch anyway in conjunction with 2 nvmes in raid0.

I installed the 4 drives into the new one using snologys online tut on it no problem. Took half an hour.

Then installed the extra drive. So far it's taken 6 days to do 70% on a volume parity consistency. So, I've not been able to install the 10g setup. I was getting 120mb on the old 1g setup.

As far as searching files. Windows is slow compared to using freecommder explorer SW even when you uncheck search file contents. I bought the donor version of freecommder which is x64 and very good.

You cannot index a NAS like your internal drives. I've seen workarounds but yet to try them out.

Yes, mapping the drive is the same as usual. Assign Z as this is the most common server/NAS drive letter. Up to you though.

Will post back when I get the WR speeds and what searching is like when this parity finishes. (pulls out hair)

[Rhodesy]

Thanks for the info and looking forward to your update. Will look in to free commander.

[3dwannab]

Parity checking FINALLY stopped so I could install the 10GbE (simple) setup with the DS1517+. I just installed the 10G NIC with the small bracket in the 1517+, installed the RJ45 10GbE NIC that came with my Zenith Extreme motherboard and used the ASUS switch mentioned in above posts. (RJ45 type)

Here's the WR speeds comparison between my new NAS speeds and an SSD. Lastest drivers might bump speeds up a bit. (I dunno).
(My old 1GbE setup was 120 Write and around 80 read in Seq.)
https://imgur.com/a/Ia2z0
This new setup is roughly 5.5 times better performance than the old also. Not exactly 10 times but I'm sure that's because I'm running SHR RAID. I've more testing to do.

Took about a minute in FreeCommander to finish searching a folder for one filename around 1minute.

Folder was quite large @ 455GB, 19105 folders & 102219 files.

Hope this info helps.

[Juraj]

Btw, a bunch of Asus XG C100C 10gbit cards arrived at my office :- ). Really nice, single-port, passively cooled and you can get them for 90 euros each. Imho better than the Intel card, let's see if drivers are fine.

[Rhodesy]

They must be the same as the one packaged with the Zenith mobo. Yes great to have the option of a cheaper single socket card. Its more expensive in the UK of course!

Thanks for the update 3Dwannab. You have done well to get 5.5X performance. Most of the other reviews I have read give about a 3-4 X performance over 1GB speeds but thinking about it that was just with standard NAS drives. Im confused I thouight you had to chose either the 10GB card or PCI based SSD in that unit as there was only one slot?

[3dwannab]

They must be the same as the one packaged with the Zenith mobo. Yes great to have the option of a cheaper single socket card. Its more expensive in the UK of course!

Thanks for the update 3Dwannab. You have done well to get 5.5X performance. Most of the other reviews I have read give about a 3-4 X performance over 1GB speeds but thinking about it that was just with standard NAS drives. Im confused I thouight you had to chose either the 10GB card or PCI based SSD in that unit as there was only one slot?

Yeah, they prob slapped the outer casing on mine and rebranded it.

No, my workflow is two nvmes in RAID0 (2TB), which backs to the NAS every so often, then external offsite every weekend. I went ahead and got the 10g setup anyway.

Why the hell not, I like blowing my money away ;)

Btw, a bunch of Asus XG C100C 10gbit cards arrived at my office :- ). Really nice, single-port, passively cooled and you can get them for 90 euros each. Imho better than the Intel card, let's see if drivers are fine.

Same as the one I have only rebranded like I said probably. Let me know your performance please.

[3dwannab]

10Gbe UPDATE:

I was getting these results in CDM below.



But in transferring files, I was only getting 80MB/s. Not sure what I did to get this but I swapped the LAN port on the back of the i10Gb X540-T2 and get results that were closer to the CDM results.
I thought that the two ports on that were 10GbE. Or else the top one of the two is faulty.

I'm getting real file transfer speeds (when copying a 16GB file)

NAS & HD SSD:
500MB/s from NAS to HD SSD
350MB/s from HD SSD to NAS.

NVMe SSD (960 PRO) & NAS:
430MB/s from NAS to NVMe SSD
350MB/s from NVMe SSD to NAS

What stumps me is the transfers are better to the HD SSD. The NVMe is a little bit worse. I wonder what the bottleneck would be.

[3dwannab]

I have to say I've completely overlooked a CMD command called robocopy for my backup procedure!!

It's quite amazing.

I've created a W Drive Backup.cmd file (W Drive Backup can be any filename you like) and wrote up this little beaut. One click and it runs the copy with options via the switches. I just need to find a program that can run this every hour or so.

Just don't name the file robocopy.cmd or you're in for a world of pain resulting in a continous loop of the .cmd file.

For the schedule, windows built-in 'Task Scheduler' will probably do the trick to start it upon login and the /MON:n AND /MOT:m switches will monitor changes and by time. Please refer to code for these two switches.

Code: [Select]

REM title Backing Up Made Easy. [%~nx0] by 3dwannab
@ECHO off

SET SOR_PATH1=W:
SET DES_PATH1=Z:\Backups\Drive W Backup
Start /Min "JOB: %DES_PATH1% Job" robocopy "%SOR_PATH1%" "%DES_PATH1%" /MON:50 /MOT:30 /XO /MIR /FFT /Z /XA:H /R:10 /W:10 /MT:5 /XD "$RECYCLE.BIN" "System Volume Information" /XF "thumbs.db"
attrib -s -a -h "%DES_PATH1%"

::::::::::::::::::::::::::::::::::::::::

SET SOR_PATH2=C:
SET DES_PATH2=Z:\Backups\Drive C Backup
Start /Min "JOB: %DES_PATH2% Job" robocopy "%SOR_PATH2%" "%DES_PATH2%" /MON:50 /MOT:60 /XO /MIR /FFT /Z /XA:H /R:10 /W:10 /MT:5 /XD "$RECYCLE.BIN" "System Volume Information" "*Windows*" "*microsoft*" "*dropbox*" "nvidia" "temp" "C:\Windows" "C:\ffmpeg" "C:\PerfLogs" "C:\Python34" "C:\Swsetup" "C:\temp" /XF "C:\*" "thumbs.db" "*.thumb" "*.bak" "*.sv$"
attrib -s -a -h "%DES_PATH2%"

REM @pause

:: More Info here:
:: https://social.technet.microsoft.com/wiki/contents/articles/1073.robocopy-and-a-few-examples.aspx
:: https://ss64.com/nt/robocopy.html

:: NOTES
:: ---------------------------------
:: start /min runs robocopy in minimised mode.
:: /MT[:n] where n = no. of threads being used.
:: /MIR specifies that Robocopy should mirror the source directory and the destination directory. Note that this will delete files at the destination if they were deleted at the source.
:: /FFT uses fat file timing instead of NTFS. This means the granularity is a bit less precise. For across-network share operations this seems to be much more reliable - just don't rely on the file timings to be completely precise to the second.
:: /Z ensures Robocopy can resume the transfer of a large file in mid-file instead of restarting.
:: /XA:H makes Robocopy ignore hidden files, usually these will be system files that we're not interested in.
:: /R:<n> Specifies the number of retries on failed copies. The default value of n is 1,000,000 (one million retries max).
:: /W:<n> Specifies the wait time between retries, in seconds. The default value of n is 30 (wait time 30 seconds).
:: /MON:n : MONitor source; run again when more than n changes seen.
:: /MOT:m : MOnitor source; run again in m minutes Time, if changed.

:: EXAMPLES
:: ---------------------------------
:: #8 Mirror directory excl. deletion
:: To mirror the directory "C:\directory" to "\\server2\directory" excluding \\server2\directory\dir2" from being deleted (since it isn't present in C:\directory) use the following command:

:: Robocopy "C:\Folder" "\\Machine2\Folder" /MIR /XD  \\server2\ directory\dir2"
:: Robocopy can be setup as a simply Scheduled Task that runs daily, hourly, weekly etc. Note that Robocopy also contains a switch that will make Robocopy monitor the source for changes and invoke synchronization each time a configurable number of changes has been made. This may work in your scenario, but be aware that Robocopy will not just copy the changes, it will scan the complete directory structure just like a normal mirroring procedure. If there are a lot of files & directories, this may hamper performance.
Basic explanation of code shown after :: in code above with resource links.
This will back up my W Drive (Work Folder) and C Drive with a few omissions as you can see.
The only thing I'll change after the initial backup is the thread count of the copies to maybe 6 so it's less in your face when running every hour.

If you want to autoclose the CMD delete the @pause or block it by adding :: to it like so ::@pause