mdraid and the 200000k speed limit
By default md-raid will limit its operations to 200000k/sec - which is plenty for most desktop and 2 - 3 disk machines, but when you have more than 3 - 4 disks and there is enough cpu and i/o bandwith available, it makes sense to increase that limit.
to find out what the limit on your machine is :
$ cat /proc/sys/dev/raid/speed_limit_max
200000
Setting it to something higher :
echo 500000 >/proc/sys/dev/raid/speed_limit_max
So whats a good speed to set ? That depends on what it is that you are looking to achieve, eg: if you dont mind max'ing out your hardware platform ( cpu / io / disks ) then set it to something very high, like 2000000. On the other hand, if you want to keep some cpu and io resources back from md-raid ( like when doing a raid-1 rebuild on a production machine ) you might want to actually lower it down a bit.
The three main issues to consider when working out a raid max speed :
- Number of disks: for aggressive sync's I tend to go with 50 - 70 M/sec per disk, so on a 4 disk system the 200000 number is mostly ok, but on a 8 or 12 disk system I'd look to make that much higher. For conservative rates, or when machine resources are required elsewhere as well, 10 - 12M/sec per disk.
- Interface: What interface you use is also going to make a big difference. So consider the implications of using IDE / SATA / SCSI.
- CPU: the raid jobs,specially when run for large disks or over many disks, will be fairly cpu intensive. So workout what sort of speeds work best for the loads you have. Usually this isnt something one needs to consider unless the machine is already under load or expected to be used during the raid operation. Over the last few years, AMD's have been able to deliver slightly better throughputs than Intel's - but in the recent past, much of that has changed. So dont just go with what you hear or opinions around the place : test it yourself.
Finally, while speed_limit_max sets the rates md-raid is going to try and reach, there is the speed_limit_min - which is the rate that md-raid will try and maintain as an 'atleast' limit. I tend to be a bit more conservative about that number. Usually aiming for 25 - 30 M/sec per disk for a very aggressive run. Or 10 - 15 M/sec for a more toned down run. If you have i/o intensive ops running on the machine you might need to reduce this even further - however the default of 1M/s for the whole machine, irrespective of disk count is something I feel too low for a modern machine.
I find many people are unaware of this small detail, hopefully this post will help.
2 comments
I think this is only good for tweaking rebuilt process.
Thanks
-Raj
24/Sep/2009 07:23:40 am, 