rageek

A place for Unix Thoughts and Ideas

Strickly Limiting ZFS arc cache size

On the majority of my servers I use ZFS just for the root filesystem and allowing the arc to grow uncheck is counterproductive for tracking server utilization and running some applications.

Consequently I severely limit the amount of memory used and set it at 100MB.

If your going to limit the arc cache, just about every ZFS tuning guide suggests capping the arc cache limit via zfs:zfs_arc_max

However, I was digging into the memory utilization of one of my Tibco servers and noticed that the ZFS arc cache was quite a bit larger than value specified in /etc/system

root@testserver # kstat zfs:0:arcstats:size | grep size | awk '{printf "%2dMB\n",  $2/1024/1024+0.5}'
1990MB

I actually noticed this before in the past, but didn’t research any further due since it was insignificant compared to the free ram on that server.

I checked a couple other servers and noticed that it was consistently around 2GB for most of my servers.

root@testserver # grep zfs /etc/system
set zfs:zfs_arc_max = 104857600

In checking Kstat, I noticed a minimum parameter for zfs that I hadn’t noticed before that looked very similar to my arc size.

root@testserver # kstat -p zfs:0:arcstats | head -4
zfs:0:arcstats:c        2101237248
zfs:0:arcstats:class    misc
zfs:0:arcstats:c_max    104857600
zfs:0:arcstats:c_min    2101237248

Referring the Oracle Solaris Tunable guide, the zfs_arc_min parameter is set to 1/32nd of the physical memory or a minimum of 64MB. 2GB on a 64GB system, 4GB on 128GB one.

So I now include the maximum and minimum values in /etc/system and now the limits are occurring as predicted.

root@testserver # grep zfs_arc /etc/system
set zfs:zfs_arc_max = 104857600
set zfs:zfs_arc_min = 104857600

root@testserver # kstat -p zfs:0:arcstats | head -3
zfs:0:arcstats:c        104857600
zfs:0:arcstats:c_max    104857600
zfs:0:arcstats:c_min    104857600

root@testserver # kstat zfs:0:arcstats:size | grep size | awk '{printf "%2dMB\n",  $2/1024/1024+0.5}'
100MB

Now that I have realized that the majority of my servers have arc caches set to 1/32 of ram, I can take a good look at whether I should increase my intended defaults or leave them as is.

Advertisements

One response to “Strickly Limiting ZFS arc cache size

  1. irraz January 11, 2013 at 1:05 am

    Thank served me very helpful, had not found good references on this subject.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: